Skip to content

feat: add Bedrock Knowledge Base retriever to amazon-bedrock integration - #3582

Merged
davidsbatista merged 10 commits into
deepset-ai:mainfrom
PVidyadhar:bmkb-managed-kb-support
Aug 5, 2026
Merged

feat: add Bedrock Knowledge Base retriever to amazon-bedrock integration#3582
davidsbatista merged 10 commits into
deepset-ai:mainfrom
PVidyadhar:bmkb-managed-kb-support

Conversation

@PVidyadhar

Copy link
Copy Markdown
Contributor

feat: add Bedrock Knowledge Base retriever to amazon-bedrock integration

Related Issues

N/A — new feature adding Knowledge Base retrieval to the existing Amazon Bedrock integration.

Proposed Changes:

Adds BedrockKnowledgeBaseRetriever component to the amazon-bedrock-haystack integration, enabling document retrieval from Amazon Bedrock Managed Knowledge Bases.

  • Created BedrockKnowledgeBaseRetriever as a Haystack @component with run(query) method
  • Returns List[Document] with content, metadata (source URI, score)
  • Supports both Managed (recommended) and Vector knowledge base types
  • Agentic retrieval (query decomposition + managed reranking) with automatic fallback to standard Retrieve API
  • Includes to_dict/from_dict serialization for pipeline persistence
  • Required IAM permissions: bedrock:Retrieve, bedrock:AgenticRetrieve
  • SDK requirement: boto3 >= 1.43

How did you test it?

  • Unit tests included (mocked boto3 calls)
  • Manual E2E: Created Haystack Pipeline with retriever component → pipe.run() returned 3 Documents with scores. Tested against managed KB in us-west-2.
  • Serialization verified via to_dict() round-trip

Notes for the reviewer

  • This extends the existing amazon-bedrock integration (which already has Embedder, Generator, Ranker) with a Retriever component
  • Follows the same pattern as existing components in integrations/amazon_bedrock/
  • managedSearchConfiguration requires boto3 >= 1.43 (released April 2026)

Checklist

@CLAassistant

CLAassistant commented Jul 13, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@github-actions github-actions Bot added integration:amazon-bedrock type:documentation Improvements or additions to documentation labels Jul 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Heads-up for maintainers

This PR is from a fork and touches integrations whose integration tests require API keys.
Those tests are skipped in CI because fork PRs don't have access to repo secrets for security reasons.

Affected integrations:

  • amazon_bedrock

Please run the integration tests locally (hatch run test:integration inside each folder) before approving.

@PVidyadhar
PVidyadhar force-pushed the bmkb-managed-kb-support branch 3 times, most recently from 53258e3 to 50ac0b9 Compare July 14, 2026 02:11
@github-actions

github-actions Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Coverage report (amazon_bedrock)

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  integrations/amazon_bedrock/src/haystack_integrations/components/retrievers/amazon_bedrock
  knowledge_base_retriever.py 27-38, 132-137, 174-188, 190-192
Project Total  

This report was generated by python-coverage-comment-action

@PVidyadhar
PVidyadhar force-pushed the bmkb-managed-kb-support branch 5 times, most recently from 48b1c26 to 1f5de93 Compare July 14, 2026 18:18
@PVidyadhar
PVidyadhar marked this pull request as ready for review July 14, 2026 18:19
@PVidyadhar
PVidyadhar requested a review from a team as a code owner July 14, 2026 18:19
@PVidyadhar
PVidyadhar requested review from davidsbatista and removed request for a team July 14, 2026 18:19
@PVidyadhar
PVidyadhar force-pushed the bmkb-managed-kb-support branch 4 times, most recently from 408586a to 772c95c Compare July 16, 2026 01:01
@davidsbatista

Copy link
Copy Markdown
Contributor

Hi @PVidyadhar - thanks for this PR.

I noticed that there are no integrations/live tests. In any case, did you actually test this Retriever against a live Amazon Bedrock Knowledge Base?

- Created BedrockKnowledgeBaseRetriever @component with run(query)
- Returns Haystack Documents with content, metadata, and scores
- Supports managed and vector search configurations
- Agentic retrieval with fallback to standard Retrieve API
- Includes to_dict/from_dict serialization for pipeline persistence
- Unit tests included
- Added BEDROCK_MANAGED_KB.md design doc
@PVidyadhar
PVidyadhar force-pushed the bmkb-managed-kb-support branch from e7e86b4 to b6bf93c Compare July 24, 2026 13:27
@PVidyadhar

Copy link
Copy Markdown
Contributor Author

Hi @PVidyadhar - thanks for this PR.

I noticed that there are no integrations/live tests. In any case, did you actually test this Retriever against a live Amazon Bedrock Knowledge Base?

Hi @davidsbatista — yes, fully tested against a live Managed KB (ZDYIO8E7GK) in us-west-2:

  • Standard Retrieve: ✅ 5 documents returned
  • Agentic Retrieve: ✅ 5 documents returned
  • User-agent confirmed on service-side logs: haystack/bedrock-kb

I've also added integration tests (test_knowledge_base_retriever_integration.py) marked with @pytest.mark.integration - they run with KNOWLEDGE_BASE_ID env var set. Push is up.

@davidsbatista davidsbatista left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution, I've done some adjustments + adding the tests to the workflow

@davidsbatista
davidsbatista merged commit 5944934 into deepset-ai:main Aug 5, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

integration:amazon-bedrock topic:CI type:documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants