Skip legacy models when selecting a Bedrock inference profile - #72521
Open
vincbeck wants to merge 1 commit into
Open
Skip legacy models when selecting a Bedrock inference profile#72521vincbeck wants to merge 1 commit into
vincbeck wants to merge 1 commit into
Conversation
get_text_inference_profile_arn() returns the first "sonnet" inference profile
in list_inference_profiles order. That order is not stable, and when it changed
the helper started returning a profile backed by a model its provider had
marked as legacy. Bedrock then rejects the request:
ValidationException: This Model is marked by provider as Legacy and you
have not been actively using the model in the last 30 days. Please upgrade
to an active model on Amazon Bedrock.
example_bedrock_batch_inference and example_bedrock_retrieve_and_generate both
use this helper, so both fail for as long as a legacy model sits first in the
listing.
The inference profile summaries do not expose a lifecycle status, so look the
legacy models up with list_foundation_models and skip any profile that
resolves to one of them. Foundation model IDs are compared rather than ARNs
because a global profile resolves to the same model in several regions.
boring-cyborg
Bot
added
area:providers
provider:amazon
ramitkataria
reviewed
Sep 4, 2026
| legacy_model_ids = { | ||
| model["modelId"] | ||
| for model in client.list_foundation_models()["modelSummaries"] | ||
| if model.get("modelLifecycle", {}).get("status") == "LEGACY" |
Contributor
There was a problem hiding this comment.
Should we filter for just ACTIVE so that if there's a new state like DEPRECATED or something else, this would still keep working?
| log.info("Legacy model IDs: %s", sorted(legacy_model_ids)) | ||
|
|
||
| profiles = client.list_inference_profiles(typeEquals="SYSTEM_DEFINED")["inferenceProfileSummaries"] | ||
| arns = [ |
Contributor
There was a problem hiding this comment.
Since the pick among non-legacy candidates still follows list_inference_profiles order, the model under test can change between runs. Would it make sense to add a deterministic tiebreak, e.g. sort the candidates and pick the oldest non-legacy Sonnet, so runs are reproducible and we don't land on a brand-new release before batch inference or RAG support it?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
get_text_inference_profile_arn()returns the first "sonnet" inference profile inlist_inference_profilesorder. That order is not stable, and when it changed the helper started returning a profile backed by a model its provider had marked as legacy. Bedrock then rejects the request:The inference profile summaries do not expose a lifecycle status, so look the legacy models up with
list_foundation_modelsand skip any profile that resolves to one of them. Foundation model IDs are compared rather than ARNs because a global profile resolves to the same model in several regions.Was generative AI tooling used to co-author this PR?
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.