Add model package support for sdkv2 - #981
Conversation
Add package-aware layout handling, loading, cache behavior, and catalog compatibility filtering.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Copilot resolve the merge conflicts in this pull request |
b75395a to
a6ceeca
Compare
a6ceeca to
88a77c5
Compare
# Conflicts: # sdk_v2/cpp/src/ep_detection/ep_detector.cc Co-authored-by: xiaoyu-work <85524621+xiaoyu-work@users.noreply.github.com>
Resolved in 2922026 — merged |
There was a problem hiding this comment.
Pull request overview
Adds C++ SDK v2 support for downloading, discovering, filtering, and loading ORT model packages.
Changes:
- Adds package layout detection and cache handling.
- Adds package metadata parsing and hardware compatibility filtering.
- Loads selected package variants through ORT GenAI and expands tests.
Reviewed changes
Copilot reviewed 29 out of 29 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
sdk_v2/cpp/CMakeLists.txt |
Builds the layout classifier. |
sdk_v2/cpp/src/util/model_layout.h |
Declares model layout classification. |
sdk_v2/cpp/src/util/model_layout.cc |
Implements layout detection. |
sdk_v2/cpp/src/model.cc |
Passes catalog EP and task metadata when loading. |
sdk_v2/cpp/src/model_info.h |
Defines typed package metadata. |
sdk_v2/cpp/src/model_info.cc |
Serializes package metadata. |
sdk_v2/cpp/src/inferencing/model_load_manager.h |
Extends the loading interface. |
sdk_v2/cpp/src/inferencing/model_load_manager.cc |
Handles package runtime metadata. |
sdk_v2/cpp/src/inferencing/generative/genai_model_instance.h |
Tracks package and modality state. |
sdk_v2/cpp/src/inferencing/generative/genai_model_instance.cc |
Loads packages through ORT GenAI. |
sdk_v2/cpp/src/ep_detection/ep_detector.h |
Adds compiled-model compatibility APIs. |
sdk_v2/cpp/src/ep_detection/ep_detector.cc |
Evaluates package compatibility. |
sdk_v2/cpp/src/download/inference_model_writer.h |
Documents package marker behavior. |
sdk_v2/cpp/src/download/inference_model_writer.cc |
Retains package root markers. |
sdk_v2/cpp/src/download/download_manager.h |
Updates download workflow documentation. |
sdk_v2/cpp/src/download/download_manager.cc |
Resolves package cache roots. |
sdk_v2/cpp/src/catalog/local_model_scanner.h |
Documents package discovery. |
sdk_v2/cpp/src/catalog/local_model_scanner.cc |
Discovers package roots without scanning variants. |
sdk_v2/cpp/src/catalog/azure_model_catalog.h |
Adds injectable catalog clients. |
sdk_v2/cpp/src/catalog/azure_model_catalog.cc |
Filters incompatible package entries. |
sdk_v2/cpp/src/catalog/azure_catalog_models.h |
Defines catalog package payloads. |
sdk_v2/cpp/src/catalog/azure_catalog_models.cc |
Parses package catalog metadata. |
sdk_v2/cpp/test/internal_api/model_load_manager_test.cc |
Tests package loading behavior. |
sdk_v2/cpp/test/internal_api/model_info_test.cc |
Tests metadata serialization. |
sdk_v2/cpp/test/internal_api/local_model_scanner_test.cc |
Tests package cache discovery. |
sdk_v2/cpp/test/internal_api/download_test.cc |
Tests package download and caching. |
sdk_v2/cpp/test/internal_api/catalog_cache_test.cc |
Tests cached package metadata. |
sdk_v2/cpp/test/internal_api/azure_catalog_test.cc |
Tests parsing and compatibility filtering. |
model-package-foundry-local-integration.md |
Documents the integration design and scope. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| const bool is_multimodal = | ||
| package_is_multimodal || (genai_config.model && genai_config.model->IsMultiModal()) || IsTaskMultiModal(task); |
| if (ep == ExecutionProvider::kDefault && !info_.execution_provider.empty()) { | ||
| const auto catalog_ep = EPUtils::StringtoEP(info_.execution_provider); | ||
| if (catalog_ep != ExecutionProvider::kUnknown) { | ||
| ep = catalog_ep; | ||
| } | ||
| } |
| @@ -0,0 +1,314 @@ | |||
| # Model Package Integration into Foundry Local — Scope & Plan | |||
|
|
|||
| **Status:** Design / scoping (pre-implementation) | |||
Restrict the catalog-task multimodal fallback to model packages so flat ASR models (e.g. nemotron_speech) are not routed through OgaMultiModalProcessor. Skip the provider override for kCPU, whose GenAI provider name is empty, so default-loaded CPU catalog models are not configured with an empty provider. Move the model package design doc under sdk_v2/cpp/docs and mark Stage 1 as implemented. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e073a4e1-2902-4111-bb9b-56f152c82527
|
Copilot resolve the merge conflicts in this pull request |
# Conflicts: # sdk_v2/cpp/src/catalog/azure_model_catalog.cc # sdk_v2/cpp/src/catalog/azure_model_catalog.h Co-authored-by: xiaoyu-work <85524621+xiaoyu-work@users.noreply.github.com>
Resolved in 252ecef. Conflicts were in Verified with |
Add model package support for sdkv2