diff --git a/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/ModelGateway-Connection-Objects.md b/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/ModelGateway-Connection-Objects.md index f7a824548..486bd4fa2 100644 --- a/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/ModelGateway-Connection-Objects.md +++ b/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/ModelGateway-Connection-Objects.md @@ -37,7 +37,7 @@ All examples use `"authType": "ApiKey"` with workspace-managed credentials. The ### 1. ModelDiscovery (Dynamic Discovery) -The `modelDiscovery` object enables runtime model detection through API endpoints. Azure Agents combines this configuration with the connection's `target` URL and `credentials` to make discovery calls. +The `modelDiscovery` object enables runtime model detection through API endpoints. Endpoints can be relative to the connection `target` or absolute HTTPS URIs on the same origin as `target`. ```json { @@ -50,16 +50,43 @@ The `modelDiscovery` object enables runtime model detection through API endpoint ``` **Fields:** -- `listModelsEndpoint` - Endpoint to retrieve all available models (relative to target URL) -- `getModelEndpoint` - Endpoint to get specific model details with `{deploymentName}` placeholder +- `listModelsEndpoint` - Relative endpoint or allowed absolute URI used to retrieve all available models +- `getModelEndpoint` - Relative endpoint or allowed absolute URI used to get specific model details; it must contain exactly one `{deploymentName}` placeholder in the path - `deploymentProvider` - Provider format for response parsing. **Supported values: `"OpenAI"` and `"AzureOpenAI"`** (exactly 2 formats) -**How Azure Agents Uses It:** -1. Constructs full URL: `{target}{listModelsEndpoint}` +**How ModelGateway Uses It:** +1. Constructs full URL: `{target}{listModelsEndpoint}` when relative endpoint is passed in, or uses the `{listModelsEndpoint}` absolute endpoint as-is 2. Adds authentication headers from `credentials` 3. Makes HTTP request to discover available models 4. Parses response based on `deploymentProvider` format (OpenAI or AzureOpenAI) +#### Absolute discovery endpoints + +Absolute endpoints support providers whose inference and discovery APIs share an origin but use sibling paths: + +```json +{ + "target": "https://contoso.services.ai.azure.com/openai/v1", + "metadata": { + "modelDiscovery": { + "listModelsEndpoint": "https://contoso.services.ai.azure.com/openai/deployments?api-version=2022-12-01", + "getModelEndpoint": "https://contoso.services.ai.azure.com/openai/deployments/{deploymentName}?api-version=2022-12-01", + "deploymentProvider": "AzureOpenAI" + } + } +} +``` + +Requirements: + +- `target` and each absolute discovery endpoint must use HTTPS. +- Scheme, hostname, and effective port must match exactly. The discovery path can differ from the target path. +- User information, fragments, path traversal, and encoded path separators are rejected. +- `getModelEndpoint` must contain exactly one `{deploymentName}` placeholder in its path. +- Existing query parameters are preserved. An endpoint `api-version` takes precedence over `deploymentAPIVersion`; duplicate `api-version` parameters are rejected. + +Relative endpoints remain supported and do not require this feature. + **Supported DeploymentProvider Formats:** We support exactly **2 deployment API formats** for model discovery: diff --git a/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/README.md b/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/README.md index c70a08d30..d3b527922 100644 --- a/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/README.md +++ b/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/README.md @@ -11,6 +11,7 @@ This folder contains Azure Bicep templates for creating ModelGateway connections ## 📚 Documentation - **[Setup Guide](./modelgateway-setup-guide-for-agents.md)** - Complete configuration guide for ModelGateway connections +- **[Connection Objects](./ModelGateway-Connection-Objects.md)** - ModelGateway metadata schema and examples - **[Troubleshooting Guide](./troubleshooting-guide.md)** - Common issues and solutions. ## Prerequisites @@ -79,6 +80,20 @@ az deployment group create \ --parameters apiKey= ``` +### Absolute Dynamic Discovery Endpoints + +Use absolute discovery endpoints when inference and discovery use sibling paths on the same HTTPS origin. This approach can connect one Foundry project to models in another Foundry project. For example, the connection target can end in `/openai/v1` while discovery uses `/openai/deployments`. + +```bash +# 1. Edit samples/parameters-dynamic-absolute.json with your resource IDs and URLs +# 2. Deploy with your API key +az deployment group create \ + --resource-group \ + --template-file connection-modelgateway.bicep \ + --parameters @samples/parameters-dynamic-absolute.json \ + --parameters apiKey= +``` + ### Static Models ModelGateway Connection ```bash # 1. Edit samples/parameters-static.json with your resource IDs @@ -158,6 +173,7 @@ The template includes built-in validation: - `samples/parameters-foundryopenai.json`: For Foundry AzureOpenAI connection - `samples/parameters-foundryanthropic.json`: For Foundry Anthropic connection - `samples/parameters-dynamic.json`: For dynamic discovery connections with API key authentication +- `samples/parameters-dynamic-absolute.json`: For same-origin absolute dynamic discovery endpoints - `samples/parameters-static.json`: For static model list connections with placeholder models - `samples/parameters-custom-auth-config.json`: For custom authentication and headers configuration - `samples/parameters-oauth2.json`: For OAuth2 authentication connections @@ -170,7 +186,7 @@ The `connection-modelgateway.bicep` template supports all ModelGateway connectio 1. **Basic Configuration**: Required deploymentInPath and inferenceAPIVersion 2. **Deployment API Version**: Optional deploymentAPIVersion for deployment management -3. **Dynamic Discovery**: Automatic model discovery using API endpoints (listModelsEndpoint, getModelEndpoint, deploymentProvider) +3. **Dynamic Discovery**: Automatic model discovery using relative or same-origin absolute HTTPS endpoints (listModelsEndpoint, getModelEndpoint, deploymentProvider) 4. **Static Model List**: Predefined list of available models in staticModels array 5. **Custom Headers**: Custom HTTP headers as key-value pairs in customHeaders object 6. **Custom Auth Config**: Flexible authentication configuration with authConfig object diff --git a/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/connection-modelgateway.bicep b/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/connection-modelgateway.bicep index bdc49661e..4a57ae771 100644 --- a/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/connection-modelgateway.bicep +++ b/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/connection-modelgateway.bicep @@ -53,8 +53,8 @@ param inferenceAPIVersion string = '' // Required: API version for model infe param deploymentAPIVersion string = '' // Optional: API version for deployment management // 3. OPTIONAL - Dynamic Discovery Configuration -param listModelsEndpoint string = '' // Optional: Endpoint for listing available models -param getModelEndpoint string = '' // Optional: Endpoint for getting model details +param listModelsEndpoint string = '' // Optional: Relative or same-origin absolute HTTPS endpoint for listing available models +param getModelEndpoint string = '' // Optional: Relative or same-origin absolute HTTPS endpoint for getting model details param deploymentProvider string = '' // Optional: Provider type (e.g., OpenAI) // 4. OPTIONAL - Static Model List diff --git a/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/modelgateway-setup-guide-for-agents.md b/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/modelgateway-setup-guide-for-agents.md index 6f953e35f..894d0ccb8 100644 --- a/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/modelgateway-setup-guide-for-agents.md +++ b/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/modelgateway-setup-guide-for-agents.md @@ -269,6 +269,23 @@ If you choose dynamic discovery, make these endpoints available on your gateway: } ``` +**Absolute endpoints for sibling discovery paths**: + +Use absolute endpoints when the inference target and discovery API are on the same HTTPS origin but cannot be represented by appending relative paths. The discovery path does not need to share the target path prefix. + +```json +{ + "target": "https://contoso.services.ai.azure.com/openai/v1", + "modelDiscovery": { + "listModelsEndpoint": "https://contoso.services.ai.azure.com/openai/deployments?api-version=2022-12-01", + "getModelEndpoint": "https://contoso.services.ai.azure.com/openai/deployments/{deploymentName}?api-version=2022-12-01", + "deploymentProvider": "AzureOpenAI" + } +} +``` + +Both absolute endpoints must use HTTPS and match the target's hostname and effective port. Different hosts, schemes, or ports; user information; fragments; path traversal; and encoded path separators are rejected. If an endpoint includes `api-version`, it takes precedence over `deploymentAPIVersion`. + **Supported DeploymentProvider Values:** - `"AzureOpenAI"`: **Recommended** - For Azure OpenAI ARM resource response format with detailed model information - `"OpenAI"`: For OpenAI-compatible response format @@ -330,7 +347,7 @@ Note any API versions query param (api-version) your endpoints require: Based on your choice in Step 3: **For Static Models**: Prepare your model list -**For Dynamic Discovery**: Note your discovery endpoint paths +**For Dynamic Discovery**: Note your relative endpoint paths or same-origin absolute HTTPS endpoint URIs #### 🔍 5. Authentication @@ -369,6 +386,7 @@ Before creating your ModelGateway connection in Azure AI Foundry, follow these s ### 1. **Choose your parameter file** based on your gateway type: - `samples/parameters-static.json` - For gateways with predefined static models - `samples/parameters-dynamic.json` - For gateways with dynamic model discovery + - `samples/parameters-dynamic-absolute.json` - For dynamic discovery on a sibling path of the target's HTTPS origin - `samples/parameters-oauth2.json` - For OAuth2 authentication (requires `clientId`, `tokenUrl`, `scopes`) - `samples/parameters-custom-auth-config.json` - For custom authentication headers - `samples/parameters-foundryopenai.json` - For Azure OpenAI Foundry connections diff --git a/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/samples/parameters-dynamic-absolute.json b/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/samples/parameters-dynamic-absolute.json new file mode 100644 index 000000000..1de6d4434 --- /dev/null +++ b/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/samples/parameters-dynamic-absolute.json @@ -0,0 +1,43 @@ +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#", + "contentVersion": "1.0.0.0", + "parameters": { + "projectResourceId": { + "value": "/subscriptions/YOUR-SUBSCRIPTION-ID/resourceGroups/YOUR-RG/providers/Microsoft.CognitiveServices/accounts/YOUR-AI-FOUNDRY-ACCOUNT/projects/YOUR-PROJECT" + }, + "targetUrl": { + "value": "https://your-resource.services.ai.azure.com/openai/v1" + }, + "gatewayName": { + "value": "YOUR-GATEWAY-NAME" + }, + "connectionName": { + "value": "YOUR-CONNECTION-NAME" + }, + "authType": { + "value": "ApiKey" + }, + "isSharedToAll": { + "value": false + }, + "listModelsEndpoint": { + "value": "https://your-resource.services.ai.azure.com/openai/deployments?api-version=2022-12-01" + }, + "getModelEndpoint": { + "value": "https://your-resource.services.ai.azure.com/openai/deployments/{deploymentName}?api-version=2022-12-01" + }, + "deploymentProvider": { + "value": "AzureOpenAI" + }, + "deploymentAPIVersion": { + "_comment": "The endpoint api-version takes precedence. This value is retained as a fallback for endpoints without api-version.", + "value": "2025-03-01" + }, + "deploymentInPath": { + "value": "false" + }, + "inferenceAPIVersion": { + "value": "" + } + } +} \ No newline at end of file diff --git a/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/test_model_gateway_connection.py b/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/test_model_gateway_connection.py index ae25e314d..9bb588e29 100644 --- a/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/test_model_gateway_connection.py +++ b/infrastructure/infrastructure-setup-bicep/01-connections/model-gateway/test_model_gateway_connection.py @@ -20,6 +20,7 @@ from typing import Dict, Any, Optional, Tuple, List from enum import Enum from dataclasses import dataclass +from urllib.parse import parse_qsl, quote, unquote, urlsplit, urlunsplit # Color constants for terminal output class Colors: @@ -321,13 +322,18 @@ def _validate_dynamic_structure(self, config: ValidationConfig) -> bool: return False # Validate getModelEndpoint template - if "{deploymentName}" not in config.get_model_endpoint: - self.print_colored("❌ Error: getModelEndpoint must contain '{deploymentName}' template", Colors.RED) + if config.get_model_endpoint.count("{deploymentName}") != 1: + self.print_colored("❌ Error: getModelEndpoint must contain exactly one '{deploymentName}' template", Colors.RED) self.print_colored(f"Current value: {config.get_model_endpoint}", Colors.NC) print() self.print_colored("📋 Fix in your Bicep parameter file:", Colors.BLUE) print("Update getModelEndpoint to include the {deploymentName} placeholder:") - print("Example: \"/deployments/{deploymentName}\" or \"/models/{deploymentName}\"") + print("Example: \"/deployments/{deploymentName}\" or \"https://gateway.example/deployments/{deploymentName}\"") + return False + + endpoint_parts = urlsplit(config.get_model_endpoint) + if "{deploymentName}" in endpoint_parts.netloc or "{deploymentName}" in endpoint_parts.query or "{deploymentName}" in endpoint_parts.fragment: + self.print_colored("❌ Error: {deploymentName} must appear in the getModelEndpoint path", Colors.RED) return False print("✅ Dynamic discovery structure valid") @@ -389,9 +395,15 @@ def _test_list_models_endpoint(self, config: ValidationConfig, headers: Dict[str """Test the list models endpoint""" self.print_colored("🔍 2a. List Models Endpoint Test:", Colors.YELLOW) - list_url = f"{base_url}{config.list_models_endpoint}" - if config.deployment_api_version: - list_url += f"?api-version={config.deployment_api_version}" + try: + list_url = self._build_discovery_url( + base_url, + config.list_models_endpoint, + config.deployment_api_version + ) + except ValueError as error: + self.print_colored(f"❌ Invalid listModelsEndpoint: {error}", Colors.RED) + return False print(f"Testing: {list_url}") @@ -463,9 +475,16 @@ def _test_get_model_endpoint(self, config: ValidationConfig, headers: Dict[str, self.print_colored("❌ Error: getModelEndpoint missing {deploymentName} template", Colors.RED) return False - get_url = f"{base_url}{config.get_model_endpoint.replace('{deploymentName}', config.deployment_name)}" - if config.deployment_api_version: - get_url += f"?api-version={config.deployment_api_version}" + try: + get_url = self._build_discovery_url( + base_url, + config.get_model_endpoint, + config.deployment_api_version, + config.deployment_name + ) + except ValueError as error: + self.print_colored(f"❌ Invalid getModelEndpoint: {error}", Colors.RED) + return False print(f"Testing: {get_url}") @@ -547,6 +566,70 @@ def _test_get_model_endpoint(self, config: ValidationConfig, headers: Dict[str, return True + def _build_discovery_url(self, base_url: str, endpoint: str, + deployment_api_version: str, + deployment_name: Optional[str] = None) -> str: + """Build a relative or absolute discovery URL using runtime query precedence.""" + configured_endpoint_parts = urlsplit(endpoint) + if configured_endpoint_parts.scheme and configured_endpoint_parts.netloc: + self._validate_absolute_discovery_endpoint(base_url, configured_endpoint_parts) + + resolved_endpoint = endpoint + if deployment_name is not None: + resolved_endpoint = resolved_endpoint.replace( + "{deploymentName}", quote(deployment_name, safe="") + ) + + endpoint_parts = urlsplit(resolved_endpoint) + if endpoint_parts.scheme and endpoint_parts.netloc: + resolved_url = resolved_endpoint + else: + resolved_url = f"{base_url.rstrip('/')}{resolved_endpoint}" + + url_parts = urlsplit(resolved_url) + api_version_count = sum( + 1 for name, _ in parse_qsl(url_parts.query, keep_blank_values=True) + if name.lower() == "api-version" + ) + if api_version_count > 1: + raise ValueError("the endpoint contains duplicate api-version parameters") + + query = url_parts.query + if api_version_count == 0 and deployment_api_version: + separator = "&" if query else "" + query = f"{query}{separator}api-version={quote(deployment_api_version, safe='')}" + + return urlunsplit((url_parts.scheme, url_parts.netloc, url_parts.path, query, url_parts.fragment)) + + def _validate_absolute_discovery_endpoint(self, target: str, endpoint_parts) -> None: + """Reject absolute discovery endpoints outside the target's HTTPS origin.""" + target_parts = urlsplit(target) + if target_parts.scheme.lower() != "https" or endpoint_parts.scheme.lower() != "https": + raise ValueError("absolute discovery endpoints and targetUrl must use HTTPS") + if target_parts.username or target_parts.password or endpoint_parts.username or endpoint_parts.password: + raise ValueError("user information is not allowed") + if endpoint_parts.fragment: + raise ValueError("fragments are not allowed") + + try: + target_origin = (target_parts.hostname.encode("idna").decode("ascii").lower(), target_parts.port or 443) + endpoint_origin = (endpoint_parts.hostname.encode("idna").decode("ascii").lower(), endpoint_parts.port or 443) + except (AttributeError, UnicodeError, ValueError) as error: + raise ValueError("targetUrl and endpoint must contain valid hosts and ports") from error + if target_origin != endpoint_origin: + raise ValueError("absolute discovery endpoints must use the same origin as targetUrl") + + path_segments = endpoint_parts.path.split("/") + for _ in range(3): + decoded_segments = [unquote(segment) for segment in path_segments] + if any(segment in (".", "..") for segment in decoded_segments): + raise ValueError("path traversal segments are not allowed") + if any("/" in segment or "\\" in segment for segment in decoded_segments): + raise ValueError("encoded path separators are not allowed") + if decoded_segments == path_segments: + break + path_segments = decoded_segments + # ============================================================================ # 3. MODEL VALIDATION MODULE # ============================================================================