Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ All examples use `"authType": "ApiKey"` with workspace-managed credentials. The

### 1. ModelDiscovery (Dynamic Discovery)

The `modelDiscovery` object enables runtime model detection through API endpoints. Azure Agents combines this configuration with the connection's `target` URL and `credentials` to make discovery calls.
The `modelDiscovery` object enables runtime model detection through API endpoints. Endpoints can be relative to the connection `target` or absolute HTTPS URIs on the same origin as `target`.

```json
{
Expand All @@ -50,16 +50,43 @@ The `modelDiscovery` object enables runtime model detection through API endpoint
```

**Fields:**
- `listModelsEndpoint` - Endpoint to retrieve all available models (relative to target URL)
- `getModelEndpoint` - Endpoint to get specific model details with `{deploymentName}` placeholder
- `listModelsEndpoint` - Relative endpoint or allowed absolute URI used to retrieve all available models
- `getModelEndpoint` - Relative endpoint or allowed absolute URI used to get specific model details; it must contain exactly one `{deploymentName}` placeholder in the path
- `deploymentProvider` - Provider format for response parsing. **Supported values: `"OpenAI"` and `"AzureOpenAI"`** (exactly 2 formats)

**How Azure Agents Uses It:**
1. Constructs full URL: `{target}{listModelsEndpoint}`
**How ModelGateway Uses It:**
1. Constructs full URL: `{target}{listModelsEndpoint}` when relative endpoint is passed in, or uses the `{listModelsEndpoint}` absolute endpoint as-is
2. Adds authentication headers from `credentials`
3. Makes HTTP request to discover available models
4. Parses response based on `deploymentProvider` format (OpenAI or AzureOpenAI)

#### Absolute discovery endpoints

Absolute endpoints support providers whose inference and discovery APIs share an origin but use sibling paths:

```json
{
"target": "https://contoso.services.ai.azure.com/openai/v1",
"metadata": {
"modelDiscovery": {
"listModelsEndpoint": "https://contoso.services.ai.azure.com/openai/deployments?api-version=2022-12-01",
"getModelEndpoint": "https://contoso.services.ai.azure.com/openai/deployments/{deploymentName}?api-version=2022-12-01",
"deploymentProvider": "AzureOpenAI"
}
}
}
```

Requirements:

- `target` and each absolute discovery endpoint must use HTTPS.
- Scheme, hostname, and effective port must match exactly. The discovery path can differ from the target path.
- User information, fragments, path traversal, and encoded path separators are rejected.
- `getModelEndpoint` must contain exactly one `{deploymentName}` placeholder in its path.
- Existing query parameters are preserved. An endpoint `api-version` takes precedence over `deploymentAPIVersion`; duplicate `api-version` parameters are rejected.

Relative endpoints remain supported and do not require this feature.

**Supported DeploymentProvider Formats:**

We support exactly **2 deployment API formats** for model discovery:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ This folder contains Azure Bicep templates for creating ModelGateway connections
## 📚 Documentation

- **[Setup Guide](./modelgateway-setup-guide-for-agents.md)** - Complete configuration guide for ModelGateway connections
- **[Connection Objects](./ModelGateway-Connection-Objects.md)** - ModelGateway metadata schema and examples
- **[Troubleshooting Guide](./troubleshooting-guide.md)** - Common issues and solutions.

## Prerequisites
Expand Down Expand Up @@ -79,6 +80,20 @@ az deployment group create \
--parameters apiKey=<your-api-key>
```

### Absolute Dynamic Discovery Endpoints

Use absolute discovery endpoints when inference and discovery use sibling paths on the same HTTPS origin. This approach can connect one Foundry project to models in another Foundry project. For example, the connection target can end in `/openai/v1` while discovery uses `/openai/deployments`.

```bash
# 1. Edit samples/parameters-dynamic-absolute.json with your resource IDs and URLs
# 2. Deploy with your API key
az deployment group create \
--resource-group <your-resource-group> \
--template-file connection-modelgateway.bicep \
--parameters @samples/parameters-dynamic-absolute.json \
--parameters apiKey=<your-api-key>
```

### Static Models ModelGateway Connection
```bash
# 1. Edit samples/parameters-static.json with your resource IDs
Expand Down Expand Up @@ -158,6 +173,7 @@ The template includes built-in validation:
- `samples/parameters-foundryopenai.json`: For Foundry AzureOpenAI connection
- `samples/parameters-foundryanthropic.json`: For Foundry Anthropic connection
- `samples/parameters-dynamic.json`: For dynamic discovery connections with API key authentication
- `samples/parameters-dynamic-absolute.json`: For same-origin absolute dynamic discovery endpoints
- `samples/parameters-static.json`: For static model list connections with placeholder models
- `samples/parameters-custom-auth-config.json`: For custom authentication and headers configuration
- `samples/parameters-oauth2.json`: For OAuth2 authentication connections
Expand All @@ -170,7 +186,7 @@ The `connection-modelgateway.bicep` template supports all ModelGateway connectio

1. **Basic Configuration**: Required deploymentInPath and inferenceAPIVersion
2. **Deployment API Version**: Optional deploymentAPIVersion for deployment management
3. **Dynamic Discovery**: Automatic model discovery using API endpoints (listModelsEndpoint, getModelEndpoint, deploymentProvider)
3. **Dynamic Discovery**: Automatic model discovery using relative or same-origin absolute HTTPS endpoints (listModelsEndpoint, getModelEndpoint, deploymentProvider)
4. **Static Model List**: Predefined list of available models in staticModels array
5. **Custom Headers**: Custom HTTP headers as key-value pairs in customHeaders object
6. **Custom Auth Config**: Flexible authentication configuration with authConfig object
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,8 @@ param inferenceAPIVersion string = '' // Required: API version for model infe
param deploymentAPIVersion string = '' // Optional: API version for deployment management

// 3. OPTIONAL - Dynamic Discovery Configuration
param listModelsEndpoint string = '' // Optional: Endpoint for listing available models
param getModelEndpoint string = '' // Optional: Endpoint for getting model details
param listModelsEndpoint string = '' // Optional: Relative or same-origin absolute HTTPS endpoint for listing available models
param getModelEndpoint string = '' // Optional: Relative or same-origin absolute HTTPS endpoint for getting model details
param deploymentProvider string = '' // Optional: Provider type (e.g., OpenAI)

// 4. OPTIONAL - Static Model List
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -269,6 +269,23 @@ If you choose dynamic discovery, make these endpoints available on your gateway:
}
```

**Absolute endpoints for sibling discovery paths**:

Use absolute endpoints when the inference target and discovery API are on the same HTTPS origin but cannot be represented by appending relative paths. The discovery path does not need to share the target path prefix.

```json
{
"target": "https://contoso.services.ai.azure.com/openai/v1",
"modelDiscovery": {
"listModelsEndpoint": "https://contoso.services.ai.azure.com/openai/deployments?api-version=2022-12-01",
"getModelEndpoint": "https://contoso.services.ai.azure.com/openai/deployments/{deploymentName}?api-version=2022-12-01",
"deploymentProvider": "AzureOpenAI"
}
}
```

Both absolute endpoints must use HTTPS and match the target's hostname and effective port. Different hosts, schemes, or ports; user information; fragments; path traversal; and encoded path separators are rejected. If an endpoint includes `api-version`, it takes precedence over `deploymentAPIVersion`.

**Supported DeploymentProvider Values:**
- `"AzureOpenAI"`: **Recommended** - For Azure OpenAI ARM resource response format with detailed model information
- `"OpenAI"`: For OpenAI-compatible response format
Expand Down Expand Up @@ -330,7 +347,7 @@ Note any API versions query param (api-version) your endpoints require:
Based on your choice in Step 3:

**For Static Models**: Prepare your model list
**For Dynamic Discovery**: Note your discovery endpoint paths
**For Dynamic Discovery**: Note your relative endpoint paths or same-origin absolute HTTPS endpoint URIs

#### 🔍 5. Authentication

Expand Down Expand Up @@ -369,6 +386,7 @@ Before creating your ModelGateway connection in Azure AI Foundry, follow these s
### 1. **Choose your parameter file** based on your gateway type:
- `samples/parameters-static.json` - For gateways with predefined static models
- `samples/parameters-dynamic.json` - For gateways with dynamic model discovery
- `samples/parameters-dynamic-absolute.json` - For dynamic discovery on a sibling path of the target's HTTPS origin
- `samples/parameters-oauth2.json` - For OAuth2 authentication (requires `clientId`, `tokenUrl`, `scopes`)
- `samples/parameters-custom-auth-config.json` - For custom authentication headers
- `samples/parameters-foundryopenai.json` - For Azure OpenAI Foundry connections
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"projectResourceId": {
"value": "/subscriptions/YOUR-SUBSCRIPTION-ID/resourceGroups/YOUR-RG/providers/Microsoft.CognitiveServices/accounts/YOUR-AI-FOUNDRY-ACCOUNT/projects/YOUR-PROJECT"
},
"targetUrl": {
"value": "https://your-resource.services.ai.azure.com/openai/v1"
},
"gatewayName": {
"value": "YOUR-GATEWAY-NAME"
},
"connectionName": {
"value": "YOUR-CONNECTION-NAME"
},
"authType": {
"value": "ApiKey"
},
"isSharedToAll": {
"value": false
},
"listModelsEndpoint": {
"value": "https://your-resource.services.ai.azure.com/openai/deployments?api-version=2022-12-01"
},
"getModelEndpoint": {
"value": "https://your-resource.services.ai.azure.com/openai/deployments/{deploymentName}?api-version=2022-12-01"
},
"deploymentProvider": {
"value": "AzureOpenAI"
},
"deploymentAPIVersion": {
"_comment": "The endpoint api-version takes precedence. This value is retained as a fallback for endpoints without api-version.",
"value": "2025-03-01"
},
"deploymentInPath": {
"value": "false"
},
"inferenceAPIVersion": {
"value": ""
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
from typing import Dict, Any, Optional, Tuple, List
from enum import Enum
from dataclasses import dataclass
from urllib.parse import parse_qsl, quote, unquote, urlsplit, urlunsplit

# Color constants for terminal output
class Colors:
Expand Down Expand Up @@ -321,13 +322,18 @@ def _validate_dynamic_structure(self, config: ValidationConfig) -> bool:
return False

# Validate getModelEndpoint template
if "{deploymentName}" not in config.get_model_endpoint:
self.print_colored("❌ Error: getModelEndpoint must contain '{deploymentName}' template", Colors.RED)
if config.get_model_endpoint.count("{deploymentName}") != 1:
self.print_colored("❌ Error: getModelEndpoint must contain exactly one '{deploymentName}' template", Colors.RED)
self.print_colored(f"Current value: {config.get_model_endpoint}", Colors.NC)
print()
self.print_colored("📋 Fix in your Bicep parameter file:", Colors.BLUE)
print("Update getModelEndpoint to include the {deploymentName} placeholder:")
print("Example: \"/deployments/{deploymentName}\" or \"/models/{deploymentName}\"")
print("Example: \"/deployments/{deploymentName}\" or \"https://gateway.example/deployments/{deploymentName}\"")
return False

endpoint_parts = urlsplit(config.get_model_endpoint)
if "{deploymentName}" in endpoint_parts.netloc or "{deploymentName}" in endpoint_parts.query or "{deploymentName}" in endpoint_parts.fragment:
self.print_colored("❌ Error: {deploymentName} must appear in the getModelEndpoint path", Colors.RED)
return False

print("✅ Dynamic discovery structure valid")
Expand Down Expand Up @@ -389,9 +395,15 @@ def _test_list_models_endpoint(self, config: ValidationConfig, headers: Dict[str
"""Test the list models endpoint"""
self.print_colored("🔍 2a. List Models Endpoint Test:", Colors.YELLOW)

list_url = f"{base_url}{config.list_models_endpoint}"
if config.deployment_api_version:
list_url += f"?api-version={config.deployment_api_version}"
try:
list_url = self._build_discovery_url(
base_url,
config.list_models_endpoint,
config.deployment_api_version
)
except ValueError as error:
self.print_colored(f"❌ Invalid listModelsEndpoint: {error}", Colors.RED)
return False

print(f"Testing: {list_url}")

Expand Down Expand Up @@ -463,9 +475,16 @@ def _test_get_model_endpoint(self, config: ValidationConfig, headers: Dict[str,
self.print_colored("❌ Error: getModelEndpoint missing {deploymentName} template", Colors.RED)
return False

get_url = f"{base_url}{config.get_model_endpoint.replace('{deploymentName}', config.deployment_name)}"
if config.deployment_api_version:
get_url += f"?api-version={config.deployment_api_version}"
try:
get_url = self._build_discovery_url(
base_url,
config.get_model_endpoint,
config.deployment_api_version,
config.deployment_name
)
except ValueError as error:
self.print_colored(f"❌ Invalid getModelEndpoint: {error}", Colors.RED)
return False

print(f"Testing: {get_url}")

Expand Down Expand Up @@ -547,6 +566,70 @@ def _test_get_model_endpoint(self, config: ValidationConfig, headers: Dict[str,

return True

def _build_discovery_url(self, base_url: str, endpoint: str,
deployment_api_version: str,
deployment_name: Optional[str] = None) -> str:
"""Build a relative or absolute discovery URL using runtime query precedence."""
configured_endpoint_parts = urlsplit(endpoint)
if configured_endpoint_parts.scheme and configured_endpoint_parts.netloc:
self._validate_absolute_discovery_endpoint(base_url, configured_endpoint_parts)

resolved_endpoint = endpoint
if deployment_name is not None:
resolved_endpoint = resolved_endpoint.replace(
"{deploymentName}", quote(deployment_name, safe="")
)

endpoint_parts = urlsplit(resolved_endpoint)
if endpoint_parts.scheme and endpoint_parts.netloc:
resolved_url = resolved_endpoint
else:
resolved_url = f"{base_url.rstrip('/')}{resolved_endpoint}"

url_parts = urlsplit(resolved_url)
api_version_count = sum(
1 for name, _ in parse_qsl(url_parts.query, keep_blank_values=True)
if name.lower() == "api-version"
)
if api_version_count > 1:
raise ValueError("the endpoint contains duplicate api-version parameters")

query = url_parts.query
if api_version_count == 0 and deployment_api_version:
separator = "&" if query else ""
query = f"{query}{separator}api-version={quote(deployment_api_version, safe='')}"

return urlunsplit((url_parts.scheme, url_parts.netloc, url_parts.path, query, url_parts.fragment))

def _validate_absolute_discovery_endpoint(self, target: str, endpoint_parts) -> None:
"""Reject absolute discovery endpoints outside the target's HTTPS origin."""
target_parts = urlsplit(target)
if target_parts.scheme.lower() != "https" or endpoint_parts.scheme.lower() != "https":
raise ValueError("absolute discovery endpoints and targetUrl must use HTTPS")
if target_parts.username or target_parts.password or endpoint_parts.username or endpoint_parts.password:
raise ValueError("user information is not allowed")
if endpoint_parts.fragment:
raise ValueError("fragments are not allowed")

try:
target_origin = (target_parts.hostname.encode("idna").decode("ascii").lower(), target_parts.port or 443)
endpoint_origin = (endpoint_parts.hostname.encode("idna").decode("ascii").lower(), endpoint_parts.port or 443)
except (AttributeError, UnicodeError, ValueError) as error:
raise ValueError("targetUrl and endpoint must contain valid hosts and ports") from error
if target_origin != endpoint_origin:
raise ValueError("absolute discovery endpoints must use the same origin as targetUrl")

path_segments = endpoint_parts.path.split("/")
for _ in range(3):
decoded_segments = [unquote(segment) for segment in path_segments]
if any(segment in (".", "..") for segment in decoded_segments):
raise ValueError("path traversal segments are not allowed")
if any("/" in segment or "\\" in segment for segment in decoded_segments):
raise ValueError("encoded path separators are not allowed")
if decoded_segments == path_segments:
break
path_segments = decoded_segments

# ============================================================================
# 3. MODEL VALIDATION MODULE
# ============================================================================
Expand Down
Loading