Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
1396a0d
Document AutoUpdateSchema property for external delta tables
shanisolomon Jun 26, 2026
fd05254
Add AutoUpdateSchema prerequisite note to delete and disable QAP docs
shanisolomon Jun 26, 2026
f4d4356
Update data-explorer/kusto/management/alter-query-acceleration-policy…
shanisolomon Jun 29, 2026
c759ce7
Update data-explorer/kusto/management/query-acceleration-policy.md
shanisolomon Jun 29, 2026
1b4b937
Update data-explorer/kusto/management/external-tables-delta-lake.md
shanisolomon Jun 29, 2026
eb526b2
Update data-explorer/kusto/management/delete-query-acceleration-polic…
shanisolomon Jun 29, 2026
1caf672
Merge branch 'main' into shanisolomon/auto-update-schema-docs
shanisolomon Jun 29, 2026
417f740
Address PR feedback: mention AutoUpdateSchema updates are periodic
shanisolomon Jun 30, 2026
1a5e61f
[C & G] | Correct erroneous ms.services for include files
markingmyname Jul 27, 2026
05ae967
Merge pull request #7561 from markingmyname/bam-iot
prmerger-automator[bot] Jul 27, 2026
5ca02d8
Merge pull request #7562 from MicrosoftDocs/main
learn-build-service-prod[bot] Jul 27, 2026
4118264
Freshness review: 11 Azure Data Explorer articles (#7564)
spelluru Jul 29, 2026
275b47f
Merge branch 'main' into shanisolomon/auto-update-schema-docs
shanisolomon Jul 29, 2026
106d62c
Fix leftover merge conflict markers from prior merge and resolve merg…
shanisolomon Jul 29, 2026
c9d7ea7
Merge pull request #7528 from shanisolomon/shanisolomon/auto-update-s…
prmerger-automator[bot] Jul 29, 2026
a7b2406
Remove preview labels and refresh metadata for graph docs (#7565)
royoMS Jul 29, 2026
93b8280
Merge pull request #7566 from MicrosoftDocs/main
learn-build-service-prod[bot] Jul 29, 2026
617f62c
Merging changes synced from https://github.com/MicrosoftDocs/dataexpl…
Jul 29, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions data-explorer/dashboard-parameters.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ The parameter can be seen in the **Parameters** side pane, but aren't currently

#### Use a single-selection fixed-values parameter

1. Run a sample query using the new *Event TypTope* parameter by using the `_eventType` variable name:
1. Run a sample query that uses the new *Event Type* parameter by using the `_eventType` variable name:

```kusto
StormEvents
Expand Down Expand Up @@ -249,13 +249,13 @@ Query-based parameter values are retrieved during dashboard loading by executing

### Multiple-selection query-based parameters

Query-based parameter values are derived at dashboard load time by executing the user specified query. The following example shows how to can create a multiple-selection query-based parameter:
Query-based parameter values come from running the user-specified query when the dashboard loads. The following example shows how to create a multiple-selection query-based parameter:

#### Create a multiple-selection query-based parameter

1. Select **Parameters** to open the **Parameters** pane and select **+ New parameter**.

1. 1. Fill in the details as mentioned in [Use a single-selection fixed-values parameter](#use-a-single-selection-fixed-values-parameter) with the following changes:
1. Fill in the details as described in [Use a single-selection fixed-values parameter](#use-a-single-selection-fixed-values-parameter) with the following changes:

* **Label**: State
* **Parameter type**: Multiple selection
Expand Down
13 changes: 7 additions & 6 deletions data-explorer/data-explorer-insights.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
---
title: Azure Data Explorer Clusters insights
description: This article describes how to use Azure Data Explorer Clusters Insights.
ms,reviewer: guregini
description: This article describes how to use Azure Data Explorer Clusters insights.
ms.reviewer: guregini
ms.topic: concept-article
ms.date: 05/24/2022
ai-usage: ai-assisted
ms.custom:
- subject-monitoring
- sfi-image-nochange
---

# Azure Data Explorer Clusters Insights
# Azure Data Explorer Clusters insights

Azure Data Explorer Clusters Insights provides comprehensive monitoring of your clusters by delivering a unified view of your cluster performance, operations, usage, and failures.

Expand Down Expand Up @@ -102,7 +103,7 @@ The **Key Metrics** tab shows a unified view of some of the cluster's metrics. T

### Usage tab

The **Usage** tab allows users to deep dive into the performance of the cluster's commands and queries. On this tab, you can:
The **Usage** tab provides detailed insights into the performance of the cluster's commands and queries. On this tab, you can:

- See which workload groups, users, and applications are sending the most queries or consuming the most CPU and memory. You can then understand which workloads are submitting the heaviest queries for the cluster to process.
- Identify top workload groups, users, and applications by failed queries.
Expand All @@ -121,7 +122,7 @@ The **Tables** tab shows the latest and historical properties of tables in the c

### Cache tab

The **Cache** tab allows users to analyze their actual queries' lookback window patterns and compare them to the configured cache policy (for each table). You can identify tables used by the most queries and tables that aren't queried at all, and adapt the cache policy accordingly.
The **Cache** tab lets you analyze your actual queries' lookback window patterns and compare them to the configured cache policy for each table. You can identify tables used by the most queries and tables that aren't queried at all, and adapt the cache policy accordingly.

You might get cache policy recommendations on specific tables in Azure Advisor. Currently, cache recommendations are available only from the [main Azure Advisor dashboard](azure-advisor.md#use-azure-advisor-recommendations). They're based on actual queries' lookback window in the past 30 days and an unoptimized cache policy for at least 95 percent of the queries.

Expand All @@ -140,7 +141,7 @@ On this tab, you can select a metric tile and deep dive to understand its trend

### Ingestion tab (preview tab)

The **Ingestion** tab allows you to monitor the status of queued ingestion operations to the cluster.
The **Ingestion** tab lets you monitor the status of queued ingestion operations to the cluster.

In the [queued ingestion](ingest-data-overview.md#continuous-data-ingestion) process, data ingestion is optimized for high throughput. This is done by grouping small chunks of incoming data into batches based on a configurable [ingestion batching policy](/kusto/management/batching-policy?view=azure-data-explorer&preserve-view=true).

Expand Down
25 changes: 13 additions & 12 deletions data-explorer/end-to-end-blob-ingestion.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
---
title: 'End-to-end blob ingestion into Azure Data Explorer'
description: In this article, you learn how to ingest blobs into Azure Data Explorer with an end-to-end example.
title: 'Configure Azure Data Explorer to ingest from Azure Blob Storage'
description: In this tutorial, you learn how to ingest blobs into Azure Data Explorer with an end-to-end example.
ms.reviewer: lugoldbe
ms.topic: tutorial
ms.topic: how-to
ms.date: 05/10/2023
ai-usage: ai-assisted
ms.custom:
- devx-track-arm-template
- sfi-ropc-nochange
---

# End-to-end blob ingestion into Azure Data Explorer
# Configure Azure Data Explorer to ingest from Azure Blob Storage

Azure Data Explorer is a fast and scalable data exploration service for log and telemetry data. This article gives you an end-to-end example of how to ingest data from Azure Blob storage into Azure Data Explorer.

Expand Down Expand Up @@ -191,7 +192,7 @@ In this article, you use an Azure Resource Manager (ARM) template to create a re
"variables": {
},
"resources": [{
"apiVersion": "2017-04-01",
"apiVersion": "2026-01-01",
"type": "Microsoft.EventHub/namespaces",
"name": "[parameters('eventHubNamespaceName')]",
"location": "[parameters('location')]",
Expand All @@ -205,7 +206,7 @@ In this article, you use an Azure Resource Manager (ARM) template to create a re
"maximumThroughputUnits": 0
}
}, {
"apiVersion": "2017-04-01",
"apiVersion": "2026-01-01",
"type": "Microsoft.EventHub/namespaces/eventhubs",
"name": "[concat(parameters('eventHubNamespaceName'), '/', parameters('eventHubName'))]",
"location": "[parameters('location')]",
Expand All @@ -218,7 +219,7 @@ In this article, you use an Azure Resource Manager (ARM) template to create a re
"type": "Microsoft.Storage/storageAccounts",
"name": "[parameters('storageAccountName')]",
"location": "[parameters('location')]",
"apiVersion": "2018-07-01",
"apiVersion": "2026-04-01",
"sku": {
"name": "[parameters('storageAccountType')]"
},
Expand All @@ -227,7 +228,7 @@ In this article, you use an Azure Resource Manager (ARM) template to create a re
{
"name": "[concat('default/', parameters('containerName'))]",
"type": "blobServices/containers",
"apiVersion": "2018-07-01",
"apiVersion": "2026-04-01",
"dependsOn": [
"[parameters('storageAccountName')]"
],
Expand All @@ -245,15 +246,15 @@ In this article, you use an Azure Resource Manager (ARM) template to create a re
"tier": "Standard",
"capacity": 2
},
"apiVersion": "2019-09-07",
"apiVersion": "2025-02-14",
"location": "[parameters('location')]",
"tags": {
"Created By": "GitHub quickstart template"
}
}, {
"name": "[concat(parameters('kustoClusterName'), '/', parameters('kustoDatabaseName'))]",
"type": "Microsoft.Kusto/clusters/databases",
"apiVersion": "2019-09-07",
"apiVersion": "2025-02-14",
"location": "[parameters('location')]",
"dependsOn": ["[resourceId('Microsoft.Kusto/clusters', parameters('kustoClusterName'))]"],
"properties": {
Expand All @@ -262,7 +263,7 @@ In this article, you use an Azure Resource Manager (ARM) template to create a re
}
}, {
"type": "Microsoft.Kusto/Clusters/principalAssignments",
"apiVersion": "2019-11-09",
"apiVersion": "2025-02-14",
"name": "[concat(parameters('kustoClusterName'), '/', parameters('clusterPrincipalAssignmentName'))]",
"dependsOn": ["[resourceId('Microsoft.Kusto/clusters', parameters('kustoClusterName'))]"],
"properties": {
Expand All @@ -273,7 +274,7 @@ In this article, you use an Azure Resource Manager (ARM) template to create a re
}
}, {
"type": "Microsoft.Kusto/Clusters/Databases/principalAssignments",
"apiVersion": "2019-11-09",
"apiVersion": "2025-02-14",
"name": "[concat(parameters('kustoClusterName'), '/', parameters('kustoDatabaseName'), '/', parameters('databasePrincipalAssignmentName'))]",
"dependsOn": ["[resourceId('Microsoft.Kusto/clusters/databases', parameters('kustoClusterName'), parameters('kustoDatabaseName'))]"],
"properties": {
Expand Down
12 changes: 6 additions & 6 deletions data-explorer/how-it-works.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: How Azure Data Explorer works
description: 'Learn more about how Azure Data Explorer works.'
ms.reviewer: avnera
ms.topic: how-to
ms.topic: concept-article
ms.date: 07/23/2026
ai-usage: ai-assisted
---
Expand Down Expand Up @@ -63,11 +63,11 @@ For more information on specifying data sort order, see [Row order policy](/kust

Azure Data Explorer uses distributed data query technology intended for fast ad hoc analytics on large unstructured data sets. Key features of this technology include:

* Query-generated temporary data is stored in aggregated RAM
* Relevant extents are marked on a query plan, providing snapshot isolation
* Fast and efficient queries are prioritized with short [default timeouts](/kusto/set-timeout-limits?view=azure-data-explorer&preserve-view=true)
* Native support for [cross-cluster queries](/kusto/query/cross-cluster-or-database-queries?view=azure-data-explorer&preserve-view=true) that minimizes inter-cluster data exchange
* Queries are just-in-time compiled into highly efficient machine code, using data statistics from all extents and tailored to column encoding specifics
* Query-generated temporary data is stored in aggregated RAM.
* Relevant extents are marked on a query plan, providing snapshot isolation.
* Fast and efficient queries are prioritized with short [default timeouts](/kusto/set-timeout-limits?view=azure-data-explorer&preserve-view=true).
* Native support for [cross-cluster queries](/kusto/query/cross-cluster-or-database-queries?view=azure-data-explorer&preserve-view=true) that minimizes inter-cluster data exchange.
* Queries are just-in-time compiled into highly efficient machine code, using data statistics from all extents and tailored to column encoding specifics.

> [!NOTE]
> Azure Data Explorer is designed to work with the [Kusto Query Language (KQL)](/kusto/query/index?view=azure-data-explorer&preserve-view=true), custom-built for Azure Data Explorer. Additionally, [T-SQL](t-sql.md) is supported.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
description: include file
services: iot-hub
author: spelluru
ms.service: iot-hub
ms.service: azure-iot-hub
ms.topic: include
ms.date: 09/07/2018
ms.author: spelluru
Expand Down
2 changes: 1 addition & 1 deletion data-explorer/includes/iot-hub-include-create-hub.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: include file
description: include file
ms.service: iot-hub
ms.service: azure-iot-hub
services: iot-hub
ms.topic: include
ms.date: 02/14/2020
Expand Down
2 changes: 1 addition & 1 deletion data-explorer/includes/iot-hub-pii-note-naming-device.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
ms.service: iot-hub
ms.service: azure-iot-hub
ms.topic: include
ms.date: 10/26/2018
---
Expand Down
2 changes: 1 addition & 1 deletion data-explorer/includes/iot-hub-pii-note-naming-hub.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
ms.service: iot-hub
ms.service: azure-iot-hub
ms.topic: include
ms.date: 10/26/2018
---
Expand Down
4 changes: 2 additions & 2 deletions data-explorer/ingest-data-event-grid-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
title: Ingest from storage using Event Grid subscription - Azure Data Explorer
description: This article describes Ingest from storage using Event Grid subscription in Azure Data Explorer.
ms.reviewer: leshalev
ms.topic: how-to
ms.date: 08/11/2025
ms.topic: concept-article
ms.date: 07/28/2026
ms.custom:
- devx-track-azurepowershell
- sfi-ropc-nochange
Expand Down
13 changes: 7 additions & 6 deletions data-explorer/k2bridge.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@ title: Visualize data from Azure Data Explorer using Kibana
description: In this article, you learn how to set up Azure Data Explorer as a data source for Kibana
ms.reviewer: guregini
ms.topic: how-to
ms.date: 03/12/2020
ms.date: 07/28/2026
ms.custom: sfi-image-nochange
ai-usage: ai-assisted
---

# Visualize data from Azure Data Explorer in Kibana with the K2Bridge open-source connector
Expand Down Expand Up @@ -46,10 +47,10 @@ Before you can visualize data from Azure Data Explorer in Kibana, have the follo
* An Azure subscription. Create a [free Azure account](https://azure.microsoft.com/pricing/purchase-options/azure-account?cid=msft_learn).
* An [Azure Data Explorer cluster and database](create-cluster-and-database.md). You will need the cluster's URL and the database name.
* [Helm v3](https://github.com/helm/helm#install), the Kubernetes package manager.
* Azure Kubernetes Service (AKS) cluster or any other Kubernetes cluster. Use version 1.21.2 or newer, with a minimum of three Azure Kubernetes Service nodes. Version 1.21.2 has been tested and verified. If you need an AKS cluster, see how to deploy an AKS cluster [using the Azure CLI](/azure/aks/kubernetes-walkthrough) or [using the Azure portal](/azure/aks/kubernetes-walkthrough-portal).
* Azure Kubernetes Service (AKS) cluster or any other Kubernetes cluster. Use version 1.21.2 or newer, with a minimum of three Azure Kubernetes Service nodes. Version 1.21.2 is tested and verified. If you need an AKS cluster, see how to deploy an AKS cluster [using the Azure CLI](/azure/aks/learn/quick-kubernetes-deploy-cli) or [using the Azure portal](/azure/aks/learn/quick-kubernetes-deploy-portal).
* A Microsoft Entra service principal authorized to view data in Azure Data Explorer, including the client ID and client secret. Alternatively, you can use a [system-assigned managed identity](/azure/aks/use-managed-identity).

If you choose to use a Microsoft Entra service principal, you will need to [create a Microsoft Entra service principal](/azure/active-directory/develop/howto-create-service-principal-portal#create-an-azure-active-directory-application). For the installation, you will need the ClientID and a Secret.
If you choose to use a Microsoft Entra service principal, you need to [create a Microsoft Entra service principal](/entra/identity-platform/howto-create-service-principal-portal). For the installation, you need the ClientID and a Secret.
We recommend a service principal with viewer permission and discourage you from using higher-level permissions. To assign permissions, see [Manage database permissions in the Azure portal](manage-database-permissions.md) or use management commands to [Manage database security roles](/kusto/management/manage-database-security-roles?view=azure-data-explorer&preserve-view=true).

If you choose to use a system assigned identity, you will need to get the agent pool managed identity **ClientID** (located in the generated "[_MC_xxxx_]" [resource group](/azure/aks/faq#why-are-two-resource-groups-created-with-aks))
Expand Down Expand Up @@ -93,7 +94,7 @@ By default, the Helm chart of K2Bridge references a publicly available image loc
```

> [!NOTE]
> When using a managed identity, the ADX_CLIENT_ID value is the client ID of the managed identity, located in the generated "[_MC_xxxx_]" resource group. For more information, see [MC_ resource group](/azure/aks/faq#why-are-two-resource-groups-created-with-aks). The ADX_SECRET_ID is only required if you use a Microsoft Entra service principal.
> When you use a managed identity, the `ADX_CLIENT_ID` value is the client ID of the managed identity, located in the generated `[_MC_xxxx_]` resource group. For more information, see [MC_ resource group](/azure/aks/faq#why-are-two-resource-groups-created-with-aks). The `ADX_CLIENT_SECRET` is only required if you use a Microsoft Entra service principal.

1. Optionally, enable Application Insights telemetry. If you're using Application Insights for the first time, [create an Application Insights resource](/azure/azure-monitor/app/create-new-resource). [Copy the instrumentation key](/azure/azure-monitor/app/create-new-resource#copy-the-instrumentation-key) to a variable.

Expand All @@ -118,7 +119,7 @@ By default, the Helm chart of K2Bridge references a publicly available image loc
helm install k2bridge charts/k2bridge -n k2bridge --set settings.adxClusterUrl="$ADX_URL" --set settings.adxDefaultDatabaseName="$ADX_DATABASE" --set settings.aadClientId="$ADX_CLIENT_ID" --set settings.useManagedIdentity=true --set settings.aadTenantId="$ADX_TENANT_ID" [--set image.tag=7.16_latest] [--set settings.collectTelemetry=$COLLECT_TELEMETRY]
```

In [Configuration](https://github.com/microsoft/K2Bridge/blob/master/docs/configuration.md), you can find the complete set of configuration options.
In [Configuration](https://github.com/microsoft/K2Bridge/blob/main/docs/configuration.md), you can find the complete set of configuration options.

1. <a name="install-kibana-service"></a> The previous command's output suggests the next Helm command to deploy Kibana. Optionally, run this command:

Expand Down Expand Up @@ -204,7 +205,7 @@ When Azure Data Explorer is configured as a data source for Kibana, you can use
> Only kibana's Lucene query syntax is supported. Do not use the KQL option, which stands for Kibana Query Language.

> [!Tip]
> In [Searching](https://github.com/microsoft/K2Bridge/blob/master/docs/searching.md), you can find more search rules and logic.
> In [Searching](https://github.com/microsoft/K2Bridge/blob/main/docs/searching.md), you can find more search rules and logic.

1. To filter your search results, use the **Available field** list. The field list is where you can see:

Expand Down
10 changes: 6 additions & 4 deletions data-explorer/kusto-emulator-install.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,12 @@ ms.custom: sfi-image-nochange

# Install the Azure Data Explorer Kusto emulator

The Kusto emulator is a local environment that runs the Azure Data Explorer query engine in a Docker container. You can develop and test queries without provisioning an Azure cluster.

You can install the Azure Data Explorer Kusto emulator in the following ways:

- **On your own device**: Consider using this option if you need to provision a local development environment
- **On a CI/CD agent virtual machine (VM)**: Use this option if you require a CI/CD pipeline for running automated tests
- **On your own device**: Use this option if you need to set up a local development environment.
- **On a CI/CD agent virtual machine (VM)**: Use this option if you need a CI/CD pipeline for running automated tests.

The emulator is available as a *Linux* Docker container image.

Expand All @@ -36,7 +38,7 @@ This article focuses on how to install the Linux Docker container on a Windows c
- Any Linux distribution that supports Docker Client for Linux
- A processor that supports SSE4.2/AVX2 instruction sets
- At least 2 GB of RAM (4 GB or more recommended)
- [Docker Client for Linux](https://docs.docker.com/desktop/install/linux-install/) or [Docker Client for Windows](https://docs.docker.com/desktop/windows/install/)
- [Docker Client for Linux](https://docs.docker.com/desktop/setup/install/linux/) or [Docker Client for Windows](https://docs.docker.com/desktop/setup/install/windows-install/)

> [!IMPORTANT]
> Linux distros only support Linux container images.
Expand Down Expand Up @@ -209,7 +211,7 @@ MyIngestedSample
| summarize sum(Id), avg(Id)
```

## Stopping the container
## Stop the container

1. You can stop the container by running the following command to obtain the container ID:

Expand Down
Loading