In this article, I will explain how to export your data from Log Analytics to Azure Data Explorer. The retention in Microsoft Sentinel is typically 3-12 months retention for your SOC users. Then, all data will be sent to Azure Data Explorer and be retained for longer term. Azure Data Explorer is cheaper than Microsoft Sentinel.

Using Azure Data Explorer is useful when you want to query logs with Kusto Query Language (KQL) to search the data. On top of that, you can use the same queries in Azure Log Analytics and Azure Data Explorer.

Getting Started

Open the Azure portal and go to Azure Data Explorer to create the ADX cluster :

 Azure Data Explorer

Enter the cluster name and click Create

Click Create

Wait until the creation is done

The creation is done

Next, you can create the database, click Add Database

Add Database

Enter the database name and the retention period (in days)

Enter the database name and the retention period

Now, we will create the Event Hub, search for Event Hubs and click Create

Event Hubs

Enter the namespace name and click Create

Create Namespace

Then, copy the resource ID, you will need it later

Copy the resource ID

Now we need to configure the data export in Azure Sentinel’s Log Analytics workspace. First, we need to enable the resource providers called Microsoft.insights

You can use the following command:

Resource Providers

Then, go to the Log Analytics workspace and copy the name, the resource group and the workspace ID.

Log Analytics

Open the Cloud Shell from the Azure portal and copy/paste the following commands:

PowerShell

You can adjust the tables you want to export in the previous command.

Now we need to create table mapping. I used this script to simplify the process https://github.com/javiersoriano/sentinel-scripts/blob/main/ADX/Create-TableInADX.ps1

Open the Azure portal, click Cloud Shell. Then, click Upload/Download files

Upload/Download files

Upload the previous script

Upload the previous script

Then, you just need to run the script and enter the table + the workspace ID

Workspace ID

The script will output the commands to populate the ADX databse

ADX databse

Copy/paste these commands and go back to ADX cluster, Databases and choose Query. Paste the commands and click Run

Query

The tables will be populated

Create a new data ingestion connection in ADX from Event Hub

Create a new data ingestion connection

You may need to wait few minutes before creating the connection because It may take some time to appear in the following event hub field

Event Hub

Once it is updated, you should have “am-signinlogs” event hub available.

Graphical user interface, text, application, email Description automatically generated

Wait a few minutes and you should see incoming and outgoing message

Incoming and outgoing message

You can see the size of the database which should normally grow up

Database

StarWind Backup Appliance (BA) is an industry-first all-NVMe backup appliance that provides unprecedented levels of backup and recovery speed. It comes as a tiny, pre-configured, ready-to-backup solution that eliminates the backup server storage bottleneck and fits even the strictest RTPO. Consisting of the best commodity hardware, your chosen hypervisor, and a StarWind SDS engine backed by 24/7/365 ProActive Support, StarWind BA ushers in a new age of a future-proof, eco-friendly, transparent, and reasonably priced backup infrastructure. Explore Backup Appliance from StarWind StarWind Backup Appliance Datasheet

To finish, we can query data directly in Azure Data Explorer instead of Azure Log Analytics

Azure Data Explorer

Back to blog