Search
StarWind is a hyperconverged (HCI) vendor with focus on Enterprise ROBO, SMB & Edge

Using Azure Data Explorer to store Microsoft Sentinel logs

  • March 31, 2022
  • 5 min read
IT Production Manager. Nicolas is primarily focused on Microsoft technologies, he is a Microsoft MVP in Cloud and Datacenter Management.
IT Production Manager. Nicolas is primarily focused on Microsoft technologies, he is a Microsoft MVP in Cloud and Datacenter Management.


In this article, I will explain how to export your data from Log Analytics to Azure Data Explorer. The retention in Microsoft Sentinel is typically 3-12 months retention for your SOC users. Then, all data will be sent to Azure Data Explorer and be retained for longer term. Azure Data Explorer is cheaper than Microsoft Sentinel.

Using Azure Data Explorer is useful when you want to query logs with Kusto Query Language (KQL) to search the data. On top of that, you can use the same queries in Azure Log Analytics and Azure Data Explorer.

Getting Started

Open the Azure portal and go to Azure Data Explorer to create the ADX cluster :

 Azure Data Explorer

Enter the cluster name and click Create

Click Create

Wait until the creation is done

The creation is done

Next, you can create the database, click Add Database

Add Database

Enter the database name and the retention period (in days)

Enter the database name and the retention period

Now, we will create the Event Hub, search for Event Hubs and click Create

Event Hubs

Enter the namespace name and click Create

Create Namespace

Then, copy the resource ID, you will need it later

Copy the resource ID

Now we need to configure the data export in Azure Sentinel’s Log Analytics workspace. First, we need to enable the resource providers called Microsoft.insights

You can use the following command:

Resource Providers

Then, go to the Log Analytics workspace and copy the name, the resource group and the workspace ID.

Log Analytics

Open the Cloud Shell from the Azure portal and copy/paste the following commands:

PowerShell

You can adjust the tables you want to export in the previous command.

Now we need to create table mapping. I used this script to simplify the process https://github.com/javiersoriano/sentinel-scripts/blob/main/ADX/Create-TableInADX.ps1

Open the Azure portal, click Cloud Shell. Then, click Upload/Download files

Upload/Download files

Upload the previous script

Upload the previous script

Then, you just need to run the script and enter the table + the workspace ID

Workspace ID

The script will output the commands to populate the ADX databse

ADX databse

Copy/paste these commands and go back to ADX cluster, Databases and choose Query. Paste the commands and click Run

Query

The tables will be populated

Create a new data ingestion connection in ADX from Event Hub

Create a new data ingestion connection

You may need to wait few minutes before creating the connection because It may take some time to appear in the following event hub field

Event Hub

Once it is updated, you should have “am-signinlogs” event hub available.

Graphical user interface, text, application, email Description automatically generated

Wait a few minutes and you should see incoming and outgoing message

Incoming and outgoing message

You can see the size of the database which should normally grow up

Database

To finish, we can query data directly in Azure Data Explorer instead of Azure Log Analytics

Azure Data Explorer

Found Nicolas’s article helpful? Looking for a reliable, high-performance, and cost-effective shared storage solution for your production cluster?
Dmytro Malynka
Dmytro Malynka StarWind Virtual SAN Product Manager
We’ve got you covered! StarWind Virtual SAN (VSAN) is specifically designed to provide highly-available shared storage for Hyper-V, vSphere, and KVM clusters. With StarWind VSAN, simplicity is key: utilize the local disks of your hypervisor hosts and create shared HA storage for your VMs. Interested in learning more? Book a short StarWind VSAN demo now and see it in action!