Moving custom logs to the Logs Ingestion API takes real planning

Over the past few months I've come across a lot of Azure environments that still push custom logs through the classic Azure Monitor HTTP Data Collector API. It might be a PowerShell script, a Logic App or something the team built in-house, but the pattern rarely changes. You POST to the API with the workspace shared key, and the data shows up in a custom table. That convenience has a cost. With the official retirement set for September 2026, this needs to become a proper migration project. Moving to the Logs Ingestion API means more than rewriting a few calls, because authentication, table structure and pipeline security all change.

Microsoft replaces shared keys with a more secure ingestion stack

Microsoft has two main reasons for retiring the Data Collector API. The shared key model no longer holds up against modern compliance requirements. The old API also can't support features such as built-in transformations or granular, identity-based access. The Logs Ingestion API brings a different stack. Data goes through Data Collection Endpoints (DCE) and Data Collection Rules (DCR), and you authenticate through Microsoft Entra with either a managed identity or an app registration. Custom tables also follow a new format, with the _CL suffix and a schema you define up front.

In practice, a lot of existing pipelines are little more than background jobs with a workspace key hardcoded in the app settings. That works fine until someone asks for an audit trail or you need to revoke a key. In most tenants I get called into, finding out who (or what) still uses shared keys for ingestion is half the work.

Pro tip: Azure Policy can detect and prevent shared key ingestion. Look for the built-in policy "Azure Monitor Log Analytics workspaces should not use shared keys for ingestion" in the Policy catalog and put it to use.

Every ingestion workflow changes in four places

Whichever ingestion workflow you migrate, you'll have to deal with these changes:

• Data no longer goes straight to the workspace but through a Data Collection Endpoint and a Data Collection Rule.
• You authenticate with Entra ID (managed identity or app registration) instead of a workspace key.
• Custom tables have to be created in the new format, with explicit columns and the _CL suffix.
• Transformations such as renaming, filtering or enrichment happen in the DCR itself, written in KQL.

The diagram below shows the new flow. Data source, authentication, endpoint, rule and table are clearly separated, and that affects how you design permissions. Automation accounts and Logic Apps that used to get by with just a key are hit hardest.

flowchart TD
    Source["Source (script, app, Logic App)"] --> Auth["Authentication (Entra app or managed identity)"]
    Auth --> DCE["Data Collection Endpoint"]
    DCE --> DCR["Data Collection Rule (DCR)"]
    DCR --> Table["Custom Table (_CL)"]

Figure: The modern log pipeline, from the source through DCE and DCR to the new custom table.

The Data Collection Endpoint and Rule come first

This is how I do it in real projects. I start by creating a Data Collection Endpoint (DCE), either in the Azure portal or with ARM/Bicep, and that becomes the new entry point for ingestion. Next comes the Data Collection Rule (DCR). It links to the DCE, points to the Log Analytics workspace and the target table, and can carry KQL-based transformations if you need them. I really like these transformations. You can standardize, filter or enrich incoming data directly in the rule, no matter which source sent it.

The target table has to use the new format. In the Azure portal, open your Log Analytics workspace, go to Tables → Create → Custom log and define your schema. A simple example:

{
  "tableName": "CustomEvents_CL",
  "columns": [
    {"name": "EventTime", "type": "datetime"},
    {"name": "Level", "type": "string"},
    {"name": "Details", "type": "dynamic"}
  ]
}

In a real deployment, create the table in the portal and adjust the columns to fit your data.

The identity that writes the data (managed identity or app) needs the Monitoring Metrics Publisher role on the DCR. I assign it through IAM on the DCR resource so permissions stay tight and auditable.

Scripts and Logic Apps need the most rework

This is where most teams get stuck. Every custom client, script or Logic App that uses the workspace shared key has to be reworked. For PowerShell, .NET or Python you can use the official SDKs (Azure.Monitor.Ingestion, for example) or call the REST API directly, this time with an OAuth2 token for https://monitor.azure.com/. In Logic Apps, the HTTP action supports Entra authentication. Not every security vendor or SaaS integration supports the new API yet, though. In those cases you may have to file a feature request or put an Azure Function in between as a bridge.

Calls to the legacy Data Collector API are easy to recognize. They go to URLs like https://<workspace-id>.ods.opinsights.azure.com/api/logs?api-version=... and send the X-API-Key header. The new API talks to your DCE endpoint (e.g. https://<endpoint-name>.<region>.ingest.monitor.azure.com) and needs a bearer token. Microsoft's overview of the Logs Ingestion API is short and practical. If you're starting from legacy custom logs, read the migration guide as well.

Shared key consumers rarely show up in a single inventory

In most environments nobody has a full list of everything that uses workspace shared keys. I start in the Azure portal. In the Log Analytics workspace, "Workspace summary" → "Shared key usage" gives you a quick overview. If you run Microsoft Sentinel, look closely at custom connectors. They often still use the old API and have to move to the Logs Ingestion API, provided the vendor supports it.

KQL can't show you shared key usage, unfortunately, because authentication happens outside the workspace. That's why every migration project includes going through each script, pipeline and integration and making sure it gets updated. If a SaaS app sends you custom logs, ask the vendor what their migration plans are.

Early planning saves you pain later

I don't see the move to the Logs Ingestion API as a compliance checkbox. It's a real security upgrade. Shared keys disappear from your code, permissions are set explicitly on the DCR, and transformations sit where you actually need them. My advice is to run this as a structured project rather than a last-minute scramble. Take stock of every data source, give the migration a clear owner and bring in app teams and vendors early. Once Microsoft Sentinel and automated governance come into play, you'll be glad you did. The documentation is good, automation with Bicep and Terraform works well, and by 2026 this will be the only supported option. Better to get started now.