Recommend a Solution for Routing Logs — Lesson

This lesson focuses on one of the most overlooked yet operationally critical Azure design decisions: how logs get from where they are produced to where they need to be consumed. You have already chosen a workspace topology (LO-1); now the question is plumbing — which routing mechanism connects each telemetry source to the correct sink, and how do you enforce that plumbing at scale? The lesson covers diagnostic settings, the three destination types (Log Analytics workspace, Azure Storage, Event Hubs), Data Collection Rules (DCR), Azure Monitor Agent (AMA), partner-solution integrations, and Azure Policy enforcement. It does not cover workspace design (LO-1) or alert/dashboard configuration (LO-3).

Reference: Ch. 1, §1.1, p. 3–8 of the AZ-305 exam book.

Why This Matters

Picture a scenario that keeps coming up on AZ-305: Contoso deploys 200 resources across three subscriptions but never configures diagnostic settings. Six months later the security team discovers that Key Vault access logs, virtual network flow logs, and SQL audit logs were never captured — a major compliance gap. The resources were running fine, platform metrics showed green dashboards, but no resource-level logs were ever routed anywhere. The root cause is simple: Azure does not route resource logs by default. Every byte of diagnostic data that reaches a Log Analytics workspace, a Storage account, or an Event Hub does so because an architect explicitly designed the routing pipeline. That architect is the role the AZ-305 exam is testing you for.

Getting log routing wrong has three direct consequences: blind spots during incident response (increased MTTD), compliance audit failures (missing retention evidence), and runaway costs (duplicate routing, over-collected data). Getting it right means every resource emits exactly the data you need, to the right destination, at the right retention tier, enforced by policy so no new deployment slips through.

Prerequisites

• Log Analytics workspace topology — Can you explain the difference between centralised, decentralised, and hybrid workspace designs? Self-check: which topology lets you use a single KQL query across all resources?
• Azure Resource Manager (ARM) hierarchy — Do you understand how management groups, subscriptions, and resource groups scope policy assignments? Self-check: at which scope would you assign a policy to enforce diagnostic settings across all production resources?
• Azure RBAC basics — Are you familiar with the Monitoring Contributor and Log Analytics Contributor roles? Self-check: which role lets you create a diagnostic setting on a resource you don't own?
• JSON and Bicep fundamentals — Can you read a Bicep resource declaration and a JSON policy rule? Self-check: what does the DeployIfNotExists effect do in Azure Policy?

Learning Objectives

By the end of this lesson you will be able to:

1. Design a diagnostic settings routing plan that maps each Azure resource type to the appropriate sink (Log Analytics, Storage, Event Hub) based on query, compliance, and cost requirements.
2. Evaluate the three destination types and recommend when to use each — or when to fan out to multiple sinks simultaneously.
3. Recommend a Data Collection Rules (DCR) configuration for VM guest-OS telemetry via Azure Monitor Agent (AMA), including multi-workspace routing.
4. Construct an Azure Policy strategy that enforces diagnostic settings at scale using DeployIfNotExists policies and policy initiatives.
5. Analyse the cost implications of routing decisions — duplicate routing, data-volume filtering, and transformation rules in DCRs.

Building Blocks

Diagnostic settings — Analogy: think of each Azure resource as a building with security cameras; the cameras are always there, but the footage only reaches your monitoring station if you run a cable from the building to the station. A diagnostic setting is that cable. → Formal definition: a sub-resource (Microsoft.Insights/diagnosticSettings) attached to an Azure resource that routes platform metrics and resource logs to one or more sinks — Log Analytics workspace, Azure Storage account, or Event Hub. Each resource supports up to 5 diagnostic settings, and each must target a distinct sink. → Why it matters: without a diagnostic setting, most resource-level logs are simply discarded by the platform. This is the single most common observability gap in Azure environments.

Log categories and category groups — Analogy: if a diagnostic setting is a cable, log categories are the channels on that cable — you choose which channels to transmit. → Formal definition: each Azure resource type exposes a set of log categories (e.g., SQLSecurityAuditEvents, KeyVaultAuditEvent, NetworkSecurityGroupFlowEvent). Category groups like allLogs and audit bundle related categories for convenience. → Why it matters: selecting only the categories you need is the primary lever for controlling ingestion volume and cost.

Azure Storage account (as a log sink) — Analogy: a filing cabinet in a locked room — you can store documents for decades, but you cannot search them without pulling them out first. → Formal definition: when configured as a diagnostic-setting destination, logs are written as JSON blobs organised by resource ID, date, and category. You can apply immutability policies for WORM (Write Once, Read Many) compliance. → Why it matters: Storage is the cheapest long-term archive for logs that must be retained for regulatory reasons but do not need real-time query capability.

Event Hub (as a log sink) — Analogy: a conveyor belt in a factory — items (log events) arrive in order and are picked up by one or more workers (consumers) downstream. → Formal definition: an Azure Event Hubs namespace configured as a diagnostic-setting destination. Logs stream in near real-time and are consumed by external tools (Splunk, Datadog, a custom Azure Function). Each consumer gets its own consumer group. → Why it matters: Event Hubs are the only way to get Azure platform logs to third-party SIEMs and partner solutions in near real-time.

Data Collection Rules (DCR) — Analogy: a recipe card that tells the kitchen staff (AMA) exactly which ingredients (data streams) to collect, how to prepare them (transformations), and which plates (workspaces) to serve them on. → Formal definition: an ARM resource (Microsoft.Insights/dataCollectionRules) that defines data sources (performance counters, Windows Event Log, Syslog, custom text logs), optional KQL transformations, and one or more destinations. DCRs are associated with VMs via Data Collection Rule Associations (DCRAs). → Why it matters: DCRs replace the legacy workspace-level configuration of the MMA agent and give you per-stream, per-destination control — including the ability to filter, transform, and route different streams to different workspaces from a single VM.

Azure Monitor Agent (AMA) — Analogy: a postal carrier who picks up mail from your house (VM) and delivers each envelope to the address on the label (destination in the DCR). → Formal definition: the unified, cross-platform data-collection agent (Windows and Linux) that replaces the legacy Log Analytics agent (MMA/OMS). AMA reads its instructions from one or more DCRs and supports multi-homing natively. → Why it matters: AMA is the only supported agent path going forward — the legacy agent reached end-of-support in August 2024. Every VM-based log routing design must use AMA + DCR.

Deep Dive

Diagnostic Settings — The Platform Routing Layer

Diagnostic settings are the foundation of log routing for PaaS and control-plane telemetry. Every Azure resource type that emits logs exposes a set of log categories and metrics categories. You create a diagnostic setting to select which categories flow to which sink.

A single diagnostic setting targets exactly one sink. To route the same logs to multiple sinks you create multiple diagnostic settings, each pointing to a different destination. A resource can have up to 5 diagnostic settings.

For a second diagnostic setting routing the same categories to Storage:

[!WARNING] If two diagnostic settings route the same log category to the same sink, one of them silently fails. The portal does not warn you. Always verify with az monitor diagnostic-settings list --resource <id> after deployment.

The Activity Log is a special case — it is a subscription-level resource, not a per-resource one. You route it by creating a diagnostic setting at the subscription scope:

[!TIP] Use the audit category group (available on many resource types) instead of allLogs when you only need security-relevant events. This can reduce ingestion volume by 40–70% on chatty resources like Azure SQL and Key Vault.

Data Collection Rules and Azure Monitor Agent — The Guest-OS Routing Layer

Diagnostic settings handle PaaS and platform telemetry. For IaaS (VMs, VMSS, Arc-enabled servers), you need Azure Monitor Agent (AMA) configured by Data Collection Rules (DCR). This is the only supported path since the legacy MMA agent reached end-of-support.

A DCR has three sections:

1. Data sources — what to collect (performance counters, Windows Event Logs, Syslog, custom text/JSON logs, IIS logs).
2. Destinations — where to send it (one or more Log Analytics workspaces, Azure Monitor Metrics).
3. Data flows — which data source maps to which destination, optionally with a KQL transformation applied inline.

The key design insight: a single VM can be associated with multiple DCRs, and AMA evaluates all of them independently. This lets you split routing — security events to a central Sentinel workspace, performance counters to a regional workspace — without duplicating the agent.

[!IMPORTANT] DCR transformations use ingestion-time KQL (a subset of full KQL). You can filter rows, project columns, and compute new fields, but you cannot use join, union, or external-data operators. Transformations reduce ingestion cost by dropping unwanted rows before they hit the workspace.

DCR deployment at scale follows this pattern:

Multi-Sink Fan-Out and Partner Solutions

Many enterprise designs require the same log data to reach multiple consumers: a Log Analytics workspace for operational analytics, a Storage account for compliance archive, and an Event Hub for a third-party SIEM. This is called a fan-out pattern.

For platform logs (PaaS resources, Activity Log), fan-out is achieved by creating multiple diagnostic settings — one per sink — on the same resource. Remember the 5-setting limit per resource.

For guest-OS logs (AMA/DCR), fan-out is achieved by defining multiple destinations in the DCR's dataFlows section, or by using workspace data export rules that continuously export selected tables from Log Analytics to Storage or Event Hubs.

Azure also supports partner solutions in Azure Monitor. Providers like Datadog, Elastic, Logz.io, and Dynatrace have native integrations that route telemetry directly from Azure resources to the partner platform, bypassing Log Analytics entirely. These are configured in the Azure portal under Monitor > Partner Solutions or via the Microsoft.Datadog/monitors resource type.

[!NOTE] Partner-solution routing can coexist with diagnostic settings. You can route allLogs to Log Analytics for internal use and simultaneously route a subset to Datadog for the operations team's existing dashboards.

Enforcing Log Routing at Scale with Azure Policy

Manual creation of diagnostic settings does not scale — every new resource deployment is a potential gap. Azure Policy with the DeployIfNotExists (DINE) effect closes this gap by automatically creating diagnostic settings on any resource that lacks them.

Microsoft provides built-in policy definitions for most resource types:

Group related definitions into a Policy Initiative (also called a policy set) for easier assignment and compliance tracking:

The policy assignment specifies the target workspace as a parameter:

[!TIP] After assigning a DINE policy, run a remediation task to back-fill diagnostic settings on existing resources. New resources are covered automatically, but existing resources require an explicit remediation scan.

The compliance dashboard then becomes your audit evidence: 100% compliance means every in-scope resource has the required diagnostic settings.

Cost Implications of Routing Decisions

Every routing path has a cost component. The primary levers:

Use this query monthly to identify tables that should be moved to the Basic Logs tier or filtered with DCR transformations.

Worked Examples

Easy — Single Resource Diagnostic Setting

Problem: Contoso has an Azure Key Vault in West Europe. They need audit logs retained for 90 days for compliance and want to query them with KQL for incident investigation. No streaming to external tools is required.

Step-by-step solution:

1. Confirm the target Log Analytics workspace exists with at least 90-day interactive retention configured.
2. Create a diagnostic setting on the Key Vault:
   • Log category group: audit (captures AuditEvent category — the only category Key Vault exposes).
   • Metrics: AllMetrics enabled.
   • Destination: the Log Analytics workspace.
3. Verify with az monitor diagnostic-settings list --resource <keyvault-resource-id>.

[!NOTE] Key Vault is one of the simpler resources — it has only one log category (AuditEvent). The audit and allLogs category groups resolve to the same data here.

Medium — Multi-Sink Fan-Out for Compliance

Problem: Fabrikam's Azure SQL Database must route audit logs to three places: a Log Analytics workspace for the DBA team's operational queries, a Storage account with a 365-day immutability policy for compliance, and an Event Hub for their Splunk-based SOC. All three must receive the SQLSecurityAuditEvents category.

Step-by-step solution:

1. Create three diagnostic settings on the SQL Database — one per sink:
   • diag-sql-la: routes SQLSecurityAuditEvents + AllMetrics → Log Analytics workspace.
   • diag-sql-storage: routes SQLSecurityAuditEvents → Storage account (immutability policy pre-configured).
   • diag-sql-eh: routes SQLSecurityAuditEvents → Event Hub namespace.
2. Configure the Splunk Azure Event Hub input with a dedicated consumer group on the Event Hub.
3. On the Storage account, verify the immutability policy is set at the container level with a 365-day retention interval and legal-hold disabled (unless required).

[!NOTE] This uses 3 of the 5 allowed diagnostic settings. The remaining 2 could route additional categories (e.g., QueryStoreRuntimeStatistics) to a different workspace for performance tuning.

Hard — Enterprise-Scale VM Routing with AMA and DCR

Problem: Woodgrove Bank has 500 VMs across 3 Azure regions. The security team needs Windows SecurityEvent logs (specifically logon events 4624/4625) centralised in a Sentinel-enabled workspace in East US. The infrastructure team needs performance counters (% Processor Time, Available MBytes, % Free Space) in regional workspaces to avoid cross-region egress. Custom application logs written to /var/log/app/*.log on Linux VMs should go to the central workspace but be filtered to only include lines containing "ERROR" or "CRITICAL" to reduce volume.

Step-by-step solution:

1. Deploy AMA on all 500 VMs via the built-in Azure Policy initiative Enable Azure Monitor for VMs (with AMA).
2. Create DCR-Security (one per region, targeting Sentinel workspace):
   • Data source: windowsEventLogs with XPath filter Security!*[System[(EventID=4624 or EventID=4625)]].
   • Destination: la-sentinel (East US).
3. Create DCR-Perf (one per region, targeting the regional workspace):
   • Data source: performanceCounters with 60-second sampling for the three specified counters.
   • Destination: la-{region} (same region as the VMs).
4. Create DCR-AppLogs (one per region, targeting Sentinel workspace):
   • Data source: logFiles pointing at /var/log/app/*.log.
   • Transformation (ingestion-time KQL): source | where RawData has "ERROR" or RawData has "CRITICAL".
   • Destination: la-sentinel (East US).
5. Associate each VM with all applicable DCRs using DCR Associations (deployed via the same policy or Bicep modules).

[!NOTE] The ingestion-time transformation on custom logs is the key cost optimisation here. Without it, every line of the application log would be ingested — potentially hundreds of GB/day across 500 VMs. Filtering to ERROR/CRITICAL could reduce volume by 80–95%.

Visual Explanations

Log Routing Decision Tree (Mermaid)

Diagnostic Settings Architecture (TikZ)

Sink Comparison Table

DCR Data Flow Architecture

Routing Mechanism by Telemetry Source

Common Mistakes

❌ Myth: "Azure resources automatically send logs to Log Analytics once I create a workspace." ✅ Reality: Creating a workspace creates the destination. You must explicitly create a diagnostic setting on each resource to route data there. Without the setting, no resource logs flow — only platform metrics appear in Azure Metrics Explorer (which is a separate, automatic data store). Why it's tricky: The Azure portal shows metric charts for resources with no diagnostic settings configured, creating the illusion that observability is already in place. But metrics and logs are separate pipelines.

❌ Myth: "I can just install AMA on my VMs and logs will flow automatically." ✅ Reality: AMA requires at least one Data Collection Rule (DCR) and a DCR Association linking the VM to the rule. Without a DCR, AMA sits idle. Without a DCRA, the DCR exists but AMA on that VM does not know about it. Why it's tricky: The legacy MMA agent was configured at the workspace level — you pointed the agent at a workspace and it collected a default set of data. AMA's DCR model is more flexible but requires explicit configuration per data stream.

❌ Myth: "Routing the same logs to two Log Analytics workspaces costs the same as routing to one." ✅ Reality: Ingestion is charged per workspace. If you route SecurityEvent to both a regional workspace and a Sentinel workspace, you pay ingestion cost twice. This is by design — each workspace independently indexes the data. Use workspace data export rules or DCR transformations to avoid unintended duplication. Why it's tricky: The Azure documentation describes multi-homing as a feature of AMA, which it is — but nowhere on the configuration page does it warn you about the double billing.

❌ Myth: "Event Hub is a good primary log store — it's the cheapest option." ✅ Reality: Event Hub is a transient message broker, not a store. Events have a configurable TTL (default 1 day, max 90 days with premium). After the TTL expires, events are gone. Event Hub is for streaming to a downstream consumer, not for retention or querying. Why it's tricky: The Event Hub Capture feature writes events to Storage, which looks like retention — but Capture is an add-on and the stored data is in Avro format, not directly queryable with KQL.

Practice Exercises

🟢 Easy — You have an Azure Key Vault and need its audit logs in a Log Analytics workspace. What is the minimum configuration required?

▶💡 Hint

Only one routing mechanism is needed for a PaaS resource.

▶✅ Solution

Create a single diagnostic setting on the Key Vault with the audit category group enabled, targeting the Log Analytics workspace. No agent, no DCR — diagnostic settings are the PaaS routing mechanism.

🟢 Easy — What is the maximum number of diagnostic settings per Azure resource, and what constraint applies to each?

▶💡 Hint

Think about the relationship between settings and destinations.

▶✅ Solution

5 diagnostic settings per resource. Each must target a different sink — you cannot have two settings pointing to the same Log Analytics workspace, even if they select different log categories.

🟡 Medium — Contoso needs Windows Security Events from 100 VMs routed to a Sentinel workspace, but performance counters should go to a separate regional workspace. How do you design this with AMA?

▶💡 Hint

A VM can be associated with more than one DCR.

▶✅ Solution

Create two DCRs: (1) DCR-Security with windowsEventLogs data source targeting the Sentinel workspace, and (2) DCR-Perf with performanceCounters data source targeting the regional workspace. Associate both DCRs with each VM via DCR Associations. AMA evaluates both rules independently and routes each stream to its designated destination.

🟡 Medium — Fabrikam wants to reduce Log Analytics ingestion cost for their ContainerLog table, which ingests 50 GB/day but is only queried during incident investigations (approximately twice per month). What two options should you recommend?

▶💡 Hint

Think about the table plan and the retention tier.

▶✅ Solution

1. Switch ContainerLog to the Basic Logs table plan — this reduces per-GB ingestion cost by approximately 60% (trade-off: limited KQL, no alerts, 8-day interactive retention).
2. Configure an archive policy on the table beyond 8 days — archived data costs approximately 90% less than interactive retention and can be queried via search jobs when needed. Combined, these two changes could reduce the annual cost of this table from approximately $$50{,}000$ to approximately $$10{,}000$.

🔴 Hard — Woodgrove Bank must prove to auditors that every resource in the Production subscription has diagnostic settings routing allLogs to the central workspace. How do you implement and evidence this?

▶💡 Hint

You need both a corrective control (auto-fix gaps) and a reporting mechanism (prove compliance).

▶✅ Solution

1. Create a Policy Initiative containing DeployIfNotExists policy definitions for every resource type in the Production subscription (Key Vault, SQL, App Service, NSG, etc.).
2. Assign the initiative at the subscription scope with the central workspace resource ID as a parameter.
3. Run a remediation task to back-fill diagnostic settings on existing resources.
4. Export the Azure Policy Compliance report showing 100% compliant. This report — timestamped and filterable by policy assignment — is the auditor-ready evidence.
5. For continuous proof, create a Log Search Alert that queries AzureActivity | where OperationNameValue == "MICROSOFT.INSIGHTS/DIAGNOSTICSETTINGS/DELETE" and fires if anyone deletes a diagnostic setting.

🔴 Hard — An architect proposes routing all VM logs to both a regional workspace and a Sentinel workspace using AMA with two DCRs. The estimated ingestion is 200 GB/day per region across 3 regions. What are the cost implications, and how could you optimise?

▶💡 Hint

Consider which streams truly need to be in both workspaces and whether DCR transformations or workspace data export could help.

▶✅ Solution

Dual-routing 200 GB/day across 3 regions means $1{,}200$ GB/day total ingestion (600 GB regional + 600 GB Sentinel). At approximately $$2.76$/GB, that is roughly $$3{,}312$/day or $$100{,}000$/month.

Optimisations:

1. Route only security-relevant streams (SecurityEvent, Syslog auth) to Sentinel. Keep Perf and custom logs regional only. If security events are 20% of volume, Sentinel ingestion drops from 600 GB to 120 GB.
2. Use DCR transformations to filter security events — e.g., only logon events (4624/4625) rather than all of Security!*. This could reduce the security stream by 50–70%.
3. Enable Commitment Tiers on both workspace types — 500 GB/day on regional, 200 GB/day on Sentinel — for approximately 25–30% savings.
4. Use workspace data export from the regional workspace to stream a copy of security tables to Sentinel instead of double-collecting at the source. Export itself is free; you pay only Sentinel ingestion.

Combined, these optimisations could reduce monthly cost from $100,000 to approximately $30,000–$40,000.

Summary & Concept Map

• Diagnostic settings are the primary routing mechanism for PaaS and platform telemetry — they are disabled by default and must be explicitly created per resource.
• The three sink types serve distinct purposes: Log Analytics for real-time analytics, Storage for compliance archive, and Event Hub for SIEM streaming.
• Data Collection Rules (DCR) paired with Azure Monitor Agent (AMA) are the only supported path for VM guest-OS telemetry — the legacy MMA agent is end-of-support.
• A single VM can be associated with multiple DCRs, enabling split routing (security to Sentinel, perf to regional workspace) without duplicate agents.
• DCR transformations (ingestion-time KQL) are a powerful cost lever — filter and project data before it incurs ingestion charges.
• Azure Policy with DeployIfNotExists enforces diagnostic settings at scale and provides auditor-ready compliance evidence.
• Every routing decision has a cost dimension: category selection, dual-routing duplication, Basic Logs tier, and archive policies are the primary optimisation levers.