I Built an AI SOC Analyst Out of Logic Apps. Here’s What It Taught Me About Limits.

Ten workflows, nine of them workers, all of them living inside a ninety-second response budget. The AI was the easy part — the architecture is what the constraints wrote.

Microsoft SentinelAzure Logic AppsDefender XDRAzure OpenAISOAR~12 min read

A note on this post

This was written by AI from my own working notes and the real workflow definitions, then reviewed by me — so some details may be imprecise or incomplete. I publish these to show what’s possible and what I learned building it, not as a step-by-step guide to reproducing it.

Contents

1. The shape of it
2. The limit that shaped everything
3. Failing open, deliberately
4. Two scores, not one
5. Deduplication
6. Everything else that has a ceiling
7. What I’d do differently
8. What it actually cost
9. The takeaway

We use a third-party SOC. They’re good at what they do, but they see our environment through a keyhole: an alert fires, a ticket appears, and someone who has never met our environment decides whether it matters. Meanwhile the incident lands in Microsoft Sentinel with a title, a severity, and a handful of entities — a username, a hostname, maybe a URL — and nothing else. Everything that would let you actually judge it lives somewhere else. Sign-in history is in one blade. Device telemetry is in another. Reputation data is on three different vendor websites. Whether the user’s password changed last Tuesday is a Graph call away.

So the first twenty minutes of every investigation is the same twenty minutes. Open tabs. Copy IOCs. Paste. Read. Repeat.

I built a system that does those twenty minutes automatically, in about a minute, before a human ever looks at the incident. By the time an analyst opens the ticket, it already contains a written analysis, ten enrichment tables, and every KQL query — pre-populated with the incident’s real time windows and entity values — needed to reproduce the findings.

It’s built entirely out of Azure Logic Apps. That constraint turned out to be the most interesting part of the project.

The shape of it

There are ten workflows. One orchestrator and nine workers.

The orchestrator is triggered by Sentinel incident creation. It pulls the incident, resolves its entities, and fans out to the workers in parallel. Each worker owns exactly one enrichment domain, does its job, and returns a JSON envelope. The orchestrator fans back in, assembles everything into one context object, sends it to an LLM, and writes the result back to Sentinel as comments — plus, conditionally, an email and a ServiceNow ticket.Sentinel incident createdORCHESTRATORentity extraction · payload compositionparallel POST · shared-key auth8 workers — parallel, independent 90-second budgetsVirusTotalurlscan.ioNetskopeAbuseIPDBAccountsHostsMDEEmail / TIfan-in · fail-openanchored KQL replayAI analysis — 1 or 2 callsSentinel commentschunked, reverse-postedHigh-risk emailconditionalServiceNow ticketconditional · 9th workerFan-out, fan-in. The ServiceNow worker is the ninth — it runs at the end, only on the operational-review path.

What each worker does:

Worker	Input entities	What it fetches
Accounts	accounts	Graph user profile, interactive + non-interactive sign-ins, identity risk, CloudAppEvents, IdentityLogonEvents
Hosts	hosts	Defender device record, plus a ±30-minute DeviceProcessEvents / DeviceNetworkEvents / DeviceFileEvents / DeviceLogonEvents timeline
MDE	hosts	Device risk/exposure/onboarding posture, active Defender alerts (7d), and a wider ±2-hour process/network/file telemetry sweep
VirusTotal	URLs, IPs, hashes	Reputation reports, with live scan submission for unseen URLs
urlscan.io	URLs, IPs, hashes	Existing scan lookup, or a fresh scan submission and poll
Netskope	URLs	URL categorisation, plus UrlClickEvents and DeviceNetworkEvents for who actually clicked it
AbuseIPDB	IPs	Abuse confidence score, report counts, ISP, usage type
Email/TI	URLs, hashes, IPs	EmailEvents / EmailUrlInfo / EmailAttachmentInfo delivery analysis, and ThreatIntelligenceIndicator matches
ServiceNow	the finished analysis	OAuth, deduplication query, ticket creation

Every worker is a Logic App with an HTTP request trigger and the same three-part skeleton: validate a shared key, do the work, respond with { someTableHTML, kqlSectionHTML, results }. The results array is machine-readable and goes to the LLM. The HTML is display-ready and goes to humans. The KQL section is the receipts.

That last one matters more than it sounds. Every query a worker runs, it also returns as text — with the literal timestamps and entity values baked in. The analyst gets a “here’s how I know” section they can paste straight into Log Analytics. Automated analysis you can’t verify is just a rumour with better formatting.

The limit that shaped everything

A Logic App with an HTTP request trigger has to send its response inside a fixed, short window. I designed against 90 seconds. Whether the true documented ceiling for your plan is 90 or 120 doesn’t really matter — what matters is that it’s fixed, it’s short, and you cannot negotiate with it. Blow through it and the caller gets nothing. Not a partial result. Nothing.

That single fact is responsible for most of the architecture.

It’s why there are nine workers instead of one. A single workflow doing VirusTotal and urlscan and five KQL queries and Graph would blow the budget on a bad day and return nothing at all. Nine workers means nine independent 90-second budgets running concurrently. The wall clock is the slowest worker, not the sum of all of them.

It’s why the orchestrator sets DisableAsyncPattern on every worker call. Without it, Logic Apps sees a 202 and starts polling — the orchestrator would happily wait, and the worker’s carefully-composed response would never be read as a response body.

It’s why the VirusTotal worker caps itself at five scans. VT’s free tier allows four requests per minute, so a submit-then-fetch cycle needs a 15-second wait between the submission and the report. Five of those is 75 seconds. The Until loop that drives the queue has "timeout": "PT1M15S" hard-coded — the worker will abandon its own queue rather than miss the response deadline. It reports how many it skipped and returns what it has.

It’s why every KQL query ends in take 20 or take 25. Not because twenty rows is analytically ideal, but because that’s what comes back fast enough and fits in a prompt.

It’s why the foreach loops run sequentially. Parallel iteration inside a worker gets you throttled by the upstream API faster than it gets you speed, and a 429 storm mid-run is a worse failure than a slow run.

The general lesson: when your platform gives you a hard wall-clock budget, that budget becomes your architecture. You don’t design the system and then check whether it fits. You design around the fit from the first line.

Failing open, deliberately

The orchestrator never lets a worker take down a run. Every worker call is followed by a Compose that looks like this:

"runAfter": {
  "Call_Worker_VT": [ "Succeeded", "Failed", "TimedOut", "Skipped" ]
},
"inputs": "@coalesce(body('Call_Worker_VT'), json('{\"htmlTable\":\"\",\"results\":[]}'))"

Two things are happening. The runAfter accepts every terminal state, so a dead worker doesn’t halt the branch. The coalesce substitutes an empty envelope with the exact shape the downstream steps expect, so nothing further has to know the worker failed.

The same pattern repeats about forty times through the workflow — entity extraction, KQL execution, each AI call, each comment post. If VirusTotal is rate-limited, you get an analysis without VirusTotal. If Defender’s API is having a morning, you get an analysis without device telemetry. You always get an analysis.

For a triage tool this is unambiguously the right trade. A degraded answer in sixty seconds beats a perfect answer that never arrives — and beats a red “Failed” run that a human has to notice and rerun.

Two scores, not one

The genuinely useful idea in this system isn’t the enrichment. It’s the scoring model.

Most AI triage produces one number: how bad is this? That number is built for the SOC, and the SOC’s job is to close things. But a meaningful share of incidents get correctly closed as non-threats while still being symptoms of something broken. A deployment script that trips Defender on two hundred files isn’t an attack — it’s a deployment script that needs fixing. A 300% spike in failed logins traced to a password change that never propagated to a service account isn’t a breach — it’s a ticket for the identity team.

Those incidents close clean and the underlying problem recurs next week.

So the model returns two independent 1–10 scores:

• Security Risk Score — the real threat, after weighting false-positive likelihood.
• Operational Review Need — how much an internal team should look at this regardless of whether it’s a threat.

The system prompt calibrates them with worked examples, and the crucial ones are the divergent pairs:

Defender blocked 200 files on a single endpoint due to a new deployment script

Security2

Ops review9

Unknown process lateral movement with confirmed C2 beaconing

Security10

Ops review4

Two numbers, two audiences, two workflows. High security risk with low false-positive likelihood pages me. High operational review need triggers a second AI call that writes a different document entirely — an operations memo aimed at a helpdesk supervisor, with a probable root cause, the specific internal teams to involve, numbered remediation actions, and a recurrence assessment. That memo becomes a ServiceNow ticket.

The security analysis and the operations memo are written by the same model from the same evidence, but they are not the same document, because they are not for the same person.

Deduplication, or: how to stop generating the same ticket forever

Automated ticket creation without deduplication is a denial-of-service attack against your own service desk. A misconfigured script generating an incident every ten minutes will generate a ticket every ten minutes.

I ended up with two fingerprints doing different jobs.

The hard fingerprint is deterministic string assembly: the analytic rule name (capped at 80 characters), then up to three account names, two hostnames, two IPs and two URLs — normalised to lowercase, joined, then run through a sanitiser that strips newlines, tabs, non-breaking spaces, zero-width characters, smart quotes and em-dashes, and finally truncated to 160 characters. It becomes the ServiceNow short_description, and dedup is a query for that exact string on any ticket not in a resolved/closed/cancelled state. Same rule, same entities, open ticket already exists → skip.

That sanitiser looks like paranoia. It isn’t. Smart quotes and zero-width characters come in from copy-pasted alert descriptions, they’re invisible in every UI, and they silently break exact-match dedup forever. That chain of twelve nested replace() calls is the scar tissue from a real incident.

The AI fingerprint is a separate, cheap LLM call (400 output tokens) that normalises the incident into a rigid taxonomy — incident_type from a closed list of thirteen values, environment_scope from four, likely_cause_category from nine, source_system from six. Returns nothing but raw JSON. It’s the classification layer: what kind of thing is this, in language that’s stable across incidents whose titles aren’t.

The pattern worth taking away: use deterministic logic for the identity check, and the model only for normalisation. If the model had authored the dedup key directly, the key would drift between runs and dedup would silently stop working. Deterministic where correctness matters, probabilistic where judgment does.

Everything else that has a ceiling

Once you start looking for hard limits in this stack, they’re everywhere, and each one leaves a fingerprint in the code.

Sentinel comments have a size cap. The analysis doesn’t fit. So the orchestrator measures the finished HTML, divides by 28,000, and builds an array of chunks labelled [Part 1 of 3] — then posts them via the ARM API in an Until loop that walks the index backwards, from last chunk to first. Sentinel renders comments newest-first, so posting in reverse is the only way they read in order. Three separate payloads get this treatment: the AI analysis, the enrichment tables, and the KQL reference.

Model context has a ceiling — and you can’t measure it directly. The orchestrator measures the assembled context and, above a threshold, splits into two parallel LLM calls with disjoint section assignments — sections 1–8 plus 18–19 (incident, accounts, hosts, device telemetry) in one, sections 9–17 (all the threat-intel enrichment) in the other — then concatenates the results.

The catch is that the constraint is denominated in tokens and the only thing a Logic App can count is characters. There is no tokenizer in workflow definition language, so the threshold is length(string(context)) and the conversion is a guess. And it’s a bad guess, because characters-per-token isn’t a constant — it’s a property of the content. English prose runs around four characters per token. This payload is JSON packed with SHA256 hashes, GUIDs, IP addresses, URLs, command lines and raw KQL, none of which sit in a tokenizer’s vocabulary the way words do. Effective density drops toward two or three characters per token, and it varies incident to incident depending on which workers returned data.

So the split threshold isn’t derived from anything. It was walked downward over several iterations, each time an incident with unusually dense enrichment overflowed the window at a value that had been fine the week before. It’s an empirical number defending against a limit I can only estimate.

The orchestrator also emails me whenever a split fires, which is the part that makes this tolerable: I get told every time the system hits the ceiling, so the threshold gets tuned from observed failures rather than from a formula that was never going to hold. When you can only measure a proxy for the thing that’s actually constrained, that feedback loop is the design.

Output tokens are capped at 3,500. Nineteen sections in 3,500 tokens is roughly 180 tokens per section. That is a real constraint on how discursive the model can be, and it’s a large part of why the prompt is so prescriptive about formatting.

Sentinel comments accept HTML, not Markdown. Hence the most aggressively negative section of the system prompt: no #, no **, no backticks, no hyphens as bullets, <br> for every line break, the literal • character for lists. Left unconstrained the model emits Markdown, and Markdown renders in a Sentinel comment as exactly what it is — punctuation.

API rate limits are wall-clock costs. VirusTotal’s free tier is the binding constraint on the whole VT worker, as described above. urlscan.io needs a submit-then-poll cycle with waits between. Every one of those seconds comes out of the same 90.

What I’d do differently

I’d be a poor engineer if I published this as though it were finished. Three things are wrong with it, and I know exactly what they are.

Control flow depends on string-matching LLM prose. The high-risk email fires on a condition containing six contains checks against the model’s output — "severity: high", "severity assessment</strong><br>high", ">high<", and so on. The ServiceNow branch has eighteen, covering every way the model might render “Operational Review Need: 8”. Each of those is a phrasing the model actually produced at some point, discovered by watching it fail.

This works. It is also indefensible. The model is being asked for prose and the workflow is trying to parse structure back out of it. The fix is well understood: a second constrained call that returns {"security_risk": 8, "operational_review": 3} as strict JSON, and branch on integers. The AI fingerprint call in this same workflow already proves the pattern works. The scoring calls just haven’t been migrated yet.

Incident data flows into prompts, and model output flows into control flow. This is a security tool that ingests attacker-influenced strings — URLs, filenames, email subjects, command lines — concatenates them into an LLM prompt, and then uses that model’s output to decide who gets paged and what gets ticketed. The output is also written into Sentinel comments and HTML emails without escaping. A sufficiently thoughtful adversary who can control a filename has a path to influencing both the analysis text and the branch conditions. Structured outputs would close most of it. HTML-escaping the model’s response before rendering closes the rest. Neither is done today.

Secrets are in workflow parameters. They should be in Key Vault, referenced at runtime. Every workflow in this repo has been rotated and scrubbed before publication, but the right answer was never “scrub before sharing” — it was “never have them there.”

There’s also a smaller class of things I’d tidy: the workers all run their KQL against a hard-coded workspace rather than a parameter, and the Hosts and MDE workers overlap meaningfully in their Defender queries — Hosts pulls a ±30-minute timeline, MDE pulls ±2 hours plus device posture, and the union is more than either needs.

What it actually cost

The whole thing runs on Logic Apps Consumption, one Azure OpenAI deployment, and free-tier API keys for VirusTotal, urlscan.io and AbuseIPDB. Typical incident: two LLM calls, three on the operational path. Wall clock from Sentinel trigger to first comment is roughly a minute.

There is no container, no function app, no dependency file, and nothing to patch. It is JSON in a portal. That is either the most or the least appealing thing about it depending on your priors — but it means the operational cost of owning it is close to zero, which for a small security team is the number that actually decides whether a tool survives its first quarter.

The takeaway

The interesting engineering in this project wasn’t the AI. Sending context to a model and getting an analysis back is the easy part, and it gets easier every month.

The interesting engineering was everything downstream of a single unnegotiable fact: you have ninety seconds. Ninety seconds is why there are nine workers. It’s why they fail open. It’s why VirusTotal scans five URLs and tells you it skipped the rest. It’s why every query takes twenty rows. And ninety seconds sits alongside a comment size cap, a context window, an output token limit, a renderer that only speaks HTML, and three vendor rate limits — each one of which left something in the code that would look arbitrary if you didn’t know what it was defending against.

Good automation isn’t the thing that works when everything is available. It’s the thing that still returns something useful when half of it isn’t — and tells you honestly which half was missing.

Built for a retail environment running Microsoft Sentinel, Defender XDR, Entra ID and Netskope, with a third-party SOC. All credentials, tenant identifiers and internal addresses have been removed or rotated. The workflow definitions in this repository are the real ones, sanitized.