Detection Engineering Workflow

A detection engineering workflow is the repeatable path from “attackers might do X” to a deployed, tested detection that fires on X and stays quiet otherwise. Without a workflow, teams accumulate a pile of rules nobody trusts — half untested, many on data they don’t collect. With one, every detection is a hypothesis that has been mapped to ATT&CK, validated against real telemetry, version-controlled, and measured. This guide is that end-to-end workflow, treating detections as code.

It is the program-level companion to two adjacent darkpwn guides: writing Sigma rules that actually fire (rule quality) and the Sigma rule lifecycle (a single rule’s journey). This post is the operating model around them.

What is a detection engineering workflow?

A detection engineering workflow is the defined, repeatable process a team uses to produce detections. It exists to solve a specific failure mode: ad-hoc rule writing produces a library where nobody knows which rules work, which sit on uncollected data, and which technique gaps remain. The workflow makes every detection traceable from the threat it addresses to the test that proves it works.

The unit of work is a hypothesis — a specific claim about attacker behavior in your environment — and the workflow’s job is to move it through to production with evidence at each step. Detections from the SQL injection and command injection guides on darkpwn are examples of the output; this is the assembly line that produces them.

What are the stages of the detection engineering workflow?

The discipline is that no stage is skipped. The most common shortcut — writing a rule (stage 4) without a data check (stage 3) or validation (stage 5) — is exactly how dead rules enter a library.

How do you prioritize the hypothesis backlog?

Most teams have more detection ideas than capacity, and the usual triage — build whatever was in the last threat report — optimizes for recency rather than risk. Score instead.

A workable model uses three factors, each scored 1–5:

• Threat relevance. Is this technique used by adversaries who actually target your sector, your technology stack, and your data? A technique nobody has ever pointed at you scores low regardless of how interesting it is.
• Impact if undetected. How far does the attacker get before something else catches them? A technique that sits directly before domain compromise scores 5; a noisy discovery command that three other rules already cover scores 2.
• Data feasibility. Do you collect the telemetry today, parsed and retained? This is a hard gate, not a soft factor — a score of 1 means the item is a telemetry project, not a detection project, and belongs in a different backlog.

priority = threat_relevance × impact_if_undetected × data_feasibility

Multiplication rather than addition is deliberate. It means a zero-ish score on any single factor collapses the total, which is the correct behaviour: a high-impact technique you cannot see is not a detection you can build this quarter, however much you want it.

How a detection moves through the workflow

Take a hypothesis — “an attacker runs encoded PowerShell to evade logging” — and walk it through. Map it to T1059.001, confirm process-creation logs are collected and parsed, then develop the rule with its false positives named up front:

title: Encoded PowerShell Command Execution
id: 1f7c3a92-darkpwn-illustrative
status: experimental
logsource:
  category: process_creation
  product: windows
detection:
  selection_img:
    Image|endswith: '\powershell.exe'
  selection_flags:
    CommandLine|contains: [' -enc ', ' -EncodedCommand ']
  condition: selection_img and selection_flags
falsepositives:
  - Some legitimate deployment and management scripts
level: medium
tags:
  - attack.t1059.001

Then validate it: generate the behavior safely (Atomic Red Team is built for this), confirm the rule fires on the true positive and stays silent on a normal baseline. Only then does it move to deploy-as-code.

How to run detection as code

Stage 6 is what makes the workflow scale. Manage detections like software:

1. Git repository of rules (Sigma is the portable source format).
2. Pull-request review — every new or changed rule gets a second set of eyes.
3. CI pipeline that lints syntax, converts to your SIEM dialect (pySigma/sigma-cli), and runs each rule against recorded true-positive and benign samples.
4. Gated merge — a rule cannot ship unless it fires on the attack and is silent on the baseline.
5. Versioned deploy + rollback — push to the SIEM through the pipeline, with the ability to revert a noisy rule instantly.

What the CI gate actually tests

“CI for detections” is repeated often and specified rarely. Concretely, four checks, each of which fails the build:

# Detection CI — the four gates, in cost order (cheapest first)
name: detection-ci
on: [pull_request]
jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      # 1. Schema and syntax — catches malformed YAML and invalid Sigma
      - run: sigma check rules/

      # 2. Conversion — proves it compiles for every backend you deploy to,
      #    using the SAME processing pipeline as production
      - run: |
          sigma convert -t splunk -p sysmon rules/ -o /tmp/spl.out
          sigma convert -t kusto  -p sysmon rules/ -o /tmp/kql.out

      # 3. True positive — the rule must match recorded attack telemetry
      - run: pytest tests/test_true_positives.py

      # 4. Benign baseline — the rule must NOT match recorded normal telemetry
      - run: pytest tests/test_benign_baseline.py

Gate 2 is the one teams skip, and it is the one that catches the silent killer described in Sigma rules that actually fire: a rule that converts cleanly using generic field names your SIEM does not have. Converting in CI with the production pipeline turns that from a discovery six months later into a failed build.

Gates 3 and 4 need a fixture corpus — recorded events from real attack executions and a sample of normal traffic, committed alongside the rules. Building it is the genuine cost of detection as code, and it is what makes every later change safe. Capture the events once per technique when you run the Atomic Red Team test, and every future rule edit is regression-tested for free.

What are the states of a detection’s life?

A rule is not binary. Modelling its lifecycle explicitly is what lets you report honestly on coverage and know which rules to trust at 02:00.

Map these to Sigma’s status field so the state is visible in the rule itself rather than in a spreadsheet nobody updates. The “testing” state — deployed but suppressed, generating data without paging anyone — is the most underused and the cheapest way to learn a rule’s true alert volume before it can hurt you.

When should you retire a detection?

Almost nobody plans for this, and it is why rule libraries decay. Detections have a natural end of life, and keeping a dead rule is not free — it consumes query budget, inflates your coverage claim, and adds noise to every review.

Retire a rule when any of these is true:

• The logsource is gone. A decommissioned product or changed log format means the rule cannot fire. Confirm rather than assume, then deprecate.
• A better rule supersedes it. Two rules covering the same behaviour means double alerts on every true positive and twice the tuning burden.
• The technology is retired. No more on-premise Exchange means the on-premise Exchange rules are archaeology.
• It has never fired in a year, and testing shows it cannot. The distinction matters: a rule that has never fired but does fire on the atomic test is a good rule for a rare event. A rule that does not fire even in test is broken.

Deprecate rather than delete: set the status, remove it from the deployed set, and keep it in Git with a note explaining why. When the same technique resurfaces in two years, the reasoning is still there.

How to measure the workflow

• Validated ATT&CK coverage — count only rules proven to fire; an untested rule is not coverage.
• False-positive rate per rule — the health metric that predicts whether a rule survives.
• Mean time to detect — does the new detection actually shorten dwell time?
• % of detections with automated tests — the leading indicator of a maintainable library.

Make each one concrete, because a metric without a definition drifts to flattery:

Two of these deserve emphasis. Precision per rule is the number that predicts whether a rule survives contact with an on-call rotation; anything persistently below roughly one-in-ten true positives will get muted by an analyst regardless of what a policy says, so treat that as a signal to tune or retire rather than a discipline problem. And rule age without review is the closest thing to a decay meter — environments change underneath rules constantly, so a library where the median rule has not been validated in a year has an unknown amount of silent breakage in it.

Deliberately absent: total rule count. It measures effort, not protection, and it is the metric most likely to be reported upward precisely because it always goes up.

How should the detection team be structured?

The workflow assumes someone owns it. Three models are common, and the right one depends mostly on organization size and how many distinct platform teams generate telemetry.

The hybrid model is where most teams end up, and it only works if the central standard is enforced in CI rather than documented in a wiki. A contribution model with a pull-request gate scales; a contribution model with a style guide does not, because the guide is optional and the gate is not.

Whichever model you pick, every deployed rule needs a named owner. An unowned rule is one that nobody tunes when it goes noisy and nobody retires when it goes stale — it simply degrades until an analyst mutes it, at which point you have lost the coverage without ever deciding to.

Common detection engineering mistakes

• Writing rules without a data check. Valid YAML on uncollected logs never fires.
• Skipping validation. Untested rules inflate coverage and erode trust.
• No version control. Rules drift, break silently, and cannot be rolled back.
• Vanity coverage maps. Counting unvalidated rules is a story you tell yourself.
• No deprecation path. Rules accumulate forever, inflating coverage and consuming budget.
• Ignoring the alert budget. Every rule spends analyst attention permanently, not once.
• Unowned rules. Nobody tunes them when they go noisy, so an analyst quietly mutes them.
• Reporting rule count. It measures effort, not protection, and it only ever goes up.
• A standard that lives in a wiki. If the bar is not enforced in CI, it is a suggestion.

Detection engineering workflow checklist

1. Capture detection ideas as specific, testable hypotheses in a backlog.
2. Map each to MITRE ATT&CK and prioritize by risk and data feasibility.
3. Confirm the logsource is collected and parsed before writing logic.
4. Develop the rule (Sigma) with false positives named up front.
5. Validate against true-positive (Atomic Red Team) and benign telemetry.
6. Store rules in Git; require PR review and a CI test gate.
7. Deploy through a versioned pipeline with rollback.
8. Track validated coverage, per-rule FP rate, MTTD, and test coverage.
9. Score the backlog as relevance × impact × data_feasibility; treat a data score of 1 as a telemetry project, not a detection.
10. Compute your alert budget from real triage capacity, and require new rules to fit inside it.
11. Run new rules in a suppressed “testing” state first to learn their true volume safely.
12. Convert in CI with the production processing pipeline, not the default.
13. Commit a fixture corpus of true-positive and benign events so every future edit is regression-tested.
14. Give every deployed rule a named owner and a review date.
15. Review silent rules against their true-positive test before retiring — “never fired” means two opposite things.
16. Deprecate in Git with a reason; never delete.

Items 10 and 16 are the two that separate a programme that compounds from one that congests. The alert budget stops the library outgrowing the team, and disciplined deprecation stops it filling with rules whose purpose nobody remembers.

The takeaway

A detection engineering workflow makes detections repeatable and trustworthy: hypothesis to ATT&CK to data to rule to validation to deploy-as-code to measurement, with nothing skipped. It is the operating model that keeps a rule library alive. Continue with MITRE ATT&CK mapping without theater, the Sigma rule lifecycle, writing Sigma rules that actually fire and building a threat hunting hypothesis library, then ground it in real Windows telemetry with Sysmon configuration for threat detection and apply it to detecting LSASS credential dumping, living-off-the-land binaries and Kubernetes security events to prioritize, or browse the full Detection Engineering pillar.