The NCSC, the UK National Cyber Security Centre, has watched well-funded security operation centres lose the ability to spot real attacks because their leadership scored them on the wrong things, according to its newest advisory. The most damaging case the agency cites in its newest advisory: an analyst working a ticket queue where 99% of alerts were noise, then graded on tickets-closed-per-hour. Every false-positive click moves the number up. Some of those clicks closed live intrusions.
- 99% false-positive rates in ticket-focused SOCs make speed-to-close metrics directly hostile to detection.
- Rule-count and log-volume metrics produce alert inflation and unread feeds: one SOC the NCSC visited had collected only the first 30 characters of its largest log source for years without noticing.
- Time-to-detect (TTD) and time-to-respond (TTR) are the only outward-facing SOC metrics that prove the function works.
- Red teaming and purple teaming, plus MITRE ATT&CK-decomposed test cases, give a SOC a defensible TTD number when real intrusions are rare.
The 99% false-positive SOC and the metric that broke it
The advisory describes a recurring pattern across SOC engagements the NCSC has run: skilled analysts, capable tooling, expensive log ingestion, and almost no ability to detect a live attacker — a gap the AI-SOC vendor debate has been arguing around for two years. The shared cause is governance rather than capability. SOC metrics in these environments are inherited from the IT service desk, customer support, and development teams that share the same ticketing system. Tickets-processed-per-shift and time-to-close are intuitive numbers for non-security executives, and they fit cleanly on a board dashboard. They are also the wrong things to optimize for an analyst whose alert queue runs 99% false positives. A reward for fast closure becomes a reward for closing real attacks alongside the noise.
The second class of broken metrics is volume. Number of detection rules incentivises analysts to write a rule per Indicator of Compromise (IOC), including individual IP addresses, which the NCSC observed in production. Volume of logs collected looks like coverage but is uncorrelated with detection unless the SOC is writing alerts and threat hunts against those logs; the 30-character-truncated log feed went undiscovered because no one was measured on whether the feed was useful.
Time-to-detect: the only one of these SOC metrics that survives red teaming
The NCSC sanctions only one outward-facing measure: whether the SOC detects and responds to attacks in a timely manner, expressed as time-to-detect (TTD) or time-to-respond (TTR). The problem with TTD is that for a healthy organisation, real successful attacks should be rare; the metric needs a denominator that the SOC incident queue does not provide. The advisory manufactures that denominator through controlled adversarial testing. Red teaming reproduces an attacker covert posture and tests both detection and response. Purple teaming sacrifices covertness for coverage and lets the SOC see, alongside the testing team, exactly which steps fired alerts and which did not. MITRE ATT&CK-decomposed test cases, where each adversary technique runs as an isolated step, give a SOC a reproducible TTD per technique rather than a single aggregate number.
The contrarian observation in the NCSC guidance is that any metric reported outwards changes analyst behaviour, even health-monitoring metrics the SOC manager intends only for internal tracking. The agency recommendation is sharper than the usual report-different-metrics version: ticket counts and rule counts should not be reported at all, inward or outward, because once analysts know the number is being watched they optimise it. Defining TTD/TTR as the single board-facing measure removes the incentive to game the upstream counters.
How to rebuild a SOC around detection instead of tickets
The agency recommendations sequence from culture to coverage: give analysts the time and authority to investigate first, then prove the detection coverage with adversary simulation. Each step assumes the metric reset above has happened.
Run hypothesis-led threat hunting on a recurring cadence. An analyst forms a hypothesis from threat intelligence about a likely attacker technique, then searches logs for evidence. Most hunts find nothing; the real output is the new detection rule or hardening recommendation the analyst writes afterwards. This is the activity that ticket-throughput SOC metrics make impossible.
Set hard false-positive thresholds on detection rules. The NCSC worked example is a rule for PowerShell execution by anyone outside an IT role, refined by working down the small set of remaining false positives until any new PowerShell execution is either an attack or a documented exception. Fortnightly or monthly false-positive reviews convert this rule is noisy into this rule needs different logic, rather than ignore the alerts it generates.
Track threat awareness, tool expertise, and organisational fluency as analyst-development metrics, not throughput. Use ATT&CK technique coverage and red-team escape rates rather than rule counts. Track analyst engagement with IT operations and business owners, because analysts who do not know what normal looks like cannot detect abnormal. A SOC manager internal dashboard can carry these signals; the board dashboard carries only TTD.
Validate the rebuild with a purple-team exercise on the techniques most likely to target your sector. Pick adversary playbooks relevant to your organisation, run each ATT&CK step in isolation, and measure how many fire alerts an analyst escalates within the SOC stated TTD. The NCSC caution is to use the exercise to hone the SOC rather than score it, so analysts find the search for passw variant when only search for password was alerted. The 99%-false-positive SOC the NCSC originally diagnosed ran a purple-team exercise after rebuilding around detection-coverage metrics, and the analyst whose queue had once been the noise floor detected the simulated adversary at hours-not-days TTD on every priority ATT&CK step — a rebuild driven by replacing the wrong SOC metrics with the one that matters.
Join our LinkedIn group Information Security Community!
