01. The Cost of False Positives
A false positive in threat intelligence fusion is not a minor inconvenience. It is an analyst's hour. It is a suppressed alert. It is a training signal telling the system, and the operator, that a particular pattern of activity is safe. In a high-volume environment, where hundreds of signals are being fused and assessed per hour, false positives are not noise around the signal. They are a load-bearing part of the system. They determine what gets reviewed and what does not.
We have spent considerable time with this problem during the development of SENTINEL-Ω, our threat intelligence fusion engine. The challenge is not simply reducing false positives, it is understanding their distribution. A system with a 5% false positive rate sounds reasonable until you discover that those 5% are not randomly distributed across the inputs. They cluster. They have structure. And when we first mapped that structure, the immediate reaction was: this looks like it was designed.
It was not. That realization, and what it implies, is the subject of this note.
// OBSERVATION - SENTINEL-Ω LOG, CYCLE 21
"Initial false positive rate on fusion outputs: 6.3%. On examination, 71% of false positives originated from inputs sharing three structural characteristics (source, user, and a score provided by our sentiment engine). The system had learned a bias we had not designed. We had not looked for it until the distribution made it visible."
02. What Geometry Means Here
When we say the geometry of false positives, we mean something specific. We are not speaking metaphorically about the shape of the problem. We are describing a literal spatial and statistical structure, a distribution of errors across the input space that has identifiable properties: clustering, directionality, correlation with specific input features.
Most error analysis in machine learning focuses on aggregate metrics. Accuracy. Precision. Recall. These are useful numbers, but they collapse the distribution of errors into single values, losing the geometric information entirely. A system with 94% precision and a clustered false positive distribution is a fundamentally different system from one with 94% precision and a uniformly random error distribution. The aggregate metrics do not distinguish them. The geometry does.
"Aggregate error metrics are averages of a distribution. Understanding where errors live, not just how many there are, is a different kind of analysis. It is also, we found, a more useful one."
In SENTINEL-Ω's case, the geometry revealed something we had not designed and had not anticipated: the false positives were not scattered randomly across input types. They were concentrated at the intersection of specific signal combinations, inputs that shared structural features that the fusion model had learned to associate with legitimate activity, but which an adversary could, in principle, deliberately replicate.
03. The Cluster Problem
The cluster problem is this: when errors are not uniformly distributed, the system's behavior is predictable in ways that are potentially exploitable.
A uniform false positive distribution means the system is equally wrong across the input space. This is recoverable. Operators learn to apply consistent skepticism, calibration can be applied globally, and there is no structural pattern that a sophisticated adversary can exploit. The errors are noise.
A clustered distribution is different. The clusters are signal about what the system has learned, what it has over-generalized, and where its decision boundaries are soft. If an adversary can identify those clusters, they have a map of where the system is most likely to be wrong, and therefore where activity is most likely to be classified as benign.
// WHY CLUSTERING IS THE PROBLEM, SPECIFICALLY
- - Clustered false positives create predictable blind spots rather than unpredictable noise
- - Blind spots can be reverse-engineered from system behavior without access to the model
- - Training on operational data can reinforce clusters rather than correct them, if the feedback loop is not carefully designed
- - Analysts adapt to known false positive patterns, which means clusters affect human behavior as well as system behavior
We have taken to calling this the orientation problem: a clustered error distribution orients both the system and its operators in ways that are systematic and therefore structural. It is not that the system occasionally gets fooled. It is that there are specific directions in which the system is reliably more foolable.
04. What the Structure Revealed
When we mapped SENTINEL-Ω's false positive distribution plotting errors against input feature space and looking for density patterns, three clusters emerged clearly. Each cluster had a characteristic input signature: a combination of source type, signal cadence, and behavioral pattern that the model had learned to weight toward benign.
The first cluster: signals from sources with long historical clean records, arriving in regular cadence, exhibiting low behavioral variance. The model had learned, correctly, that this combination was historically low-risk. It had generalized this learning too broadly, applying it even when other contextual indicators were elevated.
The second cluster: signals that had been manually reviewed and cleared by analysts in prior cycles. The model had incorporated analyst verdicts as strong positive signal but had over-weighted the source-cleared label relative to the signal content. A signal that had been cleared once was substantially less likely to be flagged again, regardless of content change.
The third cluster was the one that gave us pause: signals whose structure closely resembled known training data examples of benign activity. The model had not merely learned that certain activity was benign, it had learned specific structural patterns associated with benign examples, and was generalizing inputs that shared those structural features, without sharing the actual behavioral context.
"The third cluster looked almost like a template. Input a signal that structurally resembles a known-benign pattern, and the fusion engine tends to classify it as benign. We had not designed a template. We had accidentally created one."
05. Intentional vs. Emergent
The immediate question, when we saw the third cluster, was: is this a designed system behavior or an emergent property of the training process? The answer matters enormously for several reasons.
If the structure is designed - meaning someone deliberately created or exploited it. The implications are serious and immediate. The system has been compromised in a specific and targeted way. The structure is a weapon, not a byproduct.
If the structure is emergent - an unintended consequence of the training data, the model architecture, and the feedback loops. The implications are different. The system has not been compromised. It has developed a systematic bias as a consequence of how it was built and trained. This is fixable, though the fix is not simple. It is also, in its own way, a kind of failure: we built a system with a predictable blind spot without knowing we were doing so.
After careful analysis of the training data, the model architecture, and the operational history, we concluded: the third cluster is emergent. The structural pattern that concentrates false positives there is not the result of deliberate manipulation. It is the result of a training distribution that over-represented certain benign structural patterns and under-represented variants.
This conclusion is reassuring in one direction and sobering in another. The system has not been attacked. The system has developed a limitation that makes it exploitable in principle, by anyone who studies its behavior carefully enough.
// THE DISTINCTION IN OPERATIONAL TERMS
"An emergent structural bias and a deliberately planted one produce identical behavioral signatures. You cannot tell them apart from the outside. From the inside, from analysis of training data and model weights, they are distinguishable. But most operational environments do not run this analysis routinely. The bias is invisible until someone maps the geometry."
06. Adversarial Implications
We use the word "honest" deliberately, and we think it is the right word.
The implications of a predictable, structured false positive distribution in a threat intelligence system are not abstract. They translate directly into adversarial operational advantage.
If an adversary can identify that a fusion system tends to under-flag signals with a particular structural signature: long clean history, regular cadence, low variance. They can engineer their signals to match that signature. This is not a sophisticated attack. It requires observation of system behavior over time, which is available to any patient adversary operating in the environment the system monitors.
This is a variant of what security researchers call adversarial evasion, deliberately crafting inputs to exploit known model behaviors. The difference in the intelligence fusion context is that the adversary does not need access to the model. They need access to the environment the model monitors. If they can observe which of their activities get flagged and which do not, they can map the decision boundary from the outside.1Szegedy et al., "Intriguing Properties of Neural Networks," ICLR 2014. The foundational adversarial examples paper. The geometric intuition of decision boundaries begins here.
The practical consequence: a fusion system with a clustered false positive distribution is not merely imprecise, it is a system whose imprecision is potentially legible to a sophisticated adversary. The errors tell a story. The story can be read.2Biggio & Roli, "Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning," Pattern Recognition, 2018. Useful survey of adversarial evasion.
Designing against this requires more than reducing false positives. It requires designing error distributions that are genuinely hard to predict, which means understanding the geometry of errors and actively working against clustering, not just toward aggregate metric improvement.
07. Designing for Honest Error
What does it mean to design for honest error distribution? We have been working on this in SENTINEL-Ω development with a set of specific interventions:
01 -Error distribution monitoring as a first-class metric. We now track clustering coefficients on false positive distributions alongside precision and recall. If the cluster coefficient rises, it is treated as a regression regardless of aggregate metric performance.4Sculley et al., "Hidden Technical Debt in Machine Learning Systems," NeurIPS 2015. The feedback loop problems apply directly to how analyst corrections interact with model training.
02 -Deliberate input diversity in training. We audit training data for structural over-representation and actively supplement under-represented input patterns. The goal is not just accuracy on the training distribution but a training distribution that approximates the full operational input space.
03 -Analyst feedback integration with structural weighting. When analysts correct false positives, we capture not just the verdict but the structural features of the input. This allows us to track whether corrections are correcting specific clusters or distributed randomly.
04 -Adversarial input augmentation. We now generate synthetic inputs designed to match the structural signatures of false positive clusters and include them in evaluation suites. The question is not just "does the system perform well on the test set" but "does the system perform well on inputs designed to exploit its known error structure."
05 -Regular geometry audits. Every major model update is accompanied by a full mapping of false positive distribution. Changes in cluster structure, even if aggregate metrics improve, are flagged for review. A model that reduces false positives while concentrating them more tightly in a smaller cluster is not an improvement in the relevant dimension.3SENTINEL-Ω Experiment Log, Cycles 19–21. Internal.
"The goal is not zero false positives. The goal is false positives whose distribution contains no useful information for an adversary. Random noise is harder to exploit than structured error."
08. Open Questions
We do not have this solved. To be direct about where the work stands:
"Is it possible to certify that a false positive distribution is non-exploitable, or is this fundamentally an asymmetric information problem?"
// Status: We believe asymmetric. An adversary with patience has more time to map the distribution than we have to audit it.
"How does false positive geometry change over time as the operational environment shifts? Is cluster structure stable or does it migrate?"
// Status: Early indicators suggest clusters migrate. Tracking this over longer time series is a current research priority.
"Can deliberate error distribution design be quantified and verified, or is it necessarily a qualitative goal?"
// Status: Partial progress. The clustering coefficient is one quantitative measure but it does not fully characterize the adversarial risk of a given distribution.
"At what point does reducing false positive clustering introduce a different kind of bias, randomizing errors in ways that make feedback harder to act on?"
// Status: Open. There is a real tension between predictable errors (useful for correction) and unpredictable errors (harder to exploit).
"Is the 'almost intentional' appearance of emergent structure a reliable signal that the structure is adversarially significant?"
// Status: We think yes but cannot yet formalize the test.
09. Notes & References
This note draws on work across several fields. References below are not exhaustive, they represent the specific sources that shaped our thinking on this problem.
[01] Szegedy et al., "Intriguing Properties of Neural Networks," ICLR 2014. The foundational adversarial examples paper. The geometric intuition of decision boundaries begins here.
[02] Biggio & Roli, "Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning," Pattern Recognition, 2018. Useful survey of adversarial evasion.
[03] SENTINEL-Ω Experiment Log, Cycles 19–21. Internal.
[04] Sculley et al., "Hidden Technical Debt in Machine Learning Systems," NeurIPS 2015. The feedback loop problems apply directly to how analyst corrections interact with model training.
[05] Helios Prime, "Automated Reasoning and the Confidence Problem," Lab Notes, June 2026. The calibration problem and the geometry problem are related, both concern what aggregate metrics conceal
[06] Boyd, J., "Patterns of Conflict," 1986. Not about ML. About how predictable patterns create exploitable structure. Surprisingly applicable.
* A note on scope: this document describes our thinking as of the time of this writing. The field moves. Our opinions may move with it. We will update this note when they do.
* We are aware that "almost intentional" is not a formal category. We are working on formalizing it. In the meantime: you know it when you see it, and seeing it in your own system's outputs is an unusual experience.