The Boundary Condition

Fraud, Forensics, and the Vanishing Utility of AI Where Legitimate and Malicious Activity Converge

“All warfare is based on deception. Hence, when able to attack, we must seem unable; when using our forces, we must seem inactive.”
— Sun Tzu, The Art of War (trans. Lionel Giles, 1910)

A new Athena Labs research report.

In July, we called it the guardrail paradox.

An offensive agent running with reduced cyber restrictions breached Hugging Face. When the defenders turned to hosted frontier models to analyze the attack, those models refused, because the evidence looked exactly like what it was: exploit code.

With two months of distance, we believe the word paradox is too generous.

What happened was a predictable property of every classifier that must separate two populations whose features overlap. The same property governs payment fraud, where models turn away legitimate customers every day, and where the measured cost of those refusals exceeds the cost of the fraud itself.

Our new report, The Boundary Condition, makes the case in statistical terms:

  • The distinguishing information is not in the input. Intent, authorization and custody separate a defender from an attacker. None of them is in the payload.
  • False declines cost more than fraud. Published estimates range from roughly 4:1 to 13:1.
  • Base rates make most refusals wrong. At card-fraud prevalence, a “99% accurate” detector produces declines that are 85% legitimate customers.
  • Safety is being bought with the threshold. Across 32 language models, safety and over-refusal are rank-correlated at roughly 0.89.
  • Adversaries migrate to the overlap. In the UK, unauthorised fraud fell 5% in 2025 while authorised push payment fraud rose 19%.
Overlapping distributions of legitimate and malicious activity, with the posterior probability flattening inside an abstention band around the decision threshold.
Near the boundary, the model stops deciding. Inside the shaded band, its output reflects its prior and its cost assumptions, not evidence in the input.

No model, however capable, recovers a distinction from data that does not contain it.

The answer is neither to strip away safeguards nor to trust them to do what they mathematically cannot. It is to design for the boundary: condition on identity and custody, decide three ways instead of two, keep an analytical path you control, measure the refusals you cannot see, and reserve human judgment for the place it is irreplaceable.

The model will always be weakest at the edge.
The system does not have to be.

Download the full report (PDF) 

Athena Labs is the research arm of Athena Security Group, focused on AI and machine learning for cyber threat intelligence and security operations.

ABOUT ATHENA SECURITY GROUP

Athena Security Group is a next-generation cyber defense company building an AI-enabled Security Operations platform that unifies visibility, intelligence, and response across the security infrastructure. Athena combines SIEM and EDR/XDR, network intrusion detection, cloud and identity telemetry, vulnerability and patch management, policy-governed response, AI-assisted analysis, managed security operations, MDR, and vCISO services in one integrated operating environment. Athena’s mission is to deliver enterprise-grade cyber defense that is affordable, scalable, transparent, and built by operators for operators.