AI / FRAUD LAB
The deterministic evaluation failed before any result was shown.
Give Sweat historical payments and the policy protecting them. It finds attacks that pass, searches bounded replacements, selects a challenger, and tests whether it holds up on payments it never saw.
The institution fixes permissions, customer-friction limits, queue capacity, and the evaluator. Sweat audits point-in-time evidence and the positive-unlabeled risk before it searches attacks, features, or challenger policy.
All 144 executions are fraud in this synthetic test. Baseline@17 sends 14 to review and leaves 130 in the unlabeled pool. If training treats that pool as legitimate, the next model can learn the same blind spot.
The replay runs 144 synthetic fraud executions of one trusted-device takeover mutation. Dark squares are attempts, green outlined squares are surfaced by the current policy, and orange diamonds are hidden positives left unlabeled by that policy. 14 are surfaced and 130 remain unlabeled. The consequence model then narrows the unlabeled stream from 130 to 122 at mule hop one, 74 at mule hop two, and 59 at cash-out.
Synthetic truth verifies this pattern. Sweat proposes decision-time signals, rejects future-derived evidence, and compiles an inspectable rule without changing the institution’s limits or evaluator.
Choose how much customer friction and review work the institution can tolerate. Sweat evaluates the same 260 executable policies, maps the feasible trade-offs, and selects the best fit without opening the sealed test.
CH-256 selected under ≤0.80% false alerts and ≤60 / 1,344 reviews.
After the challenger wins search, the red-team probes it with 384 later synthetic events the optimizer never saw. Capture rises, but a relay variant still escapes—so the lab keeps searching instead of declaring victory.
Only after selection do we open 1,664 unseen synthetic payments: surface rate moves 33.1% → 69.1%; good-payment flags 21 → 15; reviews 42 → 108.
It surfaces +$33,282 more synthetic fraud value for review, reduces good-payment flags by 28.6%, and adds +66 higher-yield reviews.
One miss does not retrain a model. The synthetic episode ledger starts with no active pattern record, keeps the relay candidate unverified, proposes a point-in-time feature, and queues LAB-RUN-043. It writes no production label or policy.
From one synthetic control gap to a simulated Shadow-test package: attack the baseline, compile bounded defenses, search under institutional limits, test on unseen data, and turn every residual into a new experiment.