The threshold is a price, not a score.

Fictional scenarios and synthetic data. Reviewed 6 November 2026.

Hollow Oak Supplements wants its risk score threshold raised from 60 to 80. Their support lead says the queue is full of real customers. Tomas pulls the last month and finds that raising it would have let 20 more fraudulent orders through. Ines asks neither "is 80 safer" nor "is 60 stricter". She asks what each one costs.

What is actually being decided.

A threshold is the point on a score at which an observation becomes an action. Moving it does one thing and one thing only: it trades one kind of error for the other. Lower it and you catch more fraud and interrupt more customers. Raise it and the reverse. There is no setting that reduces both, which is why the decision cannot be made by looking at the score alone.

The merchant carries both errors in money: goods and dispute fees for the fraud that passes, lost orders and support time for the customers who are stopped. The PSP carries the merchant's ratio with the acquirer. The analyst carries the choice, and should be able to show the arithmetic behind it. A rule is a claim gives the rule its sentence; this piece prices its fourth part.

Four cells, two prices.

Google's machine-learning crash course defines the four outcomes of any yes-or-no decision: true positives (flagged and fraud), false positives (flagged and legitimate), false negatives (passed and fraud) and true negatives (passed and legitimate). It defines precision as true positives over everything flagged, and recall as true positives over all actual fraud. We use those definitions and nothing else from the page.

A cost matrix adds one thing: a price on each cell. In this illustration we price a false negative at €80 (synthetic: goods shipped, the dispute fee and the handling time) and a false positive at €6 (the cost of a manual review and a small chance of an abandoned order). We leave the true cells unpriced to keep the sum short; in your own version, add the handling cost of true positives, because it is real.

Here is a synthetic month at Hollow Oak: 10,000 orders, of which 100 turn out, after disputes settle, to be fraud.

At 60: precision 70 / 2100 = 0.23, recall 70 / 100 = 0.70, false negatives 30. At 80: precision 50 / 120 = 0.42, recall 50 / 100 = 0.50, false negatives 50.

By precision, 80 is the better rule. By recall, 60 is. Neither tells you which to choose. The matrix does.

At 60: 230 false positives × €6 = €1,380, plus 30 false negatives × €80 = €2,400. Total €3,780. At 80: 70 false positives × €6 = €420, plus 50 false negatives × €80 = €4,000. Total €4,420.

The lower threshold costs €640 less a month, despite the fuller queue, because at these prices a missed fraud is worth about thirteen interruptions. The support lead's queue is real. It is also cheaper than the alternative.

Change the action and the price changes.

Now suppose Hollow Oak does not have a review queue. Flagged orders are declined outright. A declined legitimate customer does not come back, and the price of a false positive rises. In this illustration we set it at €15 (synthetic: lost margin on the order plus a share of lost repeat business).

At 60: 230 × €15 = €3,450, plus €2,400. Total €5,850. At 80: 70 × €15 = €1,050, plus €4,000. Total €5,050.

The answer flips. At €15 per interruption, the higher threshold saves €800 a month. Nothing changed in the fraud, the score or the data. Only the action changed, and with it the price of being wrong about a customer. This is the whole argument of this article: the threshold is not a property of the rule. It is a property of the rule and the action and the prices.

The alternative we keep alive: the prices themselves are estimates, and the false-negative price is the shakier one. Dispute outcomes arrive weeks after the order; Visa’s dispute-resolution guidance describes the lifecycle from the cardholder's claim through evidence submission to the issuer's decision, and the elapsed time varies. Your €80 is a cohort figure that will be revised. A chargeback arrives in its own time says how to wait for it.

Why accuracy cannot carry the decision alone.

In the synthetic month, the threshold-60 rule gets 70 fraud right and 9,670 legitimate orders right, which is an accuracy of 9,740 / 10,000 = 97.4 per cent. A rule that flags nothing at all gets 9,900 / 10,000 = 99.0 per cent. The crash course makes the same point in general terms: when the thing you are looking for is rare, accuracy rewards doing nothing. A rule that reports accuracy to a merchant is flattering itself. Report the four cells and the two prices.

When the matrix needs a third price.

Some errors do not cost money directly; they cost ratio. A fraudulent order that becomes a fraud report counts against the merchant in scheme monitoring whether or not the goods were worth much. Two ratios, two denominators, one portfolio covers the programmes. For a merchant near a threshold, it is honest to add a line to the false-negative price that represents the programme exposure, and to say openly that it is a judgement rather than a measurement. A matrix with a labelled guess in it is better than a matrix that pretends the guess does not exist.

A failure mode to watch.

The threshold is moved to make a complaint go away, and the price of the other error is never written down. A merchant's support lead sees the queue; nobody sees the fraud that will arrive as disputes in six weeks. Six weeks later the threshold is moved back, and the merchant's customers see a second change they were not told about. Both moves were made on one price. The matrix forces both prices onto the same page at the same time.

A way to work it.

1. Agree the two prices with the merchant, in writing, with the assumptions beside them. A rough number both sides have seen beats a precise number one side invented.
2. Take the last complete month with settled outcomes. For each candidate threshold, count the four cells.
3. Multiply the two error cells by their prices and sum. Put the sums side by side.
4. Repeat for the action actually in use. A review queue, a step-up and a hard decline each have their own false-positive price.
5. Choose the cheapest threshold, and write the rule sentence with its purpose and budget as in A rule is a claim.
6. Set a review date two dispute cycles out. Rerun the cells on the new cohort and revise the prices before you revise the threshold.

Take it to the desk.

What does one interrupted legitimate customer cost us, in money, and who on the team agreed that number?

Try it: A rule stops a customer who was not a fraudster Look up: precision · false positive · cohort

The Volume 01 lesson Measuring errors introduces the four cells for a reader who has not met them. The Volume 02 case vamp-portfolio-call is where the third price becomes the whole decision.

Edited 6 November 2026. Primary references are listed above; fictional scenarios are not claims about Antifrog customers or operating experience.

Fictional learning material. Any numerical examples are synthetic. Use only authorised, appropriately minimised records for workplace exercises; do not upload customer information to this learning site.