typed decisions instead of text

Most of what software needs from a model is not prose. It is a decision: which queue does this go to, is this urgent, how angry is this person, does this draft contradict the record. A chat model can be made to answer those by asking for JSON and parsing it, but it is slow, it costs a thousand times what the decision is worth, and it can answer with something that is not one of the options.

A decision model takes a state (text or structured data) and a set of typed questions — choose one of these, score this on that scale, is this true — and returns each answer with a probability and a confidence, all questions in one pass, in a fraction of a second. It cannot answer outside the options because it has no way to write text. That is the whole trick, and it is a good one.

Only that the answer is always one of the options. The model can still be confidently wrong. Its probabilities are calibrated against whatever it was trained on, not against your task; before a probability is used to decide anything, it has to be measured on your own outcomes. We score every decision model we use against realised results and two cheap baselines, in public where we can.

sam is our own decision model. Version one is deliberately small: one calibrated classifier per question, trained on our own labelled states, reported with held-out accuracy, Brier score and calibration error, and answering in under a millisecond on a four-core server. It cannot answer a question it was not trained on; that is the price of owning it. The next version is an encoder with question heads, a few hundred million parameters, milliseconds a decision, on our own hardware. It gets built when version one has shown, on real tasks, exactly where it falls short.

Rule for the house: for typed decisions, the model we build and run ourselves comes first. Outside models are for labelling data and for comparison, never for the decision itself.

Routing and triage inside our receptionist product, the next-minute call on the market page, and the agents in the survival experiment. Each use writes its outcomes back, which is how sam gets its training data.