What are AI Decision Models? How They Work and Key Examples

Imagine you sit down at a restaurant and the waiter asks, “Still or sparkling?” You don’t reply with a paragraph about your hydration preferences; you pick one of the two options you were given.

Now think about how we use large language models (LLMs) in most apps today. We ask them questions like “Is this support ticket about billing, shipping, or returns?” and the model writes us a full sentence back, which we then have to parse with some code. Realistically, there were only three possible answers, but we paid for an entire thought process anyway.

This is the gap this new type of model is trying to fill. In September 2026, a startup called TypeSafe AI released Jev, a model that does not write text at all.

Instead, it scores a fixed set of answers and tells you how likely each one is. Within a few weeks, OpenAI, Cloudflare, Perplexity, and Microsoft had all shipped their own versions, which shows how fast this field moves.

In this article, I want to explore what these AI decision models are, how they work, and where the idea actually comes from (it is a lot older than all this hype suggests). I will also look at what independent benchmarks found.

Decision Models in a Nutshell

• An AI decision model takes an input and a list of allowed answers, then returns a probability for each answer instead of generating text.
• Because the model scores answers in one pass, TypeSafe reports Jev responding in 70 to 500 milliseconds, and the result can go straight into an if statement.
• The idea is not new. Zero-shot classifiers, rerankers, and verifiers have scored candidates for years, and the 2026 releases from TypeSafe, OpenAI, Cloudflare, Perplexity, and Microsoft mainly package it better.
• In Red Hat’s independent guardrail tests, decision models did not give faster, cheaper, or better answers than LLM-as-a-judge, and a small trained classifier came close on prompt injection.
• They work best as fast, cheap gates around an LLM. Test their probabilities on your own data before you set thresholds.

What Are AI Decision Models?

An AI decision model is a model that takes an input and a set of allowed answers and returns the probability of each answer, without generating any free text.

The main idea of these models is that the output is already structured.

You never have to parse a sentence because the model can only hand back one of the answers you gave it. This means the result can go straight into an if statement in your code.

I like to think of the difference as two types of exams.

A large language model takes an essay exam, writing its answer out word by word, whereas a decision model takes a multiple-choice exam and also tells you how sure it is about every box (a confidence score).

The three question types

Jev and most of the Jev alternatives support three types of questions:

• Choice: pick one option from a list (Jev allows up to 255 options), with a probability for each.
• Noul: the probability that a statement is true, which is TypeSafe’s name for a yes/no question.
• Score: a position on an ordered scale, like a 1 to 5 rubric.

TypeSafe also has an experimental Box type for locating things in images, which I will skip here.

Let me give you an example from TypeSafe’s own documentation.

You can send a state (which is just the text you want the model to look at) and a map of questions. For a choice question, each option comes with a short description in a field called criteria.

{
  "model": "jev-latest",
  "state": "My running shoes arrived in the wrong size. Can I swap them for a size 10?",
  "questions": {
    "department": {
      "type": "choice",
      "instructions": "Which team should handle this?",
      "criteria": {
        "returns": "Exchanges, wrong or damaged items",
        "shipping": "Delivery status, delays, lost packages",
        "billing": "Charges, invoices, payment problems"
      }
    }
  }
}

And this was the respective response:

{
  "model": "jev-1.13.0",
  "answers": {
    "department": {
      "type": "choice",
      "choice": "returns",
      "confidence": 1.0,
      "probabilities": {
        "shipping": 0.0,
        "returns": 1.0,
        "billing": 0.0
      }
    }
  }
}

There is no text to parse.

You get the chosen option, a probability for every option, and a confidence value derived from the top probability, so a flatter distribution means lower confidence. In the documentation, a messier ticket that mentions both a wrong size and a double charge splits the probability between returns (0.61) and billing (0.35), and the confidence drops to 0.42.

Why TypeSafe calls it a “System One” model

TypeSafe describes Jev as a “System One Model,” borrowing from Daniel Kahneman’s book Thinking, Fast and Slow. Why is that the case?

In very simple language, System 1 is your fast, gut-feeling judgment, and System 2 is your slow, step-by-step reasoning.

A decision model is meant to be the fast one.

One caveat: System 1 is also the part of our brain that makes quick mistakes, and to TypeSafe’s credit, its own docs openly list where Jev gets things wrong. We will come back to this later.

Clearing up the name

One thing that confused me when I first read about this is that “decision model” already means several other things. It can refer to:

• Decision tables in business rules software.
• The Decision Transformer in reinforcement learning, which actually generates actions, so it is almost the opposite of what we are discussing.
• Choice models in economics.

So just to be clear, in this article, a “decision model” means only the new category of models released in late 2026 that score typed answers instead of writing text.

How Token-by-Token Generation Works

Before getting into how decision models work, it helps to see how a normal LLM produces its answer. I believe this makes the difference easier to see.

A large language model works in a loop.

It turns our text into tokens, predicts a probability for every token in its vocabulary, picks one, adds it to the text, and then repeats.

Every extra token in the answer means another forward pass through the model.

Figure 1: An LLM needs one forward pass for every token it writes. Original image by author.

This loop works well for writing emails, code, or explanations, but when it comes to simple decisions, it can cause a few problems:

• You pay to generate words you will throw away.
• The model may reply “Yes, because…” when you asked for just “yes” or “no”, which breaks your parser.
• Small mistakes can build up over a long output.
• If you ask an LLM how confident it is, the number it writes back often has little to do with reality, as John Berryman at Arcturus Labs points out in his write-up of Jev.

Decision models are trying to fix exactly these problems.

How Decision Models Work: Scoring Outputs

Now let’s get into the actual mechanics.

There are three ideas worth understanding here:

1. One pass instead of a loop
2. Where the probabilities come from
3. What “calibrated” actually means

Let’s look at each of them in some detail.

One pass instead of a loop

A decision model reads your input and the questions and produces a score for each allowed answer.

Those scores are typically turned into probabilities using a softmax function. As you can probably tell, there is no long loop because there is nothing to write.

TypeSafe’s documentation also says that questions in the same request are evaluated in parallel, so adding extra questions barely changes the response time.

This is where most of the speed comes from, and TypeSafe says Jev typically responds in around 70 to 500 milliseconds.

Figure 2: The same ticket, answered by an LLM and by a decision model. Original image by author.

Where the probabilities come from

There is no single “decision model architecture.”

TypeSafe hasn’t published how Jev works, but there are roughly three ways you could get these scores:

• Reading an LLM’s own token probabilities. You list the answers as A, B, and C, run the model, and look only at how likely it thinks “A”, “B”, or “C” is as the next token. John Berryman at Arcturus Labs first guessed Jev was an encoder model with a custom output head, then pointed to an X post from Harsha Gundala that replicated something similar with token probabilities. Neither guess is confirmed.
• Adding a scoring head. A smaller encoder model gets an output layer that scores each option directly. The open-source Laya model uses ModernBERT this way.
• Using a separate scorer. A second model, like a reranker or reward model, rates each candidate.

Don’t worry about which one is used where: all three end in the same place, a number for each allowed answer.

What “calibrated” means

This is the part I think matters the most, so let me give you an example.

Imagine a model says “70% likely” on 100 different questions. If the model is well calibrated, about 70 of those answers should turn out to be correct.

Calibration matters because calibrated probabilities let us set safe thresholds in our code, like “approve automatically above 90%, otherwise send it to a human.”

If the probabilities don’t match reality, those thresholds are meaningless.

Vendors train for this using loss functions that punish overconfidence.

TypeSafe calls its method Reinforcement Learning for Calibrated Decisions (RLCD), whereas Cloudflare says it pairs cross-entropy with a Brier loss for its Clef models.

Decision Models Are Not a New Idea

This might come as a surprise to you, but the idea of scoring candidates instead of generating them has been around for years.

In my opinion, decision models are the latest and most convenient version of it rather than a brand new invention. Here are the main families of models it builds on.

Figure 3: Scoring instead of generating goes back to 2019, and decision models are the 2026 version. Original image by author.

Zero-shot classifiers

Back in 2019, models like BART-large-mnli could already sort text into labels they were never trained on.

They did this by scoring statements such as “this text is about billing” for each possible label.

Red Hat’s AI safety team suggests that decision models are arguably just these zero-shot classifiers under a new name.

Rerankers and cross-encoders

If you have built a RAG system, you may have used one of these. You first quickly retrieve a shortlist (for example, 50 documents), then a cross-encoder reads your question and each document together and assigns each pair a relevance score.

Nothing is generated; instead, the documents are simply reordered by their score.

Reward models and verifiers

These are scorers that rate answers that an LLM has already generated.

In 2021, OpenAI researchers showed that adding a verifier to a model gave a boost on grade-school math roughly equal to making the model 30 times bigger, which I find striking.

Two years later, the “Let’s Verify Step by Step” paper showed that scoring each reasoning step works even better than scoring only the final answer.

Best-of-N and generate-then-filter

The simple version is to generate many answers, score each one, and keep the best. DeepMind’s AlphaCode (2022) took this to the extreme by generating up to a million programs per problem, throwing away about 99% using example tests, and then submitting just 10.

In simulated evaluations on past Codeforces contests, it ranked on average in the top 54.3% of human competitors.

One warning: if you push too hard on an imperfect scorer, quality eventually goes down because the generator ends up exploiting the scorer’s mistakes.

The technical term for this is reward overoptimization.

LLM-as-a-judge

This is when a large LLM grades the output of another model.

A 2023 study found that GPT-4 agreed with human judges over 80% of the time but also had biases, like preferring longer answers or whichever answer was shown first.

Decision models are partly being sold as a cheaper, structured replacement for these judges.

Energy-based models and JEPA

An energy-based model gives a low “energy” score to an input and answer that fit well together and a high score to pairs that don’t.

Picking an answer then just means finding the lowest energy, which is scoring candidates again.

Yann LeCun has argued for years that generating token by token is not the path to real intelligence, and that energy-based approaches and JEPA are a better direction.

Key Decision Models in 2026

Now I want to talk about the models that have actually been released. Things are moving very quickly here, so I will keep each one short.

TypeSafe’s Jev

TypeSafe AI is a San Francisco startup founded by Diogo Almeida, a former OpenAI researcher, and it came out of stealth on September 15, 2026 with Jev and a $40 million seed round led by DCVC.

Jev is a hosted API. In production it takes text only; image input is experimental and needs an opt-in header. Pricing is charged on input tokens, and the outputs are free, which makes sense when the output is just a few numbers.

Jev 1.14 followed on October 14, although the jev-latest alias still points to 1.13.0, the version Red Hat tested later in this article.

There was a lot of buzz around the launch. Vercel said around 13% of its paid teams tried Jev through its AI Gateway within the first 24 hours, although it was free on the gateway at the time.

OpenAI’s Decisions API

OpenAI announced its Decisions API at DevDay on September 29, 2026.

It works with user-defined questions that have a fixed set of answers and is built on GPT-6 Luna. It moved to public beta on October 6, and it can also take images, which Jev only supports experimentally.

Cloudflare’s Clef

Cloudflare released Clef and a smaller Clef-flash on October 1, 2026, with open weights and hosting on Workers AI. Both are built on Qwen models and accept Jev-style requests, so code written for Jev can be pointed at Clef with very few changes.

Other models

What really stood out to me while putting this section together is that most of the open and big-tech models here are fine-tuned versions of normal LLMs. Laya is the exception: it is an encoder with a decision head. Jev’s internals are undisclosed.

In my opinion, this supports the idea that the real innovation is the packaging (i.e., the interface and the training for calibration) rather than just a brand new architecture.

Decision Models vs LLMs vs Classifiers

The clearest way to decide when to use one is a comparison:

When to use which

My rough rule of thumb is this:

• Bounded answers and no training data: a decision model is a great fit.
• One task, lots of labeled data and tight speed limits: a fine-tuned classifier is hard to beat.
• Open-ended output: use an LLM.
• A hard problem with a cheap way to check answers: use an LLM plus a verifier.

I do believe that the best systems mix all of these.

I can see decision models acting as fast “gates” around an LLM. A gate decides:

• Which tool to call.
• Whether an input is safe.
• Whether a result needs a human.

That way, the expensive LLM only runs when something actually needs to be written.

What independent tests show

This section matters most, because most of the big numbers you will see online come from the vendors themselves. Even TypeSafe admits that its headline speed and cost gains are on the higher end of what you would see in real use.

The Red Hat guardrail benchmark

In October 2026, Red Hat’s AI safety team compared nine guardrail setups on two tasks: catching prompt injection and catching unsafe content.

DiffusionGemma is a Gemma text-diffusion model served through vLLM.

Here are the most interesting results:

Jev came first on content safety, ahead of DiffusionGemma by 0.67 percentage points (the Qwen judge was third), and the Qwen judge came first on prompt injection.

That split is why Red Hat can still say decision models showed no quality gain over LLM judges overall.

But the bit I find most interesting is that a 200-million-parameter classifier finished only 0.3 percentage points behind on prompt injection, with a median latency under 100 milliseconds compared with about 348 for Jev.

Jev’s latency did include a trip across the Atlantic, since the tests ran from the UK and TypeSafe’s servers are in the US.

That hop only explains the gap to the local CPU models. Qwen, Nemotron, and DiffusionGemma also ran on a US cluster, and Qwen was still faster than Jev.

Even so, Red Hat concluded that decision models did not give faster, cheaper, or better answers than LLM-as-a-judge in their tests, except for Laya, whose small size is a real advantage if you can prompt around its limits.

The “Just Ask Jev” paper

A research paper from university researchers, posted to arXiv in September 2026, tested Jev on 44 alignment benchmarks.

The results were mixed in an interesting way:

• Jev was good at ranking alignment failures above non-failures, with a median AUROC of 0.886 on the 31 benchmarks that support a generic yes/no question. AUROC measures how well a score ranks positive examples above negative ones, where 0.5 is chance and 1 is perfect ranking.
• It cost about $0.30 for work that cost about $18.96 with LLM judges, across the 19 benchmarks that use an API judge.
• It was well calibrated when all benchmarks were pooled together, but not within individual benchmarks.
• Keeping only the half of its answers furthest from 50% raised median accuracy from 0.793 to 0.933.
• The authors note limits: one Jev version, English only, small 2B to 7B target models, and labels mostly generated by existing scorers.

I think the selective-prediction point is really useful in practice.

It shows that a simple design where the decision model handles the easy cases and passes the uncertain ones to an LLM or a person is a good strategy.

Hands-On: Build a Decision Model From a Small LLM

Let’s now build a decision model ourselves.

We are going to turn a small open LLM into a decision model using the token probability trick from earlier and then check whether we can actually trust its probabilities.

First, we need to install the packages using this (all of this can be run on a free Colab notebook):

pip install transformers torch datasets sentence-transformers numpy

Step 1: Turn an LLM into a scorer

We will use Qwen2.5-0.5B-Instruct, which is small enough to run on a CPU.

The idea is simple: we label the options with letters, run the model once, and only look at the probabilities of those letters as the next token.

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "Qwen/Qwen2.5-0.5B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
model.eval()

def option_logits(state, question, options):
    letters = [chr(ord("A") + i) for i in range(len(options))]
    option_text = "\n".join(f"{l}. {o}" for l, o in zip(letters, options))
    messages = [
        {"role": "system", "content": "Answer with a single letter only."},
        {"role": "user", "content": f"{state}\n\nQuestion: {question}\n{option_text}"},
    ]
    prompt = tokenizer.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )
    inputs = tokenizer(prompt, return_tensors="pt")

    with torch.no_grad():
        next_token_logits = model(**inputs).logits[0, -1].float()

    letter_ids = [tokenizer.encode(l, add_special_tokens=False)[0] for l in letters]
    return next_token_logits[letter_ids]

def decide(state, question, options, temperature=1.0):
    logits = option_logits(state, question, options)
    probs = torch.softmax(logits / temperature, dim=-1)
    return {o: round(p.item(), 3) for o, p in zip(options, probs)}

Notice that we never call generate(). We do one forward pass, keeping the scores for “A”, “B”, “C” and so on, and then throw everything else away.

print(decide(
    state="My running shoes arrived in the wrong size. Can I swap them for a size 10?",
    question="Which team should handle this?",
    options=["returns", "shipping", "billing"],
))

print(decide(
    state="The app crashes every time I open the settings page.",
    question="Does this message describe a software bug?",
    options=["yes", "no"],
))

And that really is it. We have just created a working decision model in about 30 lines.

A yes/no question is simply a choice question with two options.

Step 2: Spot the quirks

Now let’s see where this simple version starts to struggle.

Try shuffling the order of the options and run the same question again:

options = ["returns", "shipping", "billing"]
state = "My shoes arrived late and in the wrong size, and I was charged twice."
question = "Which team should handle this?"

print(decide(state, question, options))
print(decide(state, question, list(reversed(options))))

On messy inputs like this one, we will often see the probabilities shift just because the options moved.

This is called order bias, and TypeSafe actually lists a lean toward the first option as one of Jev’s own known weaknesses.

You may also see the model give near-100% answers quite often. Berryman ran a nice test on Jev itself, where he told it a coin lands heads 60% of the time, and the yes/no question returned 0.58, which is close.

But the choice question (“heads or tails?”) returned 0.99 for heads, because the probability collapsed onto the more likely answer.

Step 3: Measure and fix calibration

This is the step I think everyone should run. We will take 400 news items from the AG News dataset (each is a headline plus a short description), which has four topics, and check whether our probabilities match reality.

import numpy as np
from datasets import load_dataset

data = load_dataset("fancyzhx/ag_news", split="test").shuffle(seed=42).select(range(400))
labels = ["World", "Sports", "Business", "Science and technology"]
question = "What is the topic of this news headline?"

all_logits, targets = [], []
for row in data:
    all_logits.append(option_logits(row["text"], question, labels))
    targets.append(row["label"])

all_logits = torch.stack(all_logits)
targets = torch.tensor(targets)

Next, we need a way to measure calibration. Expected calibration error (ECE) groups the predictions into confidence bins and checks how far the average confidence in each bin is from the actual accuracy.

Just to be clear, lower is better:

def expected_calibration_error(probs, targets, n_bins=10):
    confidences, predictions = probs.max(dim=-1)
    correct = (predictions == targets).float()
    bins = np.linspace(0, 1, n_bins + 1)
    ece = 0.0
    for lo, hi in zip(bins[:-1], bins[1:]):
        mask = (confidences > lo) & (confidences <= hi)
        if mask.any():
            gap = abs(correct[mask].mean() - confidences[mask].mean())
            ece += mask.float().mean() * gap
    return float(ece), float(correct.mean())

Now here is the trick.

We split our data in two, use the first half to find a temperature that makes the probabilities more honest, and then test on the second half.

A temperature above 1 softens overconfident predictions, while a temperature below 1 sharpens underconfident ones.

calib_logits, test_logits = all_logits[:200], all_logits[200:]
calib_targets, test_targets = targets[:200], targets[200:]

def nll(temperature):
    probs = torch.softmax(calib_logits / temperature, dim=-1)
    picked = probs[torch.arange(len(calib_targets)), calib_targets]
    return -torch.log(picked + 1e-12).mean().item()

temperatures = [float(t) for t in np.linspace(0.5, 5.0, 46)]
best_t = min(temperatures, key=nll)

before = expected_calibration_error(torch.softmax(test_logits, dim=-1), test_targets)
after = expected_calibration_error(torch.softmax(test_logits / best_t, dim=-1), test_targets)

print(f"Best temperature: {best_t:.2f}")
print(f"Before: ECE={before[0]:.3f}, accuracy={before[1]:.3f}")
print(f"After:  ECE={after[0]:.3f}, accuracy={after[1]:.3f}")

The accuracy doesn’t change, because dividing by a temperature never changes which option ranks first.

Only the confidence changes.

Your exact numbers will depend on your run, but the ECE on the test half should usually drop. With only 200 test items, the estimate is noisy, so don’t read much into small differences.

The lesson here is simple: a model being calibrated on someone else’s data does not mean it is calibrated on yours.

Step 4: Score candidates with a cross-encoder

To show you how common this “score, don’t generate” idea actually is, here is the same thing in a RAG setting.

A cross-encoder scores each passage against a question and we just sort them:

from sentence_transformers import CrossEncoder

reranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2")

query = "How do I get a refund for a double charge?"
passages = [
    "Our app supports dark mode on all devices.",
    "If you were charged twice, contact billing and we will refund the duplicate payment.",
    "Refunds are processed within five business days.",
    "You can change your password from the settings page.",
]

for result in reranker.rank(query, passages):
    print(round(float(result["score"]), 3), passages[result["corpus_id"]])

Again, nothing is generated: the model just gives each candidate a number.

Decision Model Use Cases

These are the use cases that come up most often:

The robotics point is the one I personally like the most.

Berryman suggests a model like Jev could sit between a slow LLM making high-level plans and the kinematic models that move a robot’s joints.

It is worth knowing, though, that TypeSafe’s well-known Doom demo fed Jev text telemetry rather than raw images.

Decision Model Limitations

Decision models are useful, but I don’t want to oversell them.

Here are the main limitations I found to keep in mind:

• It can only pick from your list. If the right answer is missing from your options, the model can’t invent it, which is why TypeSafe recommends adding an “other” option, or a companion yes/no question that checks whether anything fits at all.
• Confidently wrong is still wrong. No off-format answers does not mean no mistakes.
• Calibration doesn’t transfer automatically. Test your thresholds on a small labeled sample of your own data.
• Prompts don’t always transfer. Red Hat found a prompt tuned for Laya raised its accuracy by almost 18 percentage points but made Jev about 3.7 percentage points worse.
• There is no explanation. This makes decisions harder to audit, so log your inputs and keep an LLM for cases that need a reason.
• There are known weak spots. TypeSafe’s own docs list math, dates, long irrelevant context, adversarial text, and a bias toward the first option.

Final Thoughts

AI decision models take a very old idea of scoring candidates instead of generating answers, and make it much easier to use.

You don’t need training data; you get typed answers back, and you can ask many questions at once.

In my opinion, this packaging is the real innovation.

I expect decision models to sit around LLMs as fast, cheap gates that handle the simple decisions, so the expensive model only runs when something actually needs to be written.