Skip to main content

Gemini Decisions API: Testing Google’s New Decision Model on Latency and Cost

Learn how Gemini Decision 1 scores fixed options without generating text, build a banking intent classifier with the Decisions API, and compare it with Gemini 3.5 Flash-Lite on accuracy, latency, calibration, and cost.
· 15 min read

Explore with AI

Most LLM apps are full of small decisions: Which tool should the agent call? Which model should handle this query? Does this message break a policy?

Today, many teams answer these questions by asking a generative model to write the answer as text, then parsing that text to make a decision.

Google's Gemini Decisions API takes a different approach. It uses the Gemini Decision 1 model to score a fixed set of options in a single forward pass and returns a probability for each option, with zero output tokens billed.

In this tutorial, we'll run the same intent classification task on two setups:

  • Gemini Decision 1 through the Decisions API
  • Gemini 3.5 Flash-Lite through the Interactions API, using structured JSON output

We'll compare them on accuracy, latency, token usage, cost, and calibration, and then push both from 1 to 100 questions in a single request.

By the end of this tutorial, you'll know how to:

  • Call the Decisions API with choice questions and read its probabilities and confidence scores

  • Build a generative baseline with the Interactions API and a JSON schema

  • Measure latency, token usage, and cost for both approaches

  • Check whether a model's confidence can be trusted, using a reliability diagram and expected calibration error (ECE)

  • Decide when a decision model fits your app better than a generative one

You can run the complete code in this Google Colab.

Decision Models vs. Generative Models

Before we look at how each one works, let's define the two terms.

  • Generative model: This model produces new content. You give it a prompt, and it writes a response one token at a time. That response can be anything like an answer, a summary, code, or a label. Gemini 3.5 Flash-Lite, like most LLMs we use in chat apps, is a generative model.
  • Decision model: A decision model picks from the options you define. You give it an input and a fixed list of possible answers, and it tells you the probability of each option without writing a single line of text. Gemini Decision 1 is a decision model.

Here's an example. 

A customer writes to a bank: "My card hasn't arrived yet. When will it come?"

If you ask a generative model, "What does this customer want?", it might reply:

“The customer is waiting for a new card and wants to know when it will arrive.”

That's a good answer for a human to read. But if your app needs to route the message to the right team, your code now has to turn that sentence into a label. You can force a label with a JSON schema, but the model still writes it token by token.

However, if you ask a decision model the same thing, you first give it the options, such as card_arrival, lost_or_stolen_card, and exchange_rate. It returns a probability for each one:

card_arrival         0.91
lost_or_stolen_card  0.06
exchange_rate        0.03

Your code can then act on that probability directly by routing to the card team if card_arrival is above a threshold, and sending it to a human if not. In short:

 

Generative model

Decision model

Returns

Text it writes

A probability for each option you define

Possible answers

Open ended

Fixed set

Good for

Answers, summaries, explanations, and extraction

Routing, tool selection, policy checks, and grading

In practice, many apps use a generative model for both jobs. That’s what we’ll test in the next section: whether a decision model does the classification part better than a generative model.

When you use a generative model as a classifier, it writes the answer one token at a time. For a JSON answer like ours, the output looks like this as it is generated:

{
{"intent":
{"intent": "card_arrival",
{"intent": "card_arrival", "confidence": 0.95}

Each token is a separate decoding step, and each output token is billed. If you want a confidence score, the model has to write that number too. That number is the model's own guess about itself, not a probability measured from its internals.

A decision model, however, skips this writing step. You give it the options up front, and it scores all of them against the input in one forward pass. The result is a probability for every option:

card_arrival            0.12
card_delivery_estimate  0.85
lost_or_stolen_card     0.01
...

This changes three things you can measure:

  • Latency: There is no decoding loop, so latency should not grow with the length of the answer.
  • Cost: There are no output tokens to bill. You pay only for input tokens, which include your question text and option descriptions.
  • Confidence: The confidence comes from the probabilities themselves, so it should track how often the model is right.

The trade-off is that you can only ask closed-set questions. A decision model can't explain its answer, extract a value, or write free text.

Demo: Building a Banking Intent Classifier

In this demo, we’ll build a banking intent classifier. We'll use the test split of Banking77, a public dataset of customer messages labeled with 77 intents, and keep 12 intents that are easy to confuse with each other:

  • card_arrival and card_delivery_estimate

  • pending_card_payment, declined_card_payment, and card_payment_fee_charged

  • pending_transfer, transfer_not_received_by_recipient, and balance_not_updated_after_bank_transfer

  • lost_or_stolen_card, card_not_working, exchange_rate, and top_up_failed

Similar intents make the task harder, which makes the confidence scores more interesting to study.

We'll run the classifier with both models in two ways:

  • One message per request: 180 messages (15 per intent), sent to each model one at a time.
  • Many questions per request: N messages placed in one shared input, with one question per message, for N = 1, 5, 10, 25, 50, and 100.

Here are the settings we keep the same for both models:

Setting

Value used

Why 

Decision model

gemini-decision-1

The model under test

Generative model

gemini-3.5-flash-lite

The cost-efficient generative baseline used in the launch post comparison

Flash-Lite temperature

0.0

Removes sampling noise from the baseline

Flash-Lite thinking level

minimal

Keeps the comparison about decoding, not reasoning

Output format

JSON schema with an enum of the 12 labels

Flash-Lite can only return a valid label

Option descriptions

One short sentence per intent

Both models see the same label definitions

Warmup calls

2 per model

Keeps the connection setup out of the measurements

Repeats (many questions per request)

3 per setting

We report the median, not a single run

Note: These are working settings for this demo, not tuned settings for either model.

Step 1: Prerequisites and setup

To follow along, you need:

  • A Gemini API key from Google AI Studio

  • A Colab notebook (or any Python 3 environment)

  • google-genai version 3.2 or later

The Decisions API has a no-cost tier in Google AI Studio for prototyping, so you can try it before paying. I ran this demo in Colab, but feel free to use any Python 3 environment. 

Let’s start by installing the packages:

!pip install -q -U "google-genai>=3.2" pandas matplotlib

Here, google-genai is Google's Python SDK for the Gemini API. It gives us the client.decisions.create and client.interactions.create methods, which send requests to the Decisions API and the Interactions API. We use pandas to load the dataset and organize results, and matplotlib to plot the figures.

Next, we create the client. The Decisions API needs the v1beta API version:

import os, time, json
import numpy as np
import pandas as pd
from google import genai
client = genai.Client(
    api_key=os.environ["GEMINI_API_KEY"],
    http_options={"api_version": "v1beta"},
)

DECISION_MODEL = "gemini-decision-1"
GEN_MODEL = "gemini-3.5-flash-lite"

Here, genai.Client() reads the API key from the GEMINI_API_KEY set under the secrets in Colab or as an environment variable, and http_options points it at the v1beta API, where the Decisions API lives. 

Then we’ll load the Banking77 dataset and define our 12 intents. Each intent gets a one-sentence description that both models will see:

BANKING77_URL = (
    "https://raw.githubusercontent.com/PolyAI-LDN/task-specific-datasets/master/banking_data/test.csv"
)
INTENTS = {
    "card_arrival": "The customer is asking where their new card is or why it has not arrived yet.",
    "card_delivery_estimate": "The customer wants to know how long card delivery usually takes.",
    "lost_or_stolen_card": "The customer's card was lost or stolen.",
    # ... 9 more intents
}
LABELS = list(INTENTS)

df = pd.read_csv(BANKING77_URL)
df = df[df["category"].isin(LABELS)].reset_index(drop=True)

sample = (
    df.groupby("category")
      .sample(n=15, random_state=42)
      .sample(frac=1, random_state=42)
      .reset_index(drop=True)
)

The above code snippet downloads the Banking77 test split as a CSV file and keeps only the rows labeled with one of our 12 intents. It then samples 15 messages per intent with a fixed random seed, so you get the same sample on every run. The final .sample(frac=1) shuffles the rows so the intents are mixed.

This gives us 180 messages, 15 per intent.

Step 2: Classify with the Decisions API

A choice question takes a dictionary that maps each option ID to a description. We pass the message as input and ask one question, keyed intent:

def decide_intent(text):
    start = time.perf_counter()
    result = client.decisions.create(
        model=DECISION_MODEL,
        input=text,
        questions={
            "intent": {
                "type": "choice",
                "instructions": "Which intent best describes this banking customer message?",
                "criteria": INTENTS,
            }
        },
    )
    latency = time.perf_counter() - start
    ans = result.answers["intent"]
    return {
        "pred": ans.choice,
        "confidence": float(ans.confidence),
        "probabilities": dict(ans.probabilities),
        "latency_s": latency,
        "input_tokens": result.usage.total_input_tokens,
        "output_tokens": result.usage.total_output_tokens,
    }

The decide_intent() function sends one message as the input and one choice question with our 12 intents as options. We time the call with time.perf_counter(), then read the top choice, confidence, all option probabilities, and token counts from the response.

Each answer carries three fields:

  • choice: The option with the highest probability

  • probabilities: One probability per option, summing to 1

  • confidence: The estimated probability that the top answer is correct

The usage object reports input tokens, and total_output_tokens is always 0.

Here's the result for the first message in our sample:

The model confused two similar intents, but its confidence of 0.846 shows some uncertainty. The good part is that the correct intent, card_arrival, came second with almost all of the remaining probability (about 0.15). All other options were close to zero.

Step 3: Classify with Flash-Lite

For the generative baseline, we use structured output with an enum of the same 12 labels, so Flash-Lite can only return a valid label. We also ask for a confidence between 0 and 1, which is how most apps get a confidence score from a generative model:

LABEL_LIST = "\n".join(f"- {k}: {v}" for k, v in INTENTS.items())

GEN_SCHEMA = {
    "type": "object",
    "properties": {
        "intent": {"type": "string", "enum": LABELS},
        "confidence": {"type": "number"},
    },
    "required": ["intent", "confidence"],
}

def generate_intent(text):
    prompt = (
        "Which intent best describes this banking customer message?\n\n"
        f"Intents:\n{LABEL_LIST}\n\n"
        f"Message: {text}\n\n"
        "Return the intent and your confidence from 0 to 1."
    )
    start = time.perf_counter()
    interaction = client.interactions.create(
        model=GEN_MODEL,
        input=prompt,
        response_format={
            "type": "text",
            "mime_type": "application/json",
            "schema": GEN_SCHEMA,
        },
        generation_config={"temperature": 0.0, "thinking_level": "minimal"},
        store=False,
    )
    latency = time.perf_counter() - start
    parsed = json.loads(interaction.output_text)
    return {
        "pred": parsed["intent"],
        "confidence": float(parsed["confidence"]),
        "latency_s": latency,
        "input_tokens": interaction.usage.total_input_tokens,
        "output_tokens": interaction.usage.total_output_tokens,
        "thought_tokens": interaction.usage.total_thought_tokens,
    }

The generate_intent() function builds a prompt with the 12 intents and their descriptions. It calls Flash-Lite with response_format set to our JSON schema, so the output must be valid JSON with one of the 12 labels. Temperature is set to 0 with minimal thinking level to keep the answer stable and short. We then parse interaction.output_text with json.loads() and read the token counts from interaction.usage.

I tested the same message as in the previous step with Flash-Lite, and it made the same mistake with higher confidence:

Thinking tokens are 0, which confirms that minimal thinking kept Flash-Lite to writing the answer only. Flash-Lite used fewer input tokens than Decision 1 (277 vs 337) because the Decisions API counts the option descriptions inside the question as input.

Step 4: Classify one message per request

Now, let's send all 180 messages to both models, one request at a time, after two warmup calls each:

rows = []
for i, r in sample.iterrows():
    for model_name, fn in (("decision", decide_intent), ("generative", generate_intent)):
        out = fn(r["text"])
        rows.append({
            "model": model_name, "gold": r["category"], "pred": out["pred"],
            "confidence": out["confidence"], "latency_s": out["latency_s"],
            "input_tokens": out["input_tokens"], "output_tokens": out["output_tokens"],
        })

exp1 = pd.DataFrame(rows)
exp1["correct"] = exp1["pred"] == exp1["gold"]

This loop sends each message to both functions and saves the prediction, confidence, latency, and token counts as one row. We then turn the rows into a DataFrame and add a correct column that marks whether each prediction matches the true label.

Here are the results:

Metric

Gemini Decision 1

Gemini 3.5 Flash-Lite

Accuracy

84.4%

83.9%

Median latency

325 ms

1,068 ms

p90 latency

412 ms

6,483 ms

Slowest request

1.0 s

101.6 s

Requests over 5 s

0 of 180

33 of 180

Mean input tokens

339

279

Mean output tokens

0

25

Here are three things we learned from these metrics:

  • Accuracy: Both models were about equally accurate.
  • Latency: Decision 1 was about 3x faster at the median and much more consistent, while Flash-Lite had a long tail of slow requests.
  • Tokens: Decision 1 used more input tokens because the option descriptions count as input, but it generated no output tokens.

Note: Our timing includes the network round-trip from Colab, so treat that gap as setup-dependent.

Step 5: Check calibration

Accuracy tells us how often a model is right, but calibration tells us whether its confidence means anything or not. A model is well calibrated when its confidence matches its accuracy, i.e., if it says 0.8 on a group of answers, about 80% of them should be correct.

To measure this, we group predictions into 10 confidence bins and compare each bin's average confidence with its accuracy. Expected calibration error (ECE) is the average gap, weighted by how many predictions fall in each bin. Lower is better.

def ece(conf, correct, n_bins=10):
    conf = np.asarray(conf, dtype=float)
    correct = np.asarray(correct, dtype=float)
    edges = np.linspace(0, 1, n_bins + 1)
    ids = np.clip(np.digitize(conf, edges[1:-1]), 0, n_bins - 1)
    total = 0.0
    for b in range(n_bins):
        mask = ids == b
        if mask.sum():
            total += mask.mean() * abs(conf[mask].mean() - correct[mask].mean())
    return total

The ece() function places each prediction into one of 10 bins by its confidence (0 to 0.1, 0.1 to 0.2, and so on). For each bin, it takes the gap between average confidence and accuracy, weights it by the share of predictions in that bin, and adds up the results.

Here's how the two models compare:

Metric

Gemini Decision 1

Gemini 3.5 Flash-Lite

Accuracy

84.4%

83.9%

Mean confidence

0.867

0.957

ECE

0.050

0.124

Mean confidence when correct

0.901

0.966

Mean confidence when wrong

0.686

0.912

Figure: Reliability diagrams for both models

Decision 1's confidence is close to its accuracy, with an ECE of 0.050. It is also clearly lower when the model is wrong (0.686) than when it is right (0.901).

Flash-Lite is overconfident. It reports 0.957 on average while being right 83.9% of the time. It used only 9 distinct confidence values across 180 answers, and 158 of them were 0.95 or higher. Even its wrong answers averaged 0.912. The number looks like a probability, but it is the model's own estimate written as text.

Calibration matters when you act on the score. A common pattern is to accept answers above a confidence threshold and send the rest to a fallback model or a human:

Threshold

Decision 1: share kept

Decision 1: accuracy on kept

Flash-Lite: share kept

Flash-Lite: accuracy on kept

0.50

94.4%

85.9%

99.4%

84.4%

0.70

78.3%

92.2%

98.9%

84.3%

0.80

74.4%

93.3%

98.9%

84.3%

0.90

63.3%

95.6%

88.9%

87.5%

0.95

53.9%

96.9%

87.8%

87.3%

With Decision 1, a higher threshold works as expected. At 0.9, it keeps 63% of messages at 95.6% accuracy. With Flash-Lite, the threshold barely helps, because most of its wrong answers also have high confidence. The official docs recommend choosing a threshold from labeled examples of your own traffic.

Step 6: Classify 100 messages in one request

The Decisions API can answer up to 1,000 questions about one shared input in a single request. To test this, we put N customer messages into one input and ask one question per message.

For Decision 1, that means N choice questions keyed m1 to mN:

def build_batch(n):
    msgs = pool.iloc[:n]
    shared = "Banking customer messages:\n" + "".join(
        f"Message {i + 1}: {t}\n" for i, t in enumerate(msgs["text"])
    )
    return shared, list(msgs["category"])

def decide_batch(n):
    shared, gold = build_batch(n)
    questions = {
        f"m{i + 1}": {
            "type": "choice",
            "instructions": f"Which intent best describes Message {i + 1}?",
            "criteria": INTENTS,
        }
        for i in range(n)
    }
    result = client.decisions.create(
        model=DECISION_MODEL, input=shared, questions=questions
    )
    preds = [result.answers[f"m{i + 1}"].choice for i in range(n)]
    return preds, gold, result.usage

Here, the build_batch() function numbers the first N messages and joins them into one shared input. Then decide_batch() creates one choice question per message, keyed m1 to mN, sends them all in a single request, and reads back one answer per key.

For Flash-Lite, we send the same shared input with a JSON schema that has N fields, each limited to 12 labels. Each N uses the same messages for both models, and each setting runs three times.

Questions per request

Decision 1: median latency

Flash-Lite: median latency

Decision 1: input tokens

Flash-Lite: input / output tokens

1

318 ms

888 ms

344

291 / 14

5

353 ms

858 ms

1,554

369 / 64

10

398 ms

1,063 ms

3,056

456 / 129

25

546 ms

2,753 ms

7,570

710 / 350

50

592 ms

Failed

15,100

Failed

100

59,095 ms

Failed

30,309

Failed

Up to 25 questions, Decision 1's median latency stayed under 550 ms. Flash-Lite's median grew with N, from under 1s at 1 to 5 questions to 2.75 s at 25, because it has to write every answer as output tokens. Even at 50 questions, Decision 1's median was 592 ms, though one of the three runs took 70 s.

Here are two key points to ponder:

  • Flash-Lite failed at 50 and 100 questions: The "Failed" rows in the table above are all the same 400 error. It shows that the schema "has too many states for serving." A schema with 50 or more fields, each limited to 12 labels, is more than the API accepts.
  • Decision 1 slowed down at 100 questions: Each question repeats all 12 option descriptions, about 300 tokens per question, so the request reached 30,309 tokens, while Google's flat latency claim covers contexts up to 16K tokens.

Step 7: Trim the requests

Now, an obvious question arises: can we fix the slowdown and the failures? For Decision 1, we use empty option descriptions (""). For Flash-Lite, we ask for one list of N labels instead of N separate fields:

# Decision 1: option IDs only, no descriptions
"criteria": {k: "" for k in LABELS}

# Flash-Lite: one list field instead of N fields
schema = {
    "type": "object",
    "properties": {"intents": {
        "type": "array",
        "items": {"type": "string", "enum": LABELS},
        "minItems": n, "maxItems": n,
    }},
    "required": ["intents"],
}

Here's how Decision 1 did with empty descriptions:

Questions per request

Median latency

Accuracy

Input tokens

25

357 ms

84.0%

3,995

50

649 ms

78.0%

7,950

100

1,603 ms

83.0%

16,009

Dropping the descriptions roughly halved the input tokens. At 100 questions, the request came down to about 16K tokens, and median latency fell from 59 s to 1.6 s. Accuracy dropped a little at 25 and 50 questions, so short descriptions are a better middle ground than none, especially for similar intents.

The list schema didn't help the Flash-Lite model. It worked at 25 questions, but still failed at 50 and 100.

Step 8: Compare cost

Finally, we compute the cost from the token counts each API returns, using paid-tier prices:

Model

Input price per 1M tokens

Output price per 1M tokens

Gemini Decision 1

$0.10

$0

Gemini 3.5 Flash-Lite

$0.30

$2.50 (including thinking tokens)

def cost_per_request(model, input_tokens, output_tokens, thought_tokens):
    if model == "decision":
        return input_tokens * 0.10 / 1e6
    return (input_tokens * 0.30 + (output_tokens + thought_tokens) * 2.50) / 1e6

The cost_per_request() function multiplies each token count by its price per 1M tokens. For Decision 1, only input tokens count. For Flash-Lite, thinking tokens are billed at the output price, so we add them to the output tokens.

Here's the cost per 1,000 requests for each setup:

Setup

Gemini Decision 1

Gemini 3.5 Flash-Lite

Flash-Lite / Decision 1

1 message per request

$0.034

$0.147

4.3x

5 questions per request

$0.155

$0.271

1.7x

10 questions per request

$0.306

$0.459

1.5x

25 questions per request

$0.757

$1.088

1.4x

25 questions, trimmed setups (Step 7)

$0.400

$0.831

2.1x

50 questions per request

$1.510

Failed

n/a

100 questions, empty descriptions

$1.601

Failed

n/a

On single messages, Decision 1 was 4.3x cheaper, but the gap shrinks as you batch more questions into one request. The shared input is billed once in both APIs. But each Decision 1 question carries its own instruction and all 12 option descriptions, while Flash-Lite sees the label list only once and pays for a short output per answer. At 25 questions, Decision 1 used 7,570 input tokens against Flash-Lite's 710 input and 350 output tokens.

Note: The Decisions API doesn't support context caching or the Batch API. If you send the same input in several requests, it is billed in full each time. So when you have many questions about the same input, ask them in one request rather than spreading them across calls.

When to Use Each Approach

Based on the above runs, a decision model is the better fit when:

  • The answer comes from a fixed set of options, like intents, routes, tools, or policy labels
  • The decision sits in the request path, where every user waits for it
  • You want to act on confidence, like, by sending unclear cases to a fallback or a human
  • You need to ask many independent questions about the same input in one request

A generative model is still the better fit when:

  • You need an explanation, an extracted value, or any free text
  • The set of possible answers is open or changes per request
  • You already made a generative call on that input and can add the classification to it

Use the Decisions API when you need

Use the Interactions API when you need

A yes/no probability, one option from a fixed set, or a rating on a fixed scale

Generated text, summaries, or explanations

Calibrated probabilities that you can threshold

Extracted values or objects that follow your own JSON schema

Many independent checks over the same input in one call

Tool calls with arguments

Low latency and zero output token cost

Multi-step reasoning

Final Thoughts

In this tutorial, we built a banking intent classifier with Gemini Decision 1 and Gemini 3.5 Flash-Lite, then calculated accuracy, latency, calibration, and cost to compare the two models. Here are four takeaways from the tutorial:

  • Accuracy was a tie: 84.4% for Decision 1 and 83.9% for Flash-Lite, with the same verdict on 163 of 180 messages.
  • Decision 1 was faster and more predictable: Its median latency was 325 ms against 1,068 ms, and its slowest request took 1.0 s against 101.6 s.
  • Decision 1's confidence was usable: Its ECE was 0.050 against 0.124, and raising the threshold to 0.9 lifted accuracy on kept answers from 84.4% to 95.6%. Flash-Lite's self-reported confidence barely separated right answers from wrong ones.
  • Cost and scaling depend on how you write the questions: Decision 1 was 4.3x cheaper on single messages, but every question pays for its option descriptions. So, keep the total input under about 16K tokens and keep option descriptions short.

These results come from one task, one dataset, and a small number of runs from a Colab notebook. You can treat them as observations from a demo rather than general performance claims. 

Aashi Dutt

Author

Aashi Dutt

I am a Google Developers Expert in ML(Gen AI), a Kaggle 3x Expert, and a Women Techmakers Ambassador with 3+ years of experience in tech. I co-founded a health-tech startup in 2020 and am pursuing a master's in computer science at Georgia Tech, specializing in machine learning.

FAQs

What is the difference between the Decisions API and a generative API?

A generative API writes its answer token by token and bills each output token. The Decisions API scores a fixed set of options in one forward pass and returns a probability for each one. It generates no output tokens, so you pay only for input.

Is Gemini Decision 1 more accurate than Gemini 3.5 Flash-Lite?

Not in our test. On 180 banking messages, accuracy was 84.4% for Decision 1 and 83.9% for Flash-Lite. The differences we measured were in latency, calibration, and cost.

Why does Decision 1 use more input tokens than Flash-Lite?

Each question's instruction and option descriptions count as input. With 12 described options, that added about 300 tokens per question in our test. Short or empty option descriptions reduce this.

Does latency stay flat as I add more questions?

In our runs, the median stayed under 650 ms up to 50 questions. At 100 questions with full option descriptions, the request reached about 30K tokens and took 48 to 59 s. With empty descriptions, the same 100 questions fit in about 16K tokens and took 1.6 s.

What question types does the Decisions API support?

Decisions API supports three question types, including noul for yes/no checks, choice for picking one option from 2 to 255, and score for grading on an ordered rubric. You can mix all three in one request.

Topics

Related