Gemini Decisions API: Testing Google’s New Decision Model on Latency and Cost
Learn how Gemini Decision 1 scores fixed options without generating text, build a banking intent classifier with the Decisions API, and compare it with Gemini 3.5 Flash-Lite on accuracy, latency, calibration, and cost.
Most LLM apps are full of small decisions: Which tool should the agent call? Which model should handle this query? Does this message break a policy?
Today, many teams answer these questions by asking a generative model to write the answer as text, then parsing that text to make a decision.
Google's Gemini Decisions API takes a different approach. It uses the Gemini Decision 1 model to score a fixed set of options in a single forward pass and returns a probability for each option, with zero output tokens billed.
In this tutorial, we'll run the same intent classification task on two setups:
Gemini Decision 1 through the Decisions API
Gemini 3.5 Flash-Lite through the Interactions API, using structured JSON output
We'll compare them on accuracy, latency, token usage, cost, and calibration, and then push both from 1 to 100 questions in a single request.
By the end of this tutorial, you'll know how to:
Call the Decisions API with choice questions and read its probabilities and confidence scores
Build a generative baseline with the Interactions API and a JSON schema
Measure latency, token usage, and cost for both approaches
Check whether a model's confidence can be trusted, using a reliability diagram and expected calibration error (ECE)
Decide when a decision model fits your app better than a generative one
You can run the complete code in this Google Colab.
Decision Models vs. Generative Models
Before we look at how each one works, let's define the two terms.
Generative model: This model produces new content. You give it a prompt, and it writes a response one token at a time. That response can be anything like an answer, a summary, code, or a label. Gemini 3.5 Flash-Lite, like most LLMs we use in chat apps, is a generative model.
Decision model: A decision model picks from the options you define. You give it an input and a fixed list of possible answers, and it tells you the probability of each option without writing a single line of text. Gemini Decision 1 is a decision model.
Here's an example.
A customer writes to a bank: "My card hasn't arrived yet. When will it come?"
If you ask a generative model, "What does this customer want?", it might reply:
“The customer is waiting for a new card and wants to know when it will arrive.”
That's a good answer for a human to read. But if your app needs to route the message to the right team, your code now has to turn that sentence into a label. You can force a label with a JSON schema, but the model still writes it token by token.
However, if you ask a decision model the same thing, you first give it the options, such as card_arrival, lost_or_stolen_card, and exchange_rate. It returns a probability for each one:
card_arrival 0.91
lost_or_stolen_card 0.06
exchange_rate 0.03Obsługiwane przez
Your code can then act on that probability directly by routing to the card team if card_arrival is above a threshold, and sending it to a human if not. In short:
Generative model
Decision model
Returns
Text it writes
A probability for each option you define
Possible answers
Open ended
Fixed set
Good for
Answers, summaries, explanations, and extraction
Routing, tool selection, policy checks, and grading
In practice, many apps use a generative model for both jobs. That’s what we’ll test in the next section: whether a decision model does the classification part better than a generative model.
When you use a generative model as a classifier, it writes the answer one token at a time. For a JSON answer like ours, the output looks like this as it is generated:
{
{"intent":
{"intent": "card_arrival",
{"intent": "card_arrival", "confidence": 0.95}Obsługiwane przez
Each token is a separate decoding step, and each output token is billed. If you want a confidence score, the model has to write that number too. That number is the model's own guess about itself, not a probability measured from its internals.
A decision model, however, skips this writing step. You give it the options up front, and it scores all of them against the input in one forward pass. The result is a probability for every option:
card_arrival 0.12
card_delivery_estimate 0.85
lost_or_stolen_card 0.01
...Obsługiwane przez
This changes three things you can measure:
Latency: There is no decoding loop, so latency should not grow with the length of the answer.
Cost: There are no output tokens to bill. You pay only for input tokens, which include your question text and option descriptions.
Confidence: The confidence comes from the probabilities themselves, so it should track how often the model is right.
The trade-off is that you can only ask closed-set questions. A decision model can't explain its answer, extract a value, or write free text.
Demo: Building a Banking Intent Classifier
In this demo, we’ll build a banking intent classifier. We'll use the test split of Banking77, a public dataset of customer messages labeled with 77 intents, and keep 12 intents that are easy to confuse with each other:
card_arrival and card_delivery_estimate
pending_card_payment, declined_card_payment, and card_payment_fee_charged
pending_transfer, transfer_not_received_by_recipient, and balance_not_updated_after_bank_transfer
lost_or_stolen_card, card_not_working, exchange_rate, and top_up_failed
Similar intents make the task harder, which makes the confidence scores more interesting to study.
We'll run the classifier with both models in two ways:
One message per request: 180 messages (15 per intent), sent to each model one at a time.
Many questions per request: N messages placed in one shared input, with one question per message, for N = 1, 5, 10, 25, 50, and 100.
Here are the settings we keep the same for both models:
Setting
Value used
Why
Decision model
gemini-decision-1
The model under test
Generative model
gemini-3.5-flash-lite
The cost-efficient generative baseline used in the launch post comparison
Flash-Lite temperature
0.0
Removes sampling noise from the baseline
Flash-Lite thinking level
minimal
Keeps the comparison about decoding, not reasoning
Output format
JSON schema with an enum of the 12 labels
Flash-Lite can only return a valid label
Option descriptions
One short sentence per intent
Both models see the same label definitions
Warmup calls
2 per model
Keeps the connection setup out of the measurements
Repeats (many questions per request)
3 per setting
We report the median, not a single run
Note: These are working settings for this demo, not tuned settings for either model.
The Decisions API has a no-cost tier in Google AI Studio for prototyping, so you can try it before paying. I ran this demo in Colab, but feel free to use any Python 3 environment.
Let’s start by installing the packages:
!pip install-q-U"google-genai>=3.2" pandas matplotlibObsługiwane przez
Here, google-genai is Google's Python SDK for the Gemini API. It gives us the client.decisions.create and client.interactions.create methods, which send requests to the Decisions API and the Interactions API. We use pandas to load the dataset and organize results, and matplotlib to plot the figures.
Next, we create the client. The Decisions API needs the v1beta API version:
import os, time, json
import numpy as np
import pandas as pd
from google import genai
client = genai.Client(
api_key=os.environ["GEMINI_API_KEY"],
http_options={"api_version":"v1beta"},)
DECISION_MODEL ="gemini-decision-1"
GEN_MODEL ="gemini-3.5-flash-lite"Obsługiwane przez
Here, genai.Client() reads the API key from the GEMINI_API_KEY set under the secrets in Colab or as an environment variable, and http_options points it at the v1beta API, where the Decisions API lives.
Then we’ll load the Banking77 dataset and define our 12 intents. Each intent gets a one-sentence description that both models will see:
BANKING77_URL =("https://raw.githubusercontent.com/PolyAI-LDN/task-specific-datasets/master/banking_data/test.csv")
INTENTS ={"card_arrival":"The customer is asking where their new card is or why it has not arrived yet.","card_delivery_estimate":"The customer wants to know how long card delivery usually takes.","lost_or_stolen_card":"The customer's card was lost or stolen.",# ... 9 more intents}
LABELS =list(INTENTS)
df = pd.read_csv(BANKING77_URL)
df = df[df["category"].isin(LABELS)].reset_index(drop=True)
sample =(
df.groupby("category").sample(n=15, random_state=42).sample(frac=1, random_state=42).reset_index(drop=True))Obsługiwane przez
The above code snippet downloads the Banking77 test split as a CSV file and keeps only the rows labeled with one of our 12 intents. It then samples 15 messages per intent with a fixed random seed, so you get the same sample on every run. The final .sample(frac=1) shuffles the rows so the intents are mixed.
This gives us 180 messages, 15 per intent.
Step 2: Classify with the Decisions API
A choice question takes a dictionary that maps each option ID to a description. We pass the message as input and ask one question, keyed intent:
defdecide_intent(text):
start = time.perf_counter()
result = client.decisions.create(
model=DECISION_MODEL,input=text,
questions={"intent":{"type":"choice","instructions":"Which intent best describes this banking customer message?","criteria": INTENTS,}},)
latency = time.perf_counter()- start
ans = result.answers["intent"]return{"pred": ans.choice,"confidence":float(ans.confidence),"probabilities":dict(ans.probabilities),"latency_s": latency,"input_tokens": result.usage.total_input_tokens,"output_tokens": result.usage.total_output_tokens,}Obsługiwane przez
The decide_intent() function sends one message as the input and one choice question with our 12 intents as options. We time the call with time.perf_counter(), then read the top choice, confidence, all option probabilities, and token counts from the response.
Each answer carries three fields:
choice: The option with the highest probability
probabilities: One probability per option, summing to 1
confidence: The estimated probability that the top answer is correct
The usage object reports input tokens, and total_output_tokens is always 0.
Here's the result for the first message in our sample:
The model confused two similar intents, but its confidence of 0.846 shows some uncertainty. The good part is that the correct intent, card_arrival, came second with almost all of the remaining probability (about 0.15). All other options were close to zero.
Step 3: Classify with Flash-Lite
For the generative baseline, we use structured output with an enum of the same 12 labels, so Flash-Lite can only return a valid label. We also ask for a confidence between 0 and 1, which is how most apps get a confidence score from a generative model:
LABEL_LIST ="\n".join(f"- {k}: {v}"for k, v in INTENTS.items())
GEN_SCHEMA ={"type":"object","properties":{"intent":{"type":"string","enum": LABELS},"confidence":{"type":"number"},},"required":["intent","confidence"],}defgenerate_intent(text):
prompt =("Which intent best describes this banking customer message?\n\n"f"Intents:\n{LABEL_LIST}\n\n"f"Message: {text}\n\n""Return the intent and your confidence from 0 to 1.")
start = time.perf_counter()
interaction = client.interactions.create(
model=GEN_MODEL,input=prompt,
response_format={"type":"text","mime_type":"application/json","schema": GEN_SCHEMA,},
generation_config={"temperature":0.0,"thinking_level":"minimal"},
store=False,)
latency = time.perf_counter()- start
parsed = json.loads(interaction.output_text)return{"pred": parsed["intent"],"confidence":float(parsed["confidence"]),"latency_s": latency,"input_tokens": interaction.usage.total_input_tokens,"output_tokens": interaction.usage.total_output_tokens,"thought_tokens": interaction.usage.total_thought_tokens,}Obsługiwane przez
The generate_intent() function builds a prompt with the 12 intents and their descriptions. It calls Flash-Lite with response_format set to our JSON schema, so the output must be valid JSON with one of the 12 labels. Temperature is set to 0 with minimal thinking level to keep the answer stable and short. We then parse interaction.output_text with json.loads() and read the token counts from interaction.usage.
I tested the same message as in the previous step with Flash-Lite, and it made the same mistake with higher confidence:
Thinking tokens are 0, which confirms that minimal thinking kept Flash-Lite to writing the answer only. Flash-Lite used fewer input tokens than Decision 1 (277 vs 337) because the Decisions API counts the option descriptions inside the question as input.
Step 4: Classify one message per request
Now, let's send all 180 messages to both models, one request at a time, after two warmup calls each:
rows =[]for i, r in sample.iterrows():for model_name, fn in(("decision", decide_intent),("generative", generate_intent)):
out = fn(r["text"])
rows.append({"model": model_name,"gold": r["category"],"pred": out["pred"],"confidence": out["confidence"],"latency_s": out["latency_s"],"input_tokens": out["input_tokens"],"output_tokens": out["output_tokens"],})
exp1 = pd.DataFrame(rows)
exp1["correct"]= exp1["pred"]== exp1["gold"]Obsługiwane przez
This loop sends each message to both functions and saves the prediction, confidence, latency, and token counts as one row. We then turn the rows into a DataFrame and add a correct column that marks whether each prediction matches the true label.
Here are the results:
Metric
Gemini Decision 1
Gemini 3.5 Flash-Lite
Accuracy
84.4%
83.9%
Median latency
325 ms
1,068 ms
p90 latency
412 ms
6,483 ms
Slowest request
1.0 s
101.6 s
Requests over 5 s
0 of 180
33 of 180
Mean input tokens
339
279
Mean output tokens
0
25
Here are three things we learned from these metrics:
Accuracy: Both models were about equally accurate.
Latency: Decision 1 was about 3x faster at the median and much more consistent, while Flash-Lite had a long tail of slow requests.
Tokens: Decision 1 used more input tokens because the option descriptions count as input, but it generated no output tokens.
Note: Our timing includes the network round-trip from Colab, so treat that gap as setup-dependent.
Step 5: Check calibration
Accuracy tells us how often a model is right, but calibration tells us whether its confidence means anything or not. A model is well calibrated when its confidence matches its accuracy, i.e., if it says 0.8 on a group of answers, about 80% of them should be correct.
To measure this, we group predictions into 10 confidence bins and compare each bin's average confidence with its accuracy. Expected calibration error (ECE) is the average gap, weighted by how many predictions fall in each bin. Lower is better.
defece(conf, correct, n_bins=10):
conf = np.asarray(conf, dtype=float)
correct = np.asarray(correct, dtype=float)
edges = np.linspace(0,1, n_bins +1)
ids = np.clip(np.digitize(conf, edges[1:-1]),0, n_bins -1)
total =0.0for b inrange(n_bins):
mask = ids == b
if mask.sum():
total += mask.mean()*abs(conf[mask].mean()- correct[mask].mean())return totalObsługiwane przez
The ece() function places each prediction into one of 10 bins by its confidence (0 to 0.1, 0.1 to 0.2, and so on). For each bin, it takes the gap between average confidence and accuracy, weights it by the share of predictions in that bin, and adds up the results.
Here's how the two models compare:
Metric
Gemini Decision 1
Gemini 3.5 Flash-Lite
Accuracy
84.4%
83.9%
Mean confidence
0.867
0.957
ECE
0.050
0.124
Mean confidence when correct
0.901
0.966
Mean confidence when wrong
0.686
0.912
Figure: Reliability diagrams for both models
Decision 1's confidence is close to its accuracy, with an ECE of 0.050. It is also clearly lower when the model is wrong (0.686) than when it is right (0.901).
Flash-Lite is overconfident. It reports 0.957 on average while being right 83.9% of the time. It used only 9 distinct confidence values across 180 answers, and 158 of them were 0.95 or higher. Even its wrong answers averaged 0.912. The number looks like a probability, but it is the model's own estimate written as text.
Calibration matters when you act on the score. A common pattern is to accept answers above a confidence threshold and send the rest to a fallback model or a human:
Threshold
Decision 1: share kept
Decision 1: accuracy on kept
Flash-Lite: share kept
Flash-Lite: accuracy on kept
0.50
94.4%
85.9%
99.4%
84.4%
0.70
78.3%
92.2%
98.9%
84.3%
0.80
74.4%
93.3%
98.9%
84.3%
0.90
63.3%
95.6%
88.9%
87.5%
0.95
53.9%
96.9%
87.8%
87.3%
With Decision 1, a higher threshold works as expected. At 0.9, it keeps 63% of messages at 95.6% accuracy. With Flash-Lite, the threshold barely helps, because most of its wrong answers also have high confidence. The official docs recommend choosing a threshold from labeled examples of your own traffic.
Step 6: Classify 100 messages in one request
The Decisions API can answer up to 1,000 questions about one shared input in a single request. To test this, we put N customer messages into one input and ask one question per message.
For Decision 1, that means N choice questions keyed m1 to mN:
defbuild_batch(n):
msgs = pool.iloc[:n]
shared ="Banking customer messages:\n"+"".join(f"Message {i +1}: {t}\n"for i, t inenumerate(msgs["text"]))return shared,list(msgs["category"])defdecide_batch(n):
shared, gold = build_batch(n)
questions ={f"m{i +1}":{"type":"choice","instructions":f"Which intent best describes Message {i +1}?","criteria": INTENTS,}for i inrange(n)}
result = client.decisions.create(
model=DECISION_MODEL,input=shared, questions=questions
)
preds =[result.answers[f"m{i +1}"].choice for i inrange(n)]return preds, gold, result.usageObsługiwane przez
Here, the build_batch() function numbers the first N messages and joins them into one shared input. Then decide_batch() creates one choice question per message, keyed m1 to mN, sends them all in a single request, and reads back one answer per key.
For Flash-Lite, we send the same shared input with a JSON schema that has N fields, each limited to 12 labels. Each N uses the same messages for both models, and each setting runs three times.
Questions per request
Decision 1: median latency
Flash-Lite: median latency
Decision 1: input tokens
Flash-Lite: input / output tokens
1
318 ms
888 ms
344
291 / 14
5
353 ms
858 ms
1,554
369 / 64
10
398 ms
1,063 ms
3,056
456 / 129
25
546 ms
2,753 ms
7,570
710 / 350
50
592 ms
Failed
15,100
Failed
100
59,095 ms
Failed
30,309
Failed
Up to 25 questions, Decision 1's median latency stayed under 550 ms. Flash-Lite's median grew with N, from under 1s at 1 to 5 questions to 2.75 s at 25, because it has to write every answer as output tokens. Even at 50 questions, Decision 1's median was 592 ms, though one of the three runs took 70 s.
Here are two key points to ponder:
Flash-Lite failed at 50 and 100 questions: The "Failed" rows in the table above are all the same 400 error. It shows that the schema "has too many states for serving." A schema with 50 or more fields, each limited to 12 labels, is more than the API accepts.
Decision 1 slowed down at 100 questions: Each question repeats all 12 option descriptions, about 300 tokens per question, so the request reached 30,309 tokens, while Google's flat latency claim covers contexts up to 16K tokens.
Step 7: Trim the requests
Now, an obvious question arises: can we fix the slowdown and the failures? For Decision 1, we use empty option descriptions (""). For Flash-Lite, we ask for one list of N labels instead of N separate fields:
# Decision 1: option IDs only, no descriptions"criteria":{k:""for k in LABELS}# Flash-Lite: one list field instead of N fields
schema ={"type":"object","properties":{"intents":{"type":"array","items":{"type":"string","enum": LABELS},"minItems": n,"maxItems": n,}},"required":["intents"],}Obsługiwane przez
Here's how Decision 1 did with empty descriptions:
Questions per request
Median latency
Accuracy
Input tokens
25
357 ms
84.0%
3,995
50
649 ms
78.0%
7,950
100
1,603 ms
83.0%
16,009
Dropping the descriptions roughly halved the input tokens. At 100 questions, the request came down to about 16K tokens, and median latency fell from 59 s to 1.6 s. Accuracy dropped a little at 25 and 50 questions, so short descriptions are a better middle ground than none, especially for similar intents.
The list schema didn't help the Flash-Lite model. It worked at 25 questions, but still failed at 50 and 100.
Step 8: Compare cost
Finally, we compute the cost from the token counts each API returns, using paid-tier prices:
Model
Input price per 1M tokens
Output price per 1M tokens
Gemini Decision 1
$0.10
$0
Gemini 3.5 Flash-Lite
$0.30
$2.50 (including thinking tokens)
defcost_per_request(model, input_tokens, output_tokens, thought_tokens):if model =="decision":return input_tokens *0.10/1e6return(input_tokens *0.30+(output_tokens + thought_tokens)*2.50)/1e6Obsługiwane przez
The cost_per_request() function multiplies each token count by its price per 1M tokens. For Decision 1, only input tokens count. For Flash-Lite, thinking tokens are billed at the output price, so we add them to the output tokens.
Here's the cost per 1,000 requests for each setup:
Setup
Gemini Decision 1
Gemini 3.5 Flash-Lite
Flash-Lite / Decision 1
1 message per request
$0.034
$0.147
4.3x
5 questions per request
$0.155
$0.271
1.7x
10 questions per request
$0.306
$0.459
1.5x
25 questions per request
$0.757
$1.088
1.4x
25 questions, trimmed setups (Step 7)
$0.400
$0.831
2.1x
50 questions per request
$1.510
Failed
n/a
100 questions, empty descriptions
$1.601
Failed
n/a
On single messages, Decision 1 was 4.3x cheaper, but the gap shrinks as you batch more questions into one request. The shared input is billed once in both APIs. But each Decision 1 question carries its own instruction and all 12 option descriptions, while Flash-Lite sees the label list only once and pays for a short output per answer. At 25 questions, Decision 1 used 7,570 input tokens against Flash-Lite's 710 input and 350 output tokens.
Note: The Decisions API doesn't support context caching or the Batch API. If you send the same input in several requests, it is billed in full each time. So when you have many questions about the same input, ask them in one request rather than spreading them across calls.
When to Use Each Approach
Based on the above runs, a decision model is the better fit when:
The answer comes from a fixed set of options, like intents, routes, tools, or policy labels
The decision sits in the request path, where every user waits for it
You want to act on confidence, like, by sending unclear cases to a fallback or a human
You need to ask many independent questions about the same input in one request
A generative model is still the better fit when:
You need an explanation, an extracted value, or any free text
The set of possible answers is open or changes per request
You already made a generative call on that input and can add the classification to it
Use the Decisions API when you need
Use the Interactions API when you need
A yes/no probability, one option from a fixed set, or a rating on a fixed scale
Generated text, summaries, or explanations
Calibrated probabilities that you can threshold
Extracted values or objects that follow your own JSON schema
Many independent checks over the same input in one call
Tool calls with arguments
Low latency and zero output token cost
Multi-step reasoning
Final Thoughts
In this tutorial, we built a banking intent classifier with Gemini Decision 1 and Gemini 3.5 Flash-Lite, then calculated accuracy, latency, calibration, and cost to compare the two models. Here are four takeaways from the tutorial:
Accuracy was a tie: 84.4% for Decision 1 and 83.9% for Flash-Lite, with the same verdict on 163 of 180 messages.
Decision 1 was faster and more predictable: Its median latency was 325 ms against 1,068 ms, and its slowest request took 1.0 s against 101.6 s.
Decision 1's confidence was usable: Its ECE was 0.050 against 0.124, and raising the threshold to 0.9 lifted accuracy on kept answers from 84.4% to 95.6%. Flash-Lite's self-reported confidence barely separated right answers from wrong ones.
Cost and scaling depend on how you write the questions: Decision 1 was 4.3x cheaper on single messages, but every question pays for its option descriptions. So, keep the total input under about 16K tokens and keep option descriptions short.
These results come from one task, one dataset, and a small number of runs from a Colab notebook. You can treat them as observations from a demo rather than general performance claims.
FAQs
What is the difference between the Decisions API and a generative API?⌄
A generative API writes its answer token by token and bills each output token. The Decisions API scores a fixed set of options in one forward pass and returns a probability for each one. It generates no output tokens, so you pay only for input.
Is Gemini Decision 1 more accurate than Gemini 3.5 Flash-Lite?⌄
Not in our test. On 180 banking messages, accuracy was 84.4% for Decision 1 and 83.9% for Flash-Lite. The differences we measured were in latency, calibration, and cost.
Why does Decision 1 use more input tokens than Flash-Lite?⌄
Each question's instruction and option descriptions count as input. With 12 described options, that added about 300 tokens per question in our test. Short or empty option descriptions reduce this.
Does latency stay flat as I add more questions?⌄
In our runs, the median stayed under 650 ms up to 50 questions. At 100 questions with full option descriptions, the request reached about 30K tokens and took 48 to 59 s. With empty descriptions, the same 100 questions fit in about 16K tokens and took 1.6 s.
What question types does the Decisions API support?⌄
Decisions API supports three question types, including noul for yes/no checks, choice for picking one option from 2 to 255, and score for grading on an ordered rubric. You can mix all three in one request.
Analyze data smarter with Gemini in Google Sheets. Use AI-powered insights, formula suggestions, and automation to simplify spreadsheets and boost productivity.
Most LLM apps are full of small decisions: Which tool should the agent call? Which model should handle this query? Does this message break a policy?
Today, many teams answer these questions by asking a generative model to write the answer as text, then parsing that text to make a decision.
Google's Gemini Decisions API takes a different approach. It uses the Gemini Decision 1 model to score a fixed set of options in a single forward pass and returns a probability for each option, with zero output tokens billed.
In this tutorial, we'll run the same intent classification task on two setups:
We'll compare them on accuracy, latency, token usage, cost, and calibration, and then push both from 1 to 100 questions in a single request.
By the end of this tutorial, you'll know how to:
Call the Decisions API with
choicequestions and read its probabilities and confidence scoresBuild a generative baseline with the Interactions API and a JSON schema
Measure latency, token usage, and cost for both approaches
Check whether a model's confidence can be trusted, using a reliability diagram and expected calibration error (ECE)
Decide when a decision model fits your app better than a generative one
You can run the complete code in this Google Colab.
Decision Models vs. Generative Models
Before we look at how each one works, let's define the two terms.
Here's an example.
A customer writes to a bank: "My card hasn't arrived yet. When will it come?"
If you ask a generative model, "What does this customer want?", it might reply:
“The customer is waiting for a new card and wants to know when it will arrive.”
That's a good answer for a human to read. But if your app needs to route the message to the right team, your code now has to turn that sentence into a label. You can force a label with a JSON schema, but the model still writes it token by token.
However, if you ask a decision model the same thing, you first give it the options, such as
card_arrival,lost_or_stolen_card, andexchange_rate. It returns a probability for each one:Your code can then act on that probability directly by routing to the card team if
card_arrivalis above a threshold, and sending it to a human if not. In short:Generative model
Decision model
Returns
Text it writes
A probability for each option you define
Possible answers
Open ended
Fixed set
Good for
Answers, summaries, explanations, and extraction
Routing, tool selection, policy checks, and grading
In practice, many apps use a generative model for both jobs. That’s what we’ll test in the next section: whether a decision model does the classification part better than a generative model.
When you use a generative model as a classifier, it writes the answer one token at a time. For a JSON answer like ours, the output looks like this as it is generated:
Each token is a separate decoding step, and each output token is billed. If you want a confidence score, the model has to write that number too. That number is the model's own guess about itself, not a probability measured from its internals.
A decision model, however, skips this writing step. You give it the options up front, and it scores all of them against the input in one forward pass. The result is a probability for every option:
This changes three things you can measure:
The trade-off is that you can only ask closed-set questions. A decision model can't explain its answer, extract a value, or write free text.
Demo: Building a Banking Intent Classifier
In this demo, we’ll build a banking intent classifier. We'll use the test split of Banking77, a public dataset of customer messages labeled with 77 intents, and keep 12 intents that are easy to confuse with each other:
card_arrivalandcard_delivery_estimatepending_card_payment,declined_card_payment, andcard_payment_fee_chargedpending_transfer,transfer_not_received_by_recipient, andbalance_not_updated_after_bank_transferlost_or_stolen_card,card_not_working,exchange_rate, andtop_up_failedSimilar intents make the task harder, which makes the confidence scores more interesting to study.
We'll run the classifier with both models in two ways:
Here are the settings we keep the same for both models:
Setting
Value used
Why
Decision model
gemini-decision-1The model under test
Generative model
gemini-3.5-flash-liteThe cost-efficient generative baseline used in the launch post comparison
Flash-Lite temperature
0.0
Removes sampling noise from the baseline
Flash-Lite thinking level
minimalKeeps the comparison about decoding, not reasoning
Output format
JSON schema with an
enumof the 12 labelsFlash-Lite can only return a valid label
Option descriptions
One short sentence per intent
Both models see the same label definitions
Warmup calls
2 per model
Keeps the connection setup out of the measurements
Repeats (many questions per request)
3 per setting
We report the median, not a single run
Note: These are working settings for this demo, not tuned settings for either model.
Step 1: Prerequisites and setup
To follow along, you need:
A Gemini API key from Google AI Studio
A Colab notebook (or any Python 3 environment)
google-genaiversion 3.2 or laterThe Decisions API has a no-cost tier in Google AI Studio for prototyping, so you can try it before paying. I ran this demo in Colab, but feel free to use any Python 3 environment.
Let’s start by installing the packages:
Here,
google-genaiis Google's Python SDK for the Gemini API. It gives us theclient.decisions.createandclient.interactions.createmethods, which send requests to the Decisions API and the Interactions API. We usepandasto load the dataset and organize results, andmatplotlibto plot the figures.Next, we create the client. The Decisions API needs the v1beta API version:
Here,
genai.Client()reads the API key from theGEMINI_API_KEYset under the secrets in Colab or as an environment variable, andhttp_optionspoints it at the v1beta API, where the Decisions API lives.Then we’ll load the Banking77 dataset and define our 12 intents. Each intent gets a one-sentence description that both models will see:
The above code snippet downloads the Banking77 test split as a CSV file and keeps only the rows labeled with one of our 12 intents. It then samples 15 messages per intent with a fixed random seed, so you get the same sample on every run. The final
.sample(frac=1)shuffles the rows so the intents are mixed.This gives us 180 messages, 15 per intent.
Step 2: Classify with the Decisions API
A
choicequestion takes a dictionary that maps each option ID to a description. We pass the message asinputand ask one question, keyedintent:The
decide_intent()function sends one message as theinputand onechoicequestion with our 12 intents as options. We time the call withtime.perf_counter(), then read the top choice, confidence, all option probabilities, and token counts from the response.Each answer carries three fields:
choice: The option with the highest probabilityprobabilities: One probability per option, summing to 1confidence: The estimated probability that the top answer is correctThe
usageobject reports input tokens, andtotal_output_tokensis always 0.Here's the result for the first message in our sample:
The model confused two similar intents, but its confidence of 0.846 shows some uncertainty. The good part is that the correct intent,
card_arrival, came second with almost all of the remaining probability (about 0.15). All other options were close to zero.Step 3: Classify with Flash-Lite
For the generative baseline, we use structured output with an
enumof the same 12 labels, so Flash-Lite can only return a valid label. We also ask for a confidence between 0 and 1, which is how most apps get a confidence score from a generative model:The
generate_intent()function builds a prompt with the 12 intents and their descriptions. It calls Flash-Lite withresponse_formatset to our JSON schema, so the output must be valid JSON with one of the 12 labels. Temperature is set to0withminimalthinking level to keep the answer stable and short. We then parseinteraction.output_textwithjson.loads()and read the token counts frominteraction.usage.I tested the same message as in the previous step with Flash-Lite, and it made the same mistake with higher confidence:
Thinking tokens are 0, which confirms that
minimalthinking kept Flash-Lite to writing the answer only. Flash-Lite used fewer input tokens than Decision 1 (277 vs 337) because the Decisions API counts the option descriptions inside the question as input.Step 4: Classify one message per request
Now, let's send all 180 messages to both models, one request at a time, after two warmup calls each:
This loop sends each message to both functions and saves the prediction, confidence, latency, and token counts as one row. We then turn the rows into a DataFrame and add a correct column that marks whether each prediction matches the true label.
Here are the results:
Metric
Gemini Decision 1
Gemini 3.5 Flash-Lite
Accuracy
84.4%
83.9%
Median latency
325 ms
1,068 ms
p90 latency
412 ms
6,483 ms
Slowest request
1.0 s
101.6 s
Requests over 5 s
0 of 180
33 of 180
Mean input tokens
339
279
Mean output tokens
0
25
Here are three things we learned from these metrics:
Note: Our timing includes the network round-trip from Colab, so treat that gap as setup-dependent.
Step 5: Check calibration
Accuracy tells us how often a model is right, but calibration tells us whether its confidence means anything or not. A model is well calibrated when its confidence matches its accuracy, i.e., if it says 0.8 on a group of answers, about 80% of them should be correct.
To measure this, we group predictions into 10 confidence bins and compare each bin's average confidence with its accuracy. Expected calibration error (ECE) is the average gap, weighted by how many predictions fall in each bin. Lower is better.
The
ece()function places each prediction into one of 10 bins by its confidence (0 to 0.1, 0.1 to 0.2, and so on). For each bin, it takes the gap between average confidence and accuracy, weights it by the share of predictions in that bin, and adds up the results.Here's how the two models compare:
Metric
Gemini Decision 1
Gemini 3.5 Flash-Lite
Accuracy
84.4%
83.9%
Mean confidence
0.867
0.957
ECE
0.050
0.124
Mean confidence when correct
0.901
0.966
Mean confidence when wrong
0.686
0.912
Figure: Reliability diagrams for both models
Decision 1's confidence is close to its accuracy, with an ECE of 0.050. It is also clearly lower when the model is wrong (0.686) than when it is right (0.901).
Flash-Lite is overconfident. It reports 0.957 on average while being right 83.9% of the time. It used only 9 distinct confidence values across 180 answers, and 158 of them were 0.95 or higher. Even its wrong answers averaged 0.912. The number looks like a probability, but it is the model's own estimate written as text.
Calibration matters when you act on the score. A common pattern is to accept answers above a confidence threshold and send the rest to a fallback model or a human:
Threshold
Decision 1: share kept
Decision 1: accuracy on kept
Flash-Lite: share kept
Flash-Lite: accuracy on kept
0.50
94.4%
85.9%
99.4%
84.4%
0.70
78.3%
92.2%
98.9%
84.3%
0.80
74.4%
93.3%
98.9%
84.3%
0.90
63.3%
95.6%
88.9%
87.5%
0.95
53.9%
96.9%
87.8%
87.3%
With Decision 1, a higher threshold works as expected. At 0.9, it keeps 63% of messages at 95.6% accuracy. With Flash-Lite, the threshold barely helps, because most of its wrong answers also have high confidence. The official docs recommend choosing a threshold from labeled examples of your own traffic.
Step 6: Classify 100 messages in one request
The Decisions API can answer up to 1,000 questions about one shared input in a single request. To test this, we put N customer messages into one input and ask one question per message.
For Decision 1, that means N
choicequestions keyedm1tomN:Here, the
build_batch()function numbers the first N messages and joins them into one shared input. Thendecide_batch()creates onechoicequestion per message, keyedm1tomN, sends them all in a single request, and reads back one answer per key.For Flash-Lite, we send the same shared input with a JSON schema that has N fields, each limited to 12 labels. Each N uses the same messages for both models, and each setting runs three times.
Questions per request
Decision 1: median latency
Flash-Lite: median latency
Decision 1: input tokens
Flash-Lite: input / output tokens
1
318 ms
888 ms
344
291 / 14
5
353 ms
858 ms
1,554
369 / 64
10
398 ms
1,063 ms
3,056
456 / 129
25
546 ms
2,753 ms
7,570
710 / 350
50
592 ms
Failed
15,100
Failed
100
59,095 ms
Failed
30,309
Failed
Up to 25 questions, Decision 1's median latency stayed under 550 ms. Flash-Lite's median grew with N, from under 1s at 1 to 5 questions to 2.75 s at 25, because it has to write every answer as output tokens. Even at 50 questions, Decision 1's median was 592 ms, though one of the three runs took 70 s.
Here are two key points to ponder:
Step 7: Trim the requests
Now, an obvious question arises: can we fix the slowdown and the failures? For Decision 1, we use empty option descriptions (""). For Flash-Lite, we ask for one list of N labels instead of N separate fields:
Here's how Decision 1 did with empty descriptions:
Questions per request
Median latency
Accuracy
Input tokens
25
357 ms
84.0%
3,995
50
649 ms
78.0%
7,950
100
1,603 ms
83.0%
16,009
Dropping the descriptions roughly halved the input tokens. At 100 questions, the request came down to about 16K tokens, and median latency fell from 59 s to 1.6 s. Accuracy dropped a little at 25 and 50 questions, so short descriptions are a better middle ground than none, especially for similar intents.
The list schema didn't help the Flash-Lite model. It worked at 25 questions, but still failed at 50 and 100.
Step 8: Compare cost
Finally, we compute the cost from the token counts each API returns, using paid-tier prices:
Model
Input price per 1M tokens
Output price per 1M tokens
Gemini Decision 1
$0.10
$0
Gemini 3.5 Flash-Lite
$0.30
$2.50 (including thinking tokens)
The
cost_per_request()function multiplies each token count by its price per 1M tokens. For Decision 1, only input tokens count. For Flash-Lite, thinking tokens are billed at the output price, so we add them to the output tokens.Here's the cost per 1,000 requests for each setup:
Setup
Gemini Decision 1
Gemini 3.5 Flash-Lite
Flash-Lite / Decision 1
1 message per request
$0.034
$0.147
4.3x
5 questions per request
$0.155
$0.271
1.7x
10 questions per request
$0.306
$0.459
1.5x
25 questions per request
$0.757
$1.088
1.4x
25 questions, trimmed setups (Step 7)
$0.400
$0.831
2.1x
50 questions per request
$1.510
Failed
n/a
100 questions, empty descriptions
$1.601
Failed
n/a
On single messages, Decision 1 was 4.3x cheaper, but the gap shrinks as you batch more questions into one request. The shared input is billed once in both APIs. But each Decision 1 question carries its own instruction and all 12 option descriptions, while Flash-Lite sees the label list only once and pays for a short output per answer. At 25 questions, Decision 1 used 7,570 input tokens against Flash-Lite's 710 input and 350 output tokens.
Note: The Decisions API doesn't support context caching or the Batch API. If you send the same input in several requests, it is billed in full each time. So when you have many questions about the same input, ask them in one request rather than spreading them across calls.
When to Use Each Approach
Based on the above runs, a decision model is the better fit when:
A generative model is still the better fit when:
Use the Decisions API when you need
Use the Interactions API when you need
A yes/no probability, one option from a fixed set, or a rating on a fixed scale
Generated text, summaries, or explanations
Calibrated probabilities that you can threshold
Extracted values or objects that follow your own JSON schema
Many independent checks over the same input in one call
Tool calls with arguments
Low latency and zero output token cost
Multi-step reasoning
Final Thoughts
In this tutorial, we built a banking intent classifier with Gemini Decision 1 and Gemini 3.5 Flash-Lite, then calculated accuracy, latency, calibration, and cost to compare the two models. Here are four takeaways from the tutorial:
These results come from one task, one dataset, and a small number of runs from a Colab notebook. You can treat them as observations from a demo rather than general performance claims.