Gemini Embedding 2: A Guide With Python Examples

Imagine you have a folder full of photos, a few voice notes, and some PDF lecture slides, and you want to find “the photo of the whiteboard with the robot arm diagram.”

Your computer can search file names, but it has no idea what is actually inside the image or the audio clip.

To solve this, developers have needed a separate model for each type of data: one for text, one for images, one for audio, and then some glue code to make them work together.

It works, but it is very messy (and every extra model is another thing that can break).

Google’s Gemini Embedding 2 is a big attempt to change that by putting text, images, audio, video, and PDFs into one shared space.

In this article, we look at what embeddings are, what makes Gemini Embedding 2 different, and how it compares to other models.

Then we will build a small multimodal search app in Python where you can search a folder of images and audio clips with text, or search the images with another image.

What Are Embeddings?

Before we look at the model itself, I want to make sure we are all on the same page about what an embedding actually is. Let’s start by defining it:

An embedding is a list of numbers (a vector) that represents the meaning of a piece of content, so that similar content ends up with similar numbers.

From words to vectors

I like to think of embeddings as coordinates on a map.

Every piece of content gets a position, and things with similar meanings are placed close together.

So “a puppy playing in the park” and “a young dog running on grass” would sit near each other, while “quarterly tax return” would be somewhere far away.

The map is not 2D like a paper map; it has thousands of dimensions (Gemini Embedding 2 uses 3,072 by default).

We cannot picture that many dimensions, but all we need to know is that we can measure how close two vectors are using cosine similarity, which gives a score between -1 and 1, where a higher score means more similarity.

This is the idea behind semantic search, recommendation systems, and retrieval-augmented generation (RAG), which we cover in our RAG with LangChain course. Instead of matching exact keywords, you match meaning.

Why a shared multimodal space matters

Here is where it gets really interesting.

Most embedding models handle only text. If you wanted to search images, you would need a different model, and the vectors from the 2 models would live on 2 completely different maps (comparing them would be like comparing a GPS coordinate with a page number).

A multimodal embedding model puts everything on the same map.

A photo of a dog, the text “a dog,” and an audio clip of a dog barking should all land in the same neighborhood, meaning we can search images with text, find a video using a photo, or find audio clips from a written description.

Every input type ends up in the same shared latent space. Image by Author

What Is Gemini Embedding 2?

Gemini Embedding 2 is Google’s first natively multimodal embedding model. It takes in text, images, audio, video, and PDF documents and turns all of them into vectors that live in the same space, across more than 100 languages.

Release timeline

The model came out in 2 stages:

• March 10, 2026: public preview as gemini-embedding-2-preview
• April 22, 2026: general availability (stable) as gemini-embedding-2

Google’s earlier text-only model, gemini-embedding-001, is still available if you only work with text.

But the 2 models have incompatible embedding spaces, so you cannot mix their vectors (more on that later).

Native multimodality vs. stitched encoders

See, multimodal embeddings are not new. Models like CLIP have connected images and text for many years now. The difference is in how they are built.

Older approaches usually use separate encoders (one “tower” for images and another for text) that are trained to line up their outputs. Gemini Embedding 2 is built on the Gemini architecture itself, so a single model handles every input type.

Think of it this way: rather than two translators agreeing on a shared dictionary, you have one person who speaks every language.

Interleaved inputs

This is my favorite feature.

You can send several inputs in one request, such as a product photo plus its text description, and the model returns one embedding that captures them together.

Take a photo of a jacket: the image alone does not tell you it is waterproof, and the word “waterproof” alone does not tell you what the jacket looks like.

Embedding them together gives you a vector that represents the whole product, not just one piece of it.

I believe this is the real reason we should pay attention to this model, especially for smaller teams. You no longer need to host and maintain 3 or 4 models just to search your own data.

Key Features of Gemini Embedding 2

Now, let’s go through the features you will actually use when building with the model.

Supported inputs and limits

Every request shares one overall limit of 8,192 tokens, and each input type has its own rules:

There are 2 details here that are easy to miss.

• First, for video, the model ignores the audio track, so if the sound matters, you need to embed it separately.
• Second, each PDF page is 258 tokens just for its visual representation (plus the text inside it), and Google recommends one page per PDF for the best quality.

Flexible dimensions with Matryoshka Representation Learning

By default, every embedding has 3,072 numbers. That is a lot to store if you have millions of items.

To deal with this, the model is trained with Matryoshka Representation Learning (MRL).

The name comes from Russian nesting dolls, and I think the analogy fits quite well.

MRL trains the model so that the first few hundred numbers of the vector already hold most of the meaning, with each extra chunk adding finer detail.

So you can cut the vector short and still get a useful embedding.

You control this with the output_dimensionality parameter. Google recommends 768, 1,536, or 3,072. Gemini Embedding 2 also normalizes shorter vectors automatically (with gemini-embedding-001 you had to do this yourself).

Task prefixes

The same sentence can be embedded differently depending on what you want to do with it.

A search query and the document that answers it are obviously not the same thing, so the model works better when you tell it the task.

With gemini-embedding-001, you passed a task_type parameter. Gemini Embedding 2 does not support that parameter, so you add the task as a prefix inside the text itself:

Search, question answering, fact checking, and code retrieval are asymmetric (queries and documents use different formats), whereas classification, clustering, and semantic similarity are symmetric (everything uses the same format).

A few rules from Google’s docs are worth remembering. If a document has no title, use title: none. Do not use the semantic similarity format for search or retrieval, because it is intended for comparing the meaning of texts. And whichever task you pick, use it consistently: if you embed documents with the classification format, embed the queries the same way.

Google also advises not adding a task prefix to the text part of a mixed text-and-image input, as it can sometimes hurt performance.

Aggregated vs. separate embeddings

In my opinion, this one is the most common beginner mistake.

With gemini-embedding-001, passing a list of 5 strings gave you 5 embeddings back. With Gemini Embedding 2, passing a list of inputs gives you one combined embedding.

To get separate embeddings, we have to wrap each input in its own Content object (we will see exactly how later on).

Or, for large jobs, we can use the Batch API, which also costs 50% less if results are not needed straight away.

In a nutshell, a list of parts gives you one embedding, while a list of Content objects gives you one embedding each.

A list of parts gives one embedding, a list of Content objects gives one each. Image by Author

Gemini Embedding 2 vs. Other Embedding Models

Gemini Embedding 2 is, of course, not the only multimodal embedding model out there.

Here is how it compares to some of the main alternatives:

A few things stand out to me here.

Gemini Embedding 2 and Amazon Nova Multimodal Embeddings are the only models in the table that handle audio natively, and both cover documents too. Nova caps audio and video at 30 seconds per call, though, against 180 and 120 seconds for Gemini.

On the other hand, Voyage and Qwen accept much longer inputs (32,000 tokens vs. 8,192).

Google’s technical report (May 2026) shows Gemini Embedding 2 ahead of Amazon Nova and Voyage Multimodal 3.5 on most text, image, and video benchmarks, with an overall score of 77.2 against 70.0 for Voyage and 68.2 for Nova on the tasks the models share. Voyage edges ahead on ViDoRe V2 document retrieval (65.5 vs. 64.9) and on Flickr30k text-to-image retrieval (89.9 vs. 89.1).

But please do remember that these are Google-reported numbers, and on text-only benchmarks, some open-source models can match or beat it.

Also, if your data cannot leave your own servers, I would suggest an open-weights model like Qwen3-VL-Embedding, as Gemini Embedding 2 is only available through Google’s APIs (the Gemini API and the Gemini Enterprise Agent Platform, formerly Vertex AI).

How to Use Gemini Embedding 2 With the Gemini API

Before we build anything, let’s first get set up and create our first embedding.

Getting an API key

Go to Google AI Studio, sign in, and create an API key.

Then save it as an environment variable called GEMINI_API_KEY (or in a .env file, which we load in the first code snippet below), so you never have to paste it into your code:

# macOS / Linux
export GEMINI_API_KEY="your-key-here"

# Windows (PowerShell)
$env:GEMINI_API_KEY="your-key-here"

Installing the SDK

We will be using Google’s google-genai SDK as well as a few libraries for the tutorial. The tutorial uses ChromaDB 1.0 or later:

pip install google-genai "chromadb>=1.0" matplotlib pillow python-dotenvDrivs av

Your first embedding

Let’s embed a single sentence to check everything works. The client will pick up GEMINI_API_KEY from your environment automatically.

from dotenv import load_dotenv
from google import genai
from google.genai import types

load_dotenv()  # only needed if your key lives in a .env file

client = genai.Client()

result = client.models.embed_content(
    model="gemini-embedding-2",
    contents="A robot arm working on a table",
)

vector = result.embeddings[0].values
print(len(vector))
# 3072Drivs av

This output is a list of 3,072 numbers.

On their own, they mean nothing to us, but together they describe where this sentence sits in the embedding space.

Now let’s shrink it using output_dimensionality by writing this:

result = client.models.embed_content(
    model="gemini-embedding-2",
    contents="A robot arm working on a table",
    config=types.EmbedContentConfig(output_dimensionality=768),
)

print(len(result.embeddings[0].values))
# 768Drivs av

Gemini Embedding 2 Tutorial: Build a Multimodal Search App in Python

Now that the basics are out of the way, let’s build a small “search my media” app.

It will take a folder of images and audio clips, embed everything into one vector database, and then let us search it using text or another image.

Step 1: Prepare your media folder

Create a folder called media next to your notebook and put a mix of files in it. I suggest:

• Around 10 photos (JPEG or PNG) with clearly different subjects, such as animals, food, streets, and nature
• Two or three short audio clips (MP3 or WAV, under 3 minutes), for example, a piano recording and some rain sounds

Also, keep one extra photo outside the folder and call it query.jpg. We will use it later as an image query.

Step 2: Write the embedding helpers

We will need 3 small helper functions plus a MIME type lookup: one for text queries, one for files, and one to label what kind of file we have.

from pathlib import Path

MODEL = "gemini-embedding-2"
DIMS = 768

MIME_TYPES = {
    ".jpg": "image/jpeg",
    ".jpeg": "image/jpeg",
    ".png": "image/png",
    ".mp3": "audio/mpeg",
    ".wav": "audio/wav",
    ".pdf": "application/pdf",
}


def embed_text(text, task="search result", use_prefix=True, dims=DIMS):
    if use_prefix:
        text = f"task: {task} | query: {text}"
    response = client.models.embed_content(
        model=MODEL,
        contents=text,
        config=types.EmbedContentConfig(output_dimensionality=dims),
    )
    return response.embeddings[0].values


def embed_file(path, dims=DIMS):
    path = Path(path)
    mime_type = MIME_TYPES[path.suffix.lower()]
    response = client.models.embed_content(
        model=MODEL,
        contents=[types.Part.from_bytes(data=path.read_bytes(), mime_type=mime_type)],
        config=types.EmbedContentConfig(output_dimensionality=dims),
    )
    return response.embeddings[0].values


def file_kind(path):
    mime_type = MIME_TYPES[Path(path).suffix.lower()]
    return "pdf" if mime_type == "application/pdf" else mime_type.split("/")[0]

The embed_text() function adds the task prefix by default, but use_prefix lets you switch it off. Google strongly recommends prefixes for text-only tasks and advises against them on the text part of mixed text-and-image inputs. Its docs don’t cover text queries against images or audio, so we will test both in Step 5.

The file_kind() helper labels each file as an image or audio clip so we can filter results later.

Step 3: Understand merged vs. separate embeddings

Before indexing, let me quickly show you the aggregation behavior from earlier, because getting this wrong would silently break our search.

Let’s say we replace dog.jpg with any image from your folder.

image_part = types.Part.from_bytes(
    data=Path("media/dog.jpg").read_bytes(),
    mime_type="image/jpeg",
)

# A list of parts: one merged embedding
merged = client.models.embed_content(
    model=MODEL,
    contents=["A photo of my dog at the beach", image_part],
)
print("Merged:", len(merged.embeddings))

# A list of Content objects: one embedding each
separate = client.models.embed_content(
    model=MODEL,
    contents=[
        types.Content(parts=[types.Part.from_text(text="A photo of my dog at the beach")]),
        types.Content(parts=[image_part]),
    ],
)
print("Separate:", len(separate.embeddings))
# Merged: 1
# Separate: 2Drivs av

The first call merges the text and image into one vector, which is exactly what you would want for something like a product listing.

The second call wraps each input in a Content object, so you get 2 separate vectors. But for our mini-search app, we only want one vector per file, so we will just embed each file in its own call.

Step 4: Index everything in ChromaDB

Now, let’s embed every file and store it in ChromaDB, a lightweight vector database that runs locally (see our ChromaDB tutorial for a deeper look).

import time
import chromadb

# A persistent client saves the vectors to disk
chroma = chromadb.PersistentClient(path="chroma_db")


def build_index(name):
    collection = chroma.get_or_create_collection(
        name=name,
        configuration={"hnsw": {"space": "cosine"}},
    )
    files = sorted(p for p in Path("media").iterdir() if p.suffix.lower() in MIME_TYPES)
    for i, path in enumerate(files):
        collection.upsert(
            ids=[f"file-{i}"],
            embeddings=[embed_file(path)],
            metadatas=[{"path": str(path), "kind": file_kind(path)}],
        )
        time.sleep(0.5)  # stay under the rate limit
    return collection


# Run this once. In later sessions, skip it and reload the saved index with
# collection = chroma.get_collection("media")
collection = build_index("media")
print(f"Indexed {collection.count()} files")Drivs av

Setting space to cosine in the HNSW configuration tells ChromaDB to compare vectors using cosine distance, which matches how the embeddings are meant to be compared. Chroma’s default is L2, so do not skip this line.

We use a persistent client so the vectors are saved to disk, which means you do not have to pay to embed your files again every time you restart. We also store the file kind as metadata, which lets us search only images or only audio.

Step 5: Search images with text

With everything indexed, we need a function that takes a query vector and returns the closest files. ChromaDB returns a distance, so we convert it back to a similarity score (where similarity = 1 - distance) because that is easier to read.

import matplotlib.pyplot as plt
from PIL import Image


def search(collection, query_vector, n_results=3, kind=None):
    results = collection.query(
        query_embeddings=[query_vector],
        n_results=n_results,
        where={"kind": kind} if kind else None,
    )
    return [
        (meta["path"], round(1 - dist, 3))
        for meta, dist in zip(results["metadatas"][0], results["distances"][0])
    ]


def show_images(results):
    fig, axes = plt.subplots(1, len(results), figsize=(4 * len(results), 4), squeeze=False)
    for ax, (path, score) in zip(axes[0], results):
        ax.imshow(Image.open(path))
        ax.set_title(f"{Path(path).name}\nsimilarity: {score}")
        ax.axis("off")
    plt.show()

Now let’s search our photos with a plain sentence.

Change the query to match something that is actually in your folder.

Google’s docs don’t say whether a text query against images should carry a task prefix, so the loop below prints results with and without one. embed_text() uses the prefix by default, so compare the 2 lists yourself.

query = "a red car parked on a street"

for use_prefix in (True, False):
    results = search(collection, embed_text(query, use_prefix=use_prefix), kind="image")
    print(f"Prefix={use_prefix}: {results}")

show_images(search(collection, embed_text(query), kind="image"))

Step 6: Search images with an image

As images and text share the same space, a photo can be a query too. Let’s use query.jpg now, which is the photo we kept outside the folder.

results = search(collection, embed_file("query.jpg"), kind="image")
print(results)
show_images(results)

This is the same idea behind “search by photo” features in shopping apps. We do not need any text labels at all, as the model compares what the images show.

Step 7: Search audio with text

Now let’s try the really cross-modal part. We will describe an audio clip in words and see if the model finds it. In a notebook, we can also play the matched clip:

from IPython.display import Audio

results = search(collection, embed_text("someone playing the piano"), n_results=1, kind="audio")
print(results)
Audio(filename=results[0][0])

Gemini Embedding 2 Limitations

Gemini Embedding 2 is a big step forward for multimodal search, but like most things, it has some limits that are important to state.

• Short inputs only: Video is capped at 2 minutes, audio at 3 minutes, and PDFs at 6 pages. For anything longer, you need to split it into chunks and embed each one.
• Silent truncation: If a request goes over 8,192 tokens, the extra input is cut off without an error. This is easy to miss, so check your token counts for long text and multi-page PDFs.
• Video ignores sound: The audio track of a video is not embedded. If the sound matters, extract it and embed it separately.
• Re-embedding when migrating: Vectors from gemini-embedding-001 and Gemini Embedding 2 cannot be compared. Moving to the new model means re-embedding your entire dataset.
• API only: The model is proprietary, so your data is sent to Google. For sensitive data, check whether this is allowed or use an open-weights model instead.
• Vendor benchmarks: Most published results come from Google itself. Always test on your own data before committing.

Gemini Embedding 2 Use Cases

Hopefully, by now, you can see where this model fits. Here are some of the most useful applications, in my opinion:

Final Thoughts

Gemini Embedding 2 makes multimodal search much simpler. Instead of running separate models for text, images, and audio and trying to make their vectors aligned, we get one model and one embedding space for everything.

If you are starting a new project with mixed media, I would honestly give it a try first. If your data is text-only, gemini-embedding-001 or an open-source text model may be cheaper and just as good. And if you need long videos, long documents, or local hosting, then an alternative we compared is probably better.

If you want to go further, follow our multimodal RAG tutorial to connect it to an LLM and build a full multimodal RAG system.