Skip to main content

Google AI Co-Mathematician: DeepMind's Agentic Math Workbench

See how Google DeepMind's multi-agent system helps mathematicians run research end to end, and how Marc Lackenby used it to resolve an open problem.
Aug 10, 2026  · 8 min read

Explore with AI

Open in ChatGPTOpen in ClaudeOpen in Perplexity

Oxford mathematician Marc Lackenby handed an open problem to Google's AI co-mathematician. It produced a proof, and then its own review agents rejected that proof. Reading the rejected argument, Lackenby spotted a genuinely novel strategy inside it, patched the gap himself, and resolved the problem.

The co-mathematician lands in a busy year for AI and mathematics. In July 2026, Claude Fable 5 helped dent the Jacobian conjecture; GPT-5.6 Sol suggested the construction that broke the Maxwell conjecture and helped crack the Dinitz-Garg-Goemans conjecture; and OpenAI's unreleased Astra cleared ten open problems in a day.

Most of those were one-shot results: a model produces an answer, humans verify it. The co-mathematician is a different bet, built for the long, nonlinear middle of research rather than for handing over a single question. In this article, we cover what the AI co-mathematician is, how its workflow and multi-agent architecture work, what early users found, and where it still falls short.

What Is the Google DeepMind AI Co-Mathematician?

The AI co-mathematician is a research workbench from Google DeepMind that coordinates a team of specialized AI agents to help a mathematician run a whole research project, from framing the question through to a written-up proof. It is meant to orchestrate the entire messy process of research, not to answer a single question in one shot.

Introduction to AI Agents

Learn the fundamentals of AI agents, their components, and real-world use—no coding required.
Explore Course

In May 2026, Google DeepMind researchers released an arXiv preprint titled "AI co-mathematician: Accelerating mathematicians with agentic AI," describing an agentic AI workbench that orchestrates the full mathematical work cycle, including ideation, literature search, computation, proof attempts, and theory building.

The AI co-mathematician, currently in limited release to a select group of researchers, is not in itself a solver or a chatbot, but rather a stateful workspace that orchestrates the non-linear, messy mathematical workflow. It manages parallel research threads, tracks uncertainty, and preserves failed attempts for future reference. Outputs are in LaTeX, the standard document format used across mathematics and physics research.

It is built on Gemini 3.1. As of now, it uses Gemini Deep Think internally for proof attempts and integrates with Python execution for computational exploration. It does not yet incorporate specialized DeepMind mathematical engines (AlphaProof, AlphaEvolve, Aletheia) as internal components, but is designed to do so in the future.

Why Is the AI Co-Mathematician Important?

Mathematicians use LLMs to help with a number of tasks:

  • Literature review
  • Informal proof sketching
  • Boilerplate LaTeX
  • Exploratory code

They use AI tools, such as formal proof assistants, to help verify proofs mechanically. 

Although each tool used is strong in a particular part of the overall workflow, the mathematician must stitch their results together ad hoc. Up until now, there has been no infrastructure tool to orchestrate the nonlinear process mathematicians use.

How the AI Co-Mathematician Works

The workflow model is loosely adapted from AI coding tools, such as Claude Code or Codex. However, this workflow cannot be adapted directly. Software has a spec and a test suite. Mathematics research is nonlinear. The goal itself may be vague or undefined at the start, and findings along the way can redefine the question entirely.

The workflow for the co-mathematician is powered by a multi-agent system and looks like this:

  1. Initial exploration. Rather than requiring a perfect one-shot prompt, the researcher has an interactive dialogue with the system to refine high-level goals and formalize the research question. This makes the research intent explicit before the work begins.
  2. Literature review. The co-mathematician performs a literature review, where it identifies key papers and extracts relevant proofs and mathematical techniques.
  3. Computational framework. The co-mathematician creates a computational Python library, along with demonstration cases and associated tests.
  4. Search execution. After the computational framework has been created, the custom library is imported and scales the search across a cloud cluster. It gives the researcher high-level progress reports without drowning the researcher in low-level logs.
  5. Final output. The final output is a compiled LaTeX write-up. This report explains the discovery process, not just the result, and uses margin annotations to link claims to specific references or code outputs. A report is marked finalized only after all reviewer agents approve it. Reports that fail review are still surfaced to the researcher, flagged as unresolved, which is how the flawed proof reached Lackenby.

How the AI Co-Mathematician Works

Workstream Architecture of the AI Co-Mathematician

The AI Co-Mathematician's architecture is a multi-agent system with a top-level project coordinator agent and specialized sub-agents.

Workflow coordination

Before any work is done to tackle the problem, the project coordinator opens up a dialog with the researcher to refine the goal through clarifying questions. For instance, a researcher asking about upper bounds on the "sofa problem," an open question in computational geometry, might be asked whether to focus on one variant or another, or both. The precise, agreed-upon goal scopes the problem before dispatching agents.

Once the goal has been settled on, the project coordinator delegates work to agents and subagents, which work in parallel unless dependencies require otherwise. One agent performs the lit review, another builds the computational framework, and another performs the search. There is a dependency, though: the search cannot be executed until the custom library from the computational framework exists.

AI Co-Mathematician Workflow

Avoiding hallucinations

Standard agents often find invalid shortcuts, hallucinate lemmas, or prematurely declare victory. The AI co-mathematician employs mechanisms to avoid these shortcomings, including reviewer agents. 

For instance, if the search space explodes during computational exploration, the coding sub-agent cannot mark the code as finished until all tests pass and the reviewer agent accepts its results, thereby stalling the workflow. This failed exploration is logged and sent to the project coordinator, who then alerts the researcher in the chat. The researcher can then suggest another approach.

Such dead ends are not just logged; they are treated as information. They are version-traced and preserved as part of the research record. In mathematics, a failed approach often reveals a constraint or points toward a different path.

Throughout the process, the coordinator filters low-level noise by default, but the researcher can drill into any workstream and modify it if needed.

First Case Studies and Collaboration Modes

A couple of early cases reflect different modes of collaboration: a flawed proof that unlocked human insight, a marginal comment that reframed a question, and rapid dead-end detection that saved a week of misdirected effort.

Repairing a proof that the system flagged as wrong

Marc Lackenby, mathematics professor of the University of Oxford, applied the system to a problem from the Kourovka notebook (a compendium of open research problems in Group Theory). The AI made a proof attempt, a review agent spotted a flaw in the proof, and reported it to Lackenby. He knew how to repair the gap.

That handoff is the point. A human caught what the machine missed, and the machine did the heavy lifting around it. 

Human-AI Handoff

Following a margin comment to a new insight

Another mathematics professor, Gergely Bérczi researched a problem involving Stirling coefficient conjectures. For initial review, he provided the AI co-mathematician the topic, its background, currently known methods, and a potential direction suggested by AlphaEvolve experiments. 

The AI co-mathematician made proofs (currently under human mathematical review) for two of the conjectures, along with detailed computational evidence for their claims. Bérczi reported that the system helped him by giving a marginal comment in the finished document, which alerted him to a key insight, which he then followed up on in the chat interface.

Detecting a dead end before it cost a week

Finally, Semon Rezchikov posed a technical problem about Hamiltonian diffeomorphisms. He and the Project Coordinator discussed the problem until they agreed on a precise definition of the task. The write-up produced by the AI included a key lemma with proof (later verified) that essentially resolved the question. 

Along the way, an approach was explored, but it reached a dead end. The AI reached the dead end much faster than otherwise, so he did not waste a week chasing a dead end. The system compressed a week of wasted effort into hours.

FrontierMath Benchmark Results

AI Co-mathematician is a workflow tool, not a mathematical problem-solving tool per se. Nonetheless, it is important for the authors to show high benchmarks for several reasons:

  • To show that the underlying mathematical engine is capable enough to be a useful collaborator for frontier mathematics.
  • To demonstrate that benchmarks provide a controlled environment to measure how much the orchestration layer adds.
  • To provide a proxy for collaborative efficacy, since no standardized measure currently exists.

The results: AI co-mathematician scored 48% on FrontierMath Tier 4, a research-level benchmark. This is the highest score reported, including Google's own Gemini 3.1 Pro, which scored 19% on the same test, same tier. Epoch AI found errors in roughly a third of FrontierMath problems (across Tiers 1–4) and is conducting a human review.

The AI co-mathematician scored 87% on a Google internal benchmark consisting of 100 unleaked, research-level mathematics problems with code-checkable answers, sourced from professional mathematicians, while Gemini 3.1 Pro scored 57% and Gemini 3.1 Deep Think scored 70%.

Known Limitations of the AI Co-Mathematician

The paper is transparent about structural limitations:

  • The main quality control comes from reviewer agents and proof agents. The adversarial review cycle can display a reviewer-pleasing bias, where agents converge on arguments that satisfy the reviewer agent rather than being correct. This is a real structural vulnerability.
  • Polished LaTeX output can create a false sense of rigor. A well-formatted document looks finished whether or not the underlying mathematics is correct.
  • The system only works well when the researcher is truly a domain expert. There is a danger that a non-expert might hand a vague problem to the system and blindly trust its output.
  • AI proofs still require human verification. Bérczi's proofs, for example, were still in human review at publication.

Final Thoughts

The AI Co-mathematician reflects a shift from asking AI to solve a problem to working with the AI on the iterative, messy process of mathematical research. The real bottleneck is the workflow infrastructure that mirrors how mathematicians actually work.

If you want to build agentic systems like this yourself, DataCamp's Associate AI Engineer for Developers and Associate AI Engineer for Data Scientists tracks are a good place to start.

Google AI Co-Mathematician FAQs

How is the AI Co-mathematician different from ChatGPT or Gemini?

ChatGPT and Gemini are conversational tools optimized for single exchanges. The AI Co-mathematician is a stateful workspace that runs parallel research threads, tracks failed attempts, and manages a weeks-long investigation.

Does the AI Co-mathematician replace mathematicians?

No. The system works best when steered by a domain expert who can evaluate its outputs and recognize when a flawed proof contains a useful idea.

What is a multi-agent system?

It is an architecture where multiple specialized AI models collaborate on a task. In the AI Co-mathematician, separate agents handle literature search, computation, proof attempts, and adversarial review.

How does the AI Co-mathematician handle errors and dead ends?

Failed attempts are logged, version-tracked, and escalated to the researcher rather than discarded. In mathematics, a dead end often reveals a constraint or points toward a better path.

How does the output differ from a standard AI-generated response?

The system produces a compiled LaTeX document with margin annotations linking claims to specific papers or code outputs, rather than unreferenced prose. A report is marked finalized only after all reviewer agents approve it.


Mark Pedigo's photo
Author
Mark Pedigo
LinkedIn

Mark Pedigo, PhD, is a distinguished data scientist with expertise in healthcare data science, programming, and education. Holding a PhD in Mathematics, a B.S. in Computer Science, and a Professional Certificate in AI, Mark blends technical knowledge with practical problem-solving. His career includes roles in fraud detection, infant mortality prediction, and financial forecasting, along with contributions to NASA’s cost estimation software. As an educator, he has taught at DataCamp and Washington University in St. Louis and mentored junior programmers. In his free time, Mark enjoys Minnesota’s outdoors with his wife Mandy and dog Harley and plays jazz piano.

Topics

Learn About AI Agents with DataCamp!

Track

AI Agent Fundamentals

6 hr
Discover how AI agents can change how you work and deliver value for your organization!
See DetailsRight Arrow
Start Course
See MoreRight Arrow
Related

blog

What is DeepMind AlphaGeometry?

Discover AphaGeometry, an innovative AI model with unprecedented performance to solve geometry problems.
Javier Canales Luna's photo

Javier Canales Luna

8 min

Robot investigator to represent openai's deep research

blog

OpenAI's Deep Research: A Guide With Practical Examples

Learn about OpenAI's new Deep Research tool, which can perform in-depth, multi-step research.
Alex Olteanu's photo

Alex Olteanu

8 min

podcast

Can AI Agents Outperform a Data Scientist? with James Zou, Professor at Stanford University

Richie and James explore how AI scientist agents are already outperforming human experts, the Virtual Lab framework for teams of agents, a new training paradigm called "learning to discover", scaling agentic systems, and much more.

Tutorial

Gemini 3 Deep Think: A Guide to AI Reasoning

Discover how Google's newest specialized reasoning model can accelerate your data science workflows, interpret complex datasets, and write robust code.
Tim Lu's photo

Tim Lu

Tutorial

A Guide to Andrej Karpathy’s AutoResearch: Automating ML with AI Agents

Learn how Karpathy's AutoResearch runs 100+ ML experiments overnight on a single GPU. Covers the three-file architecture, ratchet loop, results, and limitations.
Bex Tuychiev's photo

Bex Tuychiev

Tutorial

Google Antigravity CLI Tutorial: Orchestrating Parallel AI Agents

Use Google's Antigravity CLI to orchestrate dynamic subagents that clean, analyze, and visualize a dataset in parallel, producing an interactive HTML dashboard.
Aashi Dutt's photo

Aashi Dutt

See MoreSee More