What Is OpenAI's O1 Pro Mode? Features, ChatGPT Pro & More

What Is ChatGPT Pro?

OpenAI has just introduced a new tier to its ChatGPT lineup: ChatGPT Pro. We're taking a close look at what this new offering brings to the table and how it compares to the existing Free and Plus plans.

ChatGPT Pro is the premium tier, providing the highest level of access to OpenAI's AI models.  This increased access comes at a cost, though: $200 per month.

While that's a significant investment, it may be worthwhile for those who require the most advanced AI tools for their work, such as researchers, engineers, and other professionals who rely heavily on these technologies.

To help you understand the differences between the tiers, we've put together this table:

However, if you're a casual user or simply exploring the capabilities of AI, the Free or Plus plans likely offer sufficient functionality.

Access to Sora

ChatGPT Pro also allows you to make the best of Sora—this is OpenAI's most advanced AI video generator. To learn more, you can read this guide on OpenAI's Sora.

You can also use Sora with ChatGPT plus, but ChatGPT Pro lets you do much more:

Now, let’s explore the most exciting part about ChatGPT Pro: the o1 pro mode model.

What Is O1 Pro Mode?

o1 pro mode is a key model you get with ChatGPT Pro. It's a significantly enhanced version of the o1 model, engineered for increased accuracy and the ability to handle greater complexity.

From what we gathered during OpenAI's live stream presentation, o1 pro mode allows you to essentially request even more computational power to tackle especially demanding problems. This is particularly useful for those pushing the limits of AI in fields like advanced mathematics, complex coding, scientific research, and other computationally intensive tasks.

It's important to note that o1 pro mode retains all the existing capabilities of o1, including multimodal inputs and advanced image understanding. So you're not losing any functionality—you're gaining a significant boost in processing power.

One thing to keep in mind is that because o1 pro mode can take longer to process requests and generate responses, ChatGPT displays a progress bar to keep you informed. This gives you a visual indication of the model's "thinking" process as it works to provide you with the most accurate and comprehensive answer possible.

Source: OpenAI presentation

How Does O1 Pro Mode Work?

O1 pro mode, as we've mentioned, is a more powerful version of the o1 model.  It inherits the core architecture and mechanisms that make o1 so effective:

1. Reinforcement learning and chain-of-thought reasoning
2. Allocation of more resources to the inference phase

Reinforcement learning and chain-of-thought reasoning

One of the defining characteristics of both o1 and o1 pro mode is their emphasis on reasoning. Instead of rushing to a response, these models take a more deliberate approach, spending more time "thinking" before answering our requests.

This deep thinking is achieved through a combination of reinforcement learning and chain-of-thought reasoning.

Reinforcement learning allows the model to learn from its mistakes and refine its approach over time, much like a human learns through trial and error. Chain-of-thought reasoning, on the other hand, enables the model to break down complex problems into smaller, more manageable steps. This is particularly useful in fields like math and coding, where arriving at the correct answer often requires a multi-step process.

More resources to the inference phase

A key factor in o1 pro mode's superior performance is its strategic allocation of computational resources. Like o1, it shifts more compute power towards the training and inference phases.

This means that not only is the model trained more effectively, but it also has more resources available when generating responses, allowing it to "think longer" and explore more possibilities.

To illustrate how crucial compute power is for these advanced models, let's take a look at some interesting data OpenAI shared about o1. These two graphs below track how o1's performance on the challenging American Invitational Mathematics Examination (AIME) improves with increased computational resources.

Source: OpenAI

As you can see, the more compute dedicated to both training and testing o1, the better it performs on these complex math problems.  What's particularly striking is how much accuracy improves when more compute is allocated during the "thinking" phase (testing or inference). This suggests that giving the model more time and resources to process information leads to significantly better results.

This underscores why o1 pro mode, with its emphasis on increased compute, is so promising. By dedicating more resources to the process, o1 pro mode can dig deeper, explore more possibilities, and ultimately arrive at more accurate and reliable solutions.

O1 Pro Mode Benchmarks

OpenAI has tested o1 pro mode on areas that require deep reasoning and problem-solving, such as math, coding, and science.

Standard evaluation

Before getting into details, let’s first take a look at the graphs below:

Source: OpenAI

Let’s explain what each benchmark means and comment on the results:

• Competition Math (AIME 2024): The American Invitational Mathematics Examination (AIME) is a challenging competition for high school students. It evaluates the model's ability to solve complex mathematical problems, requiring advanced reasoning and problem-solving skills. o1 pro mode significantly outperforms both o1 and o1-preview on this benchmark.
• Competition Code (Codeforces): Codeforces is a platform that hosts competitive programming contests. This benchmark assesses the model's coding proficiency, including its ability to understand complex code, generate efficient algorithms, and solve programming challenges. o1 pro mode achieves an impressive result on this benchmark, but it doesn’t show big improvement compared to o1.
• PhD-Level Science Questions (GPQA Diamond): This benchmark evaluates the model's ability to answer complex scientific questions at a PhD level. It tests the model's understanding of scientific concepts, its ability to extract information from scientific texts, and its capacity to reason and draw conclusions based on scientific evidence. Again, o1 pro mode demonstrates strong performance, but the differences between o1 and o1-preview are not that large.

“4/4 reliability” evaluation

To asses o1 pro mode more rigorously, OpenAI used a stricter evaluation metric called “4/4 reliability.”

What does this mean? Essentially, the model has to answer the same question correctly in four out of four attempts to be considered successful. This helps ensure that the model isn't just getting lucky or relying on chance—it has to demonstrate a deeper understanding and consistent reasoning ability.

Here's how o1 pro mode performed on the 4/4 reliability benchmarks:

Source: OpenAI

We now see bigger differences in all three areas. This indicates that o1 pro mode isn’t just capable of achieving high accuracy—it can do this reliably and consistently.

This focus on reliability is particularly important for tasks where accuracy is critical, such as in scientific research, complex coding, or legal analysis.  o1 pro mode's strong performance on these 4/4 reliability benchmarks suggests that it can be a trusted tool for professionals who need consistent and accurate results.

O1 Pro Mode Use Cases

o1 pro mode offers enhanced accuracy, complex reasoning abilities, and improved reliability compared to previous models.  This makes it well-suited for tasks that require deep analysis, careful consideration, and consistent results. Let's explore some practical applications where o1 pro mode can provide real value.

Scientific research

o1 pro mode can be a valuable asset for scientists working on challenging problems that require advanced reasoning and problem-solving skills. This could include tasks such as analyzing complex datasets, developing and testing hypotheses, and designing experiments. Its ability to process information and identify patterns could lead to breakthroughs in fields like medicine or climate research.

o1 pro mode can also automate time-consuming research tasks, such as literature reviews, data analysis, and report generation. This can free up scientists to focus on more creative and strategic aspects of their work.

Financial modeling and forecasting

Financial analysts and investors rely heavily on accurate data analysis and predictive models. o1 pro mode's ability to process complex financial data, identify trends, and generate reliable forecasts could provide a significant advantage in making informed investment decisions and managing risk effectively.

Legal research and case review

Legal professionals often need to sift through mountains of legal documents and case law to build a strong argument. o1 pro mode can assist by analyzing legal texts, identifying relevant precedents, and summarizing key information, allowing legal teams to focus on strategy and interpretation.

Medical diagnosis and treatment planning

In healthcare, accuracy can be a matter of life and death. o1 pro mode's ability to consistently analyze medical data, identify potential diagnoses, and suggest treatment plans could assist doctors in making more informed decisions, leading to better patient outcomes.

Coding

O1 pro mode can analyze code to identify performance bottlenecks in algorithms and suggest optimizations, assist in debugging by pinpointing potential errors and proposing solutions, and generate code for intricate tasks like implementing complex data structures or building software components. It can refactor code for improved readability and maintainability, automate the creation of boilerplate code and unit tests, and assist in generating clear and concise documentation.

Fraud detection and security systems

Protecting sensitive data and preventing fraud requires reliable systems that can accurately identify threats. o1 pro mode's ability to analyze patterns, detect anomalies, and make accurate predictions could enhance the effectiveness of security and fraud prevention measures.

How to Access O1 Pro Mode

Once you've subscribed to ChatGPT Pro, accessing o1 pro mode is straightforward. Simply select "o1 pro mode" from the model picker within the ChatGPT interface.  Then, you can ask your question or give your instructions as you normally would.

Keep in mind that because o1 pro mode utilizes more compute resources to generate its responses, it might take a bit longer than other models to answer complex questions.  You'll see a progress bar to keep you updated on the process. And if you happen to switch to another conversation while o1 pro mode is working, you'll receive an in-app notification when it's ready.

Source: OpenAI

O1 Pro Mode Safety

While you can read a lot more on o1’s system card, let’s briefly go through the main safety features.

One of the key safety features is the models' ability to reason contextually about safety guidelines. This means they can better understand and adhere to safety policies when responding to potentially harmful prompts. As a result, o1 models demonstrate improved performance in refusing to generate unsafe content, avoiding stereotypes, and resisting attempts to bypass safety rules.

OpenAI employs a combination of techniques to enhance safety:

• Data filtering and refinement: Rigorous filtering processes are applied to the training data to reduce personal information and prevent harmful or sensitive content from being included.  
• Moderation API and safety classifiers: These tools are used to identify and filter out inappropriate or harmful content in both the training data and the model's outputs.  
• Reinforcement learning: Through reinforcement learning, the models learn to refine their responses, recognize mistakes, and adapt their behavior to align with safety expectations.  
• Chain-of-thought reasoning: This technique allows the models to break down complex prompts and reason through their responses more carefully, reducing the likelihood of generating harmful or misleading outputs.

Extensive evaluations and red teaming exercises are conducted to identify and address potential risks before the models are released. OpenAI also emphasizes transparency and collaboration with the AI community to promote safety best practices and continually improve the safety of its models.

Conclusion

We've explored the ins and outs of o1 pro mode, and frankly, we're impressed. However, if we're being honest, o1 pro mode wasn't exactly a surprise since it was the natural successor of o1-preview. The real surprise, in our opinion, is ChatGPT Pro itself.

This new subscription tier is a bit pricey, but given o1 pro mode's reliability, it might be worth the price for those who need it.

That's all for day 1 of "12 days of Open AI." Let's see what day 2 brings to the table.