Skip to main content

OpenAI Announces the Assistants API

Discover the OpenAI Assistants API, designed to simplify AI assistant development. Explore its key features now.
Nov 8, 2023  · 5 min read

At the OpenAI Dev Day, the company made several announcements, including a new Assistants API. See the articles on GPT-4 Turbo and GPTs and the GPT Store for details of the other announcements.

The Assistants API extends the existing OpenAI API to make it easier for software developers to build AI assistants, like chatbots.

Four new features were announced:

  1. "Threads" to help manage longer conversations
  2. "Retrieval" to help store text
  3. Built-in code interpretation
  4. Improvements to the function-calling functionality.

Assistants API Key Features

Here, we'll cover each of these features in more detail.

Easier conversation management with threads

While some tasks can be performed by sending a single prompt to the API and getting a single response back, chatbots require a longer conversation (or "thread"). Until now, the onus was on the developer to keep track of the previous conversation state and decide which of the previous messages to send to the API. As conversations grow in length, this requires sending more and more text in each API call, which slows down performance.

Additionally, once the conversation exceeds the "context window" (the amount of text that GPT can remember at once), decisions need to be made: do you discard older messages, or try to summarize them, or store and make decisions at a later date on which ones are relevant and need to be included? It can quickly get fiddly and slow down application development.

The new threading tools turn the OpenAI from a "stateless" model (no memory) to a "stateful" one. Previous messages can be stored on the OpenAI side, avoiding the developer having to worry about these management issues.

Retrieval tools let you store additional text

The OpenAI presentation also mentioned retrieval tools for storing text. This was billed as a second feature, though since the threading feature appears to make use of this, it's unclear how separate this feature is.

Technical details were sparse during the presentation, though it is possible to take a guess at how this works. Currently, including generative AIino applications requires two different technologies: a large language model (LLM) like GPT, and a vector database.

Vector databases store text (or images or other unstructured data types) as numeric vectors by a process known as embedding. You can retrieve these pieces of text later to include in prompts. A typical use case includes having a store of facts related to the topic you want to discuss.

For example, if you are creating a chatbot to help answer questions about your company's products, you can store the product information in the vector database and use it to ensure that the LLM gives accurate factual responses.

The retrieval features mentioned in the presentation hint that the Assistants API might be allowing you API access to a vector database. This raises interesting possibilities beyond the capabilities for longer conversations since it would mean that you don't need to use a separate vector database such as Pinecone, Milvus, or Weaviate to store your text.

This is speculation at this point, and we shall have to wait for concrete details of the Assistants API to become available.

Code-interpretation is built-in

ChatGPT has an Advanced Data Analysis tool that allows GPT to generate and execute Python code based on natural language instructions. While details in the OpenAI presentation were somewhat vague, it was hinted that this feature will be built into the Assistants API, so you can give prompts that make GPT run Python code.

Improved function calling makes it easier to interact with other software

The function calling feature of the API allows you to write a natural language instruction and have GPT return a JSON string that represents a call to a function that you have defined. This is important for AI agents, which are designed to perform tasks based on a natural language input.

The announcement describes two improvements to this feature. Firstly, there is a "JSON mode" where the response is guaranteed to be valid JSON and should adhere more closely to the specified function signature. Until now, there was a chance that the response would be invalid, requiring thorough error detection code in your software.

The new functionality should simplify the creation of AI agents and make it easier to build natural language interfaces to software.

Summary

While the generative AI revolution in 2023 has been astounding, substantial software engineering skill has been needed to incorporate generative AI into other pieces of software. The Assistants API promises to reduce that barrier to entry somewhat, enabling more products to incorporate the technology faster.

Keep Learning

DataCamp has several courses to teach you how to use the OpenAI API. Start with Working with the OpenAI API and move on to Introduction to Embeddings with the OpenAI API.

You can also learn about the function calling API in the OpenAI Function Calling Tutorial.

Topics

Start Your OpenAI Journey Today!

course

Working with the OpenAI API

3 hr
20.7K
Start your journey developing AI-powered applications with the OpenAI API. Learn about the functionality that underpins popular AI applications like ChatGPT.
See DetailsRight Arrow
Start Course
See MoreRight Arrow
Related

blog

The 12 Best AI Coding Assistants in 2024

Explore the best AI coding assistants, including open-source, free, and commercial tools to enhance your development experience.
Abid Ali Awan's photo

Abid Ali Awan

8 min

blog

Announcing the new DataCamp AI Assistant

Discover the DataCamp AI Assistant—a new way to reach your learning goals faster
DataCamp Team's photo

DataCamp Team

3 min

blog

OpenAI Announces GPTs and ChatGPT Store

Discover the future of AI customization as OpenAI unveils GPTs and the GPT Store. Explore how you can create tailored AI models for specific tasks and learn about the innovative GPT marketplace.
Richie Cotton's photo

Richie Cotton

7 min

tutorial

OpenAI Assistants API Tutorial

A comprehensive overview of the Assistants API with our article, offering an in-depth look at its features, industry uses, setup guidance, and best practices to maximize its potential in various business applications.
Zoumana Keita 's photo

Zoumana Keita

14 min

tutorial

Introduction to Text Embeddings with the OpenAI API

Explore our guide on using the OpenAI API for creating text embeddings. Discover their applications in text classification, information retrieval, and semantic similarity detection.
Zoumana Keita 's photo

Zoumana Keita

7 min

tutorial

Introducing Google Gemini API: Discover the Power of the New Gemini AI Models

Learn how to use Gemini Python API and its various functions to build AI-enabled applications for free.
Abid Ali Awan's photo

Abid Ali Awan

13 min

See MoreSee More