8 Top Open-Source LLMs for 2024 and Their Uses

The current generative AI revolution wouldn’t be possible without the so-called large language models (LLMs). Based on transformers, a powerful neural architecture, LLMs are AI systems used to model and process human language. They are called “large” because they have hundreds of millions or even billions of parameters, which are pre-trained using a massive corpus of text data.

Start our Large Language Models (LLMs) Concepts Course today to learn more about how LLMs work.

LLM are the foundation models of popular and widely-used chatbots, like ChatGPT and Google Bard. In particular, ChatGPT is powered by GPT-4, a LLM developed and owned by OpenAI, while Google Bard is based on Google’s PaLM 2 model.

ChatGPT and Bard, as well as many other popular chatbots, have in common that their underlying LLM are proprietary. That means that they are owned by a company and can only be used by customers after buying a license. That license comes with rights, but also with possible restrictions on how to use the LLM, as well as limited information on the mechanisms behind the technology.

Yet, a parallel movement in the LLM space is rapidly gaining pace: open-source LLMs. Following rising concerns over the lack of transparency and limited accessibility of proprietary LLMs, mainly controlled by Big Tech, such as Microsoft, Google, and Meta, open-source LLMs promise to make the rapidly growing field of LMMs and generative AI more accessible, transparent, and innovative.

This article aims to explore the top open-source LLMs available in 2023. Although it’s been only a year since the launch of ChatGPT and the popularization of (proprietary) LLMs, the open-source community has already achieved important milestones, with a good number of open-source LLMs available for different purposes. Keep reading to check the most popular ones!

Benefits of Using Open-Source LLMs

There are multiple short-term and long-term benefits to choosing open-source LLMs instead of proprietary LLMs. Below, you can find a list of the most compelling reasons:

Enhanced data security and privacy

One of the biggest concerns of using proprietary LLMs is the risk of data leaks or unauthorized access to sensitive data by the LLM provider. Indeed, there have already been several controversies regarding the alleged use of personal and confidential data for training purposes.

By using open-source LLM, companies will be solely responsible for the protection of personal data, as they will keep full control of it.

Cost savings and reduced vendor dependency

Most proprietary LLMs require a license to use them. In the long term, this can be an important expense that some companies, especially SME ones, may not be able to afford. This is not the case with open-source LLMs, as they are normally free to use.

However, it’s important to note that running LLMs requires considerable resources, even only for inference, which means that you will normally have to pay for the use of cloud services or powerful infrastructure.

Code transparency and language model customization

Companies that opt for open-source LLMs will have access to the workings of LLMs, including their source code, architecture, training data, and mechanism for training and inference. This transparency is the first step for scrutiny but also for customization.

Since open-source LLMs are accessible to everyone, including their source code, companies using them can customize them for their particular use cases.

Active community support and fostering innovation

The open-source movement promises to democratize the use and access of LLM and generative AI technologies. Allowing developers to inspect the inner workings of LLMs is key for the future development of this technology. By lowering entry barriers to coders around the world, open-source LLMs can foster innovation and improve the models by reducing biases and increasing accuracy and overall performance.

Addressing the environmental footprint of AI

Following the popularization of LLMs, researchers and environmental watchdogs are raising concerns about the carbon footprint and water consumption required to run these technologies. Proprietary LLMs rarely publish information on the resources required to train and operate LLMs, nor the associated environmental footprint.

With open-source LLM, researchers have more chances to know about this information, which can open the door for new improvements designed to reduce the environmental footprint of AI.

8 Top Open-Source Large Language Models For 2024

1. LLaMA 2

Most top players in the LLM space have opted to build their LLM behind closed doors. But Meta is making moves to become an exception. With the release of its powerful, open-source Large Language Model Meta AI (LLaMA) and its improved version (LLaMA 2), Meta is sending a significant signal to the market.

Realized for research and commercial use in July 2023, LLaMA 2 is a pre-trained generative text model with 7 to 70 billion parameters. It has been fine-tuned with Reinforcement learning from human feedback (RLHF). It is a generative text model that can be used as a chatbot and can be adapted for a variety of natural language generation tasks, including programming tasks. Meta has already launched to open, customized versions of LLaMA 2, Llama Chat, and Code Llama.

To learn more about LLaMA, check out our Introduction to Meta AI’s LLaMA and our Fine-Tuning LLaMA 2 article.

2. BLOOM

Launched in 2022 following a year-long collaborative project with volunteers from 70+ countries and researchers from Hugging Face, BLOOM is an autoregressive LLM trained to continue text from a prompt on vast amounts of text data using industrial-scale computational resources.

The release of BLOOM marked an important milestone in democratizing generative AI. With 176 billion parameters, BLOOM is one of the most powerful open-source LLMs, with capabilities to provide coherent and accurate text in 46 languages and 13 programming languages.

Transparency is the backbone of BLOOM, a project where everyone can access the source code and the training data in order to run, study, and improve it.

BLOOM can be used for free through the Hugging Face ecosystem.

3. BERT

The underlying technology of LLM is a type of neural architecture called a transformer. It was developed in 2017 by Google researchers in the paper Attention is All You Need. One of the first experiments to test the potential of transformers was BERT.

Launched in 2018 by Google as an open-source LLM, BERT (stands for Bidirectional Encoder Representations from Transformers), rapidly achieved state-of-the-art performance in many natural language processing tasks.

Thanks to its innovative features back in the early days of LLMs and its open-source nature, Bert is one of the most popular and widely used LLMs. For example, in 2020, Google announced that it had adopted Bert through Google Search in over 70 languages.

There are currently thousands of open-source, free, and pre-trained Bert models available for specific use cases, such as sentiment analysis, clinical note analysis, and toxic comment detection.

Interested in the possibilities of BERT? Check out our Introduction to BERT article.

4. Falcon 180B

If the Falcon 40B already impressed the open-source LLM community (it ranked #1 on Hugging Face’s leaderboard for open-source large language models), the new Falcon 180B suggests that the gap between proprietary and open-source LLMs is rapidly closing.

Released by the Technology Innovation Institute of the United Arab Emirates in September 2023, Falcon 180B is being trained on 180 billion parameters and 3.5 trillion tokens. With this impressive computing power, Falcon 180B has already outperformed LLaMA 2 and GPT-3.5 in various NLP tasks, and Hugging Face suggests it can rival Google’s PaLM 2, the LLM that powers Google Bard.

Although free for commercial and research use, it’s important to note that Falcon 180B requires important computing resources to function.

5. OPT-175B

The release of the Open Pre-trained Transformers Language Models (OPT) in 2022 marked another important milestone in Meta’s strategy to liberate the LLM race through open source.

OPT comprises a suite of decoder-only pre-trained transformers ranging from 125M to 175B parameters. OPT-175B, one of the most advanced open-source LLMs in the market, is the most powerful brother, with similar performance to GPT-3. Both pre-trained models and the source code are available to the public.

Yet, if you’re thinking in developing an AI-driven company with LLMs, you’d better think in another one, as OPT-175B is released under a non-commercial license, allowing only the use of the model for research use cases.

6. XGen-7B

More and more companies are jumping into the LLM race. One of the last to jump into the ring was Salesforce, which launched its XGen-7B LLM in July 2023.

According to the authors, most open-source LLMs focus on providing large answers with limited information (i.e., short prompts with little context). The idea behind XGen-7B is to build a tool that supports longer context windows. In particular, the most advanced variance of XGen (XGen-7B-8K-base) allows for an 8K context window, that is, the cumulative size of the input and output text.

Efficiency is another important priority in XGen, which uses only 7B parameters for training, way less than most powerful open-source LLMs, like LLaMA 2 or Falcon.

Despite its relatively small size, XGen can still deliver great results. The model is available for commercial and research purposes, except theXGen-7B-{4K,8K}-inst variant, which has been trained on instructional data and RLHF and is released under a noncommercial license.

7. GPT-NeoX and GPT-J

Developed by researchers from EleutherAI, a non-profit AI research lab, GPT-NeoX and GPT-J are two great open-source alternatives to GPT.

GPT-NeoX has 20 billion parameters, while GPT-J has 6 billion parameters. Although most advanced LLMs can be trained with over 100 billion parameters, these two LLMs can still deliver results with high accuracy.

They have been trained with 22 high-quality datasets from a diverse set of sources that enable their use in multiple domains and many use cases. In contrast with GPT-3, GPT-NeoX and GPT-J haven’t been trained with RLHF.

Any natural language processing task can be performed with GPT-NeoX and GPT-J, from text generation and sentiment analysis to research and marketing campaign development.

Both LLMs are available for free through the NLP Cloud API.

8. Vicuna 13-B

Vicuna-13B is an open-source conversational model trained from fine-tuning the LLaMa 13B model using user-shared conversations gathered from ShareGPT.

As an intelligent chatbot, the applications of Vicuna-13B are countless, and some of them are illustrated below in different industries, such as customer service, healthcare, education, finance, and travel/hospitality.

A preliminary evaluation using GPT-4 as a judge showed Vicuna-13B achieving more than 90% quality of ChatGPT and Google Bard, then outperformed other models like LLaMa and Alpaca in more than 90% of cases.

Choosing the Right Open-Source LLM for Your Needs

The open-source LLM space is rapidly expanding. Today, there are many more open-source LLMs than proprietary ones, and the performance gap may be bridged soon as developers worldwide collaborate to upgrade current LLMs and design more optimized ones.

In this vibrant and exciting context, it may be difficult to choose the right open-source LLM for your purposes. Here is a list of some of the factors you should think about before opting for one specific open-source LLM:

• What do you want to do? This is the first thing you have to ask yourself. Open-source LLM are always open, but some of them are only released for research purposes. Hence, if you’re planning to start up a company, be aware of the possible licensing limitations.
• Why do you need a LLM? This is also extremely important. LLMs are currently in vogue. Everyone’s speaking about them and their endless opportunities. But if you can build your idea without needing LLMs, then don’t use them. It’s not mandatory (and you will probably save a lot of money and prevent further resource use).
• How much accuracy do you need? This is an important aspect. There is a direct relationship between the size and accuracy of state-of-the-art LLMs. This means, overall, that the bigger the LLM in terms of parameters and training data, the more accurate the model will be. So, if you need high accuracy, you should opt for bigger LLMs, such as LLaMA or Falcon.
• How much money do you want to invest? This is closely connected with the previous question. The bigger the model, the more resources will be required to train and operate the model. This translates into additional infrastructure to be used or a higher bill from cloud providers in case you want to operate your LLM in the cloud. LLMs are powerful tools, but they require considerable resources to use them, even open-source ones.
• Can you achieve your goals with a pre-trained model? Why invest money and energy in training your LLM from scratch if you can simply use a pre-trained model? Out there there are many versions of open-source LLMs trained for a specific use case. If your idea fits in one of these use cases, just for it.

Conclusion

Open-source LLMs are in an exciting movement. With their rapid evolution, it seems that the generative AI space won’t necessarily be monopolized by the big players who can afford to build and use these powerful tools.

We’ve only seen eight open-source LLMs, but the number is much higher and rapidly growing. We at DataCamp will continue to provide information about the latest news in the LLM space, providing courses, articles, and tutorials about LLMs. For now, check out our list of curated materials:

• Large Language Models (LLMs) Concepts Course
• How to build LLM applications with LangChain
• How to train an LLM with PyTorch
• LlamaIndex: Adding Personal Data to LLMs
• The Pros and Cons of Using LLMs in the Cloud Versus Running LLMs Locally