Skip to main content
HomeBlogArtificial Intelligence (AI)

5 Projects You Can Build with Generative AI Models (with examples)

Learn how to use Generative AI models to create an image editor, ChatGPT-like chatbot on low resources, loan approval classifier app, automate PDF Interactions, and GPT-powered voice assistant.
Apr 2023  · 10 min read

An AI juggles tasks

In this post, we will be learning about different types of generative models that we can use to create interesting projects. Moreover, we will also learn how we can use ChatGPT to create an end-to-end data science project. 

Five generative AI projects will improve your machine learning and data science portfolio and make you an attractive candidate for the job application. They will also help you understand the latest tools like Stable Diffusion Inpainting, Segment Anything, Stanford Alpaca, LoRA, LangChain, OpenAI API, and Whisper.

You can learn about the GPT series, including GPT-1, GPT-2, GPT-3, and GPT-4, by reviewing the article 'What is GPT-4 and Why Does it Matter?' You can also view some other AI projects in a separate post. 

1. StableSAM: Stable Diffusion Inpainting with Segment Anything

In this project, you will use Meta’s segment-anything, Hugging Face diffusers, and Gradio to create an app that can change the background, face, clothes or anything you select. It just wants the image, selected area, and prompt. 

Create Stable Diffusion Inpainting Pipeline

We will create a Stable Diffusion Inpainting pipeline using diffuser and model weights available on stabilityai/stable-diffusion-2-inpainting · Hugging Face. After that, we will add it to “cuda'' for GPU acceleration. 

Defining Image Mask and Inpainting Function

The image mask function is created using SAM Predictor. It takes an image, selected image section, and is_backgroud boolean value to create masked image and segmentation. 

After that, the inpainting function uses the Stable Diffusion Inpainting pipeline to change selected parts of the images. The pipelines require input image, masked image, segmented image, prompt text, and negative prompt text.  

Creating Gradio UI

You will create a row and add three image blocks. For a minimal viable product, you have to add another row with a submit button. After that, you have to modify the input image object to select pixels, generate mask and segmentation, and add action to the submit button to run the inpainting function.

Gradio is quite easy to learn. You can learn everything by reading the Gradio Docs

Improved Version of StableSAM

The improved version of StableSAM is available at hugging face, which includes a customized inpainting pipeline that uses ControlNet. It uses Runway ML Stable Diffusion Inpainting instead of Stability AI. 

StableSAM package

As you can see, the final version app looks clean with a segmentation block, clean button, background option, and negative prompt. 

StableSAM demo


2. Alpaca-LoRA: Build ChatGPT-like with Minimal Resource 

The Alpaca-LoRA provides all of the necessary components for you to create your own specialized ChatGPT-like chatbot using a single GPU. 

In this section, we will look at the initial setup, training, inference script, and native client for running inference on the CPU.

Local Setup

  1. Clone repository: tloen/alpaca-lora
  2. Install dependencies using pip install -r requirements.txt
  3. If bitsandbytes doesn't work, install it from the source


In this part, we will look at the fine-tuning script that you can run on the LLaMA model using the cleaned Stanford Alpaca model. 

You can look at the repository to tweak the hyperparameters for better performance.

python \
    --base_model 'decapoda-research/llama-7b-hf' \
    --data_path 'yahma/alpaca-cleaned' \
    --output_dir './lora-alpaca'


The inference script reads the foundation LLaMA model from Hugging Face and loads LoRA weights to run a Gradio interface. 

python \
    --load_8bit \
    --base_model 'decapoda-research/llama-7b-hf' \
    --lora_weights 'tloen/alpaca-lora-7b'

You can also use alpaca.cpp for running alpaca models on CPU or 4GB RAM Raspberry Pi 4. Furthermore, you can use Alpaca-LoRA-Serve to create a ChatGPT-style interface, as shown below. 

Alpaca-LoRA-Serve Screenshot


3. Automating PDF Interaction with LangChain and ChatGPT

Create your own ChatPDF clone using LangChain’s PDF loader, OpenAI Embeddings, and GPT-3.5. You will create a chatbot that can communicate with your book, legal documentation, and other important PDF documents. 

Loading the document

We will use the LangChain document loader to load PDFs and read the contents.

from langchain.document_loaders import PyPDFLoader

pdf_path = "./paper.pdf"
loader = PyPDFLoader(pdf_path)
pages = loader.load_and_split()

Creating embeddings and Vectorization

We will create embeddings using OpenAIEmbeddings class from LangChain API. After that, pass these embeddings to the Chroma class to create a vector database for the PDF document. 

from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import Chroma

embeddings = OpenAIEmbeddings()
vectordb = Chroma.from_documents(pages, embedding=embeddings, persist_directory=".")


Querying the PDF

We will use the ChatVectorDBChain class to interact with ChatGPT using a generated vector database. 

from langchain.chains import ChatVectorDBChain
from langchain.llms import OpenAI

pdf_qa = ChatVectorDBChain.from_llm(
    OpenAI(temperature=0.9, model_name="gpt-3.5-turbo"),

query = "What is the VideoTaskformer?"
result = pdf_qa({"question": query, "chat_history": ""})

The next step is not mentioned in the video tutorial. You can use the Gradio framework to create a web app and share it with your colleagues and friends. 

Screenshot from youtube tutorial


4. Bing-GPT Voice Assistant

Build your own AI-powered personal assistant just like J.A.R.V.I.S. For that, you will need OpenAI API, text-to-speech library, speech recognition library, and generative AI. 

After loading the required libraries, you have to provide the OpenAI API key:

openai.api_key = "[paste your OpenAI API key here]"

Wake word

Create a wake word function to activate the AI. In this case, the developer is using “‘bing” or “gpt”.

Speech Synthesis

Speech synthesis function provides text-to-speech inference. You can access the polly (text-to-speech) from boto3 and play the audio using pydub.

Transcription with Whisper

The speech recognition is done by openai/whisper. You just have to figure out the API to add it to your application. 

ChatBot with EdgeGPT

In the end, you will use acheong08/EdgeGPT and OpenAI API to create a chatbot. If the user uses the wake word “bing'' it will use the EdgeGPT model and else the ChatGPT model. 

If you are looking for a much simpler implementation of Voice Assistant, check out OpenAI Whisper, ChatGPT, TTS, and Gradio Web UI

Image from youtube tutorial


5. An End-to-End Data Science Project

In this project, we will use ChatGPT to work on end-to-end loan approval classifier applications. All we need is access to the ChatGPT interface and a personal machine to run the code. 

Project Planning

In the planning phase, we will describe the dataset and what we want from it. Sometimes the answers are not perfect, but you can tweak the response by providing follow-up prompts. 

After that, we will start following the brief plan. 

Exploratory Data Analysis (EDA) 

We will ask ChatGPT to generate Python code that will load the dataset and perform exploratory data analysis with various visualization techniques. You can even ask it to interpret the results.

Feature Engineering

We have asked ChatGPT to write a feature engineering code, and amazingly it has created two features from existing features. It means that AI now fully understands the dataset. 

Preprocessing and Balancing the Data

We have an imbalanced dataset, and in this part, we will use ChatGPT to generate preprocessing and class balancing code. 

Model Selection

We have just asked ChatGPT to write the model selection code by specifying the machine-learning models. After running the code, we will select the best-performing model. 

Hyperparameter Tuning and Model Evaluation

To improve the performance, we will ask ChatGPT to write Python code for hyperparameter tuning and model evaluation, and save the best-performing model.

Creating a Web App using Gradio

We will ask ChatGPT to write Gradio app code using the saved model and preprocessing. The AI understood the input features and output results. As a result, we got a fully functioning web app. 

Deploying the Web App on Spaces

At the final step, we have asked ChatGPT to deploy a web app on space. It has provided a few steps that we can follow and deploy in a few minutes. 

Loan Approval Classifier App

This project will also teach tips on writing effective prompts, which is becoming essential for every field. 



This is just a start of what’s to come with generative AI models. The open-source community is working hard to develop tools that will help you build any type of AI. You can use these tools to even create AGI (Artificial general AI); check out Auto-GPT (experimental open-source for creating GPT-4 fully autonomous) and babyagi (AI-powered task management system). 

In this post, we have covered the projects that can be understood by all levels and require fewer resources to get started. They all use open-source tools, models, datasets, and packages that are available for anyone to use. 

If you're new to ChatGPT, consider taking an Introduction to ChatGPT course. Alternatively, if you're already familiar with generative AI, you can improve your prompting skills by reviewing the comprehensive ChatGPT Cheat Sheet for Data Science, or by checking out the following resources:


ChatGPT vs Google Bard: A Comparative Guide to AI Chatbots

A beginner-friendly introduction to the two AI-powered chatbots everyone is talking about.
Javier Canales Luna's photo

Javier Canales Luna

17 min

10 of The Best ChatGPT Plugins to Get The Most From AI in 2023

Unlock the full potential of ChatGPT with our expert guide on the top 10 plugins for 2023. Enhance productivity, streamline workflows, and discover new functionalities to elevate your ChatGPT experience.
Matt Crabtree's photo

Matt Crabtree

12 min

Adapting to the AI Era with Jason Feifer, Editor in Chief of Entrepreneur Magazine

Jason and Adel explore AI’s role in entrepreneurship, use cases and applications of AI, AI’s impact on established business models, frameworks for navigating change and much more. 
Adel Nehme's photo

Adel Nehme

45 min

Fighting for Algorithmic Justice with Dr. Joy Buolamwini, Artist-in-Chief and President of The Algorithmic Justice League

Richie and Joy discuss her journey into AI, the ethics of AI, the Aspire Mirror and Gender Shades projects, The Algorithmic Justice League, balancing working in AI and data while being an artist, and much more.
Richie Cotton's photo

Richie Cotton

54 min

Markdown Cheat Sheet

Learn everything you need to know about Markdown in this convenient cheat sheet!
Richie Cotton's photo

Richie Cotton

Introduction to Falcon 40B: Architecture, Training Data, and Features

In this post, we will explore the architecture, training data, features, and how to run inference and fine-tune the Falcon 40B model.
Abid Ali Awan's photo

Abid Ali Awan

9 min

See MoreSee More