After doing these courses, I feel confident creating professional visualizations and dashboards
Shoebur RahmanProcurement Specialist, Giz Bangladesh
Course Description
Harness the Power of Multi-Modal AI
Dive into the cutting-edge world of multi-modal AI models, where text, images, and speech combine to create powerful applications. Learn how to leverage Hugging Face's vast repository of models that can see, hear, and understand like never before. Whether you're analyzing social media content, building voice assistants, or creating next-generation AI applications, multi-modal models are your gateway to handling diverse data types seamlessly.Master Essential Multi-Modal Techniques
Explore state-of-the-art models like CLIP for image-text understanding, SpeechT5 for voice synthesis, and the Qwen2 Vision Language model for multi-modal sentiment analysis. Through hands-on exercises, you'll master the techniques used by leading AI companies to build sophisticated multi-modal systems.Future-Proof Your AI Skills
This course will give you a robust toolkit for handling multi-modal AI tasks. You'll learn to process and combine different data modalities effectively, fine-tune pre-trained models for custom applications, and evaluate and improve model performance across modalities.Prerequisites
Curriculum
Course outline
1
Accessing Hugging Face Models and Datasets
Navigate the Hugging Face model hub, transform raw text, audio, and visual data into AI-friendly formats. Learn how to find the latest most popular models for tasks such as text generation and harness the power of pre-built pipelines.
- Hugging Face model navigation50 XP
- How many models!?100 XP
- Finding the most popular text-to-image model100 XP
- Preprocessing different modalities50 XP
- Text tokenizing100 XP
- Image preprocessing100 XP
- Audio preprocessing100 XP
- Pipeline tasks and evaluations50 XP
- Pipeline caption generation100 XP
- Passing keyword arguments100 XP
- Model evaluation on a custom dataset100 XP
2
Unimodal Vision, Audio, and Text Models
Learn to master individual modalities with state-of-the-art models. Dive into computer vision for image classification and segmentation, explore speech recognition and text-to-speech synthesis, and learn effective fine-tuning techniques. Build practical skills with pre-trained models from Hugging Face's transformers library.
3
Multi-Modal Models for Classification
Learn to fuse visual, textual, and audio information for richer AI applications. Master techniques like CLIP for zero-shot classification, build sentiment analyzers that see and read, and create emotion detectors that combine facial expressions with voice. Take your AI models beyond single-modality thinking.
4
Multi-Modal Generation
Transform ideas into reality! Master cutting-edge AI techniques to generate and manipulate visual content using text prompts. Create stunning images, edit photos intelligently, and build powerful question-answering systems for images and documents. Turn your creative vision into digital reality with multi-modal AI.
Multi-Modal Models with Hugging Face
Course
Complete
