AI Infrastructure: Storage Options

This module details the role of storage infrastructure in the AI data pipeline. It covers performance demands, key Google Cloud solutions, and the decision criteria for selecting a service based on capacity, throughput, and latency.

This module details the critical phases of data preparation and model training within the AI workflow. It covers optimizing data loading using Cloud Storage, Anywhere Cache, and the Dataflux Dataset tool, while comparing high-performance file systems like Cloud Storage FUSE and Managed Lustre. Additionally, it outlines decision criteria for efficient checkpointing strategies to ensure fault tolerance and minimize GPU idle time.

This module details strategies for AI model serving and data archiving. It covers selecting storage—Managed Lustre, Cloud Storage, or Hyperdisk ML—based on scale and latency, and optimization techniques, like GKE Image Streaming and Cloud Storage FUSE, to minimize costs and load times.

Student PDF links to all modules