What is GitOps? A Simple Guide to Automating Infrastructure Management

You have probably heard of DevOps (Developer Operations), which automates application deployment processes. If you are a data professional, you are likely familiar with MLOps (Machine Learning Operations), which streamlines model deployment. But what about GitOps?

In this blog, we will explore GitOps, why it is important, the different GitOps models, and finally, how to integrate GitOps into a large language model project. 

Image by Author

You can also learn the theory behind DevOps and MLOps concepts by taking two of our best short courses, DevOps Concepts and MLOps Concepts.

What Is GitOps?

GitOps is an operational framework that extends DevOps principles to infrastructure automation. It emphasizes key lessons from DevOps, such as version control, collaboration, compliance, CI/CD, and observability, and applies them to the provisioning and management of infrastructure, especially in modern cloud environments.

At its core, GitOps automates infrastructure management by treating configurations as code, often referred to as Infrastructure as Code (IaC). Just as development teams use source code to build application binaries consistently, operations teams use configuration files stored in Git repositories to ensure the same infrastructure environment is deployed every time. This approach guarantees consistency, reliability, and repeatability.

Key Components of a GitOps Workflow:

1. Git repository: Stores both the application’s source code and infrastructure configuration files.
2. Continuous delivery (CD) pipeline: Automates building, testing, and deploying the application and infrastructure changes.
3. Application deployment tool: Ensures applications are deployed correctly and efficiently based on configurations in the Git repository.
4. Monitoring system: Tracks application performance and health to maintain system reliability.

GitOps vs DevOps vs MLOps

Why Use GitOps?

Infrastructure management has traditionally been manual, but modern cloud requires automation to handle the speed and scale of cloud-native applications. With GitOps, infrastructure becomes elastic and reliable, enabling teams to deploy changes quickly and consistently. It eliminates manual errors, improves efficiency, and ensures that infrastructure and applications are always in sync, supporting the fast-paced needs of modern development workflows.

Key Benefits of GitOps:

1. Version control: All changes to infrastructure and applications are version-controlled in Git, allowing for easy rollbacks and comprehensive audit trails.
2. Improved collaboration: Teams can collaborate more effectively using familiar Git workflows, such as pull requests.
3. Increased reliability: Declarative configurations ensure that the desired system state can be automatically restored in the event of a failure.
4. Automation: It minimizes the need for manual intervention, thereby reducing the risk of human error.
5. Security: Git’s commit history provides an immutable record, enhancing security and traceability.

Pull-Based vs Push-Based GitOps Model

In the GitOps approach, there are two primary models for deploying and managing infrastructure and application configurations: pull-based and push-based. These models differ in how they handle configuration changes and apply them to the target environment.

Pull-Based Model (Typical GitOps)

The pull-based approach relies on storing the desired state of the system in a Git repository. A GitOps operator, such as Flux or Argo CD, continuously monitors the repository for changes. When updates are detected, the operator automatically pulls the updated configuration and applies it to the target environment. 

The pull-based model also offers drift detection and self-healing, as the cluster continuously “pulls” the desired state from Git. This ensures that any unintended changes or configuration drift in the environment are automatically corrected, maintaining consistency and stability

Push-Based Model (Using CI/CD Tools)

The push-based approach relies on tools like GitHub Actions or other CI/CD services to push updates to the cluster whenever changes are committed. Unlike the pull-based model, the push-based approach lacks continuous reconciliation, meaning there is no built-in drift detection or automated rollback. This makes it less resilient to unintended changes in the environment. However, the push-based model is often simpler to implement and provides granular control over deployments.

Integrating GitOps into the LLM Project 

We will focus on a push-based GitOps approach using GitHub Actions, as it is simpler to set up and requires minimal prior knowledge. While a push-based system may not offer all the advantages of a pull-based approach, it still provides essential GitOps features such as version-controlled configurations, automated deployments, and simplified rollbacks.

In this section, we will apply GitOps principles to the project from the tutorial How to Deploy LLM Applications Using Docker: A Step-by-Step Guide. This project includes the code for a Gradio application, Dockerfile, and requirements.txt. It demonstrates how to deploy an AI application to the cloud using Docker.

Source: How to Deploy LLM Applications Using Docker: A Step-by-Step Guide

Adopt an MLOps mindset to effectively train, document, maintain, and scale your machine learning models by enrolling in the course, Developing Machine Learning Models for Production with an MLOps Mindset.

Project structure

This GitOps project is designed to deploy Large Language Model (LLM) applications using Docker, Kubernetes, and GitHub Actions. The project organizes all LLM project files into the app folder, creates an infra folder for Kubernetes configurations, and uses .github/workflows/ for CI/CD automation with GitHub Actions.

• Docker ensures the app runs consistently across environments by packaging it with all dependencies.
• Kubernetes automates deployment, scaling, and management of the app.
• GitHub Actions automates testing, building, and deploying the app.

Take the Introduction to Kubernetes course and learn the fundamentals of Kubernetes and deploy and orchestrate containers using Manifests and kubectl instructions.

This structure ensures automated, scalable, and environment-specific deployments for your LLM application, following modern GitOps principles.

Deploying-LLM-Applications-with-Docker/
├── app/
│   ├── requirements.txt      <-- Python dependencies for your LLM app
│   ├── main.py               <-- Your LLM application code
│   └── Dockerfile            <-- Docker instructions for building the app
├── infra/
│   ├── dev/                  <-- Dev environment configs (YAML or Helm)
│   ├── staging/              <-- Staging environment configs
│   └── production/           <-- Production environment configs
└── .github/
    └── workflows/
        ├── ci.yaml           <-- GitHub Actions workflow for continuous integration
        └── cd.yaml           <-- GitHub Actions workflow for continuous deployment

Here is a simple breakdown:

• app/: Contains the application code (main.py), dependencies (requirements.txt), and a Dockerfile to build the app into a container.
• infra/: Holds Kubernetes configurations for different environments:
• dev/, staging/, and production/ folders for environment-specific settings.
• .github/workflows/: Automates CI/CD with GitHub Actions:
• ci.yaml: Builds the app and docker image.
• cd.yaml: Deploys the app by pushing Docker images and applying Kubernetes configs.

Learn the difference between Kubernetes and Docker by reading the Kubernetes vs Docker tutorial.

GitHub Actions workflow overview

This is what happens in the GitHub Actions workflow:

Developer commits code + config to GitHub
               |
               v
      GitHub Actions CI (ci.yaml)
       - Builds Docker image
       - (Optional) pushes Docker image
               |
               v
     GitHub Actions CD (cd.yaml)
       - Deploys updated app/config
       - kubectl apply or helm upgrade
               |
               v
      Kubernetes Cluster Updated

1. Developer Commits Changes: Code and configuration updates are pushed to GitHub.
2. CI Pipeline (ci.yaml):
• Builds the Docker image.
• Optionally pushes the image to the container registry.
• Run tests to validate the application.
3. CD Pipeline (cd.yaml):
• Deploys the updated app or configuration.
• Executes kubectl apply or helm upgrade to update Kubernetes manifests.
4. Kubernetes Cluster Updated: The cluster runs the new version of the application.

Learn how to automate model training, evaluation, versioning, and deployment using GitHub Actions and create a file by following A Beginner's Guide to CI/CD for Machine Learning tutorial.

Key Advantages and Trade-Offs of Using a Push-Based GitOps Approach

If you have prior experience with GitHub Actions, setting up a GitOps methodology using a push-based approach can be straightforward. All you need to do is add the infrastructure configuration files, set up CI/CD pipelines, and you are ready to go. However, while this approach is simple, there are several important trade-offs to consider.

Advantages

• Simple: You only need GitHub Actions to implement this approach—there is no need for an additional GitOps operator like Argo CD or Flux.
• One-Stop-Shop: GitHub Actions provides a single platform to build, test, and deploy your application, streamlining the entire process.

Trade-Offs

• Not true pull-based: Unlike pull-based GitOps tools like Argo CD, this approach lacks "continuous reconciliation”. This means the cluster state is not automatically monitored and reconciled with the desired state in Git.
• No drift detection: If someone manually modifies resources in the cluster, these changes won’t be automatically reverted to match the Git repository. This can lead to configuration drift over time.
• Security: To enable GitHub Actions to deploy to your cluster, you must store cluster credentials in GitHub secrets or use an OIDC-based authentication method. Both approaches require careful handling to avoid security risks.

Transitioning to a pull-based model

As your project scales or your requirements become more demanding, transitioning to a pull-based GitOps model can offer significant advantages. If your team needs features like self-healing, continuous reconciliation, or a visual dashboard, adopting a pull-based tool such as Argo CD or Flux might be the next logical step.

• Self-healing: Revert cluster states automatically if they drift.
• Continuous reconciliation: Watch your Git repo for config changes 24/7.
• Visual dashboard: For checking app statuses and logs.

You can switch to a pull-based tool like Argo CD or Flux. They will constantly monitor your repository and ensure the cluster always matches what is in Git—no manual “push” steps needed after the initial commit.

Conclusion

To effectively implement GitOps, it is best to start small and gradually adopt its technologies. Begin with a simple Dockerfile to containerize your application and deploy it to the cloud. Then, introduce Kubernetes to enable scalability and reliability through automated deployment and management. 

Once comfortable, adopt a push-based GitOps approach using tools like GitHub Actions to automate both cloud infrastructure and application deployment. Finally, as your project matures and requires production-level stability, transition to a pull-based model with tools like Argo CD or Flux to achieve continuous reconciliation, drift detection, and self-healing capabilities. 

This step-by-step progression ensures a smooth learning curve while leveraging GitOps to its full potential for modern cloud-native applications.

In this blog, we have explored GitOps, its importance, the various GitOps models, and how to integrate GitOps into an AI project. If you are new to AI, we highly recommend taking the AI Fundamentals skill track that covers AI fundamentals, delving into models like GPT-4o, and uncovering the secrets of generative AI to navigate the evolving AI landscape.