Azure Pipelines Tutorial: A Complete CI/CD Guide for Beginners

If your CI/CD pipeline keeps breaking or feels slow and clunky, you’re not alone. I’ve dealt with failed builds, mismatched environments, and teammates accidentally overwriting my work. Azure Pipelines takes away these frustrations with a cloud-hosted service that can build, test, and deploy almost any project on any platform.

That’s why in this tutorial, I’ll show you how to:

1. Create your first Azure Pipeline
2. Build your project in the cloud
3. Add automated tests to catch errors early
4. Deploy to staging or production with a single push
5. Monitor and optimize your pipeline for speed and reliability
6. Follow best practices proven on real-world projects

Let’s get started!

What is Azure Pipelines?

Azure Pipelines is a cloud-based solution that simplifies application development, testing, and deployment. 

It's a component of the Azure DevOps toolkit, which supports the entire software development lifecycle. It is also a flexible option for teams of all sizes because it supports various platforms and programming languages. GitHub, Azure Repos, and other well-known version control systems are integrated with it. 

The core features of Azure Pipelines include: 

In my experience, Azure Pipelines is helpful for anyone wanting to have proper CI/CD within their team. It is flexible, scalable, and integrates well with other Azure services.

Azure Pipelines Architecture (Source: Microsoft).

If you're looking for a broader understanding of Azure DevOps beyond pipelines, this Azure DevOps tutorial walks through the whole CI/CD lifecycle from start to finish.

Setting Up Your First Azure Pipeline

To work with the Azure Pipelines environment, we must first set up everything to foster an end-to-end build, test, and deployment process.

Prerequisites

Before diving into creating your first pipeline, ensure you have the following:

1. Azure DevOps account and organization setup: Sign up for an Azure DevOps account if you don’t already have one. Create an organization to manage your projects. 
2. Repository: You’ll need a code repository with a sample project. This can be hosted in Azure Repos or GitHub. For this tutorial, we’ll use a simple Node.js project.
3. Azure portal access: You’ll also need access to the Azure portal when you want to deploy to Azure services like Azure Kubernetes Service (AKS) and many more.

If you need to brush up on the broader Azure ecosystem, I recommend checking out the Understanding Microsoft Azure course. 

Creating a new project and repository

Now that your Azure DevOps account and organization are all set up, let’s create our first project and repository.

• Create a new project:
  • Log in to your Azure DevOps account.
  • Click on New Project, give it a name (e.g., "AzureDevOps-DataCamp-Tutorial"), and choose the visibility (public or private).
  • Click Create Project.

• Set up a repository:
  • If using Azure Repos, navigate to the Repos section and initialize a new repository.
  • Alternatively, you can link an existing GitHub repository by selecting Import Repository.
  • Name your repository, choose your Repository type (Git or TFVC), paste your cloned repository URL, and click Import.

• If you’re using GitHub, you can link your GitHub account to Azure DevOps by going to the bottom left of your project interface and selecting Project Settings > GitHub Connections > Connect your GitHub account.

For this tutorial, we’ll use Azure Repos as our code repository. If you're just starting out or don’t have a specific reason for choosing TFVC, I advise you to choose Git.

Creating your first pipeline

It’s time to create our first pipeline. Follow the steps below:

• In your Pipelines section, click Create Pipeline.

• Select Azure Repos Git as the source and choose your repository.

• You can configure your pipeline using the YAML or Classic editor. Let’s go with YAML, a code-based way of defining the pipeline.
• Save the pipeline and run it to see if everything works correctly.

Choosing YAML vs Classic Editor

Azure Pipelines provides two ways to define your pipelines:

• YAML pipelines: A code-based way to describe your pipeline. They are defined in a .yaml or .yml file in your source repository, enabling versioning and better collaboration.
• Classic Editor: Classic Editor is a user-friendly, visual drag-and-drop interface that is less flexible for complex scenarios.

In my experience, YAML pipelines provide more flexibility and are better for teams following Infrastructure as Code (IaC) practices. But this Classic Editor is perfect for fresher with a more visual way to do things. 

However, for long-term scalability, YAML pipelines are recommended over Classic Editor.

Building with Azure Pipelines

Now, let’s see how we can build with Azure Pipelines. 

In this section, we will specify the steps Azure Pipelines takes to build your code, such as installing dependencies, compiling, and running tests. We are going to express these steps in a configuration file in our source code repo using YAML, a simple, human-readable way to describe steps in a recipe.

YAML pipeline syntax

Here’s a simple example of a YAML pipeline:

# This section defines the trigger(s) for the pipeline.
# 'main' specifies that the pipeline will run whenever changes are pushed to the 'main' branch.
trigger:
- main  # automatically triggers this pipeline when changes are made to the 'main' branch.

# This defines the pool where the pipeline will run.
# 'ubuntu-latest' specifies the virtual machine image for executing tasks.
pool:
  vmImage: 'ubuntu-latest'  # This uses the latest Ubuntu image provided by Azure DevOps.

# Steps contain the individual tasks or actions that the pipeline will execute.
steps:
- script: echo Hello, world!  # A script task to run shell commands. This one prints "Hello, world!".
  displayName: 'Run my first pipeline script'  # A friendly name for this step, making it easier to identify in the pipeline logs.

Let’s break down the code step by step:

• trigger: Specifies the branch that triggers the pipeline (e.g., main).
• pool: Defines the build agent (e.g., ubuntu-latest).
• steps: Lists the tasks to execute, such as installing dependencies and running a build script.

Configuring build steps

To add build steps to your YAML pipeline, you must define the ‘script’ under the ‘steps’ section.

For a simple Node.js application, for instance, our azure-pipelines.yml file below contains code that Azure DevOps automatically created.

# Node.js
# Build a general Node.js project with npm.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://docs.microsoft.com/azure/devops/pipelines/languages/javascript

trigger:
- main

pool:
  vmImage: 'ubuntu-latest'

steps:
- task: NodeTool@0
  inputs:
    versionSpec: '20.x'
  displayName: 'Install Node.js'

- script: |
    npm install
    npm run build
  displayName: 'npm install and build'

Let’s take a look at the build steps in the YAML pipeline above: 

1. Install Node.js: Use the NodeTool@0 task to specify the Node.js version. 
2. Install dependencies: Run npm install to install project dependencies. 
3. Compile code (build the project): Run npm run build to compile the code. 

Running the pipeline

Our pipeline can be triggered manually or set to run automatically when events like branch pushes occur.

• Manual trigger: Click on Run Pipeline in the Azure DevOps interface to trigger the pipeline manually.

• Automatic trigger: Set up triggers based on repository events to push changes to the main branch to trigger the pipeline automatically. Here’s an example:

trigger:
- main

After running your pipeline, you can view logs and troubleshoot any problems.

To access logs, go to your pipeline run summary and select the specific job or task. This will display the logs for that step; just like the image below, you can choose any jobs (Build, Push, or Update) to view the raw logs.

To troubleshoot common issues, you can check your pipeline for common problems like timeouts, resource constraints, or misconfigurations.

Adding Tests to Your Pipeline

You add tests to your pipeline to ensure code quality and catch issues early in development. In Azure Pipelines, you can integrate units, integration, or other tests by adding steps in your pipeline configuration.

Configuring test steps

Include testing steps in your pipeline to guarantee code quality. You can use popular frameworks like Jest, Mocha, or NUnit. For example, here’s how to add unit tests with Jest:

- script: |
    npm test
  displayName: 'Run unit tests with Jest'

Viewing test results

There is built-in test reporting in Azure Pipelines. You can set up test result publishing for simpler tracking and view test results in the pipeline logs. This is how:

• First, navigate to Builds > Test Results.
• To publish test results, use the PublishTestResults@2 task to display results. This makes the overall tracking easier. For example:

- task: PublishTestResults@2
  inputs:
    testResultsFiles: '**/test-results.xml'
    testRunTitle: 'Unit Tests'

Deploying with Azure Pipelines

In the deployment steps in your Azure pipeline YAML file, you can automatically deploy your application to several environments, such as testing, staging, and production. In this section, we will review a few ways to deploy with Azure Pipelines.

Deploying to Azure Web Apps

Azure Web Apps is a service in the Azure Portal that enables you to develop, deploy, and manage web applications at scale. 

To deploy to Azure Web Apps, you must link the two services. Here’s how to:

• At the bottom left of your project, go to Project Settings and click Service Connections under the Pipelines section. 

• Click New Service Connection and choose Azure Resource Manager. 

• Authenticate with your Azure account.
• Deploy to Azure App Service:
  • Go back to your Pipelines and configure your pipeline.
  • In your pipeline, add a task to deploy to Azure.
  • Specify the resource group and service details. 

Here is an example of a YAML deployment pipeline for a simple Node.js app to Azure App Service:

# Define the task for deploying to an Azure Web App
- task: AzureWebApp@1
  inputs:
    # This specifies your Azure subscription service connection
    azureSubscription: 'your-azure-subscription'
    # This specifies the name of your web app
    appName: 'your-app-name'
    # This specifies the location of the app package or files to be deployed
    package: '$(Build.ArtifactStagingDirectory)'

Let’s break down the code above, so you can adapt it for your use case:

• Replace your-azure-subscription with the name of the service connection that gives access to your Azure subscription.
• Replace your-app-name with the actual name of your web app in Azure. 
• Replace $(Build.ArtifactStagingDirectory) with the directory to put the web app package or files for deployment. $(Build.ArtifactStagingDirectory) — This variable points to the folder where your build artifacts (your app files) sit after they have been built.

Setting up continuous deployment (CD)

Once everything in our code passes all the tests and checks, the next step is to set up our continuous deployment. Follow the steps described below.

Azure release pipeline deployment model (Source: Microsoft)

• Create a release pipeline:
• Go to Pipelines > Releases and click New Pipeline.

• Select the Azure App Service Deployment template.

• Configure environments:
• Add and configure Dev, Staging, QA, and Production environments where you want your application to be deployed.

• Link the artifact from your build pipeline as the source for the release pipeline by selecting Add an artifact.

• Configure approvals and gates:
• Setting up pre-deployment approvals for sensitive environments, like production environments, is necessary. For example, a team lead may be required to approve the release. You can also add gates like automated tests or monitoring checks to ensure the deployment meets quality standards.

In my experience, this phase is necessary for maintaining stability in demanding development settings. Approvals and gates provide control and ensure that only high-quality code reaches production.

Deploying to Kubernetes

Kubernetes makes it easy to run reliable and scalable containerized applications. When using Azure Kubernetes Service (AKS), you get a managed environment that simplifies deployment and operations. In this section, I’ll walk through how to deploy your application to AKS using Azure Pipelines.

Prepare your AKS Cluster

You can create an AKS cluster using the Azure portal or CLI. Here is a basic Bash CLI command to provision cluster:

az aks create \
  --resource-group <ResourceGroupName> \
  --name <AKSClusterName> \
  --node-count 3 \
  --enable-addons monitoring \
  --generate-ssh-keys

Let’s break down the command above:

• --resource-group: This indicates the Azure resource group in which the AKS cluster will be created.
• --name: This refers to the name of your AKS cluster.
• --node-count: This sets the number of nodes in the cluster (e.g., 3 in this example).
• --enable-addons monitoring: This makes monitoring and analytics possible with Azure Monitor for Containers.
• --generate-ssh-keys: This automatically generates SSH keys for secure node access if not provided.

Don’t forget to replace <ResourceGroupName> and <AKSClusterName> with your actual resource group and cluster names.

Kubectl needs to be configured after your AKS cluster is set up. This ensures your pipeline agent or local computer can use kubectl to communicate with the cluster.

You can speed up your Azure tasks using command line tools—this Azure CLI cheat sheet is a handy reference.

Create a deployment YAML file 

Let’s take a basic deployment YAML file for deploying a Docker image to Kubernetes as an example:

# This defines the API version and the kind of Kubernetes resource we are creating.
# 'apps/v1' is the API version, and 'Deployment' is the resource type.
apiVersion: apps/v1
kind: Deployment

# Metadata contains information to identify the resource uniquely.
# 'name' specifies the name of the Deployment.
metadata:
  name: my-app  # Name of the Deployment, used for identification.

# "spec" specifies the intended deployment state.
spec:
  # Number of replicas/pods we want for this Deployment.
  replicas: 3  # Ensures high availability by running three app replicas.

  # Selector is used to define the labels that this Deployment will manage.
  selector:
    matchLabels:
      app: my-app  # This ensures that only Pods with the label 'app: my-app' are managed by this Deployment.

  # Template specifies the blueprint for the pods created by the Deployment.
  template:
    metadata:
      labels:
        app: my-app  # Labels for the Pods, matching the selector above.

    # A specification for the containers of the Pods.
    spec:
      containers:
      - name: my-app  # Name of the container. This is useful for linking to the container in logs, etc.
        image: mycontainerregistry.azurecr.io/my-app:latest  # Pull the container image you will call from your container registry.
        ports:
        - containerPort: 80  # The container listens on port 80.

Deploy using Azure Pipelines

In your YAML pipeline, include a task to deploy to AKS.

# This defines a task for deploying Kubernetes manifests within an Azure DevOps pipeline.
- task: KubernetesManifest@0  # Task type for working with Kubernetes manifest files.
  inputs:
    # This configures what to do with the Kubernetes manifest files.
    action: 'deploy'  # 'deploy' action ensures the Kubernetes manifests are applied to the cluster.

    # This defines the namespace where the Kubernetes resources will be deployed.
    namespace: 'default'  # 'default' is the namespace for deploying resources. Change if needed.

    # All this does is point to the manifest files that define the desired state of your Kubernetes resources.
    manifests: '$(Build.ArtifactStagingDirectory)/manifests/*.yaml'
    # The 'manifests' variable points to the location of all the YAML files (e.g., Deployment, Service, etc.) that will be applied during the deployment.

    # To access private container registry images, use the image pull secret.
    imagePullSecrets: 'my-registry-secret'
    # This instructs Kubernetes on which secret to use when pulling images from a private container registry.
    # Ensure that 'my-registry-secret' exists in the namespace before the deploy

Commit and push the YAML file to trigger the deployment.

AKS simplifies container orchestration, making managing scaling, updates, and rollbacks easier. In my experience, it’s a must-have for modern cloud-native applications.