How to Learn Docker from Scratch: A Guide for Data Professionals

Step 3: Run your first container

Now that Docker is installed, it's time to run your first container. We'll start with the simple hello-world image, perfect for beginners.

1. Open your terminal (or Command Prompt on Windows) and run the following command:

sudo docker run hello-world

This command tells Docker to look for the hello-world image locally. If it doesn’t find it, Docker will pull the image from Docker Hub and then run it.

If Docker is installed correctly, you’ll see a message that says, “Hello from Docker!” along with details about how the Docker process works. This output confirms that Docker successfully pulled the image, created a new container, and executed the code inside it.

Step 4: Build your first Docker image

In this step, you’ll create a custom Docker image for a simple data science project. This image will include Python and common data science libraries like pandas, NumPy, and scikit-learn.

1. Create a new directory:

• • Start by creating a new directory for your project and navigating into it:

mkdir my-data-science-app
cd my-data-science-app

2. Create a Dockerfile:

Inside this directory, create a file named Dockerfile (without any file extension):

# Use an official Python runtime as a parent image
FROM python:3.8-slim
 
# Set the working directory in the container
WORKDIR /app
 
# Copy the current directory contents into the container at /app
COPY . /app
 
# Install any needed packages specified in requirements.txt
RUN pip install --no-cache-dir -r requirements.txt
 
# Make port 8888 available to the world outside this container
EXPOSE 8888
 
# Run a Jupyter notebook server when the container launches
CMD ["jupyter", "notebook", "--ip=0.0.0.0", "--port=8888", "--no-browser", "--allow-root"]

The Dockerfile does the following:

• FROM python:3.8-slim: This sets the base image to Python 3.8 in its slim variant, a minimal version of Python, ideal for a lightweight data science environment.
• WORKDIR /app: This sets the working directory inside the container to /app. All subsequent commands will be executed within this directory.
• COPY . /app: This copies all files from your current directory on the host machine and puts them into the /app directory in the container. This typically includes your data science scripts and configuration files.
• RUN pip install --no-cache-dir -r requirements.txt: This installs the Python libraries listed in requirements.txt. Common libraries for data science include pandas, NumPy, and scikit-learn.
• EXPOSE 8888: This makes port 8888 available to the host machine, the default port for Jupyter notebooks.
• CMD ["jupyter", "notebook", "--ip=0.0.0.0", "--port=8888", "--no-browser", "--allow-root"]: This command launches a Jupyter notebook server when the container starts, making it accessible from your web browser.

3. Create a requirements.txt File:

In the same directory, create a requirements.txt file to list the Python packages your application needs:

pandas
numpy
scikit-learn
matplotlib

This file lists the Python libraries that will be installed inside your Docker container.

4. Build the Docker image:

• With the Dockerfile and requirements.txt ready, build your Docker image using the following command:

sudo docker build -t my-data-science-app .

The command above does the following:

• docker build: Initiates the process of building a Docker image.
• -t my-data-science-app: The -t option tags the image as “my-data-science-app.”
• .: The dot at the end specifies the build context, which is the current directory. Docker will use the files in this directory to build the image.

5. Run the Docker image:

Once the build process is complete, you can run your new Docker image as a container:

sudo docker run -p 8888:8888 my-data-science-app

Let’s break down the command:

• docker run: Runs a container based on the specified image.
• -p 8888:8888: The -p option maps port 8888 on your local machine to port 8888 inside the container, allowing you to access the Jupyter Notebook server by navigating to localhost:8888 in your web browser.
• my-data-science-app: This specifies the image to run.

Step 5: Use Docker Compose

Docker Compose is an essential tool for managing multi-container applications. In a data science project, you might need separate containers for different components, such as a Jupyter notebook, a database for storing data, and a data visualization tool.

Docker Compose allows you to define these services in a single YAML file, making it easy to spin up your entire data science environment with one command.

1. Create a docker-compose.yml file:

• In your project directory, create a file named docker-compose.yml and write the following lines in the file:

version: '3.8'
services:
  jupyter:
        image: jupyter/scipy-notebook:latest
        volumes:
        - ./notebooks:/home/joelwembo/work
        ports:
        - "8888:8888"
        environment:
        - JUPYTER_ENABLE_LAB=yes
 
  postgres:
        image: postgres:13-alpine
        environment:
        POSTGRES_USER: myuser
        POSTGRES_PASSWORD: mypassword
        POSTGRES_DB: mydatabase
        volumes:
        - postgres_data:/var/lib/postgresql/data
        ports:
        - "5432:5432"
 
  redis:
        image: redis:alpine
        ports:
        - "6379:6379"
 
volumes:
  postgres_data:
 

Let’s break down what the Docker Compose file does:

• version: '3.8': This specifies the version of the Docker Compose file format. Version 3.8 is a commonly used version that supports many features.
• services:: The services section defines the different containers (services) that make up your application.
• jupyter:: This service represents a Jupyter notebook environment essential for interactive data analysis.
• image: jupyter/scipy-notebook:latest: This specifies the Docker image to use for the Jupyter notebook, which includes popular data science libraries like NumPy, pandas, and matplotlib.
• volumes: - ./notebooks:/home/joelwembo/work: This mounts the notebooks directory on your host machine to /home/joelwembo/work inside the container, allowing you to persist and share your notebooks. Note: This is my personal path and yours should look different.
• ports: - "8888:8888": This maps port 8888 on your host machine to port 8888 in the container, which is the default port for Jupyter notebooks.
• environment: - JUPYTER_ENABLE_LAB=yes: This enables JupyterLab, a more powerful interface for working with Jupyter notebooks.
• postgres:: This service runs a PostgreSQL database, which is often used to store and manage large datasets in data science projects.
• image: postgres:13-alpine: This specifies the Docker image for PostgreSQL version 13, based on the lightweight Alpine Linux distribution.
• environment:: These environment variables configure the database with a user, password, and database name.
• volumes: - postgres_data:/var/lib/postgresql/data: This mounts a Docker volume to persist the database data outside the container, so your data isn’t lost when the container is stopped.
• ports: - "5432:5432": This maps port 5432 on your host machine to port 5432 in the container, the default port for PostgreSQL.
• redis:: This service runs a Redis server, which can be used as a fast in-memory data store or cache.
• image: redis:alpine: This specifies the Redis image using the lightweight Alpine variant.
• ports: - "6379:6379": This maps port 6379 on your host machine to port 6379 in the container, the default port for Redis.
• volumes:: This section defines named volumes that can be shared between services or persist data across container restarts.
• postgres_data:: This creates a named volume for persisting PostgreSQL data.

To start your multi-container application, simply run the following:

sudo docker compose up

This command builds the images (if necessary) and starts the containers defined in your docker-compose.yml file. It brings up both the web application and the Redis service, allowing them to communicate with each other as defined.

You have now deployed your first Docker application! Isn’t that great? Now, let’s elaborate a plan for your continuous learning.

An Example Learning Plan for Docker

Here’s a week-by-week plan that you can follow. Feel free to tailor it to your specific needs. 

If you need a more defined path, the Containerization and Virtualization with Docker and Kubernetes skill track is perfect for you. It contains four essential courses.

Week 1: Get familiar with Docker basics

• Objective: Understand Docker's core concepts and basic commands.
• Tasks:
• Install Docker and set up Docker Desktop.
• Learn Docker commands: docker run, docker ps, docker stop, docker rm, etc.
• Run simple containers for data exploration tools like Ubuntu or Alpine and try basic commands.
• Explore official Docker images and run containers for basic visualization tools (e.g., a simple matplotlib container).
• Resources: Containerization and Virtualization Concepts course, Introduction to Docker course.

Week 2: Build and run Docker images for data tools

• Objective: Learn to create and manage custom Docker images.
• Tasks:
• Write a Dockerfile for a basic data analysis environment (e.g., Python with Pandas and NumPy).
• Build the Docker image using docker build.
• Run containers from your custom image and test data tools like Jupyter Notebook.
• Create Dockerfiles for TensorFlow and PostgreSQL, ensuring proper configuration and linking.
• Resources: Introduction to Docker course.

Week 3: Explore Docker Compose

• Objective: Understand and utilize Docker Compose for multi-container setups.
• Tasks:
• Install Docker Compose and learn its basics.
• Write docker-compose.yml files to define multi-container applications.
• Practice with examples like setting up a web application with a backend database (e.g., a Flask app with PostgreSQL).
• Experiment with different configurations and service dependencies.
• Resources: Intermediate Docker course.

Week 4: Dive deeper into Docker networking and volumes

• Objective: Master Docker’s networking and storage capabilities.
• Tasks:
• Learn about Docker networks and how to create them (bridge, host, overlay).
• Practice creating custom networks and connecting containers to them.
• Explore Docker volumes for persistent storage and practice creating and managing them.
• Implement a setup where containers communicate over a custom network and use volumes for data persistence.
• Resources: Intermediate Docker course.

Week 5: Deploy and manage Docker containers in production

• Objective: Understand container deployment and management in production.
• Tasks:
• Learn best practices for deploying Docker containers (e.g., using Docker Hub or a private registry).
• Explore container orchestration with Docker Swarm: creating and managing services.
• Understand scaling containers and managing logs.
• Experiment with deploying a containerized application to a staging or production environment.
• Resources: Intermediate Docker course.

Week 6: Introduction to Kubernetes for container orchestration

• Objective: Get acquainted with Kubernetes for scaling and managing containerized applications.
• Tasks:
• Install and set up a local Kubernetes environment (e.g., Minikube or Kind).
• Learn Kubernetes basics: pods, services, deployments, and namespaces.
• Deploy a simple containerized application on Kubernetes.
• Explore scaling and managing applications with Kubernetes, focusing on handling a containerized data tool like Jupyter.
• Resources: Introduction to Kubernetes course.

Tips and Ways to Learn Docker

Lastly, here are some tips and best practices to help you fast-track your Docker learning path.

Practice regularly

The key to mastering Docker is consistent, hands-on practice. Start by working on personal projects, like containerizing a simple website or setting up a local development environment. 

As you become more comfortable, challenge yourself with more complex tasks, such as building a multi-container application or deploying a web server. 

Regular experimentation with different use cases—like creating custom Docker images or configuring a CI/CD pipeline—will deepen your understanding and help solidify key concepts. Additionally, participating in online Docker challenges or exercises is a great way to test and enhance your skills in a structured, goal-oriented manner.

Use online resources

Online resources are invaluable for learning Docker. The official Docker documentation is an excellent place to begin, offering authoritative and up-to-date information. 

Beyond that, there are plenty of comprehensive courses available on platforms like DataCamp, which can help guide you through the learning process. 

For those who prefer visual learning, YouTube tutorials from Docker’s official channel and other educators provide useful, accessible content to reinforce your understanding.

Join the Docker community

Engaging with the Docker community is another powerful way to advance your learning. Participate in forums like Docker Community Forums and Reddit to exchange ideas, ask questions, and share experiences with other learners and professionals. 

Attending Docker meetups, webinars, or conferences is also a great way to stay current with industry trends and broaden your network.

Work on open-source projects

Contributing to open-source projects that utilize Docker can be a fantastic way to gain practical, real-world experience. 

Websites like GitHub offer opportunities to collaborate on Docker-related projects, allowing you to learn from others and apply your skills in diverse contexts. The collaboration inherent in open-source work helps expose you to different approaches and solutions, fostering a deeper understanding of Docker’s capabilities.

Stay updated

Finally, staying updated with Docker’s latest developments is essential for maintaining proficiency. Keep an eye on Docker’s official blog and release notes for updates on new features or changes. Joining community discussions on forums and social media will also inform you about emerging trends and best practices, helping you continuously refine your Docker skills.

With regular practice, active community engagement, and continuous learning, you’ll build a strong foundation in Docker and remain equipped to handle the latest developments in the field.

Conclusion

Docker has revolutionized application deployment and data management by packaging applications and their dependencies into isolated containers. This solves common challenges like environment inconsistencies and ensures workflows remain consistent, scalable, and efficient for data professionals.

This guide has provided the foundational steps to mastering Docker, from core concepts to hands-on deployments. To further deepen your knowledge, consider exploring additional resources like the Introduction to Docker and Intermediate Docker courses on DataCamp. As you advance, expanding your skills with orchestration tools like Kubernetes is equally important—check out the Introduction to Kubernetes course to get started.

Practicing regularly, using available resources, and staying engaged with the community will unlock Docker’s full potential for managing complex data environments.