Docker Compose Multi-Container Guide

What Docker Compose Actually Does

Docker Compose is a tool that lets you define and run multi-container applications using a single configuration file. Instead of running several docker run commands manually, you describe your entire application stack in one YAML file. Docker Compose then handles creating the containers, connecting them together, managing their lifecycle, and setting up the networking between services.

This approach works well for local development environments where you need a web server, database, cache, and possibly other services running at the same time. It is also useful in CI/CD pipelines to spin up the full application stack for testing, and it can occasionally appear in smaller production deployments, though tools like Docker Swarm or Kubernetes tend to be more common for larger production orchestration needs.

If you are new to Docker itself, it is worth reading a practical guide to installing Docker on Ubuntu first, as Docker Compose builds directly on the Docker daemon and assumes a working Docker installation.

The docker-compose.yml File Structure

The docker-compose.yml file sits at the heart of any Docker Compose project. It defines the services, networks, volumes, and configuration for your entire application stack. The file uses YAML syntax and follows a versioned format, with version 3.8 being current for most modern setups.

A typical docker-compose.yml for a web application includes several services that work together. Each service specifies which Docker image to use, which ports to expose, what volumes to mount, and which network to join. The depends_on directive manages startup order, ensuring that services start in the right sequence.

version: '3.8'

services:
  web:
    image: nginx:alpine
    ports:
      - "8080:80"
    volumes:
      - ./public:/var/www/html:ro
    depends_on:
      - php
      - db
    networks:
      - app-network

  php:
    image: php:8.2-fpm-alpine
    volumes:
      - ./public:/var/www/html
    networks:
      - app-network

  db:
    image: mysql:8.0
    environment:
      MYSQL_ROOT_PASSWORD: secret
      MYSQL_DATABASE: myapp
    volumes:
      - db-data:/var/lib/mysql
    networks:
      - app-network

volumes:
  db-data:

networks:
  app-network:
    driver: bridge

Run the stack with docker-compose up. Adding the -d flag runs everything in detached mode, keeping the containers running in the background so your terminal remains available.

docker-compose up -d

This Docker Compose configuration creates three services that communicate over a shared bridge network. The web service uses Nginx to serve static files, the PHP service handles server-side processing, and the MySQL database stores persistent application data. Each service has a defined role, and the depends_on declarations ensure the database starts before PHP tries to connect to it.

Essential Commands for Managing the Stack

Docker Compose provides a set of commands that cover the full lifecycle of your application stack. Understanding these commands helps you work efficiently when developing or deploying multi-container applications.

The up command creates and starts the containers. When you add -d, the containers run in the background. The --build flag forces a rebuild of images before starting, which is useful when you have changed a Dockerfile or base image configuration.

The down command stops and removes the containers, networks, and default networks. By default, volumes are preserved, which means your data remains intact. Adding the -v flag removes volumes as well, giving you a completely clean slate but deleting all data in the process.

# Start the stack in detached mode
docker-compose up -d

# Stop the stack (containers are removed, volumes are preserved)
docker-compose down

# Stop and remove volumes (data is deleted)
docker-compose down -v

# Rebuild images if you have changed Dockerfile
docker-compose up -d --build

# View logs from all services
docker-compose logs -f

# View logs from a specific service
docker-compose logs -f web

# Scale a service (run multiple replicas)
docker-compose up -d --scale php=3

The log commands are particularly useful during development. Running docker-compose logs -f without a service name shows logs from all containers, which helps you spot interactions between services. Adding a service name filters the output to just that container.

The scale command lets you run multiple instances of a service behind a load balancer. This is useful for testing how your application behaves under load, though for production deployments with proper load balancing, Docker Swarm or Kubernetes offer more robust solutions.

Environment Variables and Configuration

Environment variables let you parameterise your docker-compose.yml so the same configuration works across different environments. You can source these variables from a .env file, from your shell environment, or from explicit environment declarations within the service definition.

Storing credentials in a .env file keeps sensitive information out of version control. You should add .env to your .gitignore file to prevent accidental commits of secrets.

# .env file (keep this out of version control)
DB_ROOT_PASSWORD=secretpassword
DB_NAME=myapp
REDIS_PASSWORD=redispass

Reference these variables in your docker-compose.yml using the ${VARIABLE_NAME} syntax. This approach separates configuration from code and makes it easier to use different values across development, staging, and production environments.

services:
  db:
    image: mysql:8.0
    environment:
      MYSQL_ROOT_PASSWORD: ${DB_ROOT_PASSWORD}
      MYSQL_DATABASE: ${DB_NAME}
    env_file:
      - ./db_config.env

You can also use a separate env_file for additional configuration. This file should also be added to .gitignore to protect any sensitive data it contains.

# db_config.env (also in .gitignore)
MYSQL_PASSWORD=devpassword
MYSQL_USER=developer

Managing secrets properly matters for container security. Even in a development environment, getting into the habit of keeping credentials out of your version-controlled files prevents accidental exposure. For production deployments, you would want to explore more robust secrets management solutions, which you can learn about in a practical guide to Docker security best practices for container hardening.

Database Setup with Docker Compose

A frequent use case for Docker Compose is running a web application alongside a database. The database container needs proper initialisation, persistent storage, and network access from the application container. Getting this right prevents data loss and connection issues during development.

Persistent storage is critical for databases. Without a named volume, data lives inside the container filesystem and disappears when the container is removed. Using a named volume ensures data survives container restarts and recreations.

services:
  db:
    image: mysql:8.0
    environment:
      MYSQL_ROOT_PASSWORD: ${ROOT_PASSWORD}
      MYSQL_DATABASE: ${DB_NAME}
      MYSQL_USER: ${DB_USER}
      MYSQL_PASSWORD: ${DB_PASSWORD}
    volumes:
      - mysql-data:/var/lib/mysql
      - ./init-scripts:/docker-entrypoint-initdb.d
    networks:
      - app-net

  phpmyadmin:
    image: phpmyadmin:latest
    depends_on:
      - db
    ports:
      - "8081:80"
    environment:
      PMA_HOST: db
      PMA_USER: root
      PMA_PASSWORD: ${ROOT_PASSWORD}
    networks:
      - app-net

volumes:
  mysql-data:

networks:
  app-net:

The ./init-scripts directory contains SQL files that MySQL runs automatically when the database is first created. This feature is useful for seeding the database with initial schema and data for development, allowing new team members to get a fully working environment quickly.

Adding phpMyAdmin to your stack gives you a web-based interface for managing the database during development. It is not recommended for production use, but it can speed up troubleshooting and manual database operations while you are working locally.

The database configuration above demonstrates a practical multi-container setup where the application and database run in isolated containers but communicate over a shared Docker network. This pattern is common in development workflows and works well for evaluating when Docker is the right choice for web applications.

Building Custom Images in Compose

Pulling images from Docker Hub works for many cases, but applications often need custom configuration. Building your own images lets you install specific extensions, copy configuration files, and tailor the environment to your application requirements.

The build directive specifies the location of your Dockerfile and the build context. The context is the directory containing files that should be available during the image build process.

services:
  web:
    build:
      context: ./docker/nginx
      dockerfile: Dockerfile
    ports:
      - "8080:80"
    volumes:
      - ./public:/var/www/html

  php:
    build:
      context: ./docker/php
      dockerfile: Dockerfile
    volumes:
      - ./public:/var/www/html
    depends_on:
      - db

Your Dockerfile defines the image steps. For a PHP application, you might start with an official PHP image and add the extensions and configuration your application needs.

# docker/php/Dockerfile
FROM php:8.2-fpm-alpine

RUN docker-php-ext-install pdo pdo_mysql

RUN apk add --no-cache freetype libzip-dev \
    && docker-php-ext-configure gd --with-freetype \
    && docker-php-ext-install -j$(nproc) gd

COPY php.ini /usr/local/etc/php/conf.d/custom.ini

This approach keeps your application-specific customisation separate from the generic docker-compose.yml file. You can version control your Dockerfiles alongside your application code, and Docker Compose rebuilds the images automatically when you run docker-compose up --build.

When building custom images, it is worth understanding the security implications of your Dockerfile instructions. Layering too many tools into production images increases the attack surface. Aim for minimal base images and only include what your application genuinely needs to run.

Running Tests in Docker Compose

Docker Compose is valuable for running tests in a consistent, isolated environment. You can spin up the application stack, run your test suite, and tear everything down, all within a CI/CD pipeline. This consistency means tests run the same way locally and on the CI server.

Using tmpfs for the database stores it in memory, which makes tests run faster. It also ensures a completely clean database state for each test run, since the database is created fresh each time the container starts.

services:
  app:
    build:
      context: .
    depends_on:
      - db
    environment:
      - APP_ENV=testing
    command: php artisan test

  db:
    image: mysql:8.0
    environment:
      MYSQL_DATABASE: test_db
      MYSQL_ROOT_PASSWORD: test
    tmpfs:
      - /var/lib/mysql

This setup is particularly useful for PHPUnit test suites that create and destroy database state frequently. The in-memory database approach prevents test pollution and keeps individual test runs quick.

For teams using GitHub Actions, Docker Compose fits naturally into CI workflows. You can see how this works in practice by exploring a guide to setting up a PHP CI/CD pipeline with GitHub Actions, which covers how to run tests inside containers as part of an automated deployment workflow.

For a deeper look at automating deployment with shell scripts, you can explore how to write a bash script for application deployment to see how Docker Compose fits into broader automation workflows.

Service Dependencies and Startup Order

The depends_on directive ensures services start in the correct order. However, this directive only waits for the container to start, not for the service inside to be ready. A database container might be running but not yet able to accept connections when a dependent application service starts.

This mismatch causes connection failures during startup. Your application might try to connect to the database before MySQL has finished initialising, leading to errors that are difficult to diagnose during deployment.

The healthcheck directive solves this problem by letting you define what "healthy" means for a service. Once a health check is defined, dependent services can wait not just for the container to start, but for the health check to pass.

services:
  db:
    image: mysql:8.0
    environment:
      MYSQL_ROOT_PASSWORD: secret
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
      interval: 5s
      timeout: 5s
      retries: 10

  app:
    depends_on:
      db:
        condition: service_healthy

With condition: service_healthy, the app service waits until the database is not just running but confirmed healthy before starting. The health check runs every 5 seconds, with a 5-second timeout, and will retry 10 times before considering the service unhealthy.

Defining proper health checks makes your stack more reliable during restarts and redeployments. It is especially important when deploying to production, where containers might restart at different times due to health monitoring or resource constraints.

For more complex health checks involving multiple conditions or custom scripts, you can extend the approach with entrypoint wrappers that poll the service readiness before executing the main process. This pattern is common in production-oriented Docker configurations.

Networking Between Containers

Docker Compose automatically creates a default network for your services to communicate. Each service can reach other services by their service name as a hostname. If you define a service called db, other services can connect to it at the hostname db.

You can define custom networks to isolate services or control traffic between them. For example, you might have a frontend service that should only communicate with an application service, while the application service communicates with both the database and a cache service.

services:
  web:
    networks:
      - frontend

  api:
    networks:
      - frontend
      - backend

  db:
    networks:
      - backend

networks:
  frontend:
    driver: bridge
  backend:
    driver: bridge

This separation means the web server can talk to the API, and the API can talk to the database, but the web server cannot directly access the database. This architecture follows the principle of least privilege and makes it easier to reason about traffic flow.

Network isolation is particularly valuable when you have services with different trust levels. Your application code might be trusted, but a third-party service or a less-controlled container should have limited access to your internal resources.

Docker Compose network names are prefixed with your project name, which is usually the directory name. This prevents conflicts when running multiple projects on the same host. You can override the project name using the COMPOSE_PROJECT_NAME environment variable.

Logs, Debugging, and Troubleshooting

When something goes wrong in a multi-container setup, logs are your first debugging tool. The docker-compose logs command aggregates output from all services, with each line prefixed by the service name.

# View all logs
docker-compose logs

# Follow logs in real time
docker-compose logs -f

# View logs for a specific service
docker-compose logs -f web

# View last 100 lines
docker-compose logs --tail=100

If logs are not enough, you can open an interactive shell inside a running container using docker-compose exec. This is useful for inspecting files, checking environment variables, or running diagnostic commands.

# Open a shell in the web container
docker-compose exec web sh

# Run a command in a specific service
docker-compose exec db mysql -u root -p

For network issues, the docker-compose exec command can verify connectivity between services. If your application cannot reach the database, try running a ping or curl command from inside the application container to confirm the network path.

Common issues in Docker Compose setups include port conflicts, volume permission problems, and environment variables not loading correctly. Port conflicts occur when the host port is already in use by another service. Volume permissions can be tricky when the container user ID differs from the host user. Environment variables from .env files are only loaded at stack startup, so changes require recreating the containers.

You can inspect the full configuration that Docker Compose generates using docker-compose config. This outputs the merged configuration after variable substitution and extension processing, which helps verify that your settings are being applied as expected.

Orchestration Comparison: Docker Compose, Swarm, and Kubernetes

Docker Compose is excellent for development and single-host environments. When you move toward production with multiple servers and high availability requirements, other orchestration tools become more relevant.

Docker Swarm is built into the Docker Engine and provides basic clustering and orchestration capabilities. It shares many concepts with Docker Compose but adds service scaling, load balancing, and rolling updates across multiple hosts. If you are already using Docker Compose for development, the transition to Swarm involves learning new deployment commands rather than rewriting your configuration.

Kubernetes handles more complex orchestration scenarios including automated scaling, self-healing, and sophisticated deployment strategies. For small teams evaluating whether Kubernetes makes sense, there is a practical guide to Kubernetes for small teams that covers when the additional complexity is justified.

Docker Compose remains a valid choice for smaller production deployments where the operational complexity of Kubernetes is not warranted. Many applications run reliably on a single server using Docker Compose with a process manager like systemd to restart containers after a server reboot.

When deciding between these options, consider the operational burden each introduces. Docker Compose requires the least additional learning. Swarm adds cluster management but stays within the Docker ecosystem. Kubernetes offers the most features but requires significant investment in learning and infrastructure.

Putting It Together

Docker Compose simplifies the process of managing multi-container applications by consolidating configuration into a single file and providing commands to control the entire stack. Whether you are setting up a local development environment, running automated tests, or deploying a smaller application to production, the principles remain consistent.

Start with a clear understanding of which services your application needs and how they communicate. Define your volumes carefully to prevent data loss, use environment variables to keep configuration flexible, and add health checks to handle service startup dependencies reliably.

If you need help setting up a Docker Compose configuration for your application or want someone to review an existing setup, you can get in touch with details of your application stack, the services you need, and any specific requirements you have for the environment.