What Docker Compose Actually Does
Docker Compose is a tool that lets you define and run multi-container applications using a single configuration file. Instead of running several docker run commands manually, you describe your entire application stack in one YAML file. Docker Compose then handles creating the containers, connecting them together, managing their lifecycle, and setting up the networking between services.
This approach works well for local development environments where you need a web server, database, cache, and possibly other services running at the same time. It is also useful in CI/CD pipelines to spin up the full application stack for testing, and it can occasionally appear in smaller production deployments, though tools like Docker Swarm or Kubernetes tend to be more common for larger production orchestration needs.
If you are new to Docker itself, it is worth reading a practical guide to installing Docker on Ubuntu first, as Docker Compose builds directly on the Docker daemon and assumes a working Docker installation.
The docker-compose.yml File Structure
The docker-compose.yml file sits at the heart of any Docker Compose project. It defines the services, networks, volumes, and configuration for your entire application stack. The file uses YAML syntax and follows a versioned format, with version 3.8 being current for most modern setups.
A typical docker-compose.yml for a web application includes several services that work together. Each service specifies which Docker image to use, which ports to expose, what volumes to mount, and which network to join. The depends_on directive manages startup order, ensuring that services start in the right sequence.
version: '3.8'
services:
web:
image: nginx:alpine
ports:
- "8080:80"
volumes:
- ./public:/var/www/html:ro
depends_on:
- php
- db
networks:
- app-network
php:
image: php:8.2-fpm-alpine
volumes:
- ./public:/var/www/html
networks:
- app-network
db:
image: mysql:8.0
environment:
MYSQL_ROOT_PASSWORD: secret
MYSQL_DATABASE: myapp
volumes:
- db-data:/var/lib/mysql
networks:
- app-network
volumes:
db-data:
networks:
app-network:
driver: bridgeRun the stack with docker-compose up. Adding the -d flag runs everything in detached mode, keeping the containers running in the background so your terminal remains available.
docker-compose up -dEssential Commands for Managing the Stack
Docker Compose provides a set of commands that cover the full lifecycle of your application stack. Understanding these commands helps you work efficiently when developing or deploying multi-container applications.
The up command creates and starts the containers. When you add -d, the containers run in the background. The --build flag forces a rebuild of images before starting, which is useful when you have changed a Dockerfile or base image configuration.
The down command stops and removes the containers, networks, and default networks. By default, volumes are preserved, which means your data remains intact. Adding the -v flag removes volumes as well, giving you a completely clean slate but deleting all data in the process.
# Start the stack in detached mode
docker-compose up -d
# Stop the stack (containers are removed, volumes are preserved)
docker-compose down
# Stop and remove volumes (data is deleted)
docker-compose down -v
# Rebuild images if you have changed Dockerfile
docker-compose up -d --build
# View logs from all services
docker-compose logs -f
# View logs from a specific service
docker-compose logs -f web
# Scale a service (run multiple replicas)
docker-compose up -d --scale php=3The log commands are particularly useful during development. Running docker-compose logs -f without a service name shows logs from all containers, which helps you spot interactions between services. Adding a service name filters the output to just that container.
Environment Variables and Configuration
Environment variables let you parameterise your docker-compose.yml so the same configuration works across different environments. You can source these variables from a .env file, from your shell environment, or from explicit environment declarations within the service definition.
Storing credentials in a .env file keeps sensitive information out of version control. You should add .env to your .gitignore file to prevent accidental commits of secrets.
# .env file (keep this out of version control)
DB_ROOT_PASSWORD=secretpassword
DB_NAME=myapp
REDIS_PASSWORD=redispassReference these variables in your docker-compose.yml using the ${VARIABLE_NAME} syntax. This approach separates configuration from code and makes it easier to use different values across development, staging, and production environments.
services:
db:
image: mysql:8.0
environment:
MYSQL_ROOT_PASSWORD: ${DB_ROOT_PASSWORD}
MYSQL_DATABASE: ${DB_NAME}
env_file:
- ./db_config.envYou can also use a separate env_file for additional configuration. This file should also be added to .gitignore to protect any sensitive data it contains.
# db_config.env (also in .gitignore)
MYSQL_PASSWORD=devpassword
MYSQL_USER=developerDatabase Setup with Docker Compose
A frequent use case for Docker Compose is running a web application alongside a database. The database container needs proper initialisation, persistent storage, and network access from the application container. Getting this right prevents data loss and connection issues during development.
Persistent storage is critical for databases. Without a named volume, data lives inside the container filesystem and disappears when the container is removed. Using a named volume ensures data survives container restarts and recreations.
services:
db:
image: mysql:8.0
environment:
MYSQL_ROOT_PASSWORD: ${ROOT_PASSWORD}
MYSQL_DATABASE: ${DB_NAME}
MYSQL_USER: ${DB_USER}
MYSQL_PASSWORD: ${DB_PASSWORD}
volumes:
- mysql-data:/var/lib/mysql
- ./init-scripts:/docker-entrypoint-initdb.d
networks:
- app-net
phpmyadmin:
image: phpmyadmin:latest
depends_on:
- db
ports:
- "8081:80"
environment:
PMA_HOST: db
PMA_USER: root
PMA_PASSWORD: ${ROOT_PASSWORD}
networks:
- app-net
volumes:
mysql-data:
networks:
app-net:The ./init-scripts directory contains SQL files that MySQL runs automatically when the database is first created. This feature is useful for seeding the database with initial schema and data for development, allowing new team members to get a fully working environment quickly.
Adding phpMyAdmin to your stack gives you a web-based interface for managing the database during development. It is not recommended for production use, but it can speed up troubleshooting and manual database operations while you are working locally.
Building Custom Images in Compose
Pulling images from Docker Hub works for many cases, but applications often need custom configuration. Building your own images lets you install specific extensions, copy configuration files, and tailor the environment to your application requirements.
The build directive specifies the location of your Dockerfile and the build context. The context is the directory containing files that should be available during the image build process.
services:
web:
build:
context: ./docker/nginx
dockerfile: Dockerfile
ports:
- "8080:80"
volumes:
- ./public:/var/www/html
php:
build:
context: ./docker/php
dockerfile: Dockerfile
volumes:
- ./public:/var/www/html
depends_on:
- dbYour Dockerfile defines the image steps. For a PHP application, you might start with an official PHP image and add the extensions and configuration your application needs.
# docker/php/Dockerfile
FROM php:8.2-fpm-alpine
RUN docker-php-ext-install pdo pdo_mysql
RUN apk add --no-cache freetype libzip-dev \
&& docker-php-ext-configure gd --with-freetype \
&& docker-php-ext-install -j$(nproc) gd
COPY php.ini /usr/local/etc/php/conf.d/custom.iniThis approach keeps your application-specific customisation separate from the generic docker-compose.yml file. You can version control your Dockerfiles alongside your application code, and Docker Compose rebuilds the images automatically when you run docker-compose up --build.
Running Tests in Docker Compose
Docker Compose is valuable for running tests in a consistent, isolated environment. You can spin up the application stack, run your test suite, and tear everything down, all within a CI/CD pipeline. This consistency means tests run the same way locally and on the CI server.
Using tmpfs for the database stores it in memory, which makes tests run faster. It also ensures a completely clean database state for each test run, since the database is created fresh each time the container starts.
services:
app:
build:
context: .
depends_on:
- db
environment:
- APP_ENV=testing
command: php artisan test
db:
image: mysql:8.0
environment:
MYSQL_DATABASE: test_db
MYSQL_ROOT_PASSWORD: test
tmpfs:
- /var/lib/mysqlThis setup is particularly useful for PHPUnit test suites that create and destroy database state frequently. The in-memory database approach prevents test pollution and keeps individual test runs quick. For a deeper look at automating deployment with shell scripts, you can explore how to write a bash script for application deployment to see how Docker Compose fits into broader automation workflows.
Service Dependencies and Startup Order
The depends_on directive ensures services start in the correct order. However, this directive only waits for the container to start, not for the service inside to be ready. A database container might be running but not yet able to accept connections when a dependent application service starts.
This mismatch causes connection failures during startup. Your application might try to connect to the database before MySQL has finished initialising, leading to errors that are difficult to diagnose during deployment.
The healthcheck directive solves this problem by letting you define what "healthy" means for a service. Once a health check is defined, dependent services can wait not just for the container to start, but for the health check to pass.
services:
db:
image: mysql:8.0
environment:
MYSQL_ROOT_PASSWORD: secret
healthcheck:
test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
interval: 5s
timeout: 5s
retries: 10
app:
depends_on:
db:
condition: service_healthyWith condition: service_healthy, the app service waits until the database is not just running but confirmed healthy before starting. The health check runs every 5 seconds, with a 5-second timeout, and will retry 10 times before considering the service unhealthy.
Defining proper health checks makes your stack more reliable during restarts and redeployments. It is especially important when deploying to production, where containers might restart at different times due to health monitoring or resource constraints.
Networking Between Containers
Docker Compose automatically creates a default network for your services to communicate. Each service can reach other services by their service name as a hostname. If you define a service called db, other services can connect to it at the hostname db.
You can define custom networks to isolate services or control traffic between them. For example, you might have a frontend service that should only communicate with an application service, while the application service communicates with both the database and a cache service.
services:
web:
networks:
- frontend
api:
networks:
- frontend
- backend
db:
networks:
- backend
networks:
frontend:
driver: bridge
backend:
driver: bridgeThis separation means the web server can talk to the API, and the API can talk to the database, but the web server cannot directly access the database. This architecture follows the principle of least privilege and makes it easier to reason about traffic flow.
Logs, Debugging, and Troubleshooting
When something goes wrong in a multi-container setup, logs are your first debugging tool. The docker-compose logs command aggregates output from all services, with each line prefixed by the service name.
# View all logs
docker-compose logs
# Follow logs in real time
docker-compose logs -f
# View logs for a specific service
docker-compose logs -f web
# View last 100 lines
docker-compose logs --tail=100If logs are not enough, you can open an interactive shell inside a running container using docker-compose exec. This is useful for inspecting files, checking environment variables, or running diagnostic commands.
# Open a shell in the web container
docker-compose exec web sh
# Run a command in a specific service
docker-compose exec db mysql -u root -pFor network issues, the docker-compose exec command can verify connectivity between services. If your application cannot reach the database, try running a ping or curl command from inside the application container to confirm the network path.
Orchestration Comparison: Docker Compose, Swarm, and Kubernetes
Docker Compose is excellent for development and single-host environments. When you move toward production with multiple servers and high availability requirements, other orchestration tools become more relevant.
Docker Swarm is built into the Docker Engine and provides basic clustering and orchestration capabilities. It shares many concepts with Docker Compose but adds service scaling, load balancing, and rolling updates across multiple hosts.
Kubernetes handles more complex orchestration scenarios including automated scaling, self-healing, and sophisticated deployment strategies. For small teams evaluating whether Kubernetes makes sense, there is a practical guide to Kubernetes for small teams that covers when the additional complexity is justified.
Docker Compose remains a valid choice for smaller production deployments where the operational complexity of Kubernetes is not warranted. Many applications run reliably on a single server using Docker Compose with a process manager like systemd to restart containers after a server reboot.
Putting It Together
Docker Compose simplifies the process of managing multi-container applications by consolidating configuration into a single file and providing commands to control the entire stack. Whether you are setting up a local development environment, running automated tests, or deploying a smaller application to production, the principles remain consistent.
Start with a clear understanding of which services your application needs and how they communicate. Define your volumes carefully to prevent data loss, use environment variables to keep configuration flexible, and add health checks to handle service startup dependencies reliably.
If you need help setting up a Docker Compose configuration for your application or want someone to review an existing setup, you can get in touch with details of your application stack, the services you need, and any specific requirements you have for the environment.