Docker Compose in Practice: From Dev to Production Config Evolution
Docker Compose has one of the biggest gaps between "it works" and "it works in production." This article starts from a minimal docker-compose.yml and evolves it step by step into a production-ready configuration: logging, health checks, network isolation, database backup, and CI/CD integration. Includes production-ready config templates. [See the production-grade config →]
The Bottom Line: Compose Is the Sweet Spot for Small to Medium Projects
Docker Compose may be the most underestimated tool in the Docker ecosystem — not because it is underpowered, but because most people use only 20% of its capability.
This article starts from a “works on my machine” docker-compose.yml and evolves it step by step into a production-ready configuration. Each step explains why the change is needed and what problem it solves.
Phase 1: It Works (Dev Environment)
version: '3.8'
services:
app:
build: .
ports:
- "3000:3000"
depends_on:
- db
db:
image: postgres:16
environment:
POSTGRES_PASSWORD: secret
This works — but only for local development. Problems: plaintext password, no persistence, no health checks, no log limits, no network isolation. Take this to production and you will have issues within days.
Phase 2: Add Persistence and Config Management
version: '3.8'
services:
app:
build: .
ports:
- "3000:3000"
env_file: .env
depends_on:
db:
condition: service_healthy
networks:
- app-net
db:
image: postgres:16
env_file: .env
volumes:
- pg-data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER}"]
interval: 10s
timeout: 5s
retries: 5
networks:
- app-net
volumes:
pg-data:
networks:
app-net:
Three key changes:
-
Named volume:
pg-datais a Docker-managed named volume — data survivesdocker compose down. Safer than bind mounts (no host path dependency) and easier to back up. -
Health check:
depends_onwithcondition: service_healthyensures app starts only after the database is actually ready, not just when the container starts. Without this, the app crashes on startup because the database is not yet accepting connections. -
Network isolation: An internal
app-netnetwork allows only services in this network to communicate. External requests enter only through the app’s exposed port 3000. The database exposes no ports — a significant security improvement.
Phase 3: Add Log Rotation and Restart Policy
services:
app:
build: .
env_file: .env
restart: unless-stopped
logging:
driver: "local"
options:
max-size: "10m"
max-file: "3"
depends_on:
db:
condition: service_healthy
networks:
- app-net
db:
image: postgres:16
env_file: .env
restart: unless-stopped
logging:
driver: "local"
options:
max-size: "10m"
max-file: "3"
volumes:
- pg-data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U $${POSTGRES_USER}"]
interval: 10s
timeout: 5s
retries: 5
networks:
- app-net
Log rotation is the most overlooked configuration. Docker’s default json-file driver writes all logs to a single file with no limit. A container producing a few hundred MB of logs daily will fill up your disk in days. This is not theoretical — I have seen production outages caused by logs filling up the disk.
Setting logging.driver: "local" with max-size: 10m and max-file: 3 limits each container to 30 MB of logs with automatic rotation.
restart: unless-stopped ensures the service restarts automatically after a crash, but does not restart if you manually stopped it.
Phase 4: Add Database Backups
The daily configuration is solid, but one critical thing remains — database backups.
#!/bin/bash
# backup-db.sh — run on the host via cron
BACKUP_DIR=/var/backups/postgres
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
DB_CONTAINER=$(docker compose ps -q db)
docker exec $DB_CONTAINER pg_dump -U $POSTGRES_USER $POSTGRES_DB | gzip > $BACKUP_DIR/db_$TIMESTAMP.sql.gz
# Keep last 30 days
find $BACKUP_DIR -name "*.sql.gz" -mtime +30 -delete
Cron entry:
0 3 * * * /usr/local/bin/backup-db.sh
For critical data, add remote storage:
aws s3 cp $BACKUP_DIR/db_$TIMESTAMP.sql.gz s3://my-backups/db/
Phase 5: Add CI/CD Integration
At this point, deployment should be a single command:
# .github/workflows/deploy.yml
name: Deploy
on:
push:
branches: [main]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build and deploy
run: |
docker compose build
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
Use docker-compose.prod.yml to override production-specific settings:
# docker-compose.prod.yml
services:
app:
environment:
- NODE_ENV=production
restart: always
deploy:
resources:
limits:
memory: 512M
The production override file only contains environment-specific differences. Core configuration stays in docker-compose.yml, preventing drift between environments.
Summary: A Production-Ready Compose Checklist
| Item | Dev | Production |
|---|---|---|
| Persistence | Optional | Named volume |
| Health check | Optional | Required |
| Log rotation | Optional | Required |
| Restart policy | None | unless-stopped |
| Network isolation | Optional | Required |
| Resource limits | Optional | Recommended |
| DB backup | Optional | Required (cron + remote storage) |
| CI/CD | Optional | Recommended |
Docker Compose is not a toy — its ceiling depends on how completely you configure it. A Compose file with log rotation, health checks, network isolation, and backup strategy can run stably on a single server for a long time.
Need Docker deployment design or CI/CD pipeline setup? Contact us — tell us your service architecture and deployment environment, feasibility within 24 hours.
Related reading
- Ops Automation Script Patterns — ops automation that pairs with Docker deployment
- CI/CD Tooling: GitHub Actions vs GitLab CI — CI/CD pipeline selection that works with Docker Compose
FAQ
Is Docker Compose suitable for production?
Yes, for small to medium projects. Docker Compose is a single-machine orchestrator — not a cluster scheduler like K8s. If your project runs on a single server with 3-5 containers, Compose is sufficient and far cheaper to operate than Kubernetes. When you exceed 10 services, need multi-machine deployment, or require auto-scaling, that is when you should consider K8s or Nomad.
How do you handle sensitive data in .env files?
.env files should never be committed to Git (add to .gitignore). For production, use Docker Secrets or environment variable injection. Small teams can use a secure vault script to generate .env at deploy time. CI/CD tools like GitHub Actions Secrets can also inject environment variables directly, avoiding plaintext storage.
What happens if you do not configure log rotation?
The default json-file driver writes all logs to a single JSON file with no limit. If your container produces hundreds of MB of logs daily, disk space fills up in days — and your service goes down. Configure logging.driver as "local" or "json-file" with max-size (e.g., 10m) and max-file (e.g., 3) in docker-compose.yml. For a more thorough solution, use a log collector like Loki + Promtail or Filebeat to ship logs to centralized storage.
How do you back up a database running in a container?
Do not run mysqldump inside the container and expect it to persist. The recommended approach: ① Mount database data to a named volume for easy backup; ② Write a separate backup script on the host, executed via cron, using docker exec to run the database export, then compress and upload to object storage (S3, OSS); ③ Periodically verify backup recoverability — a backup you cannot restore is no backup at all.
This article comes from AI Enable Harness front-line delivery practice. Need a similar system or optimization service?
Subscribe to Updates
Get notified when new articles are published. No spam, occasional updates only.
Subscribe →