How to clean up unused Docker images in Linux
How to Clean Up Unused Docker Images in Linux
Docker has revolutionized containerization technology, making it easier than ever to deploy and manage applications. However, as you work with Docker over time, your system can accumulate numerous unused images, containers, volumes, and networks that consume valuable disk space. This comprehensive guide will teach you everything you need to know about cleaning up unused Docker images in Linux, from basic commands to advanced automation strategies.
Table of Contents
- [Introduction](#introduction)
- [Prerequisites](#prerequisites)
- [Understanding Docker Storage](#understanding-docker-storage)
- [Basic Docker Cleanup Commands](#basic-docker-cleanup-commands)
- [Advanced Cleanup Strategies](#advanced-cleanup-strategies)
- [Automated Cleanup Solutions](#automated-cleanup-solutions)
- [Monitoring Docker Storage Usage](#monitoring-docker-storage-usage)
- [Troubleshooting Common Issues](#troubleshooting-common-issues)
- [Best Practices](#best-practices)
- [Conclusion](#conclusion)
Introduction
Docker images can quickly accumulate on your Linux system, especially in development environments where you frequently pull, build, and test different images. These unused images, also known as "dangling" images, can consume gigabytes of disk space and impact system performance. Regular cleanup of unused Docker images is essential for maintaining an efficient Docker environment.
This guide covers multiple approaches to Docker image cleanup, from simple one-line commands to sophisticated automated solutions. You'll learn how to identify unused images, safely remove them, and implement preventive measures to keep your Docker environment clean and optimized.
Prerequisites
Before proceeding with Docker image cleanup, ensure you have:
System Requirements
- Linux operating system (Ubuntu, CentOS, RHEL, Debian, or similar)
- Docker installed and running
- Sufficient user privileges (sudo access or Docker group membership)
- Basic command-line knowledge
Checking Docker Installation
Verify your Docker installation with:
```bash
docker --version
docker info
```
User Permissions
Ensure your user can execute Docker commands without sudo:
```bash
sudo usermod -aG docker $USER
```
Note: You may need to log out and back in for group changes to take effect.
Understanding Docker Storage
Docker Storage Architecture
Docker uses a layered filesystem approach where images consist of multiple read-only layers. Understanding this architecture helps you make informed decisions about cleanup:
- Base layers: Fundamental OS layers shared across images
- Application layers: Specific to individual applications
- Dangling images: Untagged images no longer referenced by any container
- Intermediate images: Created during build processes
Checking Current Storage Usage
Before cleaning up, assess your current Docker storage usage:
```bash
Check overall Docker system usage
docker system df
Detailed breakdown of space usage
docker system df -v
List all images with sizes
docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.ID}}\t{{.Size}}"
```
The `docker system df` command provides output similar to:
```
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 15 5 2.1GB 1.2GB (57%)
Containers 8 2 125MB 100MB (80%)
Local Volumes 3 1 50MB 25MB (50%)
Build Cache 0 0 0B 0B
```
Basic Docker Cleanup Commands
Removing Dangling Images
Dangling images are untagged images that are no longer referenced by any container. These are safe to remove:
```bash
List dangling images
docker images --filter "dangling=true"
Remove all dangling images
docker image prune
Remove dangling images without confirmation prompt
docker image prune -f
```
Removing Unused Images
Remove all images not currently used by any container:
```bash
Remove all unused images (not just dangling)
docker image prune -a
Remove unused images without confirmation
docker image prune -a -f
```
Removing Specific Images
Remove individual images by name or ID:
```bash
Remove by image name and tag
docker rmi nginx:latest
Remove by image ID
docker rmi 1a2b3c4d5e6f
Force remove (even if containers are using the image)
docker rmi -f nginx:latest
Remove multiple images
docker rmi nginx:latest ubuntu:18.04 alpine:3.12
```
Comprehensive System Cleanup
Remove all unused Docker objects (images, containers, volumes, networks):
```bash
Remove all unused objects
docker system prune
Remove everything including unused images
docker system prune -a
Remove everything without confirmation
docker system prune -a -f
Remove everything including volumes
docker system prune -a --volumes -f
```
Advanced Cleanup Strategies
Time-Based Cleanup
Remove images older than a specific timeframe:
```bash
Remove images created more than 24 hours ago
docker image prune -a --filter "until=24h"
Remove images older than 7 days
docker image prune -a --filter "until=168h"
Remove images created before a specific date
docker image prune -a --filter "until=2024-01-01T00:00:00"
```
Size-Based Cleanup
Target large images for removal:
```bash
List images sorted by size (largest first)
docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}" | sort -k3 -hr
Create a script to remove images larger than 1GB
#!/bin/bash
docker images --format "{{.Repository}}:{{.Tag}} {{.Size}}" | \
grep -E '[0-9]+(\.[0-9]+)?GB' | \
while read image size; do
echo "Removing large image: $image ($size)"
docker rmi "$image"
done
```
Label-Based Cleanup
Use labels to organize and clean up images:
```bash
Remove images with specific labels
docker image prune --filter "label=environment=development"
Remove images without specific labels
docker image prune --filter "label!=environment=production"
```
Pattern-Based Removal
Remove images matching specific patterns:
```bash
Remove all images from a specific registry
docker images | grep "myregistry.com" | awk '{print $3}' | xargs docker rmi
Remove images with specific naming pattern
docker images | grep "temp-" | awk '{print $1":"$2}' | xargs docker rmi
Remove all untagged images
docker images | grep "" | awk '{print $3}' | xargs docker rmi
```
Automated Cleanup Solutions
Creating Cleanup Scripts
Basic Cleanup Script
Create a comprehensive cleanup script:
```bash
#!/bin/bash
docker-cleanup.sh
set -e
echo "Starting Docker cleanup..."
Remove stopped containers
echo "Removing stopped containers..."
docker container prune -f
Remove dangling images
echo "Removing dangling images..."
docker image prune -f
Remove unused images older than 7 days
echo "Removing unused images older than 7 days..."
docker image prune -a -f --filter "until=168h"
Remove unused volumes
echo "Removing unused volumes..."
docker volume prune -f
Remove unused networks
echo "Removing unused networks..."
docker network prune -f
Show remaining usage
echo "Current Docker disk usage:"
docker system df
echo "Docker cleanup completed!"
```
Make the script executable:
```bash
chmod +x docker-cleanup.sh
```
Advanced Cleanup Script with Logging
```bash
#!/bin/bash
advanced-docker-cleanup.sh
LOGFILE="/var/log/docker-cleanup.log"
RETENTION_DAYS=7
MIN_FREE_SPACE_GB=5
log_message() {
echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOGFILE"
}
check_free_space() {
local free_space_gb=$(df /var/lib/docker | awk 'NR==2 {print int($4/1024/1024)}')
echo $free_space_gb
}
log_message "Starting Docker cleanup process"
Check available space before cleanup
SPACE_BEFORE=$(check_free_space)
log_message "Free space before cleanup: ${SPACE_BEFORE}GB"
Perform cleanup operations
log_message "Removing stopped containers..."
REMOVED_CONTAINERS=$(docker container prune -f 2>&1 | grep "Total reclaimed space" || echo "0B")
log_message "Containers cleanup result: $REMOVED_CONTAINERS"
log_message "Removing dangling images..."
REMOVED_IMAGES=$(docker image prune -f 2>&1 | grep "Total reclaimed space" || echo "0B")
log_message "Dangling images cleanup result: $REMOVED_IMAGES"
log_message "Removing unused images older than ${RETENTION_DAYS} days..."
REMOVED_OLD_IMAGES=$(docker image prune -a -f --filter "until=${RETENTION_DAYS}d" 2>&1 | grep "Total reclaimed space" || echo "0B")
log_message "Old images cleanup result: $REMOVED_OLD_IMAGES"
Check space after cleanup
SPACE_AFTER=$(check_free_space)
SPACE_FREED=$((SPACE_AFTER - SPACE_BEFORE))
log_message "Free space after cleanup: ${SPACE_AFTER}GB (freed: ${SPACE_FREED}GB)"
Alert if still low on space
if [ $SPACE_AFTER -lt $MIN_FREE_SPACE_GB ]; then
log_message "WARNING: Still low on disk space (${SPACE_AFTER}GB remaining)"
fi
log_message "Docker cleanup process completed"
```
Cron Job Automation
Automate cleanup with cron jobs:
```bash
Edit crontab
crontab -e
Add entries for automated cleanup
Daily cleanup at 2 AM
0 2 * /path/to/docker-cleanup.sh >> /var/log/docker-cleanup.log 2>&1
Weekly comprehensive cleanup on Sundays at 3 AM
0 3 0 /path/to/advanced-docker-cleanup.sh
Hourly dangling image cleanup during business hours
0 9-17 1-5 docker image prune -f >> /var/log/docker-cleanup.log 2>&1
```
Systemd Service for Cleanup
Create a systemd service for regular cleanup:
```bash
Create service file
sudo nano /etc/systemd/system/docker-cleanup.service
```
Service file content:
```ini
[Unit]
Description=Docker Cleanup Service
Requires=docker.service
After=docker.service
[Service]
Type=oneshot
ExecStart=/usr/local/bin/docker-cleanup.sh
User=root
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target
```
Create timer file:
```bash
sudo nano /etc/systemd/system/docker-cleanup.timer
```
Timer file content:
```ini
[Unit]
Description=Run Docker Cleanup Daily
Requires=docker-cleanup.service
[Timer]
OnCalendar=daily
Persistent=true
[Install]
WantedBy=timers.target
```
Enable and start the timer:
```bash
sudo systemctl daemon-reload
sudo systemctl enable docker-cleanup.timer
sudo systemctl start docker-cleanup.timer
```
Monitoring Docker Storage Usage
Real-Time Monitoring
Monitor Docker storage usage in real-time:
```bash
Watch Docker system usage
watch -n 30 'docker system df'
Monitor specific directory sizes
watch -n 60 'du -sh /var/lib/docker/overlay2'
Custom monitoring script
#!/bin/bash
while true; do
clear
echo "=== Docker Storage Monitor ==="
echo "Timestamp: $(date)"
echo
docker system df
echo
echo "=== Top 10 Largest Images ==="
docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}" | head -11
sleep 60
done
```
Setting Up Alerts
Create alerting for low disk space:
```bash
#!/bin/bash
docker-space-alert.sh
THRESHOLD_GB=10
EMAIL="admin@example.com"
AVAILABLE_SPACE=$(df /var/lib/docker | awk 'NR==2 {print int($4/1024/1024)}')
if [ $AVAILABLE_SPACE -lt $THRESHOLD_GB ]; then
MESSAGE="WARNING: Docker storage space is running low. Only ${AVAILABLE_SPACE}GB remaining."
echo "$MESSAGE" | mail -s "Docker Storage Alert" "$EMAIL"
logger "$MESSAGE"
fi
```
Prometheus Monitoring
For advanced monitoring, integrate with Prometheus:
```yaml
docker-compose.yml for monitoring stack
version: '3.7'
services:
node-exporter:
image: prom/node-exporter:latest
ports:
- "9100:9100"
volumes:
- /proc:/host/proc:ro
- /sys:/host/sys:ro
- /:/rootfs:ro
command:
- '--path.procfs=/host/proc'
- '--path.sysfs=/host/sys'
- '--collector.filesystem.ignored-mount-points=^/(sys|proc|dev|host|etc)($$|/)'
```
Troubleshooting Common Issues
Image in Use Errors
When encountering "image is being used by running container" errors:
```bash
List containers using the image
docker ps -a --filter ancestor=IMAGE_NAME
Stop and remove containers first
docker stop $(docker ps -q --filter ancestor=IMAGE_NAME)
docker rm $(docker ps -aq --filter ancestor=IMAGE_NAME)
Then remove the image
docker rmi IMAGE_NAME
```
Permission Denied Errors
Fix permission issues:
```bash
Add user to docker group
sudo usermod -aG docker $USER
Change ownership of Docker socket (temporary fix)
sudo chmod 666 /var/run/docker.sock
Restart Docker service
sudo systemctl restart docker
```
Storage Driver Issues
Handle storage driver problems:
```bash
Check current storage driver
docker info | grep "Storage Driver"
Clean up overlay2 storage
sudo systemctl stop docker
sudo rm -rf /var/lib/docker/overlay2/*
sudo systemctl start docker
Rebuild images if necessary
docker pull IMAGE_NAME
```
Network-Related Cleanup Issues
Resolve network cleanup problems:
```bash
Force remove networks
docker network rm $(docker network ls -q)
Remove networks not used by containers
docker network prune -f
Reset Docker networking
sudo systemctl stop docker
sudo ip link delete docker0
sudo systemctl start docker
```
Recovery from Failed Cleanup
Recover from cleanup operations gone wrong:
```bash
Check Docker daemon status
sudo systemctl status docker
Restart Docker daemon
sudo systemctl restart docker
Verify image integrity
docker images --digests
Re-pull critical images
docker pull nginx:latest
docker pull ubuntu:20.04
```
Best Practices
Development Environment Practices
1. Regular Cleanup Schedule: Implement daily or weekly cleanup routines
2. Use Multi-stage Builds: Reduce final image sizes
3. Leverage .dockerignore: Exclude unnecessary files from build context
4. Tag Images Appropriately: Use meaningful tags for better organization
```dockerfile
Multi-stage build example
FROM node:16-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
FROM node:16-alpine AS runtime
WORKDIR /app
COPY --from=builder /app/node_modules ./node_modules
COPY . .
EXPOSE 3000
CMD ["npm", "start"]
```
Production Environment Practices
1. Automated Cleanup: Use systemd timers or cron jobs
2. Monitoring Integration: Set up alerts for storage usage
3. Selective Cleanup: Preserve production images while cleaning development artifacts
4. Backup Strategy: Ensure critical images are backed up before cleanup
Storage Optimization Techniques
1. Image Layer Optimization:
```dockerfile
Bad: Creates multiple layers
RUN apt-get update
RUN apt-get install -y package1
RUN apt-get install -y package2
Good: Single layer
RUN apt-get update && \
apt-get install -y package1 package2 && \
rm -rf /var/lib/apt/lists/*
```
2. Use Alpine-based Images:
```bash
Instead of ubuntu:latest (72MB)
FROM alpine:3.15 # 5.6MB
Install packages
RUN apk add --no-cache package-name
```
3. Implement Build Cache Strategies:
```bash
Use BuildKit for better caching
export DOCKER_BUILDKIT=1
docker build --cache-from myapp:cache .
```
Security Considerations
1. Scan Images Before Cleanup: Ensure you're not removing security-patched versions
2. Maintain Image Inventory: Keep track of which images are in use
3. Implement Access Controls: Restrict who can perform cleanup operations
```bash
Scan images for vulnerabilities
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
-v $HOME/Library/Caches:/root/.cache/ \
aquasec/trivy image nginx:latest
```
Performance Optimization
1. Parallel Cleanup Operations:
```bash
Remove multiple images in parallel
echo "image1 image2 image3" | xargs -n 1 -P 3 docker rmi
```
2. Optimize Cleanup Timing: Run cleanup during low-usage periods
3. Use Filters Efficiently: Target specific images rather than broad cleanup
Conclusion
Effective Docker image cleanup is crucial for maintaining a healthy Linux system. This comprehensive guide has covered everything from basic cleanup commands to advanced automation strategies. Key takeaways include:
- Regular Maintenance: Implement automated cleanup schedules to prevent storage issues
- Selective Cleanup: Use filters and targeted approaches rather than broad cleanup operations
- Monitoring: Keep track of storage usage and set up alerts for proactive management
- Best Practices: Follow development and production guidelines for optimal Docker usage
Next Steps
1. Implement Basic Cleanup: Start with simple `docker system prune` commands
2. Create Automation: Set up cron jobs or systemd timers for regular cleanup
3. Add Monitoring: Implement storage monitoring and alerting
4. Optimize Images: Review and optimize your Docker images for better space efficiency
5. Document Procedures: Create runbooks for your team's cleanup procedures
Additional Resources
- Docker Documentation: Official Docker cleanup guides
- Container Best Practices: Industry standards for container management
- Monitoring Tools: Prometheus, Grafana, and other monitoring solutions
- Security Scanning: Tools like Trivy, Clair, and Anchore for image security
By following this guide and implementing these practices, you'll maintain a clean, efficient Docker environment that performs optimally while conserving valuable disk space. Remember to always test cleanup procedures in development environments before applying them to production systems, and maintain regular backups of critical images and data.
Regular Docker maintenance is not just about freeing up disk space—it's about maintaining system performance, security, and operational efficiency. Make Docker cleanup a standard part of your Linux system administration routine, and your infrastructure will thank you for it.