How to create Docker images on Linux
How to Create Docker Images on Linux
Docker has revolutionized the way we develop, deploy, and manage applications by providing a lightweight containerization platform. Creating custom Docker images is a fundamental skill for developers and system administrators working with containerized applications. This comprehensive guide will walk you through everything you need to know about creating Docker images on Linux systems, from basic concepts to advanced techniques.
Table of Contents
1. [Introduction to Docker Images](#introduction-to-docker-images)
2. [Prerequisites and Requirements](#prerequisites-and-requirements)
3. [Understanding Docker Images vs Containers](#understanding-docker-images-vs-containers)
4. [Creating Your First Docker Image](#creating-your-first-docker-image)
5. [Dockerfile Fundamentals](#dockerfile-fundamentals)
6. [Step-by-Step Image Creation Process](#step-by-step-image-creation-process)
7. [Advanced Dockerfile Techniques](#advanced-dockerfile-techniques)
8. [Multi-stage Builds](#multi-stage-builds)
9. [Image Optimization Strategies](#image-optimization-strategies)
10. [Common Issues and Troubleshooting](#common-issues-and-troubleshooting)
11. [Best Practices and Security](#best-practices-and-security)
12. [Testing and Validating Images](#testing-and-validating-images)
13. [Conclusion and Next Steps](#conclusion-and-next-steps)
Introduction to Docker Images
Docker images serve as the foundation for containers, acting as read-only templates that contain everything needed to run an application: code, runtime, system tools, libraries, and settings. Think of a Docker image as a snapshot of a filesystem that includes your application and all its dependencies, packaged in a portable format that can run consistently across different environments.
When you create a Docker image, you're essentially building a blueprint that can be instantiated into running containers. This approach ensures that your application runs the same way whether it's on your development machine, a testing server, or in production.
Prerequisites and Requirements
Before diving into Docker image creation, ensure you have the following prerequisites:
System Requirements
- A Linux distribution (Ubuntu, CentOS, Debian, Fedora, etc.)
- Minimum 4GB RAM (8GB recommended)
- At least 20GB free disk space
- Internet connection for downloading base images and packages
Software Requirements
- Docker Engine installed and running
- Text editor (vim, nano, VS Code, etc.)
- Basic command-line knowledge
- Understanding of Linux file systems and permissions
Installing Docker on Linux
If Docker isn't already installed, here's how to install it on Ubuntu:
```bash
Update package index
sudo apt update
Install required packages
sudo apt install apt-transport-https ca-certificates curl gnupg lsb-release
Add Docker's official GPG key
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
Add Docker repository
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
Install Docker Engine
sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io
Add your user to docker group
sudo usermod -aG docker $USER
Verify installation
docker --version
```
Understanding Docker Images vs Containers
Before creating images, it's crucial to understand the distinction between Docker images and containers:
Docker Images:
- Read-only templates
- Static blueprints
- Layered file systems
- Immutable once created
- Can be shared and distributed
Docker Containers:
- Running instances of images
- Writable layer on top of image
- Can be started, stopped, and deleted
- Ephemeral by nature
- Isolated processes
Creating Your First Docker Image
Let's start with a simple example to create a basic Docker image that runs a Python web application.
Step 1: Create Project Directory
```bash
mkdir my-first-docker-image
cd my-first-docker-image
```
Step 2: Create Application Files
Create a simple Python web application:
```python
app.py
from flask import Flask
import os
app = Flask(__name__)
@app.route('/')
def hello():
return f"Hello from Docker! Container ID: {os.uname().nodename}"
@app.route('/health')
def health():
return {"status": "healthy", "version": "1.0"}
if __name__ == '__main__':
app.run(host='0.0.0.0', port=5000, debug=True)
```
Create a requirements file:
```text
requirements.txt
Flask==2.3.3
```
Step 3: Create Dockerfile
```dockerfile
Use official Python runtime as base image
FROM python:3.9-slim
Set working directory in container
WORKDIR /app
Copy requirements file
COPY requirements.txt .
Install Python dependencies
RUN pip install --no-cache-dir -r requirements.txt
Copy application code
COPY app.py .
Expose port 5000
EXPOSE 5000
Define environment variable
ENV FLASK_APP=app.py
Run the application
CMD ["python", "app.py"]
```
Step 4: Build the Image
```bash
docker build -t my-python-app:1.0 .
```
Step 5: Verify Image Creation
```bash
docker images | grep my-python-app
```
Step 6: Test the Image
```bash
docker run -p 5000:5000 my-python-app:1.0
```
Dockerfile Fundamentals
The Dockerfile is a text file containing instructions for building Docker images. Understanding each instruction is crucial for effective image creation.
Essential Dockerfile Instructions
FROM
Specifies the base image for your Docker image:
```dockerfile
Use specific version for reproducibility
FROM ubuntu:20.04
Use official language runtime
FROM python:3.9-slim
Use multi-architecture images
FROM --platform=linux/amd64 node:16-alpine
```
WORKDIR
Sets the working directory for subsequent instructions:
```dockerfile
WORKDIR /app
All subsequent commands will run in /app directory
```
COPY and ADD
Copy files from host to container:
```dockerfile
COPY is preferred for simple file copying
COPY source_file destination_file
COPY . /app
ADD has additional features (URL downloads, tar extraction)
ADD https://example.com/file.tar.gz /tmp/
```
RUN
Executes commands during image build:
```dockerfile
Install packages
RUN apt-get update && apt-get install -y \
curl \
wget \
vim \
&& rm -rf /var/lib/apt/lists/*
Create directories
RUN mkdir -p /app/logs
Set permissions
RUN chmod +x /app/start.sh
```
ENV
Sets environment variables:
```dockerfile
ENV NODE_ENV=production
ENV PORT=3000
ENV DATABASE_URL=postgresql://localhost/mydb
```
EXPOSE
Documents which ports the container listens on:
```dockerfile
EXPOSE 80
EXPOSE 443
EXPOSE 8080/tcp
```
CMD and ENTRYPOINT
Define the default command to run:
```dockerfile
CMD can be overridden at runtime
CMD ["python", "app.py"]
ENTRYPOINT cannot be overridden (but can accept parameters)
ENTRYPOINT ["python", "app.py"]
Combination: ENTRYPOINT + CMD
ENTRYPOINT ["python"]
CMD ["app.py"]
```
Step-by-Step Image Creation Process
Let's create a more comprehensive example with a Node.js application that includes multiple components.
Project Structure
```
node-docker-app/
├── Dockerfile
├── package.json
├── server.js
├── public/
│ ├── index.html
│ └── style.css
└── config/
└── database.js
```
Application Files
package.json:
```json
{
"name": "node-docker-app",
"version": "1.0.0",
"description": "Sample Node.js app for Docker",
"main": "server.js",
"scripts": {
"start": "node server.js",
"dev": "nodemon server.js"
},
"dependencies": {
"express": "^4.18.2",
"body-parser": "^1.20.2",
"cors": "^2.8.5"
},
"devDependencies": {
"nodemon": "^2.0.22"
}
}
```
server.js:
```javascript
const express = require('express');
const path = require('path');
const bodyParser = require('body-parser');
const cors = require('cors');
const app = express();
const PORT = process.env.PORT || 3000;
// Middleware
app.use(cors());
app.use(bodyParser.json());
app.use(express.static('public'));
// Routes
app.get('/', (req, res) => {
res.sendFile(path.join(__dirname, 'public', 'index.html'));
});
app.get('/api/health', (req, res) => {
res.json({
status: 'healthy',
timestamp: new Date().toISOString(),
uptime: process.uptime()
});
});
app.get('/api/info', (req, res) => {
res.json({
name: 'Node.js Docker App',
version: '1.0.0',
node_version: process.version,
platform: process.platform
});
});
app.listen(PORT, '0.0.0.0', () => {
console.log(`Server running on port ${PORT}`);
});
```
Advanced Dockerfile
```dockerfile
Multi-stage build for Node.js application
FROM node:18-alpine AS builder
Set working directory
WORKDIR /app
Copy package files
COPY package*.json ./
Install all dependencies (including dev dependencies)
RUN npm ci --only=production
Production stage
FROM node:18-alpine AS production
Create non-root user
RUN addgroup -g 1001 -S nodejs && \
adduser -S nextjs -u 1001
Set working directory
WORKDIR /app
Copy package files
COPY package*.json ./
Install only production dependencies
RUN npm ci --only=production && npm cache clean --force
Copy application files
COPY --chown=nextjs:nodejs . .
Copy node_modules from builder stage
COPY --from=builder --chown=nextjs:nodejs /app/node_modules ./node_modules
Create necessary directories
RUN mkdir -p /app/logs && chown -R nextjs:nodejs /app/logs
Switch to non-root user
USER nextjs
Expose port
EXPOSE 3000
Health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD curl -f http://localhost:3000/api/health || exit 1
Set environment variables
ENV NODE_ENV=production
ENV PORT=3000
Start application
CMD ["npm", "start"]
```
Building with Build Arguments
```dockerfile
Dockerfile with build arguments
FROM node:${NODE_VERSION:-18}-alpine
ARG BUILD_DATE
ARG VERSION
ARG VCS_REF
Labels for metadata
LABEL maintainer="your-email@example.com" \
version="${VERSION}" \
description="Node.js Docker Application" \
build-date="${BUILD_DATE}" \
vcs-ref="${VCS_REF}"
Rest of Dockerfile...
```
Build with arguments:
```bash
docker build \
--build-arg NODE_VERSION=18 \
--build-arg BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ') \
--build-arg VERSION=1.0.0 \
--build-arg VCS_REF=$(git rev-parse --short HEAD) \
-t node-docker-app:1.0.0 .
```
Advanced Dockerfile Techniques
Using .dockerignore
Create a `.dockerignore` file to exclude unnecessary files:
```
node_modules
npm-debug.log
.git
.gitignore
README.md
.env
.nyc_output
coverage
.nyc_output
.coverage
*.md
.DS_Store
```
Layer Optimization
Optimize layers for better caching:
```dockerfile
FROM python:3.9-slim
Install system dependencies first (changes rarely)
RUN apt-get update && apt-get install -y \
gcc \
g++ \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
Copy requirements first (changes less frequently than code)
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
Copy code last (changes most frequently)
COPY . .
CMD ["python", "app.py"]
```
Conditional Instructions
Use conditional logic in Dockerfiles:
```dockerfile
FROM ubuntu:20.04
ARG ENVIRONMENT=production
Install different packages based on environment
RUN if [ "$ENVIRONMENT" = "development" ]; then \
apt-get update && apt-get install -y \
vim \
curl \
git; \
fi
Set different configurations
ENV LOG_LEVEL=${ENVIRONMENT:+debug}
```
Multi-stage Builds
Multi-stage builds allow you to create smaller, more secure production images:
Go Application Example
```dockerfile
Build stage
FROM golang:1.19-alpine AS builder
WORKDIR /app
Copy go mod files
COPY go.mod go.sum ./
RUN go mod download
Copy source code
COPY . .
Build the application
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main .
Production stage
FROM alpine:3.16
Install ca-certificates for HTTPS
RUN apk --no-cache add ca-certificates tzdata
WORKDIR /root/
Copy binary from builder stage
COPY --from=builder /app/main .
Expose port
EXPOSE 8080
Run the application
CMD ["./main"]
```
React Application Example
```dockerfile
Build stage
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build
Production stage
FROM nginx:alpine
Copy built application
COPY --from=builder /app/build /usr/share/nginx/html
Copy custom nginx configuration
COPY nginx.conf /etc/nginx/nginx.conf
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
```
Image Optimization Strategies
Minimize Image Size
1. Use Alpine Linux base images:
```dockerfile
FROM python:3.9-alpine
FROM node:18-alpine
FROM nginx:alpine
```
2. Remove package caches:
```dockerfile
RUN apt-get update && apt-get install -y \
package1 \
package2 \
&& rm -rf /var/lib/apt/lists/*
```
3. Use multi-stage builds:
```dockerfile
FROM golang:1.19 AS builder
Build application
FROM scratch
COPY --from=builder /app/binary /binary
CMD ["/binary"]
```
Security Optimization
1. Use non-root users:
```dockerfile
RUN adduser -D -s /bin/sh appuser
USER appuser
```
2. Scan for vulnerabilities:
```bash
docker scout cves my-image:latest
```
3. Use specific image versions:
```dockerfile
FROM python:3.9.16-slim
Instead of FROM python:3.9-slim
```
Common Issues and Troubleshooting
Build Context Too Large
Problem: Docker build is slow due to large build context.
Solution:
```bash
Check build context size
du -sh .
Use .dockerignore to exclude files
echo "*.log" >> .dockerignore
echo "node_modules" >> .dockerignore
echo ".git" >> .dockerignore
```
Permission Denied Errors
Problem: Files created in container have wrong permissions.
Solution:
```dockerfile
Create user with specific UID/GID
RUN groupadd -r appgroup -g 1000 && \
useradd -r -g appgroup -u 1000 appuser
Set ownership
COPY --chown=appuser:appgroup . /app
USER appuser
```
Layer Caching Issues
Problem: Docker isn't using cached layers effectively.
Solution:
```dockerfile
Order instructions from least to most frequently changing
FROM python:3.9-slim
System packages (rarely change)
RUN apt-get update && apt-get install -y curl
Dependencies (change occasionally)
COPY requirements.txt .
RUN pip install -r requirements.txt
Application code (changes frequently)
COPY . .
```
Image Won't Start
Problem: Container exits immediately after starting.
Debugging steps:
```bash
Check logs
docker logs container_name
Run interactively
docker run -it image_name /bin/bash
Override entrypoint
docker run -it --entrypoint /bin/bash image_name
Check image history
docker history image_name
```
Network Connectivity Issues
Problem: Application can't connect to external services.
Solutions:
```dockerfile
Install CA certificates
RUN apt-get update && apt-get install -y ca-certificates
Set DNS servers
docker run --dns=8.8.8.8 image_name
Use host network for debugging
docker run --network host image_name
```
Best Practices and Security
Security Best Practices
1. Use official base images:
```dockerfile
FROM python:3.9-slim-bullseye
Instead of FROM ubuntu:latest
```
2. Keep images updated:
```bash
Regularly update base images
docker pull python:3.9-slim
docker build --no-cache -t my-app:latest .
```
3. Scan for vulnerabilities:
```bash
Use Docker Scout
docker scout cves my-image:latest
Use third-party tools
trivy image my-image:latest
```
4. Use secrets management:
```dockerfile
Don't embed secrets in images
Use Docker secrets or environment variables
```
Performance Best Practices
1. Optimize layer order:
```dockerfile
Dependencies first
COPY requirements.txt .
RUN pip install -r requirements.txt
Code last
COPY . .
```
2. Use specific versions:
```dockerfile
FROM python:3.9.16-slim
RUN pip install flask==2.3.3
```
3. Clean up in the same layer:
```dockerfile
RUN apt-get update && \
apt-get install -y package && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
```
Maintenance Best Practices
1. Use meaningful tags:
```bash
docker build -t my-app:v1.2.3 .
docker build -t my-app:latest .
```
2. Document your images:
```dockerfile
LABEL maintainer="team@company.com"
LABEL version="1.0"
LABEL description="Production web application"
```
3. Implement health checks:
```dockerfile
HEALTHCHECK --interval=30s --timeout=3s --retries=3 \
CMD curl -f http://localhost:8080/health || exit 1
```
Testing and Validating Images
Automated Testing
Create a test script:
```bash
#!/bin/bash
test-image.sh
IMAGE_NAME="my-app:latest"
CONTAINER_NAME="test-container"
Build image
docker build -t $IMAGE_NAME .
Start container
docker run -d --name $CONTAINER_NAME -p 8080:8080 $IMAGE_NAME
Wait for container to start
sleep 10
Test health endpoint
if curl -f http://localhost:8080/health; then
echo "Health check passed"
else
echo "Health check failed"
exit 1
fi
Test main endpoint
if curl -f http://localhost:8080/; then
echo "Main endpoint test passed"
else
echo "Main endpoint test failed"
exit 1
fi
Cleanup
docker stop $CONTAINER_NAME
docker rm $CONTAINER_NAME
echo "All tests passed!"
```
Image Analysis
```bash
Analyze image layers
docker history my-app:latest
Check image size
docker images my-app:latest
Inspect image configuration
docker inspect my-app:latest
Check for vulnerabilities
docker scout cves my-app:latest
```
Conclusion and Next Steps
Creating Docker images on Linux is a fundamental skill that enables you to package and distribute applications consistently across different environments. Throughout this guide, we've covered:
- Basic Docker image concepts and terminology
- Step-by-step image creation processes
- Advanced Dockerfile techniques and optimizations
- Multi-stage builds for smaller, more secure images
- Common troubleshooting scenarios and solutions
- Security and performance best practices
- Testing and validation strategies
Next Steps
To further enhance your Docker image creation skills:
1. Explore Container Registries: Learn to push images to Docker Hub, Amazon ECR, or Google Container Registry
2. Implement CI/CD Pipelines: Automate image building and deployment using GitHub Actions, GitLab CI, or Jenkins
3. Study Kubernetes: Understand how Docker images work in orchestrated environments
4. Learn Docker Compose: Master multi-container application development
5. Security Hardening: Implement advanced security scanning and compliance checking
6. Performance Monitoring: Set up monitoring and logging for containerized applications
Additional Resources
- [Docker Official Documentation](https://docs.docker.com/)
- [Dockerfile Best Practices](https://docs.docker.com/develop/dev-best-practices/)
- [Docker Security](https://docs.docker.com/engine/security/)
- [Multi-stage Builds](https://docs.docker.com/develop/dev-best-practices/#use-multi-stage-builds)
Remember that mastering Docker image creation is an iterative process. Start with simple images and gradually incorporate more advanced techniques as you become comfortable with the fundamentals. Regular practice and staying updated with Docker's evolving features will help you create efficient, secure, and maintainable container images for your applications.