How to vacuum old logs → journalctl --vacuum-time=7d
How to Vacuum Old Logs → journalctl --vacuum-time=7d
Table of Contents
- [Introduction](#introduction)
- [Prerequisites](#prerequisites)
- [Understanding Systemd Journal](#understanding-systemd-journal)
- [Basic Syntax and Usage](#basic-syntax-and-usage)
- [Step-by-Step Guide](#step-by-step-guide)
- [Practical Examples](#practical-examples)
- [Advanced Vacuum Options](#advanced-vacuum-options)
- [Troubleshooting Common Issues](#troubleshooting-common-issues)
- [Best Practices](#best-practices)
- [Performance Considerations](#performance-considerations)
- [Automation and Scheduling](#automation-and-scheduling)
- [Security Considerations](#security-considerations)
- [Conclusion](#conclusion)
Introduction
System log management is a critical aspect of maintaining healthy Linux systems. Over time, log files can accumulate and consume significant disk space, potentially causing system performance issues or even system failures due to full disk partitions. The `journalctl --vacuum-time=7d` command is a powerful tool for managing systemd journal logs by automatically removing entries older than a specified time period.
This comprehensive guide will teach you everything you need to know about using journalctl's vacuum functionality to efficiently manage your system logs. You'll learn how to clean up old log entries, optimize disk usage, and implement automated log rotation strategies that keep your system running smoothly while preserving important diagnostic information.
Whether you're a system administrator managing multiple servers or a Linux enthusiast maintaining your personal system, understanding how to properly vacuum old logs is essential for long-term system health and performance.
Prerequisites
Before diving into log vacuuming with journalctl, ensure you have the following:
System Requirements
- A Linux system running systemd (most modern distributions)
- systemd version 209 or later (for full vacuum functionality)
- Root or sudo privileges for system-wide log management
- Basic familiarity with command-line interface
Knowledge Prerequisites
- Understanding of Linux file systems and permissions
- Basic knowledge of systemd and its components
- Familiarity with log file concepts and importance
- Command-line experience with common Linux utilities
Verification Steps
Before proceeding, verify your system meets the requirements:
```bash
Check if systemd is running
systemctl --version
Verify journalctl is available
which journalctl
Check current journal status
sudo journalctl --disk-usage
```
Understanding Systemd Journal
What is Systemd Journal?
The systemd journal is a centralized logging system that collects and stores log data from various system components, applications, and services. Unlike traditional syslog systems that store logs in separate text files, the systemd journal uses a binary format that provides several advantages:
- Structured logging: Logs include metadata and structured fields
- Indexing: Fast searching and filtering capabilities
- Integrity: Built-in verification and tamper detection
- Compression: Efficient storage through automatic compression
Journal Storage Locations
The systemd journal stores log data in specific directories:
- System journal: `/var/log/journal/` (persistent storage)
- Runtime journal: `/run/log/journal/` (volatile storage)
- User journals: `/var/log/journal//` (per-user logs)
Why Vacuum Logs?
Log vacuuming is necessary for several reasons:
1. Disk Space Management: Prevents log files from consuming excessive storage
2. Performance Optimization: Reduces index size for faster log queries
3. System Stability: Prevents disk full conditions that can crash systems
4. Compliance: Helps meet data retention policies and regulations
5. Security: Removes old logs that might contain sensitive information
Basic Syntax and Usage
Command Structure
The basic syntax for vacuuming journal logs is:
```bash
journalctl --vacuum-time=
```
Time Specification Format
The time specification supports various formats:
- Days: `7d`, `30d`, `365d`
- Weeks: `1w`, `4w`, `52w`
- Months: `1month`, `6months`, `12months`
- Years: `1year`, `2years`
- Hours: `24h`, `168h`
- Minutes: `1440m` (for very short retention)
Permission Requirements
Most journal vacuum operations require elevated privileges:
```bash
Using sudo
sudo journalctl --vacuum-time=7d
As root user
journalctl --vacuum-time=7d
```
Step-by-Step Guide
Step 1: Assess Current Journal Usage
Before vacuuming logs, understand your current journal disk usage:
```bash
Check total journal disk usage
sudo journalctl --disk-usage
View detailed journal statistics
sudo journalctl --list-boots
Check journal configuration
sudo journalctl --header
```
Example output:
```
Archived and active journals take up 2.1G in the file system.
```
Step 2: Preview What Will Be Removed
It's good practice to understand what will be deleted before executing the vacuum command:
```bash
Check oldest log entries
sudo journalctl --since="1970-01-01" --until="7 days ago" | head -20
Count entries older than 7 days
sudo journalctl --since="1970-01-01" --until="7 days ago" | wc -l
```
Step 3: Execute the Vacuum Command
Run the vacuum command to remove logs older than 7 days:
```bash
sudo journalctl --vacuum-time=7d
```
Expected output:
```
Deleted archived journal /var/log/journal/a1b2c3d4e5f6/user-1000@a1b2c3d4e5f6-0000000000000001-0005a2b3c4d5e6f7.journal (8.0M).
Deleted archived journal /var/log/journal/a1b2c3d4e5f6/system@a1b2c3d4e5f6-0000000000000001-0005a2b3c4d5e6f8.journal (56.0M).
Vacuuming done, freed 64.0M of archived journals from /var/log/journal/a1b2c3d4e5f6.
```
Step 4: Verify the Results
Confirm the vacuum operation was successful:
```bash
Check new disk usage
sudo journalctl --disk-usage
Verify oldest remaining entries
sudo journalctl --since="8 days ago" | head -10
```
Step 5: Document the Operation
Keep records of maintenance operations:
```bash
Log the vacuum operation
echo "$(date): Vacuumed journal logs older than 7 days" | sudo tee -a /var/log/maintenance.log
```
Practical Examples
Example 1: Weekly Log Cleanup
Remove logs older than one week for regular maintenance:
```bash
Remove logs older than 7 days
sudo journalctl --vacuum-time=1w
Alternative syntax
sudo journalctl --vacuum-time=7d
```
Example 2: Monthly Archive Cleanup
For systems with moderate log generation, monthly cleanup might be sufficient:
```bash
Keep one month of logs
sudo journalctl --vacuum-time=1month
Keep 30 days of logs (equivalent)
sudo journalctl --vacuum-time=30d
```
Example 3: Aggressive Cleanup for Storage-Constrained Systems
For systems with limited storage, keep only recent logs:
```bash
Keep only 24 hours of logs
sudo journalctl --vacuum-time=24h
Keep only 3 days of logs
sudo journalctl --vacuum-time=3d
```
Example 4: Compliance-Driven Retention
For systems with specific compliance requirements:
```bash
Keep 90 days for quarterly compliance
sudo journalctl --vacuum-time=90d
Keep one year for annual compliance
sudo journalctl --vacuum-time=365d
```
Example 5: Emergency Disk Space Recovery
When disk space is critically low:
```bash
Keep only 1 hour of logs (emergency only)
sudo journalctl --vacuum-time=1h
Immediately check freed space
df -h /var/log
```
Advanced Vacuum Options
Vacuum by Size
Instead of time-based cleanup, you can limit journal size:
```bash
Keep only 100MB of journal data
sudo journalctl --vacuum-size=100M
Keep 1GB of journal data
sudo journalctl --vacuum-size=1G
```
Vacuum by Number of Files
Limit the number of journal files:
```bash
Keep only 10 journal files
sudo journalctl --vacuum-files=10
Keep only 5 journal files
sudo journalctl --vacuum-files=5
```
Combining Vacuum Options
You can combine multiple vacuum criteria:
```bash
Keep logs from last 7 days OR limit to 500MB (whichever is smaller)
sudo journalctl --vacuum-time=7d --vacuum-size=500M
```
User-Specific Journal Vacuuming
Clean up user-specific journals:
```bash
Vacuum current user's journal
journalctl --user --vacuum-time=7d
Vacuum specific user's journal (as root)
sudo journalctl --user-unit=user@1000.service --vacuum-time=7d
```
Troubleshooting Common Issues
Issue 1: Permission Denied Errors
Problem: Getting permission denied when running vacuum commands.
Symptoms:
```
Failed to vacuum: Permission denied
```
Solutions:
```bash
Ensure you're using sudo
sudo journalctl --vacuum-time=7d
Check if you're in the systemd-journal group
groups $USER
Add user to systemd-journal group (requires logout/login)
sudo usermod -a -G systemd-journal $USER
```
Issue 2: No Space Freed After Vacuum
Problem: Vacuum command runs but doesn't free expected disk space.
Symptoms:
```
Vacuuming done, freed 0B of archived journals.
```
Diagnosis:
```bash
Check if logs are within the specified timeframe
sudo journalctl --since="8 days ago" | head -1
Verify journal files exist
sudo ls -la /var/log/journal/*/
```
Solutions:
```bash
Try a more aggressive timeframe
sudo journalctl --vacuum-time=1d
Force rotation before vacuum
sudo systemctl kill --kill-who=main --signal=SIGUSR2 systemd-journald
sudo journalctl --vacuum-time=7d
```
Issue 3: Vacuum Operation Takes Too Long
Problem: Vacuum command hangs or takes excessive time.
Symptoms: Command doesn't return after several minutes.
Solutions:
```bash
Check journal service status
sudo systemctl status systemd-journald
Restart journal service if necessary
sudo systemctl restart systemd-journald
Run vacuum with verbose output
sudo journalctl --vacuum-time=7d --verbose
```
Issue 4: Journal Files Locked
Problem: Cannot vacuum because journal files are in use.
Symptoms:
```
Failed to vacuum: Device or resource busy
```
Solutions:
```bash
Rotate current journal files
sudo systemctl kill --kill-who=main --signal=SIGUSR2 systemd-journald
Wait a moment and retry
sleep 5
sudo journalctl --vacuum-time=7d
```
Issue 5: Insufficient Privileges for System Journals
Problem: Can only clean user journals, not system journals.
Solutions:
```bash
Ensure using sudo for system journals
sudo journalctl --vacuum-time=7d
Check systemd version supports vacuum
systemctl --version
Verify journal directory permissions
sudo ls -la /var/log/journal/
```
Best Practices
1. Establish Regular Maintenance Schedule
Create a consistent log maintenance routine:
```bash
Weekly cleanup script
#!/bin/bash
echo "$(date): Starting weekly log vacuum"
sudo journalctl --vacuum-time=7d
echo "$(date): Weekly log vacuum completed"
```
2. Monitor Disk Usage Before and After
Always check disk usage impact:
```bash
Pre-vacuum check
BEFORE=$(sudo journalctl --disk-usage | grep -o '[0-9.]*[KMGT]')
echo "Journal size before: $BEFORE"
Perform vacuum
sudo journalctl --vacuum-time=7d
Post-vacuum check
AFTER=$(sudo journalctl --disk-usage | grep -o '[0-9.]*[KMGT]')
echo "Journal size after: $AFTER"
```
3. Configure Persistent Journal Settings
Set up automatic log management in configuration:
```bash
Edit journal configuration
sudo nano /etc/systemd/journald.conf
Key settings to configure:
SystemMaxUse=1G
SystemMaxFileSize=100M
SystemMaxFiles=10
MaxRetentionSec=7day
```
4. Test Vacuum Operations in Non-Production
Always test vacuum procedures on development systems first:
```bash
Create test scenario
sudo journalctl --vacuum-time=1h # Very aggressive for testing
Verify system functionality
sudo systemctl status systemd-journald
sudo journalctl --since="1 hour ago" | head -10
```
5. Implement Graduated Retention Policies
Use different retention periods based on log importance:
```bash
Critical system logs - keep longer
sudo journalctl --unit=sshd --vacuum-time=30d
Application logs - shorter retention
sudo journalctl --unit=nginx --vacuum-time=7d
Debug logs - very short retention
sudo journalctl --priority=debug --vacuum-time=1d
```
6. Document Your Log Management Policy
Create clear documentation for your log retention strategy:
```markdown
Log Retention Policy
- System logs: 7 days
- Security logs: 30 days
- Application logs: 7 days
- Debug logs: 24 hours
- Emergency cleanup: 1 hour retention
```
Performance Considerations
Impact on System Resources
Vacuum operations can be resource-intensive:
- CPU Usage: Moderate during vacuum operation
- I/O Impact: High disk I/O during file deletion
- Memory Usage: Minimal additional memory required
- Network: No network impact for local operations
Optimal Timing for Vacuum Operations
Schedule vacuum operations during low-usage periods:
```bash
Run during off-hours (example: 2 AM daily)
0 2 * /usr/bin/sudo /usr/bin/journalctl --vacuum-time=7d
```
Monitoring Vacuum Performance
Track vacuum operation performance:
```bash
Time the vacuum operation
time sudo journalctl --vacuum-time=7d
Monitor system resources during vacuum
iostat -x 1 &
sudo journalctl --vacuum-time=7d
killall iostat
```
Automation and Scheduling
Creating Automated Vacuum Scripts
Develop robust automation scripts:
```bash
#!/bin/bash
/usr/local/bin/journal-vacuum.sh
LOG_FILE="/var/log/journal-vacuum.log"
RETENTION_DAYS="7d"
echo "$(date '+%Y-%m-%d %H:%M:%S'): Starting journal vacuum" >> "$LOG_FILE"
Check disk space before
DISK_BEFORE=$(df /var/log | tail -1 | awk '{print $4}')
Perform vacuum
if sudo journalctl --vacuum-time="$RETENTION_DAYS" >> "$LOG_FILE" 2>&1; then
# Check disk space after
DISK_AFTER=$(df /var/log | tail -1 | awk '{print $4}')
FREED=$((DISK_AFTER - DISK_BEFORE))
echo "$(date '+%Y-%m-%d %H:%M:%S'): Vacuum completed successfully, freed ${FREED}K" >> "$LOG_FILE"
else
echo "$(date '+%Y-%m-%d %H:%M:%S'): Vacuum failed" >> "$LOG_FILE"
exit 1
fi
```
Cron Job Setup
Configure automated execution:
```bash
Edit crontab
sudo crontab -e
Add daily vacuum at 3 AM
0 3 * /usr/local/bin/journal-vacuum.sh
Add weekly aggressive cleanup
0 3 0 /usr/bin/journalctl --vacuum-time=3d
```
Systemd Timer Alternative
Use systemd timers for more sophisticated scheduling:
```ini
/etc/systemd/system/journal-vacuum.service
[Unit]
Description=Journal Vacuum Service
After=multi-user.target
[Service]
Type=oneshot
ExecStart=/usr/bin/journalctl --vacuum-time=7d
User=root
```
```ini
/etc/systemd/system/journal-vacuum.timer
[Unit]
Description=Daily Journal Vacuum
Requires=journal-vacuum.service
[Timer]
OnCalendar=daily
Persistent=true
[Install]
WantedBy=timers.target
```
Enable the timer:
```bash
sudo systemctl enable journal-vacuum.timer
sudo systemctl start journal-vacuum.timer
```
Security Considerations
Log Integrity and Forensics
Consider security implications when vacuuming logs:
- Forensic Evidence: Old logs might contain important security information
- Compliance Requirements: Some regulations require longer retention periods
- Audit Trails: Maintain audit logs separately from general system logs
Secure Vacuum Practices
Implement secure log management:
```bash
Archive important logs before vacuum
sudo journalctl --since="7 days ago" --until="now" > /secure/archive/logs-$(date +%Y%m%d).log
Verify archive integrity
sha256sum /secure/archive/logs-$(date +%Y%m%d).log > /secure/archive/logs-$(date +%Y%m%d).log.sha256
Then perform vacuum
sudo journalctl --vacuum-time=7d
```
Access Control
Limit who can perform vacuum operations:
```bash
Create dedicated group for log management
sudo groupadd log-managers
Add specific users to the group
sudo usermod -a -G log-managers username
Configure sudo permissions
echo "%log-managers ALL=(root) NOPASSWD: /usr/bin/journalctl --vacuum-*" | sudo tee /etc/sudoers.d/journal-vacuum
```
Conclusion
Effective log management through journalctl vacuum operations is essential for maintaining healthy Linux systems. The `journalctl --vacuum-time=7d` command provides a powerful and flexible way to manage disk space while preserving important diagnostic information.
Key Takeaways
1. Regular Maintenance: Implement consistent log vacuuming schedules to prevent disk space issues
2. Flexible Options: Use time, size, or file-count based vacuum criteria based on your needs
3. Automation: Set up automated vacuum operations to reduce manual maintenance overhead
4. Monitoring: Always monitor disk usage and vacuum operation results
5. Security: Consider compliance and forensic requirements when setting retention policies
Next Steps
To implement effective log management in your environment:
1. Assess Current Usage: Analyze your current journal disk usage and growth patterns
2. Define Retention Policy: Establish clear log retention requirements based on your needs
3. Implement Automation: Set up automated vacuum operations using cron or systemd timers
4. Monitor and Adjust: Regularly review vacuum operation effectiveness and adjust retention periods as needed
5. Document Procedures: Create clear documentation for your log management processes
Final Recommendations
- Start with conservative retention periods and adjust based on experience
- Always test vacuum operations in non-production environments first
- Keep security and compliance requirements in mind when setting policies
- Monitor system performance and disk usage regularly
- Maintain backup and archival strategies for critical log data
By following the practices and techniques outlined in this guide, you'll be able to effectively manage your system logs, optimize disk usage, and maintain system performance while preserving important diagnostic and security information. Remember that log management is an ongoing process that requires regular attention and periodic review to ensure it continues to meet your evolving needs.