How to analyze disk usage with ncdu

How to Analyze Disk Usage with ncdu Table of Contents 1. [Introduction](#introduction) 2. [Prerequisites](#prerequisites) 3. [Installing ncdu](#installing-ncdu) 4. [Basic Usage and Navigation](#basic-usage-and-navigation) 5. [Advanced Features and Options](#advanced-features-and-options) 6. [Practical Examples and Use Cases](#practical-examples-and-use-cases) 7. [Command-Line Options](#command-line-options) 8. [Troubleshooting Common Issues](#troubleshooting-common-issues) 9. [Best Practices and Tips](#best-practices-and-tips) 10. [Comparison with Other Tools](#comparison-with-other-tools) 11. [Conclusion](#conclusion) Introduction Managing disk space is a critical aspect of system administration and personal computer maintenance. When your storage device starts running low on space, identifying which files and directories are consuming the most storage becomes essential. This is where ncdu (NCurses Disk Usage) proves invaluable. ncdu is a powerful, interactive command-line tool that provides a user-friendly interface for analyzing disk usage. Unlike traditional command-line utilities like `du`, ncdu offers an intuitive, navigable interface that allows you to explore your filesystem visually, making it easier to identify space-consuming files and directories quickly. In this comprehensive guide, you'll learn everything you need to know about ncdu, from basic installation and usage to advanced features and optimization techniques. Whether you're a system administrator managing servers or a regular user trying to free up space on your personal computer, this article will equip you with the knowledge to effectively analyze and manage disk usage using ncdu. Prerequisites Before diving into ncdu usage, ensure you have the following: System Requirements - A Unix-like operating system (Linux, macOS, BSD, or Windows with WSL) - Terminal or command-line access - Basic familiarity with command-line navigation - Sufficient permissions to read the directories you want to analyze Knowledge Requirements - Basic understanding of file system hierarchy - Familiarity with terminal commands - Understanding of file permissions (helpful but not mandatory) Hardware Considerations - Adequate RAM for processing large directory structures - CPU capable of handling recursive directory traversal - Storage space for temporary files (minimal requirement) Installing ncdu ncdu is available across multiple platforms and can be installed through various package managers. Linux Installation Ubuntu/Debian Systems ```bash sudo apt update sudo apt install ncdu ``` Red Hat/CentOS/Fedora Systems ```bash For RHEL/CentOS 7/8 sudo yum install ncdu For Fedora and newer RHEL versions sudo dnf install ncdu ``` Arch Linux ```bash sudo pacman -S ncdu ``` SUSE/openSUSE ```bash sudo zypper install ncdu ``` macOS Installation Using Homebrew ```bash brew install ncdu ``` Using MacPorts ```bash sudo port install ncdu ``` Windows Installation Windows Subsystem for Linux (WSL) If you're using WSL, follow the Linux installation instructions for your specific distribution. Using Windows Package Manager ```powershell winget install ncdu ``` Compiling from Source If ncdu isn't available through your package manager, you can compile it from source: ```bash Download the latest version wget https://dev.yorhel.nl/download/ncdu-2.3.tar.gz tar -xzf ncdu-2.3.tar.gz cd ncdu-2.3 Compile and install make sudo make install ``` Verifying Installation After installation, verify that ncdu is properly installed: ```bash ncdu --version ``` This should display the version information, confirming successful installation. Basic Usage and Navigation Starting ncdu The simplest way to use ncdu is to run it without any arguments, which will analyze the current directory: ```bash ncdu ``` To analyze a specific directory, provide the path as an argument: ```bash ncdu /home/username ncdu /var/log ncdu / ``` Understanding the Interface When ncdu starts, it first scans the specified directory (or current directory) to calculate disk usage. After scanning completes, you'll see an interactive interface with several components: Main Display Elements - Directory listing: Shows subdirectories and files sorted by size - Size information: Displays both apparent size and actual disk usage - Navigation path: Shows your current location in the filesystem - Status line: Provides information about selected items and available commands Size Display Format ncdu displays sizes in human-readable format by default: - B: Bytes - K: Kilobytes (1024 bytes) - M: Megabytes - G: Gigabytes - T: Terabytes Navigation Controls ncdu uses intuitive keyboard controls for navigation: Basic Movement - ↑/↓ arrows or k/j: Move up/down in the file list - Enter or →: Enter the selected directory - ← or Backspace: Go back to parent directory - Home/End: Jump to first/last item in the list Advanced Navigation - n: Sort by name (toggle ascending/descending) - s: Sort by size (default) - C: Sort by items count - M: Sort by modified time - r: Recalculate sizes (refresh) - g: Show percentage and/or graph - a: Toggle between apparent size and disk usage File Operations - d: Delete the selected file/directory (with confirmation) - t: Toggle between file and directory view - e: Show/hide hidden files - i: Show file information Other Controls - ?: Show help screen with all available commands - q: Quit ncdu Basic Example Let's walk through a basic example of using ncdu: ```bash Analyze the home directory ncdu ~ ``` After the scan completes, you'll see output similar to: ``` ncdu 2.3 ~ Use the arrow keys to navigate, press ? for help --- /home/username ----------------------------------------------- 48.2 GiB [##########] /Videos 12.1 GiB [## ] /Documents 8.9 GiB [# ] /Pictures 4.2 GiB [ ] /Downloads 2.1 GiB [ ] /Music 890.2 MiB [ ] /.cache 234.5 MiB [ ] /Desktop 89.3 MiB [ ] /.config ``` This display shows that the `Videos` directory is consuming the most space (48.2 GiB), followed by `Documents` (12.1 GiB), and so on. Advanced Features and Options Export and Import Functionality One of ncdu's powerful features is the ability to export scan results and import them later, which is particularly useful for: - Analyzing remote systems - Comparing disk usage over time - Working with slow storage devices Exporting Scan Results ```bash Export current directory scan to a file ncdu -o diskusage.txt Export specific directory scan ncdu -o /tmp/home-scan.txt /home ``` Importing Scan Results ```bash Import and analyze previously exported data ncdu -f diskusage.txt ``` This feature allows you to scan a directory once and analyze the results multiple times without rescanning. Excluding Files and Directories ncdu provides several options to exclude specific files or directories from analysis: Excluding Specific Directories ```bash Exclude multiple directories ncdu --exclude /proc --exclude /sys --exclude /dev / Using pattern matching ncdu --exclude '*.tmp' /home/username ``` Excluding Hidden Files ```bash Exclude all hidden files and directories ncdu --exclude-hidden /home/username ``` Using Exclude Files Create a file with patterns to exclude: ```bash Create exclude file echo "*.log" > exclude.txt echo "tmp/*" >> exclude.txt echo ".cache" >> exclude.txt Use exclude file ncdu --exclude-from exclude.txt /var ``` Cross-Filesystem Behavior By default, ncdu crosses filesystem boundaries. To limit analysis to a single filesystem: ```bash Stay on the same filesystem ncdu -x / ``` This is particularly useful when analyzing root filesystem without including mounted drives or network filesystems. Following Symbolic Links Control how ncdu handles symbolic links: ```bash Follow symbolic links ncdu -L /home/username Don't follow symbolic links (default) ncdu /home/username ``` Practical Examples and Use Cases Example 1: Finding Large Files in Home Directory ```bash Scan home directory ncdu ~ ``` Navigation process: 1. Start ncdu in your home directory 2. Use arrow keys to navigate to the largest directories 3. Press Enter to dive into subdirectories 4. Look for unexpectedly large files or directories 5. Use 'd' to delete unnecessary files (with caution) Example 2: System-Wide Disk Analysis ```bash Analyze entire system, excluding virtual filesystems ncdu --exclude /proc --exclude /sys --exclude /dev --exclude /run / ``` This command provides a comprehensive view of actual disk usage while excluding virtual filesystems that don't consume real disk space. Example 3: Log File Analysis ```bash Analyze log directories ncdu /var/log ``` Common findings: - Oversized log files that need rotation - Old log files that can be archived or deleted - Applications generating excessive logs Example 4: Finding Duplicate or Unnecessary Files While ncdu doesn't directly find duplicates, you can use it strategically: ```bash Check common locations for duplicates ncdu ~/Downloads ncdu ~/Desktop ncdu ~/Documents ``` Look for: - Multiple versions of the same software installers - Duplicate media files - Old backup files Example 5: Server Maintenance For server administrators, ncdu is invaluable for routine maintenance: ```bash Export scan for later analysis ncdu -o server-scan-$(date +%Y%m%d).txt / Analyze web server directories ncdu /var/www Check database directories ncdu /var/lib/mysql ``` Example 6: Development Environment Cleanup Developers can use ncdu to clean up development environments: ```bash Analyze project directories ncdu ~/projects Look for: - node_modules directories - Build artifacts - Cached dependencies - Old branches/checkouts ``` Example 7: Media Library Management For users with large media collections: ```bash Analyze media directories ncdu ~/Videos ncdu ~/Music ncdu ~/Pictures ``` Optimization strategies: - Identify low-quality duplicates - Find unorganized files - Locate large files that could be compressed Command-Line Options Complete Options Reference ```bash ncdu [OPTIONS] [DIRECTORY] ``` Scanning Options - `-o FILE`: Export scan results to file - `-f FILE`: Import scan results from file - `-r`: Read-only mode (no delete functionality) - `-e`: Enable extended information - `-x`: Stay on same filesystem - `-L`: Follow symbolic links Display Options - `-0`: Don't show progress during scan - `-1`: Don't show progress or color - `-2`: Disable color entirely - `--si`: Use base 10 (SI) prefixes instead of base 2 Filtering Options - `--exclude PATTERN`: Exclude files matching pattern - `--exclude-from FILE`: Exclude patterns from file - `--exclude-hidden`: Exclude hidden files - `--exclude-firmlinks`: Exclude firmlinks (macOS) Advanced Options - `--confirm-quit`: Require confirmation before quitting - `--color SCHEME`: Set color scheme (off/dark/default) Practical Command Combinations Complete System Analysis ```bash ncdu --exclude /proc --exclude /sys --exclude /dev --exclude /tmp -o system-scan.txt / ``` Safe Analysis Mode ```bash ncdu -r --exclude-hidden /home/username ``` Detailed Export with Exclusions ```bash ncdu -e -o detailed-scan.txt --exclude '*.tmp' --exclude-from ~/.ncdu-exclude / ``` Troubleshooting Common Issues Issue 1: Permission Denied Errors Problem: ncdu shows "Permission denied" for certain directories. Solutions: ```bash Run with sudo for system directories sudo ncdu / Or exclude inaccessible directories ncdu --exclude /root --exclude /lost+found / ``` Best Practice: Use sudo only when necessary and be extra cautious with the delete function. Issue 2: Slow Scanning Performance Problem: ncdu takes too long to scan large directories. Solutions: ```bash Exclude unnecessary directories ncdu --exclude /proc --exclude /sys / Use export/import for repeated analysis ncdu -o scan.txt /large/directory ncdu -f scan.txt # Subsequent analysis ``` Optimization tips: - Exclude virtual filesystems - Use SSD storage for better I/O performance - Increase available RAM Issue 3: Out of Memory Errors Problem: ncdu crashes or becomes unresponsive with very large directory structures. Solutions: - Analyze smaller directory subsets - Increase system swap space - Use exclusions to reduce memory footprint - Consider using ncdu on a system with more RAM Issue 4: Inaccurate Size Reporting Problem: Sizes don't match other tools like `du`. Explanation: This can occur due to: - Hard links counted differently - Sparse files - Filesystem-specific features Solutions: ```bash Toggle between apparent size and disk usage Press 'a' in ncdu interface Compare with du du -sh /path/to/directory ``` Issue 5: Terminal Display Issues Problem: ncdu interface appears corrupted or unreadable. Solutions: ```bash Disable colors ncdu --color off Use minimal interface ncdu -1 Check terminal compatibility echo $TERM ``` Issue 6: Cannot Delete Files Problem: Delete function ('d' key) doesn't work or shows errors. Causes and Solutions: - Read-only mode: Check if ncdu was started with `-r` option - Insufficient permissions: Use sudo if necessary - File in use: Close applications using the file - Immutable files: Check file attributes with `lsattr` Best Practices and Tips Performance Optimization 1. Strategic Exclusions Always exclude virtual filesystems and temporary directories: ```bash ncdu --exclude /proc --exclude /sys --exclude /dev --exclude /tmp / ``` 2. Use Export/Import for Large Scans For directories that take long to scan: ```bash Initial scan (run once) ncdu -o monthly-scan-$(date +%Y%m).txt / Analysis (run multiple times) ncdu -f monthly-scan-$(date +%Y%m).txt ``` 3. Incremental Analysis Don't analyze the entire system at once. Start with specific areas: ```bash Focus on user data first ncdu /home Then system directories ncdu /var Finally, application directories ncdu /opt ``` Safety Practices 1. Use Read-Only Mode Initially ```bash Safe exploration mode ncdu -r /important/directory ``` 2. Double-Check Before Deleting - Always verify file contents before deletion - Use the 'i' key to show detailed file information - Consider moving files to trash instead of permanent deletion 3. Backup Important Data Before cleaning up disk space: ```bash Create backups of important directories tar -czf backup-$(date +%Y%m%d).tar.gz /important/directory ``` Workflow Optimization 1. Regular Maintenance Schedule ```bash #!/bin/bash Weekly disk usage report script DATE=$(date +%Y%m%d) ncdu -o /var/log/disk-usage-$DATE.txt / echo "Disk usage report saved to /var/log/disk-usage-$DATE.txt" ``` 2. Automated Cleanup Identification Create scripts to identify cleanup candidates: ```bash #!/bin/bash Find large files for review ncdu -e -o large-files.txt /home grep "GiB\|[5-9][0-9][0-9] MiB" large-files.txt ``` 3. Documentation Keep records of: - Regular cleanup activities - Excluded directories and reasons - Disk usage trends over time Advanced Usage Patterns 1. Comparative Analysis ```bash Monthly comparison ncdu -o disk-usage-jan.txt / ncdu -o disk-usage-feb.txt / Compare files manually or with diff tools ``` 2. Multi-User Environments ```bash Analyze each user's space usage for user in $(ls /home); do echo "Analyzing $user..." ncdu -o /tmp/usage-$user.txt /home/$user done ``` 3. Server Monitoring Integration ```bash Generate reports for monitoring systems ncdu -1 -o /var/log/ncdu-report.txt / Parse output for monitoring system consumption ``` Keyboard Shortcuts Mastery Essential Shortcuts - ?: Always start here for help - r: Refresh when files change - g: Toggle graph view for visual analysis - a: Switch between apparent size and disk usage - n: Sort by name when looking for specific files Power User Shortcuts - t: Toggle file/directory view for different perspectives - e: Show/hide hidden files as needed - C: Sort by count to find directories with many files - M: Sort by modification time for temporal analysis Comparison with Other Tools ncdu vs. du | Feature | ncdu | du | |---------|------|-----| | Interface | Interactive, navigable | Command-line output only | | Sorting | Multiple sort options | Limited sorting | | Deletion | Built-in delete function | No deletion capability | | Export/Import | Yes | No | | Learning Curve | Moderate | Low | | Scripting | Limited | Excellent | ncdu vs. Graphical Tools Advantages of ncdu - Works over SSH connections - Lower resource usage - Faster for large directories - Scriptable through export/import - Available on all Unix-like systems When to Use Graphical Tools - Need visual tree maps - Prefer mouse interaction - Want integrated file management - Working locally on desktop systems ncdu vs. Other CLI Tools baobab (GNOME Disk Usage Analyzer) - ncdu advantage: Works in terminal, faster scanning - baobab advantage: Visual tree map, integrated with desktop qdirstat/kdirstat - ncdu advantage: Terminal-based, works remotely - GUI tools advantage: Visual representation, mouse interaction dust (Modern du alternative) - ncdu advantage: Mature, stable, interactive navigation - dust advantage: Colored output, modern syntax Conclusion ncdu is an indispensable tool for anyone serious about disk space management. Its combination of powerful analysis capabilities and intuitive interface makes it suitable for both casual users cleaning up their personal computers and system administrators managing complex server environments. Throughout this comprehensive guide, we've covered everything from basic installation and usage to advanced features and optimization techniques. The key takeaways include: Key Benefits of ncdu - Interactive navigation makes disk analysis intuitive and efficient - Export/import functionality enables flexible workflow management - Safety features help prevent accidental data loss - Cross-platform availability ensures consistent experience across systems - Powerful filtering options allow focused analysis of specific areas Best Practices Summary 1. Start with exclusions to improve performance and focus analysis 2. Use read-only mode when exploring unfamiliar directory structures 3. Export scan results for large directories to enable repeated analysis 4. Implement regular maintenance schedules for proactive disk management 5. Combine with other tools for comprehensive system administration Next Steps Now that you're equipped with comprehensive knowledge of ncdu, consider: 1. Implementing regular disk usage audits using ncdu in your maintenance routines 2. Creating automated scripts that leverage ncdu's export functionality 3. Exploring integration with monitoring systems for proactive space management 4. Sharing knowledge with team members to establish consistent disk management practices Final Recommendations - Practice using ncdu in safe environments before applying it to critical systems - Keep backups of important data before performing cleanup operations - Stay updated with ncdu releases for new features and improvements - Consider contributing to the ncdu project if you find it valuable Whether you're dealing with a full hard drive on your laptop or managing storage across multiple servers, ncdu provides the tools and insights needed to understand and optimize disk usage effectively. Its combination of power and simplicity makes it an essential addition to any system administrator's or power user's toolkit. Remember that effective disk space management is an ongoing process, not a one-time activity. Regular use of ncdu will help you maintain optimal system performance and prevent storage-related issues before they become critical problems.