How to monitor SSH login attempts in Linux

How to Monitor SSH Login Attempts in Linux SSH (Secure Shell) is the backbone of remote Linux server administration, but it's also a prime target for malicious attackers. Every day, thousands of automated bots and hackers attempt to breach SSH services through brute force attacks, dictionary attacks, and credential stuffing. Monitoring SSH login attempts is crucial for maintaining server security and identifying potential threats before they compromise your system. This comprehensive guide will teach you multiple methods to monitor SSH login attempts in Linux, from basic log analysis to advanced real-time monitoring solutions. Whether you're a system administrator managing multiple servers or a developer securing your personal VPS, you'll learn practical techniques to track, analyze, and respond to SSH authentication events. By the end of this article, you'll understand how to implement effective SSH monitoring strategies, set up automated alerts for suspicious activities, and create custom solutions tailored to your security requirements. Prerequisites and Requirements Before diving into SSH monitoring techniques, ensure you have the following prerequisites: System Requirements - A Linux server with SSH service running - Root or sudo access to the system - Basic command-line knowledge - Text editor familiarity (nano, vim, or emacs) Software Dependencies - OpenSSH server (usually pre-installed) - Log management tools (rsyslog or systemd-journal) - Optional: fail2ban, logwatch, or custom monitoring tools Network Access - SSH access to your Linux server - Ability to modify system configuration files - Understanding of your network topology and firewall rules Understanding SSH Logging in Linux SSH Log Locations SSH authentication attempts are logged in different locations depending on your Linux distribution and logging configuration: Common log file locations: ```bash Most distributions /var/log/auth.log # Debian/Ubuntu /var/log/secure # Red Hat/CentOS/Fedora /var/log/messages # Some older systems systemd-based systems journalctl -u ssh # systemd journal journalctl -u sshd # Alternative service name ``` SSH Log Format Understanding SSH logs contain valuable information about authentication attempts. Here's a typical log entry breakdown: ```bash Dec 15 10:30:45 server1 sshd[12345]: Failed password for invalid user admin from 192.168.1.100 port 45678 ssh2 ``` Log entry components: - Timestamp: Dec 15 10:30:45 - Hostname: server1 - Process: sshd[12345] - Event type: Failed password - Username: admin (invalid user) - Source IP: 192.168.1.100 - Source port: 45678 - Protocol: ssh2 Method 1: Basic Log Analysis with Command Line Tools Viewing Recent SSH Login Attempts Start with basic commands to examine SSH authentication logs: ```bash View recent authentication logs (Debian/Ubuntu) sudo tail -f /var/log/auth.log View recent authentication logs (Red Hat/CentOS) sudo tail -f /var/log/secure Filter SSH-specific entries sudo grep "sshd" /var/log/auth.log | tail -20 Show failed login attempts sudo grep "Failed password" /var/log/auth.log ``` Analyzing Successful Login Attempts Monitor successful SSH connections to track legitimate access: ```bash Show successful SSH logins sudo grep "Accepted password" /var/log/auth.log Show successful key-based authentication sudo grep "Accepted publickey" /var/log/auth.log Combined successful logins sudo grep -E "(Accepted password|Accepted publickey)" /var/log/auth.log ``` Identifying Failed Login Patterns Detect potential brute force attacks by analyzing failed attempts: ```bash Count failed attempts by IP address sudo grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -nr Show failed attempts for specific timeframe sudo grep "Failed password" /var/log/auth.log | grep "Dec 15" Find repeated failed attempts from same IP sudo grep "Failed password" /var/log/auth.log | grep "192.168.1.100" ``` Creating Custom Log Analysis Scripts Develop a comprehensive monitoring script: ```bash #!/bin/bash ssh_monitor.sh - SSH Login Monitoring Script LOG_FILE="/var/log/auth.log" REPORT_FILE="/tmp/ssh_report.txt" echo "SSH Login Monitoring Report - $(date)" > $REPORT_FILE echo "================================================" >> $REPORT_FILE Failed login attempts summary echo -e "\nFailed Login Attempts by IP:" >> $REPORT_FILE grep "Failed password" $LOG_FILE | awk '{print $11}' | sort | uniq -c | sort -nr | head -10 >> $REPORT_FILE Successful logins today echo -e "\nSuccessful Logins Today:" >> $REPORT_FILE grep -E "(Accepted password|Accepted publickey)" $LOG_FILE | grep "$(date '+%b %d')" >> $REPORT_FILE Invalid user attempts echo -e "\nInvalid User Attempts:" >> $REPORT_FILE grep "invalid user" $LOG_FILE | awk '{print $8, $10}' | sort | uniq -c | sort -nr | head -10 >> $REPORT_FILE Display report cat $REPORT_FILE ``` Make the script executable and run it: ```bash chmod +x ssh_monitor.sh ./ssh_monitor.sh ``` Method 2: Real-time Monitoring with systemd Journal Using journalctl for SSH Monitoring For systemd-based systems, leverage journalctl for real-time monitoring: ```bash Real-time SSH log monitoring sudo journalctl -u ssh -f Show SSH logs from last hour sudo journalctl -u ssh --since "1 hour ago" Filter by specific criteria sudo journalctl -u ssh --grep "Failed password" Show logs with full output sudo journalctl -u ssh -o verbose ``` Creating Custom journalctl Filters Set up specific filters for different monitoring needs: ```bash Monitor failed attempts in real-time sudo journalctl -u ssh -f --grep "Failed" Track specific user login attempts sudo journalctl -u ssh --grep "user root" Monitor from specific IP range sudo journalctl -u ssh --grep "192.168.1" ``` Automated journalctl Monitoring Script Create a script for continuous monitoring: ```bash #!/bin/bash journal_ssh_monitor.sh - Real-time SSH monitoring with journalctl ALERT_EMAIL="admin@example.com" FAILED_THRESHOLD=5 Function to send alert send_alert() { local ip=$1 local count=$2 echo "Alert: $count failed SSH attempts from $ip" | mail -s "SSH Security Alert" $ALERT_EMAIL } Monitor failed attempts journalctl -u ssh -f --grep "Failed password" | while read line; do ip=$(echo $line | awk '{print $11}') # Count recent attempts from this IP recent_count=$(journalctl -u ssh --since "10 minutes ago" --grep "Failed password" | grep $ip | wc -l) if [ $recent_count -ge $FAILED_THRESHOLD ]; then send_alert $ip $recent_count fi done ``` Method 3: Implementing fail2ban for Automated Protection Installing and Configuring fail2ban fail2ban is an essential tool for automated SSH protection: ```bash Install fail2ban (Debian/Ubuntu) sudo apt update sudo apt install fail2ban Install fail2ban (Red Hat/CentOS) sudo yum install epel-release sudo yum install fail2ban Start and enable fail2ban sudo systemctl start fail2ban sudo systemctl enable fail2ban ``` Configuring SSH Protection Rules Create a custom fail2ban configuration: ```bash Create local configuration file sudo nano /etc/fail2ban/jail.local ``` Add the following configuration: ```ini [DEFAULT] Ban time (in seconds) bantime = 3600 Find time window (in seconds) findtime = 600 Maximum retry attempts maxretry = 3 Email notifications destemail = admin@example.com sender = fail2ban@example.com action = %(action_mwl)s [sshd] enabled = true port = ssh filter = sshd logpath = /var/log/auth.log maxretry = 3 bantime = 3600 ``` Monitoring fail2ban Status Check fail2ban operation and banned IPs: ```bash Check fail2ban status sudo fail2ban-client status Check SSH jail status sudo fail2ban-client status sshd View banned IPs sudo fail2ban-client get sshd banip Unban specific IP sudo fail2ban-client set sshd unbanip 192.168.1.100 Check fail2ban logs sudo tail -f /var/log/fail2ban.log ``` Custom fail2ban Filters Create custom filters for specific attack patterns: ```bash Create custom filter sudo nano /etc/fail2ban/filter.d/custom-ssh.conf ``` Add custom filter rules: ```ini [Definition] Custom SSH filter for additional patterns failregex = ^%(__prefix_line)s(?:error: PAM: )?Authentication failure for . from \s$ ^%(__prefix_line)s(?:error: )?Received disconnect from : 3: .*: Auth fail$ ^%(__prefix_line)sFailed . for invalid user . from port \d+ ssh2$ ignoreregex = ``` Method 4: Advanced Monitoring with Custom Scripts Creating a Comprehensive Monitoring System Develop an advanced monitoring script with multiple features: ```bash #!/bin/bash advanced_ssh_monitor.sh - Comprehensive SSH monitoring solution CONFIG_FILE="/etc/ssh_monitor.conf" LOG_FILE="/var/log/auth.log" ALERT_LOG="/var/log/ssh_alerts.log" WHITELIST_FILE="/etc/ssh_whitelist.txt" Configuration variables FAILED_THRESHOLD=5 TIME_WINDOW=300 # 5 minutes BAN_DURATION=3600 # 1 hour EMAIL_ALERTS=true ADMIN_EMAIL="admin@example.com" Load configuration if exists if [ -f "$CONFIG_FILE" ]; then source "$CONFIG_FILE" fi Function to log alerts log_alert() { local message="$1" echo "$(date): $message" >> "$ALERT_LOG" if [ "$EMAIL_ALERTS" = true ]; then echo "$message" | mail -s "SSH Security Alert" "$ADMIN_EMAIL" fi } Function to check if IP is whitelisted is_whitelisted() { local ip="$1" if [ -f "$WHITELIST_FILE" ]; then grep -q "^$ip$" "$WHITELIST_FILE" return $? fi return 1 } Function to ban IP using iptables ban_ip() { local ip="$1" local duration="$2" if ! is_whitelisted "$ip"; then iptables -A INPUT -s "$ip" -j DROP log_alert "Banned IP $ip for $duration seconds due to excessive failed SSH attempts" # Schedule unban echo "iptables -D INPUT -s $ip -j DROP" | at now + ${duration} seconds 2>/dev/null fi } Main monitoring function monitor_ssh() { # Get failed attempts in the last TIME_WINDOW seconds local since_time=$(date -d "$TIME_WINDOW seconds ago" "+%b %d %H:%M:%S") # Extract failed attempts since the time window grep "Failed password" "$LOG_FILE" | awk -v since="$since_time" ' { log_time = $1 " " $2 " " $3 if (log_time >= since) { print $11 } }' | sort | uniq -c | while read count ip; do if [ "$count" -ge "$FAILED_THRESHOLD" ]; then ban_ip "$ip" "$BAN_DURATION" fi done } Generate monitoring report generate_report() { local report_file="/tmp/ssh_monitor_report_$(date +%Y%m%d_%H%M%S).txt" echo "SSH Monitoring Report - $(date)" > "$report_file" echo "======================================" >> "$report_file" echo -e "\nTop Failed Login Attempts (Last 24 hours):" >> "$report_file" grep "Failed password" "$LOG_FILE" | grep "$(date '+%b %d')" | \ awk '{print $11}' | sort | uniq -c | sort -nr | head -10 >> "$report_file" echo -e "\nSuccessful Logins (Last 24 hours):" >> "$report_file" grep -E "(Accepted password|Accepted publickey)" "$LOG_FILE" | \ grep "$(date '+%b %d')" | awk '{print $9, $11}' | sort | uniq >> "$report_file" echo -e "\nRecent Alerts:" >> "$report_file" tail -10 "$ALERT_LOG" >> "$report_file" echo "Report generated: $report_file" if [ "$EMAIL_ALERTS" = true ]; then cat "$report_file" | mail -s "Daily SSH Monitoring Report" "$ADMIN_EMAIL" fi } Command line options case "$1" in monitor) monitor_ssh ;; report) generate_report ;; status) echo "SSH Monitor Status:" echo "- Failed threshold: $FAILED_THRESHOLD" echo "- Time window: $TIME_WINDOW seconds" echo "- Ban duration: $BAN_DURATION seconds" echo "- Email alerts: $EMAIL_ALERTS" ;; *) echo "Usage: $0 {monitor|report|status}" exit 1 ;; esac ``` Setting up Automated Monitoring Create a cron job for continuous monitoring: ```bash Edit crontab crontab -e Add monitoring entries Run monitoring every 5 minutes /5 * /usr/local/bin/advanced_ssh_monitor.sh monitor Generate daily report at 6 AM 0 6 * /usr/local/bin/advanced_ssh_monitor.sh report ``` Configuration Management Create a configuration file for easy customization: ```bash Create configuration file sudo nano /etc/ssh_monitor.conf ``` Add configuration options: ```bash SSH Monitor Configuration FAILED_THRESHOLD=3 TIME_WINDOW=600 BAN_DURATION=7200 EMAIL_ALERTS=true ADMIN_EMAIL="security@example.com" WHITELIST_FILE="/etc/ssh_whitelist.txt" ENABLE_GEO_BLOCKING=true ALLOWED_COUNTRIES="US,CA,GB" ``` Method 5: Using Third-party Monitoring Tools Installing and Configuring Logwatch Logwatch provides comprehensive log analysis and reporting: ```bash Install logwatch (Debian/Ubuntu) sudo apt install logwatch Install logwatch (Red Hat/CentOS) sudo yum install logwatch Configure logwatch sudo nano /etc/logwatch/conf/logwatch.conf ``` Configure logwatch for SSH monitoring: ```bash Logwatch configuration for SSH LogDir = /var/log MailTo = admin@example.com MailFrom = logwatch@example.com Print = No Range = yesterday Detail = Med Service = sshd ``` Setting up OSSEC for SSH Monitoring OSSEC provides real-time log analysis and intrusion detection: ```bash Download and install OSSEC wget https://github.com/ossec/ossec-hids/archive/3.6.0.tar.gz tar -xzf 3.6.0.tar.gz cd ossec-hids-3.6.0 sudo ./install.sh ``` Configure OSSEC for SSH monitoring: ```xml syslog /var/log/auth.log sshd Failed password 5 300 SSH brute force attack detected ``` Implementing ELK Stack for SSH Log Analysis Set up Elasticsearch, Logstash, and Kibana for advanced SSH log analysis: ```yaml Logstash configuration for SSH logs input { file { path => "/var/log/auth.log" start_position => "beginning" tags => ["ssh"] } } filter { if "ssh" in [tags] { grok { match => { "message" => "%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:hostname} sshd\[%{POSINT:pid}\]: %{GREEDYDATA:ssh_message}" } } if "Failed password" in [ssh_message] { grok { match => { "ssh_message" => "Failed password for %{USERNAME:username} from %{IP:src_ip} port %{POSINT:src_port}" } add_tag => ["ssh_failed"] } } } } output { elasticsearch { hosts => ["localhost:9200"] index => "ssh-logs-%{+YYYY.MM.dd}" } } ``` Common Issues and Troubleshooting Log File Permissions and Access Issues Problem: Cannot access SSH log files Solution: ```bash Check log file permissions ls -la /var/log/auth.log Add user to appropriate group sudo usermod -a -G adm username Temporary access with sudo sudo tail -f /var/log/auth.log ``` Missing SSH Logs Problem: SSH authentication logs are not appearing Solution: ```bash Check SSH configuration sudo nano /etc/ssh/sshd_config Ensure logging is enabled LogLevel INFO SyslogFacility AUTH Restart SSH service sudo systemctl restart ssh ``` High Volume Log Management Problem: Log files growing too large Solution: ```bash Configure log rotation sudo nano /etc/logrotate.d/auth Add rotation rules /var/log/auth.log { weekly rotate 4 compress delaycompress missingok notifempty create 640 syslog adm } ``` False Positive Alerts Problem: Legitimate users triggering alerts Solution: ```bash Create whitelist file sudo nano /etc/ssh_whitelist.txt Add trusted IP addresses 192.168.1.100 10.0.0.50 203.0.113.25 Modify monitoring scripts to check whitelist if ! grep -q "^$ip$" /etc/ssh_whitelist.txt; then # Process alert fi ``` Performance Impact of Monitoring Problem: Monitoring scripts affecting system performance Solution: ```bash Use nice and ionice for resource management nice -n 10 ionice -c 3 ./ssh_monitor.sh Implement efficient log parsing Use awk instead of multiple grep commands awk '/Failed password/ {print $11}' /var/log/auth.log | sort | uniq -c ``` Best Practices and Security Tips SSH Hardening Recommendations Implement these SSH security measures alongside monitoring: ```bash SSH configuration hardening sudo nano /etc/ssh/sshd_config Recommended settings Port 2222 # Change default port PermitRootLogin no # Disable root login PasswordAuthentication no # Use key-based auth only MaxAuthTries 3 # Limit authentication attempts ClientAliveInterval 300 # Set session timeout ClientAliveCountMax 2 # Maximum alive messages AllowUsers username # Restrict allowed users ``` Monitoring Best Practices 1. Regular Log Review: Schedule daily log analysis 2. Alert Tuning: Adjust thresholds to minimize false positives 3. Backup Monitoring: Ensure monitoring systems are redundant 4. Documentation: Maintain incident response procedures 5. Testing: Regularly test monitoring and alerting systems Network Security Integration Combine SSH monitoring with network security measures: ```bash Implement network-level monitoring Use iptables for connection logging iptables -A INPUT -p tcp --dport 22 -j LOG --log-prefix "SSH_CONNECTION: " Monitor with tcpdump tcpdump -i eth0 port 22 -w ssh_traffic.pcap Analyze network patterns netstat -tn | grep :22 | awk '{print $5}' | cut -d: -f1 | sort | uniq -c ``` Incident Response Procedures Develop standardized response procedures: 1. Detection: Automated monitoring identifies suspicious activity 2. Analysis: Investigate the nature and scope of the threat 3. Containment: Block malicious IPs and limit access 4. Eradication: Remove any compromised accounts or backdoors 5. Recovery: Restore normal operations and strengthen security 6. Documentation: Record incident details and lessons learned Compliance and Auditing Ensure monitoring meets regulatory requirements: ```bash Create audit trails echo "$(date): SSH monitoring audit - $(whoami)" >> /var/log/audit.log Generate compliance reports ./ssh_monitor.sh report --compliance --format=json > compliance_report.json Implement log retention policies find /var/log -name "auth.log.*" -mtime +365 -delete ``` Advanced Monitoring Techniques Geolocation-based Monitoring Implement location-based access control: ```bash #!/bin/bash geo_ssh_monitor.sh - Geolocation-based SSH monitoring get_country() { local ip="$1" local country=$(geoiplookup "$ip" | awk -F': ' '{print $2}' | cut -d',' -f1) echo "$country" } Monitor and alert on foreign logins grep "Accepted" /var/log/auth.log | while read line; do ip=$(echo "$line" | awk '{print $11}') country=$(get_country "$ip") if [[ ! "$country" =~ ^(United States|Canada)$ ]]; then echo "Alert: SSH login from $country ($ip)" | \ mail -s "Foreign SSH Access Alert" admin@example.com fi done ``` Behavioral Analysis Implement user behavior analysis: ```bash #!/bin/bash behavior_analysis.sh - SSH user behavior monitoring analyze_user_patterns() { local username="$1" local log_file="/var/log/auth.log" echo "Analyzing behavior for user: $username" # Typical login times grep "Accepted.*for $username" "$log_file" | \ awk '{print $3}' | cut -d':' -f1 | sort | uniq -c # Login frequency grep "Accepted.*for $username" "$log_file" | \ awk '{print $1, $2}' | sort | uniq -c # Source IP patterns grep "Accepted.*for $username" "$log_file" | \ awk '{print $11}' | sort | uniq -c } Detect anomalous behavior detect_anomalies() { local username="$1" local current_hour=$(date +%H) local current_ip="$2" # Check if login time is unusual typical_hours=$(grep "Accepted.*for $username" /var/log/auth.log | \ awk '{print $3}' | cut -d':' -f1 | sort | uniq -c | \ awk '$1 > 5 {print $2}') if ! echo "$typical_hours" | grep -q "$current_hour"; then echo "Anomaly: Unusual login time for $username at hour $current_hour" fi } ``` Integration with SIEM Systems Configure integration with Security Information and Event Management systems: ```bash Syslog forwarding to SIEM echo ". @@siem-server:514" >> /etc/rsyslog.conf JSON format for structured logging logger -t ssh_monitor --rfc3164 \ "{\"event\":\"ssh_login\",\"user\":\"$username\",\"ip\":\"$ip\",\"status\":\"$status\"}" ``` Conclusion Monitoring SSH login attempts is a critical component of Linux server security that requires a multi-layered approach. Throughout this comprehensive guide, we've explored various methods ranging from basic log analysis to advanced behavioral monitoring systems. The key takeaways for effective SSH monitoring include: 1. Implement Multiple Monitoring Layers: Combine real-time monitoring, automated protection tools like fail2ban, and custom scripts for comprehensive coverage. 2. Regular Analysis and Reporting: Establish routine log analysis and generate regular reports to identify trends and potential security issues. 3. Automated Response Systems: Set up automated blocking and alerting mechanisms to respond quickly to threats while minimizing false positives. 4. Integration with Broader Security Strategy: SSH monitoring should be part of a comprehensive security strategy that includes network monitoring, intrusion detection, and incident response procedures. 5. Continuous Improvement: Regularly review and update monitoring configurations based on new threats, system changes, and lessons learned from security incidents. Next Steps To further enhance your SSH security monitoring: 1. Implement Key-based Authentication: Transition from password-based to key-based authentication where possible 2. Deploy Multi-factor Authentication: Add an additional security layer with 2FA/MFA solutions 3. Regular Security Audits: Conduct periodic security assessments and penetration testing 4. Staff Training: Ensure all team members understand SSH security best practices 5. Documentation: Maintain up-to-date documentation of monitoring procedures and incident response plans Remember that security monitoring is an ongoing process that requires regular attention and updates. Stay informed about new threats and monitoring techniques, and continuously adapt your monitoring strategy to address evolving security challenges. By implementing the techniques and best practices outlined in this guide, you'll significantly improve your ability to detect, analyze, and respond to SSH-based security threats, ultimately strengthening your overall Linux server security posture.