Troubleshooting Delivery Issues

The Ultimate Comprehensive Guide to Mastering Mail Log Analysis

Published · Updated 8 min read
Knowledge base article
Contents (47 sections)

Introduction

Mail log analysis is a critical skill for ensuring the smooth operation and security of your email server. This guide will equip you with the tools and knowledge to effectively monitor, analyze, and troubleshoot mail logs. Whether you're tracking login failures, email delivery issues, or suspicious activity, this comprehensive handbook covers everything you need to know.

Why Analyze Mail Logs

Mail logs provide valuable insights into your email server's operations, including:

  • Identifying login attempts and failures.

  • Monitoring email delivery and tracking errors.

  • Detecting and preventing unauthorized access.

  • Ensuring compliance with security and operational standards.

Prerequisites

  1. Root Access: Ensure you have root or administrative access to the server.

  2. Command-Line Tools: Familiarity with grep, tail, and awk commands.

Understanding Mail Logs

Overview of Mail Logs

Mail logs are essential for diagnosing email delivery issues, troubleshooting authentication problems, and ensuring server security. This guide covers the locations, formats, and examples of mail logs across various platforms and operating systems.

Mail Log Locations by OS

Linux-Based Servers

  • CentOS/RHEL
    Log Path: /var/log/maillog
    Default Format: Includes timestamp, service name, action, and error details.

  • Ubuntu/Debian
    Log Path: /var/log/mail.log
    Default Format: Provides detailed SMTP, IMAP, POP3, and delivery reports.

Control Panels

  • cPanel
    Logs for Mail: /var/log/maillog
    Example: SMTP, POP3, IMAP authentication logs.
    Specific Logs:

    • /var/log/exim_mainlog - Main mail transport logs.

    • /var/log/exim_rejectlog - Rejected emails.

    • /var/log/exim_paniclog - Critical Exim issues.

  • Plesk
    Logs for Mail: /var/log/maillog or /usr/local/psa/var/log/maillog
    Example: Includes postfix logs and email delivery details.

  • DirectAdmin
    Logs for Mail: /var/log/exim/mainlog
    Specific Actions Logged: Delivery statuses, rejection errors.

Windows-Based Servers

  • Microsoft Exchange Server
    Log Path: C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\Logs\MessageTracking
    Example: Detailed tracking of incoming and outgoing messages.

  • SmarterMail
    Log Path: C:\SmarterMail\Logs
    Example: Includes connection and delivery errors.

Common Log Formats

Log Example: Authentication Failure

code
Jan 5 12:00:01 server dovecot: auth: Failed login (auth failed, 2 attempts): user=<[email protected]>, method=PLAIN, rip=192.168.0.1

Details:

  • Jan 5 12:00:01: Timestamp.

  • dovecot: Service name.

  • auth: Failed login: Failure reason.

  • user=<[email protected]>: Email account targeted.

  • rip=192.168.0.1: Remote IP address.

Log Example: Rejected Email

code
Jan 5 12:15:30 server postfix/smtpd[1234]: NOQUEUE: reject: RCPT from unknown[192.168.0.2]: 554 5.7.1 Relay access denied; from=<[email protected]> to=<[email protected]>

Details:

  • postfix/smtpd[1234]: SMTP daemon logging the event.

  • 554 5.7.1 Relay access denied: Rejection reason.

Mastering Mail Log Analysis

Mail log analysis is essential for maintaining the smooth operation, security, and compliance of your email server. This guide equips you with the tools and techniques to monitor, troubleshoot, and optimize email performance effectively.

Analyzing Logs: Key Commands

1. General Authentication Failures

To locate all authentication failures:

code
grep -i "auth failed" /var/log/maillog

2. IMAP Login Failures

For isolating IMAP-specific login issues:

code
grep -i "imap-login failed" /var/log/maillog

3. SMTP Login Failures

For identifying SMTP-specific login problems:

code
grep -i "smtp-login failed" /var/log/maillog

4. Tracking Specific Email Accounts

To find failures linked to a specific email account:

code
grep -i "auth failed" /var/log/maillog | grep "[email protected]"

5. Monitoring Specific Domains

To filter logs for a specific domain:

code
grep -iE "auth failed|imap-login failed|smtp-login failed" /var/log/maillog | grep "example.com"

Daily Email Activity Summary

To analyze email activity (successful, deferred, and failed emails):

code
sudo grep "A=login" /var/log/exim/mainlog | grep "$(date +'%Y-%m-%d')" | \
awk -F"A=login:" '{print $2}' | awk '{print $1}' | sort | uniq -c | sort -nr

Command Breakdown:

  • grep "A=login": Filters for authenticated accounts.

  • grep "$(date +'%Y-%m-%d')": Retrieves logs for the current date.

  • awk -F"A=login:": Extracts the email address.

  • sort | uniq -c | sort -nr: Counts and sorts email occurrences.

Sample Output:

code
 1200 [email protected]
 800 [email protected]
 450 [email protected]

Breaking Down Email Statuses

1. Successful Emails

Command to extract successful email deliveries:

code
sudo grep "A=login" /var/log/exim/mainlog | grep "$(date +'%Y-%m-%d')" | grep "=>" | \
awk -F"A=login:" '{print $2}' | awk '{print $1}' | sort | uniq -c | sort -nr

2. Deferred Emails

Command to extract deferred emails:

code
sudo grep "A=login" /var/log/exim/mainlog | grep "$(date +'%Y-%m-%d')" | grep "defer" | \
awk -F"A=login:" '{print $2}' | awk '{print $1}' | sort | uniq -c | sort -nr

3. Failed Emails

Command to extract failed email attempts:

code
sudo grep "A=login" /var/log/exim/mainlog | grep "$(date +'%Y-%m-%d')" | grep "rejected" | \
awk -F"A=login:" '{print $2}' | awk '{print $1}' | sort | uniq -c | sort -nr

Multi-Day Analysis

To analyze email activity for the last few days:

code
for i in {0..2}; do # Change range for more days
 DATE=$(date -d "-$i day" +'%Y-%m-%d')
 echo "Logs for $DATE:"
 sudo grep "A=login" /var/log/exim/mainlog | grep "$DATE" | \
 awk -F"A=login:" '{print $2}' | awk '{print $1}' | sort | uniq -c | sort -nr
 echo "---------------------------------------------------------"
done

Command Breakdown:

  • Iterates through log files for multiple days.

  • Summarizes email activity for each day.

Sample Output:

code
Logs for 2025-01-12:
 192 [email protected]
 35 [email protected]
---------------------------------------------------------
Logs for 2025-01-11:
 150 [email protected]
 40 [email protected]
---------------------------------------------------------

Log File Locations for Control Panels

cPanel

  • Main Log Path: /var/log/exim_mainlog

DirectAdmin

  • Main Log Path: /var/log/exim/mainlog

Customizing Commands:

Update the log file path in your commands as per your control panel.

code
LOG_FILE="/var/log/exim/mainlog"
sudo grep "A=login" "$LOG_FILE" | grep "$(date +'%Y-%m-%d')" | \
awk -F"A=login:" '{print $2}' | awk '{print $1}' | sort | uniq -c | sort -nr

By mastering mail log analysis, you can:

  • Improve server performance.

  • Strengthen security.

  • Ensure compliance with operational standards. This guide equips you to handle real-world challenges efficiently. Happy logging!

Advanced Filtering

Combine all login failure types for a domain: grep -iE "auth failed|imap-login failed|smtp-login failed" /var/log/maillog | grep "example.com"

Email Delivery Analysis

1. Track Delivery Status

To analyze email delivery routes and statuses: grep -i "status=" /var/log/maillog

Status indicators:

  • status=sent: Email successfully delivered.

  • status=deferred: Temporary failure.

  • status=bounced: Permanent failure.

2. Check for Specific Recipients

Filter logs for emails sent to a specific recipient: grep -i "to=<[email protected]>" /var/log/maillog

3. Email Queue Monitoring

Check the current mail queue: postqueue -p

Security and Unauthorized Access

1. Failed Login Attempts

Locate failed login attempts: grep -iE "auth failed|invalid password" /var/log/maillog

2. Suspicious IP Activity

To identify repeated login failures from an IP: grep -i "auth failed" /var/log/maillog | awk '{print $NF}' | sort | uniq -c | sort -nr

3. Blocking Suspicious IPs

Use csf or iptables to block suspicious IPs: csf -d <ip_address> "Reason: Failed Login Attempts"

Additional Log Types to Consider

POP3 Login Failures

Analyze POP3-specific login failures:

code
 grep -i "pop3-login failed" /var/log/maillog

TLS/SSL Connection Errors

Identify issues with SSL/TLS connections:

code
 grep -i "ssl_error" /var/log/maillog

Email Queue Errors

Inspect errors in the email delivery queue:

code
 exim -bp | exiqgrep -i

SPF/DKIM/DMARC Failures

  • SPF Failures:

code
grep -i "spf check failed" /var/log/maillog

  • DKIM Failures:

    • code
    grep -i "dkim check failed" /var/log/maillog

  • DMARC Failures:

    • code
    grep -i "dmarc check failed" /var/log/maillog

    Rate-Limited Connections

    Analyze if users or IPs are hitting rate limits:

    code
     grep -i "rate limit exceeded" /var/log/maillog

    Relay Access Denied

    Logs showing attempts to relay email without permission:

    code
     grep -i "relay access denied" /var/log/maillog

    Domain-Specific Searches

    Isolate logs for a specific domain to narrow down the analysis:

    code
     grep -i "example.com" /var/log/maillog

    Outgoing Mail Analysis

    Track outgoing email failures to ensure messages are being delivered:

    code
     grep -i "rejected outgoing" /var/log/maillog

    Advanced Filtering Techniques

    Detailed Connection Logs

    Logs showing detailed SMTP session information:

    code
     grep -i "connection from" /var/log/maillog

    Advanced Filtering with awk and sed

    • Extract failed login attempts by username:

    code
    grep -i "auth failed" /var/log/maillog | awk '{print $NF}' | sort | uniq -c

  • Filter logs by date:

    • code
    grep -i "auth failed" /var/log/maillog | grep "2025-01-08"

    Suggested Enhancements

    Visualizing Logs

    Tip: Use tools like logwatch, GoAccess, or ELK Stack for graphical log representation. This can simplify monitoring and make patterns easier to identify.

    Email Reputation Monitoring

    Monitor IP reputation and avoid blacklists using tools like MXToolBox or by analyzing log data.

    Potential Extensions to the Guide

    • Add detailed steps for integrating log monitoring with centralized systems like Graylog or ELK Stack.

    • Provide examples of real-world troubleshooting scenarios for common issues.

    Advanced Monitoring Techniques

    1. Real-Time Monitoring

    Monitor logs in real-time for immediate troubleshooting: tail -f /var/log/maillog

    2. Log Rotation

    Ensure logs are rotated to prevent large file sizes: Check the configuration in /etc/logrotate.d/.

    3. Automating Log Analysis

    Create a script to analyze logs and report issues:

    code
    #!/bin/bash
grep -iE "auth failed|imap-login failed|smtp-login failed" /var/log/maillog | awk '{print $NF}' | sort | uniq -c | sort -nr

    Best Practices

    • Regularly monitor mail logs for unusual activity.

    • Use strong passwords and enable two-factor authentication.

    • Limit login attempts to prevent brute-force attacks.

    • Implement firewalls and intrusion detection systems.

    Conclusion

    Mail log analysis is an indispensable tool for server administrators. With this guide, you can effectively diagnose issues, enhance security, and ensure the smooth operation of your email services. By mastering these techniques, you'll stay ahead in maintaining a secure and reliable mail server environment.

    For more information, visit Understanding and Utilizing Mail Logs in Your Linux Server.

    Note: CentOS has reached end-of-life. If you are setting up a new server, we recommend using AlmaLinux or Rocky Linux as a drop-in replacement. The commands and procedures in this article apply equally to AlmaLinux and Rocky Linux.

    Was this article helpful?

    Your answer helps us decide what to improve next.

    Still need help? Open a support ticket and our team will reply.

    Related articles

    Prefer an app? Add this site to your home screen.Get the app
    Need Help?
    Live chatNew TicketTrack TicketKnowledge BaseContact UsWhatsApp
    The Ultimate Comprehensive Guide to Mastering Mail Log Analysis - Knowledge Base