Efficient Log Management in Linux: Best Practices and Tools

Logs tell you what your server did, who logged in and why something broke. Left alone, they also fill disks and bury the one line you need. This guide covers how logging works on a current Linux server, and the practices and tools that keep logs useful, small and safe. It applies to your own VPS or server, where you have root access.

1. Why log management matters

• Troubleshooting: the error that explains an outage is almost always in a log.
• Security: failed logins, sudo use and firewall blocks are recorded, and are the first thing to check after a suspected break-in.
• Capacity: unrotated logs are one of the most common reasons a server's disk fills up.
• Compliance: some businesses must keep certain records for a set period. In India, the DPDP Act 2023 also expects you to protect personal data, and logs often contain IP addresses, email addresses and user names. This is general information, not legal advice.

2. Where Linux logs live in 2026

Almost every current distribution (Ubuntu, Debian, AlmaLinux, Rocky Linux) runs systemd-journald, which collects messages from the kernel, services and applications. Many also run rsyslog, which writes the classic text files. Debian 12 and later no longer install rsyslog by default, so on those systems the journal may be the only system log.

Applications you install yourself often log elsewhere, for example inside their own folder, or only to standard output when they run in Docker (docker logs name).

3. Reading the journal with journalctl

journalctl filters the journal far faster than searching text files by hand:

journalctl -b -p err                 # errors since the last boot
journalctl -u nginx --since "1 hour ago"
journalctl -u ssh -f                 # follow live (the unit is sshd on AlmaLinux/Rocky)
journalctl -k                        # kernel messages
journalctl --disk-usage              # how much space the journal uses

By default the journal may live only in memory and vanish at reboot. To keep it, make sure /var/log/journal exists, or set Storage=persistent in a drop-in file, and cap its size:

# /etc/systemd/journald.conf.d/00-size.conf
[Journal]
Storage=persistent
SystemMaxUse=500M
MaxRetentionSec=1month

Apply it with sudo systemctl restart systemd-journald. To free space right away, run sudo journalctl --vacuum-size=500M.

4. Rotate text logs with logrotate

logrotate is installed and scheduled by default on all these distributions. It renames the current log, compresses old ones and deletes the oldest. Package logs already have rules in /etc/logrotate.d/; add one for each application you install yourself:

/var/www/myapp/logs/*.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    create 0640 www-data adm
    sharedscripts
    postrotate
        systemctl reload myapp >/dev/null 2>&1 || true
    endscript
}

On AlmaLinux and Rocky Linux, the web server user is apache or nginx rather than www-data. Test a rule without changing anything with sudo logrotate -d /etc/logrotate.d/myapp, and force a run with -f once it looks right.

The postrotate step tells the program to reopen its log file. If an application can't do that, use copytruncate instead, accepting that a few lines written during the copy can be lost. Older guides reload rsyslog with /etc/init.d/rsyslog reload; on systemd systems the packaged rule already handles this, so you don't need to add it.

5. Keep logs secure

6. Centralise logs from several servers

Once you have more than one server, searching each one separately stops working. The common options:

A minimal rsyslog forward over TCP to a central server looks like this:

# /etc/rsyslog.d/90-forward.conf
*.* action(type="omfwd" target="logs.example.com" port="514" protocol="tcp"
           queue.type="LinkedList" action.resumeRetryCount="-1")

Plain TCP is unencrypted; add TLS or send it through a private network or VPN for anything leaving your own network. For a full working stack, see production observability with Prometheus, Grafana and Loki.

7. Watch log growth and automate alerts

• Check sizes with sudo du -sh /var/log/* | sort -h and journalctl --disk-usage. A log that suddenly grows fast usually means something is failing in a loop.
• Alert on patterns, not on volume alone. Repeated SSH failures, database crashes and disk errors are worth an alert; routine messages are not.
• Block brute-force attempts automatically. fail2ban reads the logs and bans addresses that keep failing to log in.
• Summaries: Logwatch still emails a daily digest on many systems, which is enough for a single small server.

8. Archive what you must keep

For logs you must keep longer than the server needs them, compress and move them off the server. tar with gzip or zstd works, and object storage is a cheap place to keep the archives. Record where archives live and when they may be deleted, and check that you can actually read an old one.

9. Running this on Domain India

On a Domain India VPS you have root access, so everything in this guide applies. VPS plans are self-managed: logging, rotation and retention are yours to set up. The OS choices are listed on the VPS page; pick a current release such as Ubuntu 24.04 LTS, Debian 12 or later, or AlmaLinux or Rocky Linux 9 or later.

On shared hosting you can't change server logging, but your own website logs are available. On cPanel, the logs folder in your home directory keeps compressed monthly archives of your site's traffic. Most requests are answered by the nginx front end, so its _NGINX archives are far larger than the Apache ones; read both. If an error log grows very large, managing large error log files in WordPress explains how to find and fix the cause.

Ready to set up a server you control? Compare plans on VPS hosting, follow the VPS security and optimization checklist, or open a support ticket if you have a question first.