When a site uses far more bandwidth than its visitors explain, the access log tells you who is downloading what. This guide shows where to find your raw access logs on Domain India hosting, the few commands that turn them into a list of the heaviest IP addresses, URLs and bots, and the fixes that actually cut the traffic.
On cPanel hosting, monthly logs are in the logs folder of your home directory. Use the file ending in _NGINX.gz, because most requests are answered by the caching proxy and never reach the Apache log. Sum the bytes column by IP and by URL to find what is heavy, check whether a "Googlebot" is genuine, then fix the cause: compress media, block unwanted bots, turn on hotlink protection and fix errors that make crawlers retry. If you just need the site back after a "Bandwidth Limit Exceeded" page, start with the bandwidth guide linked below.
Getting back online, plan allowances and when to upgrade are covered in why does my website say bandwidth limit exceeded. This page is the deeper log analysis.
1. Start with the summaries
Before reading raw logs, look at the summaries in cPanel's Metrics section:
- Bandwidth shows transfer by month, split into web, FTP and email.
- Awstats lists the pages, files, visitors and robots that transferred the most. It reads both the proxy and Apache logs, so its totals are complete.
- Raw Access lets you download the current logs and controls whether monthly logs are archived in your home directory.
If Awstats already names the culprit (one large file, one robot), you may not need the raw logs at all.
2. Where the raw logs are
cPanel. Monthly archives are in ~/logs/, one set per domain and month:
example.com-ssl_log-Sep-2026.gz HTTPS requests that reached Apache
example.com-ssl_log-Sep-2026_NGINX.gz HTTPS requests handled by the front proxy
example.com-Sep-2026.gz plain HTTP, Apache
example.com-Sep-2026_NGINX.gz plain HTTP, front proxyOur cPanel servers run nginx in front of Apache and cache pages and files. A cached request is answered by nginx and never reaches Apache, so the Apache log holds only a small part of the traffic. On one site we checked in September 2026, the proxy log had about 30 times as many lines as the Apache log for the same month. Use the _NGINX files for bandwidth analysis. The Apache files are still useful to see which requests reached PHP.
DirectAdmin. Older logs are archived per domain in domains/yourdomain.com/logs/ as .tar.gz files; extract one before reading it. Webuzo: ask support where your logs are kept.
You can open the files with File Manager (see how do I use the File Manager) or download them and analyse them on your computer. With jailed SSH you can run the commands below on the server. Jailed SSH is available on every shared hosting plan; it is off by default, so ask support to enable it for your account. See enabling and accessing jailed SSH.
3. Read a log line
Both logs use the standard combined format:
203.0.113.45 - - [22/Sep/2026:10:15:32 +0530] "GET /images/banner.jpg HTTP/2.0" 200 482113 "https://example.com/" "Mozilla/5.0 ..."Split on spaces, field 1 is the client IP, field 7 the URL, field 9 the status code and field 10 the bytes sent. A - in field 10 means no body was sent.
4. Find the heaviest IPs, URLs and bots
Set the file once, then run each command:
LOG=~/logs/example.com-ssl_log-Sep-2026_NGINX.gz
# Top 20 IP addresses by data transferred (MB)
zcat "$LOG" | awk '$10 ~ /^[0-9]+$/ {b[$1]+=$10} END {for (i in b) printf "%10.1f MB %s\n", b[i]/1048576, i}' | sort -nr | head -20
# Top 20 URLs by data transferred (query strings removed)
zcat "$LOG" | awk '$10 ~ /^[0-9]+$/ {split($7,u,"?"); b[u[1]]+=$10} END {for (x in b) printf "%10.1f MB %s\n", b[x]/1048576, x}' | sort -nr | head -20
# Top 20 user agents by request count
zcat "$LOG" | awk -F'"' '{print $6}' | sort | uniq -c | sort -nr | head -20
# Status codes
zcat "$LOG" | awk '{print $9}' | sort | uniq -c | sort -nrFor DirectAdmin, extract the archive first and use cat instead of zcat.
5. Check whether "Googlebot" is really Google
Anyone can put "Googlebot" in a user agent. Verify with a reverse and forward DNS lookup:
host 66.249.66.1 # should end in googlebot.com or google.com
host crawl-66-249-66-1.googlebot.com # should return the same IPIf the name does not end in googlebot.com or google.com, or the forward lookup does not match, it is not Google and you can block it. Never block the real Googlebot from your whole site: your pages would drop out of search.
6. Fix what you found
Most fixes go in the .htaccess file of your site. mod_rewrite is enabled on our shared servers.
Block bots you don't need:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (MJ12bot|DotBot|PetalBot|AhrefsBot|SemrushBot) [NC]
RewriteRule .* - [F,L]Stop other sites hotlinking your files (cPanel also has Hotlink Protection under Security):
RewriteEngine On
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^https?://(www\.)?example\.com/ [NC]
RewriteRule \.(jpe?g|png|gif|webp|pdf|mp4|mp3|zip)$ - [F,NC,L]Block a single abusive IP. Use cPanel's IP Blocker in the Security section, or in .htaccess:
<RequireAll>
Require all granted
Require not ip 203.0.113.45
</RequireAll>IP Blocker writes an equivalent rule for you, so it is the safer choice if you are new to .htaccess.
Block xmlrpc.php on WordPress if you don't use apps or Jetpack that need it:
<Files xmlrpc.php>
Require all denied
</Files>Keep crawlers out of endless URLs such as filters, search results and session parameters, with robots.txt:
User-agent: *
Disallow: /*?s=
Disallow: /*?j=Google ignores Crawl-delay, and Search Console's old crawl-rate setting has been retired, so fix the URLs instead. Returning errors on those pages makes Google crawl them more, not less.
A typing mistake in .htaccess gives a 500 error for the whole site. Keep a copy of the working file and open the site in a private window after each change. Our servers run PHP through FPM or CGI, so php_value lines in .htaccess also cause a 500. On cPanel, add ?t=1 to the address to skip the page cache while testing.
Fix recurring 500 errors. Your site's error log shows why a page fails; see reviewing error logs in cPanel and DirectAdmin and troubleshooting 500 internal server error. On shared hosting you read your own logs; the server-wide Apache error log is not available to accounts.
Shrink what is left: resize and convert images to WebP, host videos on a video platform and embed them, move large downloads to file storage, and put a CDN in front of the site. For WordPress caching on our shared hosting, which runs Apache, use WP Super Cache or W3 Total Cache.
If usage jumped with no campaign behind it, check for a compromise first: security checklist if your website has been hacked.
7. On your own VPS
On a VPS you control the web server and firewall, so you can add rate limiting in nginx, block ranges in the firewall and read the server-wide logs. Those server-level tools apply to your own VPS, not to shared hosting, where the server configuration is managed by us.
Frequently asked questions
Where are my raw access logs on Domain India cPanel hosting?
Monthly archives are in the logs folder of your home directory, one set per domain and month. Files ending in _NGINX.gz contain the requests handled by the caching proxy and give the full picture of traffic; the other files contain only requests that reached Apache.
Why does my Apache access log show so little traffic?
On our cPanel servers nginx sits in front of Apache and answers cached requests itself, so those requests never reach Apache. Use the _NGINX log files, or Awstats, which reads both logs.
How do I find which IP addresses use the most bandwidth?
Sum the bytes field (field 10) by IP address (field 1) in the access log, for example with zcat and awk, then sort the totals. The commands are in section 4 of this guide.
How can I tell if Googlebot in my logs is genuine?
Run a reverse DNS lookup on the IP. A genuine Googlebot resolves to a name ending in googlebot.com or google.com, and that name resolves back to the same IP. Anything else can be blocked.
Can I slow Google down with Crawl-delay?
No. Google ignores Crawl-delay in robots.txt. Stop it crawling endless URLs with Disallow rules, fix pages that return errors, and let caching and a CDN reduce the load.
Do I need SSH to analyse my logs?
No. You can download the log files with File Manager or FTP and analyse them on your computer. Jailed SSH is available on every shared hosting plan if you prefer to run the commands on the server; it is off by default, so ask support to enable it.
Ready to cut the traffic? Check Metrics › Awstats first, run the commands in section 4 on this month's _NGINX log, and apply the fixes that match what you find. If you are unsure what the logs show, open a ticket with your domain and we will help you read them.
Tell us your domain and the month the usage jumped, and our team will help you find what is using the data.
Open a support ticket