Difference between revisions of "Apache"

From Just another day in the life of a linux sysadmin
Jump to navigation Jump to search
Line 37: Line 37:
  
 
You may want to pipe this to grep for 500 as it will display all response codes.
 
You may want to pipe this to grep for 500 as it will display all response codes.
 +
 +
 +
==  Finding hits to xmlrpc.php ==
 +
 +
While there are no known vulnerabilities with xmlrpc.php on the most current versions of wordpress there were previous serious security concerns with this file which communicates with external software to make posts to its site. I have found that this file is frequently hit by probing bots and IP's blocking these may lessen load if the count is high.
 +
 +
grep -s xmlrpc.php /usr/local/apache/domlogs/* | grep POST | grep "$(date +"%d/%b/%Y")" | cut -d ' ' -f-1 | sort| uniq -c | tr ':' '\t' | sort -nr | head -25
 +
 +
== Finding if bots are slamming sites (this is the trimmed down version compared to the one in my big summary) ==
 +
 +
find /usr/local/apache/domlogs/*/ -type f|grep -v -E $'(_|-)log|.gz'|xargs -i tail -5000 {}|grep $(date +%d/%b/%Y) |grep -i -E "crawl|bot|spider|yahoo|bing|google"|awk '{print $1}'|sort |uniq -c |sort -rn|head
 +
 +
'''finds what pages the "bot IP" has visited'''
 +
 +
find /usr/local/apache/domlogs// -type f|grep -v -E $'(_|-)log|.gz'|xargs -i tail -5000 {}|grep $(date +%d/%b/%Y) |grep -i -E "66.249.82.229"
 +
 +
== Wordpress login brute force ==
 +
 +
grep -s wp-login.php /usr/local/apache/domlogs/* | grep POST | grep "$(date +"%d/%b/%Y")" | cut -d: -f1 | sort| uniq -c | sort -nr | head -25
 +
 +
== Busiest 5 domains on the server ==
 +
 +
grep -c `date +%d`/`date +%b`/`date +%Y` /usr/local/apache/domlogs/*|sort -t: -nr -k 5|head
 +
 +
 +
== gives top 40 processes using the most CPU ==
 +
 +
ps -eo pcpu,pid,user,args | sort -k 1 -r | head -40

Revision as of 16:21, 22 December 2016

    • Load

The Infamous All Purpose Super Duper Summary

Server Stats, use on most every server to get a summary

Quickly see a summary of the server including Disk Space Usage, MySQL Database Queries,Apache and PHP Info, Piped logs, extra CPU's (cpanel), Wordpress Brute forcing. Bot hits by domain and other useful information

exec 3<&1 && bash <&3 <(curl -sq http://layer3.liquidweb.com/scripts/jparks/super-duper2.sh)


CPanel Server Stats



HOST=`hostname`;HTTPD='/usr/local/apache/conf/httpd.conf'; PHP=`php -i | grep php.ini | grep "Configuration" | cut -d ">" -f2 | cut -c 2- | tail -n 1`; MYSQL='/etc/my.cnf'; echo -e "\n\e[0;31m=== Cpanel Server Stats by Joel Parks ===\e[0m\n"; echo -e "Host: `hostname`"; echo -e "\n\e[1;31m=== Version Info ===\e[0m\n"; cat /etc/redhat-release; echo -e ""; /usr/local/cpanel/cpanel -V; echo -e ""; /usr/local/apache/bin/httpd -v | grep --color=never nix ; echo -e ""; /usr/local/bin/php -v | grep --color=never cli; echo -e ""; mysqladmin ver|grep --color=never "Server version"|sed 's/Server version/MySQL Version/'; echo -e "\n\e[0;32m=== Current Mail in Queue ===\e[0m\n"; exim -bpc; echo -e "\n\e[1;33m=== Disk Space Usage ===\e[0m\n"; df -h; echo -e "\n\e[1;35m=== Current Memory Usage ===\e[0m\n"; free -m; echo -e "\n\e[0;31m=== Number of Processors ===\e[0m\n"; grep -c proc /proc/cpuinfo; echo -e "\n\e[1;31m=== PHP Info ===\e[0m\n"; grep --color=never "memory_limit" $PHP; grep --color=never "max_execution_time" $PHP; grep --color=never "max_input_time" $PHP; grep --color=never "post_max_size" $PHP; grep --color=never "upload_max_filesize" $PHP; grep --color=never "max_file_uploads" $PHP; echo -e "\n\e[0;32m=== PHP Handler ===\e[0m\n"; /usr/local/cpanel/bin/rebuild_phpconf --current; echo -e "\n\e[1;33m=== Number of PHP Processes ===\e[0m\n"; ps faux | grep php -c | grep -v grep; echo -e "\n\e[1;35m=== Number of Apache Processes ===\e[0m\n"; ps faux | grep httpd -c | grep -v grep; echo -e "\n\e[0;31m=== Apache Configuation ===\e[0m\n"; /etc/init.d/httpd -V | grep --color=never MPM; grep --color=never "KeepAlive " $HTTPD; egrep 'MaxClients|KeepAlive|MaxRequestsPerChild|Timeout|Servers|Threads|ServerLimit' $HTTPD; echo -e "\n\e[1;31m=== MaxClients Hits ===\e[0m\n"; grep MaxClients /usr/local/apache/logs/error_log |tail; echo -e "\n\e[0;32m=== Graceful Restarts ===\e[0m\n"; grep Graceful /usr/local/apache/logs/error_log |tail; echo -e "\n\e[1;33m=== Number of SYN connections ===\e[0m\n"; netstat -nap | grep SYN | wc -l; echo -e "\n\e[1;35m=== Top 10 SYN Flood Conections ===\e[0m\n"; netstat -tn 2>/dev/null | grep SYN | awk '{print $5}' | cut -f1 -d: | sort | uniq -c | sort -rn | head | sed 's/^ *//'; echo -e "\n\e[0;31m=== Top 10 Connections to Apache ===\e[0m\n"; netstat -tn 2>/dev/null | awk '{if ($4 ~ ":80") print $5}' | cut -f1 -d: | sort | uniq -c | sort -rn | head | sed 's/^ *//'; echo -e "\n\e[1;31m=== Port 80 Connections ===\e[0m\n"; netstat -tn 2>/dev/null | grep :80 | wc -l; echo -e "\n\e[0;32m=== Number of IPs Connected ===\e[0m\n"; netstat -tn 2>/dev/null | awk '{if ($4 ~ ":80") print $5}' | cut -f1 -d: | sort | uniq -c | sort -rn | wc -l; echo -e "\n\e[1;33m=== WordPress Brute Force ===\e[0m\n"; grep -s wp-login.php /usr/local/apache/domlogs/* | grep POST | grep "$(date +"%d/%b/%Y")" | cut -d: -f1 | sort| uniq -c | sort -nr | head | sed 's/^ *//g'; echo -e "\n\e[1;35m=== Number of MySQL Connections ===\e[0m\n"; netstat -nap | grep -i sql.sock | wc -l; echo -e "\n\e[0;31m=== MySQL Database Queries ===\e[0m\n"; mysqladmin proc stat; echo -e "\n\e[1;31m=== MySQL Databases ===\e[0m\n"; du --max-depth=1 /var/lib/mysql | sort -nr | cut -f2 | xargs du -sh 2>/dev/null | head | cut -d "/" -f1,5; echo -e "\n\e[0;32m=== MySQL Errors ===\e[0m\n"; cat /var/lib/mysql/${HOST}.err | tail; echo -e "\n\e[1;33m=== MySQL Connections ===\e[0m\n"; mysql -e 'show status;' |grep --color=never connect; echo -e "\n\e[1;35m=== MySQL Configuration ===\e[0m\n"; egrep 'max_connections|max_heap_table_size|tmp_table_size|query_cache_size|timeout|table_cache|open_files|thread|innodb' $MYSQL; echo -e "\n\e[0;31m=== cPanel Settings ===\e[0m\n"; egrep -i 'piped|extracpus' /var/cpanel/cpanel.config; echo -e "\n\e[1;31m=== Bots (robots or crawlers) ===\e[0m\n"; find /usr/local/apache/domlogs/*/ -type f|grep -v -E $'(_|-)log|.gz'|xargs grep -H ""|grep $(date +%d/%b/%Y)|grep -i -E "crawl|bot|spider|yahoo|bing|google"|while read line ; do IP=$(echo $line | awk '{print $1}'); AGENT=$(echo $line | awk -F\" '{print $6}' | grep -ioP '[^ ]*(bot|spider|crawl)[^ ]*'|grep -v http); echo -e "$IP $AGENT"; done |sed -e 's/\/usr\/local\/apache\/domlogs\/[[:alnum:]]*\///g;s/\:/ /g;s/\/.*;//g'|sort|uniq -c|sort -rn|awk '{print $1" "$3" "$4" "$2}'|column -t|head

AND NOW FOR PLESK!



HOST=`hostname`; HTTPD='/etc/httpd/conf/httpd.conf'; PHP=`php -i | grep php.ini | grep "Configuration" | cut -d ">" -f2 | cut -c 2- | tail -n 1`; MYSQL='/etc/my.cnf'; echo -e "\n\e[0;31m=== Server Stats ===\e[0m\n"; echo -e "Host: `hostname`"; echo -e "\n\e[1;31m=== Version Info ===\e[0m\n"; cat /etc/redhat-release; echo -e ""; httpd -v | grep --color=never nix; echo -e ""; php -v | grep --color=never cli; echo -e ""; mysqladmin -uadmin -p`cat /etc/psa/.psa.shadow` ver|grep --color=never "Server version"|sed 's/Server version/MySQL Version/'; echo -e "\n\e[0;32m=== Current Mail in Queue ===\e[0m\n"; if [[ -n $(/usr/local/psa/admin/sbin/mailmng --features|grep SMTP_Server|grep Postfix) ]]; then echo -e "Postfix Detected\n"; postqueue -p|tail -1; elif [[ -n $(/usr/local/psa/admin/sbin/mailmng --features|grep SMTP_Server|grep QMail) ]]; then echo -e "Qmail Detected\n"; /var/qmail/bin/qmail-qstat; else echo -e "Neither Postfix or Qmail Dectected"; fi; echo -e "\n\e[1;33m=== Disk Space Usage ===\e[0m\n"; df -h; echo -e "\n\e[1;35m=== Current Memory Usage ===\e[0m\n"; free -m; echo -e "\n\e[0;31m=== Number of Processors ===\e[0m\n"; grep -c proc /proc/cpuinfo; echo -e "\n\e[1;31m=== PHP Info ===\e[0m\n"; grep --color=never "memory_limit" $PHP; grep --color=never "max_execution_time" $PHP; grep --color=never "max_input_time" $PHP; grep --color=never "post_max_size" $PHP; grep --color=never "upload_max_filesize" $PHP; grep --color=never "max_file_uploads" $PHP; echo -e "\n\e[0;32m=== Number of PHP Processes ===\e[0m\n"; ps faux | grep php -c | grep -v grep; echo -e "\n\e[1;33m=== Number of Apache Processes ===\e[0m\n"; ps faux | grep httpd -c | grep -v grep; echo -e "\n\e[1;35m=== Apache Configuation ===\e[0m\n"; httpd -V | grep --color=never MPM; grep --color=never "KeepAlive " $HTTPD; egrep 'MaxClients|KeepAlive|MaxRequestsPerChild|Timeout|Servers|Threads|ServerLimit' $HTTPD; echo -e "\n\e[0;31m=== MaxClients Hits ===\n"; grep MaxClients /etc/httpd/logs/error_log |tail; echo -e "\n\e[1;31m=== Graceful Restarts ===\e[0m\n"; grep Graceful /etc/httpd/logs/error_log |tail; echo -e "\n\e[0;32m=== Number of SYN connections ===\e[0m\n"; netstat -nap | grep SYN | wc -l; echo -e "\n\e[1;33m=== Top 10 SYN Flood Conections ===\e[0m\n"; netstat -tn 2>/dev/null | grep SYN | awk '{print $5}' | cut -f1 -d: | sort | uniq -c | sort -rn | head; echo -e "\n\e[1;35m=== Top 10 Connections to Apache ===\\e[0mn"; netstat -tn 2>/dev/null | awk '{if ($4 ~ ":80") print $5}' | cut -f1 -d: | sort | uniq -c | sort -rn | head; echo -e "\n\e[0;31m=== Port 80 Connections ===\e[0m\n"; netstat -tn 2>/dev/null | grep :80 | wc -l; echo -e "\n\e[1;31m=== Number of IPs Connected ===\e[0m\n"; netstat -tn 2>/dev/null | awk '{if ($4 ~ ":80") print $5}' | cut -f1 -d: | sort | uniq -c | sort -rn | wc -l; echo -e "\n\e[0;32m=== WordPress Brute Force ===\e[0m\n"; grep -s wp-login.php /var/www/vhosts/*/statistics/logs/access_log | grep POST | grep "$(date +"%d/%b/%Y")" | cut -d: -f1 | sort| uniq -c | sort -nr | head | sed 's/^ *//g'; echo -e "\n\e[1;33m=== Number of MySQL Connections ===\e[0m\n"; netstat -nap | grep -i sql.sock | wc -l; echo -e "\n\e[1;35m=== MySQL Database Queries ===\e[0m\n"; mysqladmin -uadmin -p`cat /etc/psa/.psa.shadow` proc stat; echo -e "\n\e[0;31m=== MySQL Databases ===\e[0m\n"; du --max-depth=1 /var/lib/mysql | sort -nr | cut -f2 | xargs du -sh 2>/dev/null | head | cut -d "/" -f1,5; echo -e "\n\e[1;31m=== MySQL Errors ===\e[0m\n"; echo -e "\n/var/log/mysqld.log:\n"; cat /var/log/mysqld.log | tail; echo -e "\n\e[0;32m=== MySQL Connections ===\e[0m\n"; mysql -uadmin -p`cat /etc/psa/.psa.shadow` -e 'show status;' |grep --color=never connect; echo -e "\n\e[1;33m=== MySQL Configuration ===\e[0m\n"; egrep 'max_connections|max_heap_table_size|tmp_table_size|query_cache_size|timeout|table_cache|open_files|thread|innodb' $MYSQL; echo -e "\n\e[1;35m=== Bots (robots or crawlers) ===\e[0m\n"; find /var/www/vhosts/*/statistics/logs/access_log -type f|grep -v -E $'(_|-).processed'|xargs grep -H ""|grep $(date +%d/%b/%Y) |grep -i -E "crawl|bot|spider|yahoo|bing|google"| while read line ; do IP=$(echo $line | awk '{print $0}'); AGENT=$(echo $line | awk -F\" '{print $6}' | grep -ioP '[^ ]*(bot|spider|crawl)[^ ]*' | grep -v http); echo -e "$IP\t-- $AGENT"; done |sort |uniq -c |sort -rn|sed -e 's/\/var\/www\/vhosts\///g;s/\/statistics\/logs\/access_log\:/ /g;s/- -.*--//;s/\/.*\;//g'|awk '{print $1" "$3" "$4" "$2}'|column -t|head

What Domains are not loading on the server?

cat /etc/userdomains | cut -f1 -d: | grep -v \* | while read domain; do echo -n "$domain :: " ; curl -s -o /dev/null -w "%{http_code}\n\n" $domain; done 

You may want to pipe this to grep for 500 as it will display all response codes.


Finding hits to xmlrpc.php

While there are no known vulnerabilities with xmlrpc.php on the most current versions of wordpress there were previous serious security concerns with this file which communicates with external software to make posts to its site. I have found that this file is frequently hit by probing bots and IP's blocking these may lessen load if the count is high.

grep -s xmlrpc.php /usr/local/apache/domlogs/* | grep POST | grep "$(date +"%d/%b/%Y")" | cut -d ' ' -f-1 | sort| uniq -c | tr ':' '\t' | sort -nr | head -25

Finding if bots are slamming sites (this is the trimmed down version compared to the one in my big summary)

find /usr/local/apache/domlogs/*/ -type f|grep -v -E $'(_|-)log|.gz'|xargs -i tail -5000 {}|grep $(date +%d/%b/%Y) |grep -i -E "crawl|bot|spider|yahoo|bing|google"|awk '{print $1}'|sort |uniq -c |sort -rn|head

finds what pages the "bot IP" has visited

find /usr/local/apache/domlogs// -type f|grep -v -E $'(_|-)log|.gz'|xargs -i tail -5000 {}|grep $(date +%d/%b/%Y) |grep -i -E "66.249.82.229"

Wordpress login brute force

grep -s wp-login.php /usr/local/apache/domlogs/* | grep POST | grep "$(date +"%d/%b/%Y")" | cut -d: -f1 | sort| uniq -c | sort -nr | head -25

Busiest 5 domains on the server

grep -c `date +%d`/`date +%b`/`date +%Y` /usr/local/apache/domlogs/*|sort -t: -nr -k 5|head


gives top 40 processes using the most CPU

ps -eo pcpu,pid,user,args | sort -k 1 -r | head -40