As anyone knows who has ever administered a server that is exposed to the web the Internet is a hostile place. Our servers are continually bombarded with a never ending stream of attempts to guess user ids and passwords. The source is from countless botnets and is constantly changing. I’ve tried a number of approaches to counter these attacks and I think I have come up with a solution that seems to be working.
It’s similar to the Fail2Ban https://www.fail2ban.org/wiki/index.php/Main_Page approach in that it scans the log files and creates a BLACKLIST chain of IP’s to DROP. I haven’t tried Fail2Ban as I prefer to roll my own solutions.
This is specific to the Redhat/CentOS (6) flavor of Linux although it could be easily adapted to other distros. For those not familiar with CentOS the firewall is configured in the /etc/sysconfig/iptables file. For Redhat/CenOS 7 users you might wish to disable the Firewalld system
https://fedoraproject.org/wiki/Firewalld?rd=FirewallD#Using_static_firewall_rules_with_the_iptables_and_ip6tables_services
Start by editing the iptables file.
nano /etc/sysconfig/iptables
and insert the highlighted lines
:INPUT ACCEPT [0:0] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [0:0] :RH-Firewall-1-INPUT - [0:0] :SSH_Brute_Force - [0:0] :SMTP_Brute_Force - [0:0] :CHK_FRIENDS - [0,0] :BLACKLIST - [0,0] -A INPUT -i eth0 -p tcp -m tcp -m state --state NEW -j BLACKLIST
I’ve added the highlighted lines. Note that I’m specifying eth0 which is my exposed Internet interface and I’m just checking new TCP connections. Let’s start with the CHK_FRIENDS chain. This is really important as you don’t want to lock yourself out of your own server. I speak from experience :-(. It simply is a list of IP addresses or blocks from which you want to allow access no matter what. The way it is set up here the first jump in the INPUT chain is to BLACKLIST and the first jump in it is to the CHK_FRIENDS. You can add these lines to the end of the iptables file.
-A BLACKLIST -j CHK_FRIENDS -A CHK_FRIENDS -s 123.123.123.0/24 -j ACCEPT -A CHK_FRIENDS -d 123.123.123.0/24 -j ACCEPT -A CHK_FRIENDS -s 111.222.333.31 -j ACCEPT -A CHK_FRIENDS -d 111.222.333.31 -j ACCEPT # last line in existing iptables COMMIT
Here we have a netblock of 123.123.123.0/24 and an individual IP of 111.222.333.31 that we want unrestricted access to any port on the server. DON’T put an entry that has the IP or netblock of your server in as a destination address. If you want more control then don’t put the BLACKLIST in as the first entry in the INPUT chain. Further down in the RH-Firewall-1-INPUT chain is perhaps a better location. That’s the one that the Redhat system-config-firewall utility works with (and the use of which will possibly overwrite the iptables file.)
Now restart the iptables service with
service iptables restart
Now the script part. I usually put my utilities in /root/bin
mkdir /root/bin
cd /root/bin
nano blistcron.sh
#!/bin/bash
echo iptables -F BLACKLIST > /root/bin/blistrun.sh
echo iptables -A BLACKLIST -j CHK_FRIENDS >> /root/bin/blistrun.sh
echo "# active BLACKLIST entries" >> /root/bin/blistrun.sh
iptables -L BLACKLIST -vn | egrep "^\s+[1-9]|^[1-9]+" | egrep DROP | tr -s " " | cut -d ' ' -f 9 | sort -u| sed 's/\(^.*$\)/iptables -A BLACKLIST -s \1 -j DROP -m comment --comment "active BLACKLIST"/g' >> /root/bin/blistrun.sh
echo "# SSH login failures from /var/log/secure" >> /root/bin/blistrun.sh
tail -400 /var/log/secure | egrep "Failed password for invalid user" | tr -s " " | cut -d " " -f 13 | sort -u | sed 's/\(^.*$\)/iptables -A BLACKLIST -s \1 -j DROP -m comment --comment "secure SSH failure"/g' >> /root/bin/blistrun.sh
tail -400 /var/log/secure | egrep "Failed password for(\s\w*\s)from" | tr -s " " | cut -d " " -f 11 | sort -u | egrep "^[0-9{3}\.]{3}" | sed 's/\(^.*$\)/iptables -A BLACKLIST -s \1 -j DROP -m comment --comment "secure SSH failure"/g' >> /root/bin/blistrun.sh
echo "# POP and IMAP login failures from /var/log/secure" >> /root/bin/blistrun.sh
tail -400 /var/log/secure | egrep "unix\(dovecot:auth\): authentication failure" | tr -s " " | cut -d " " -f 14 | sort -u | cut -d "=" -f 2 | egrep "^[0-9{3}\.]{3}" | sed 's/\(^.*$\)/iptables -A BLACKLIST -s \1 -j DROP -m comment --comment "secure IMAP failure"/g' >> /root/bin/blistrun.sh
echo "# SMTP login failures from /var/log/maillog" >> /root/bin/blistrun.sh
tail -400 /var/log/maillog | egrep "authentication failure" | cut -d "[" -f 3 | cut -d "]" -f 1 | sort -u | sed 's/\(^.*$\)/iptables -A BLACKLIST -s \1 -j DROP -m comment --comment "maillog SMTP failure"/g' >> /root/bin/blistrun.sh
#/root/bin/blistrun.sh
Note that I have commented out the last line for now…
chmod 700 blistcron.sh
to make it executable
/root/bin/blistcron.sh
to run it.
and
less blistrun.sh
to see the result.
iptables -F BLACKLIST iptables -A BLACKLIST -j CHK_FRIENDS # SSH login failures from /var/log/secure iptables -A BLACKLIST -s 196.196.219.18 -j DROP -m comment --comment "secure SSH failure" iptables -A BLACKLIST -s 46.29.163.138 -j DROP -m comment --comment "secure SSH failure" iptables -A BLACKLIST -s 68.183.178.162 -j DROP -m comment --comment "secure SSH failure" # POP and IMAP login failures from /var/log/secure # SMTP login failures from /var/log/maillog iptables -A BLACKLIST -s 103.241.243.214 -j DROP -m comment --comment "maillog SMTP failure" iptables -A BLACKLIST -s 176.97.49.115 -j DROP -m comment --comment "maillog SMTP failure" iptables -A BLACKLIST -s 89.39.95.129 -j DROP -m comment --comment "maillog SMTP failure" # active BLACKLIST entries iptables -A BLACKLIST -s 68.183.178.162 -j DROP -m comment --comment "active BLACKLIST" iptables -A BLACKLIST -s 78.128.113.68 -j DROP -m comment --comment "active BLACKLIST"
If you actually got anything resembling the above, well, congratulations! There are a couple more modifications that we have to make to your system.
First off, sendmail doesn’t log SMTP login failures by default. In order to enable that edit /etc/mail/sendmail.mc and change the log level to 10.
dnl # default logging level is 9, you might want to set it higher to dnl # debug the configuration dnl # dnl define(`confLOG_LEVEL', `9')dnl define(`confLOG_LEVEL', `10')dnl dnl #
And do a
service sendmail restart
That should log authentication failures in the /var/log/maillog file that look like
Jul 26 12:04:11 mail sendmail[22014]: x6QJ45kl022014: AUTH failure (PLAIN): authentication failure (-13) SASL(-13): authentication failure: Password verification failed, relay=[94.74.144.187]
If you are sure that everything is working properly then add the blistcron.sh to your crontab and have it run every 1/2 hour.
nano /etc/crontab
and add the last line
# Example of job definition: # .---------------- minute (0 - 59) # | .------------- hour (0 - 23) # | | .---------- day of month (1 - 31) # | | | .------- month (1 - 12) OR jan,feb,mar,apr ... # | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat # | | | | | # * * * * * user-name command to be executed */30 * * * * root /root/bin/blistcron.sh
Make the generated blistrun.sh script executable.
chmod 700 /root/bin/blistrun.sh
And test it by running it (cross your fingers)
/root/bin/blistrun.sh
Now take a look at the resulting firewall.
iptables -L BLACKLIST -vn
And it should look like.
Chain BLACKLIST (1 references)
pkts bytes target prot opt in out source destination
633 81462 CHK_FRIENDS all -- * * 0.0.0.0/0 0.0.0.0/0
3 180 DROP all -- * * 110.164.67.47 0.0.0.0/0 /* secure SSH failure */
0 0 DROP all -- * * 51.255.174.215 0.0.0.0/0 /* secure SSH failure */
3 180 DROP all -- * * 67.205.135.188 0.0.0.0/0 /* secure SSH failure */
0 0 DROP all -- * * 103.241.243.214 0.0.0.0/0 /* maillog SMTP failure */
0 0 DROP all -- * * 91.246.209.143 0.0.0.0/0 /* maillog SMTP failure */
0 0 DROP all -- * * 91.82.40.237 0.0.0.0/0 /* maillog SMTP failure */
0 0 DROP all -- * * 94.74.144.187 0.0.0.0/0 /* maillog SMTP failure */
0 0 DROP all -- * * 110.164.67.47 0.0.0.0/0 /* active BLACKLIST */
0 0 DROP all -- * * 51.255.131.51 0.0.0.0/0 /* active BLACKLIST */
0 0 DROP all -- * * 68.183.178.162 0.0.0.0/0 /* active BLACKLIST */
92 5520 DROP all -- * * 78.128.113.68 0.0.0.0/0 /* active BLACKLIST */
If you are happy with the results wait for at least one cron period so that blistcron.sh creates a new blistrun.sh, check it for correctness, and then edit blistcron.sh and remove the comment from the last line so that it runs the blistrun.sh script.
In the next post I’ll go into the details of how it works and also describe the SSH_Brute_Force and SMTP_Brute_Force chains.
One thought on “Blocking dictionary attacks with an adaptive iptables firewall. (CentOS)”
Comments are closed.