Firewall/fail2ban thresholds?


#1

This morning I awoke to heavy load on my server. Investigating the logs in real time I could see the same IP hammering the WP login page for one of my sites.

This wasn’t a distributed attack but the pings were coming every 1-2 seconds.

The firewall didn’t kick in and I manually ‘touched’ the IP to the blacklist.d folder, which stopped it dead.

I have fail2ban installed. Is there a threshold I can set? Admittedly, a ping on the same address every 1-2 seconds isn’t fast by machine standards but it’s certainly not human. Am I missing something?


#2

My understanding is that fail2ban simply watches log files for regex matches against its rules. I don’t think pings are logged anywhere so I doubt fail2ban is applicable to this issue. If you can find the events logged somewhere then a rule could be added, try:

# grep -r ip.ad.dr.es /var/log

Obviously replacing ip.ad.dr.es for the actual IP Address, and see if anything comes up. (You may need to tweak the above depending on your distro.)

If you search iptables ping you’ll find loads of advice.

Chris.


#3

Thanks.

It’s quite unusual to see such an attack from a single IP although they obviously still happen.

Do you know anything about the firewall? It used to block very effectively (even blocked me a couple of times when I’d failed to log in correctly a few times) so I don’t know why it’s let hundreds of pings from this IP go through.

I’ve a couple of .auto files in the blacklist folder from 7th and 8th November so presumably it has been working to some extent.


#4

Do you know anything about the firewall? It used to block very effectively so I don’t know why it’s let hundreds of pings from this IP go through.

fail2ban is not a firewall, iptables is the firewall, which fail2ban will use when banning. fail2ban is a great tool for checking your log files for possible attacks and banning the source accordingly, but that is all it is, it certainly isn’t complete security solution.

When you fail to login (I assume by ssh), the system will create log entries in /var/log/auth that the fail2ban ssh rule will spot and ban you as appropriate. So yes it will have protected you from ssh (and possibly other) attacks, but probably no protection at all against ping.

iptables has no configuration by default as far as I know. You can see your current configuration with:

# iptables --list

It will probably have a list of your manually blocked addresses.

If you’re on an Ubuntu system, then you may have ufw installed as a simpler way of controlling iptables.

# ufw status

Would tell you if it is running and what rules are in place. If it is running then search the web for “ufw ping protection” or similar. This article seems quite a good simple introduction to ufw.

Needless to say with a firewall you do need to be very careful not to lock yourself out of your system.

(even blocked me a couple of times when I’d failed to log in correctly a few times)

To protect yourself from being locked out by fail2ban (but not the firewall), you can add:

ignoreip = your.ip.address.here

to your [DEFAULT] section in your /etc/fail2ban/jail.local or /etc/fail2ban/jail.conf files (on debian based systems).


#5

Apologies - I’ve just realised that this is the Symbiosis users section - there could be some default set-up in there about which I am not aware - others would know better.

I was responding to the Bytemark forum email, and hadn’t noticed the Symbiosis users at the top.


#6

Sorry, I was combining f2b and the Symbiosis firewall in general as I wasn’t sure if either should be catching these issues.

These aren’t SSH logins, which do appear to be caught. These are attempted web logins via a single web page (the WordPress platform).

I am sure high frequency loading or form submission on Apache has been caught in the past - there’s no way a human needs to be refreshing a page every second - certainly not a log in page. There are WP plugins which can go some way to mitigating this but I’d like to catch these issues at as low a level as possible, as when it gets to WP it’s already using way more server resources.

Also I don’t want to be playing catchup with specific IP addresses if possible as they change on a regular basis, so frequency of single-IP attacks from any IP is what I’d like. Won’t solve distributed attacks but that’s another issue. I can lock down the appropriate files to only allow access from our own IPs if required but I was avoiding that as our own IPs do change from time to time.


#7

For what it’s worth I am running fail2ban w/ worpdress following instructions here:

which mostly seem to be working, but I am having issues with “already banned” IP addreses showing up in fail2ban logs which I think is because debian has such an old version of fail2ban with it …


#8

Hi,

I saw that article but wanted to avoid getting as far as WP in the first place, although I suppose if it can add the rules after a few attacks it then gets back to a lower server level from thereon.

I’ve tried adding my own rule/filter to fail2ban based on a couple of articles I’ve read but it doesn’t always seem to be banning those IPs ‘live’. It does pick them up from the logs if I restart fail2ban, but seemingly not at the time.

I think I’m fairly close to a solution but may be missing something.

By the way, does that WP plugin now handle IPv6 addresses, as these are much more commonly appearing in my logs now.


#9

Newer version of fail2ban support IP6 but Debian version doesn’t. At some point I’ll have to upgrade (probably over Christmas when I am bored),


#10

@andymerrett I’ve not tried fail2ban but have looked at the stock symbiosis blacklist pattern matching. By default, the firewall blacklist script runs every 15 minutes and bans access to specified ports for ip addresses with 25+ hits. (At 100, all-ports). Configuration is via /etc/symbiosis/firewall/patterns.d/*.patterns. I suspect a few of the files are out-of-date and you’ll almost certainly need custom entries to catch failed logins for web applications (probably including roundcube, possibly squirrelmail).

Here’s a test file…

# /etc/symbiosis/firewall/patterns.d/x-test.patterns

#  file to scan 
file = /var/log/x-symbiosis-abuse-pattern-test.log

# ports 
# might be able to use this to identify which ruleset (pattern file) has fired (<100 hits only)
# "all" or comma and/or space separated list
ports = 8888, 8889, 8890

# patterns
# __IP__ matches ipv4 & 6 addresses 
# N.B. use at least one anchor or '$' will be appended

^__IP__ did something very bad

… and dummy data…

# test file for symbiosis custom firewall patterns 
# /var/log/x-symbiosis-abuse-pattern-test.log
# 
# sets up barring for 93.184.216.34 (example.com) via a custom pattern
# 25 hits in the last hour (or is it 15m?) should be enough to trigger the ban

93.184.216.34 did something very bad and it might be time to raise the drawbridge
93.184.216.34 did something very bad and it might be time to raise the drawbridge
[...]

You can track incidence by looking in /var/lib/symbiosis/firewall-blacklist-count.db (Firefox’s SQLite Manager plug-in, although marked legacy, works well here).


Symbiosis firewall blacklist auto entries not updated
#11

Thanks, I’ll look into it.

What I need is a very basic block for IPs which are spamming “POST /wp-login.php” - and most of the attacks I’ve seen are coming from a single IP at any one time. It will be easiest for me if I can get it to analyse the domain access log files.


#12

Ah! Cool, so if I have an /var/log/auth.log file that looks something like this:

Dec 4 01:14:38 lastmanstanding wordpress(xxxxxx.co.uk)[23715]: Authentication attempt for unknown user xx from xxx.xxx.xxx.xxx
Dec 4 01:14:39 lastmanstanding wordpress(xxxxxx.co.uk)[3443]: Authentication attempt for unknown user xx from xxx.xxx.xxx.xxx
Dec 4 01:14:39 lastmanstanding wordpress(xxxxxx.co.uk)[13658]: Authentication attempt for unknown user xx from xxx.xxx.xxx.xxx
Dec 4 01:14:40 lastmanstanding wordpress(xxxxxx.co.uk)[24694]: Authentication attempt for unknown user xx from xxx.xxx.xxx.xxx
Dec 4 01:14:40 lastmanstanding wordpress(xxxxxx.co.uk)[24003]: Authentication attempt for unknown user xx from xxx.xxx.xxx.xxx

Do you know I would write the catch message?

[IP] rejected Authentication attempt for $

or do I need something more sophisticated?


#13

Ah, ignore that question. It turns that there is this thing called “documentation” that explains these things! (http://symbiosis.bytemark.co.uk/docs/symbiosis.html) :smiley:

So I’ve created a patterns file that looks like this:

and put that into /etc/symbiosis/firewall/patterns.d but does anyone know if there is a way of validating that? Are banned ip addresses logged anywhere? or is just a matter of coming back in a few hours and looking through auth.log to see if any ips made it into /etc/symbiosis/blacklist.d?


#14

Hi @bwaymark1

# Catch message
#
[^ ]+ rejected Authentication attempt for [^ ]+ from __IP__

There’s a gotcha that might be worth mentioning: in the absence of a start-or-end of line anchor, “$” is appended to the pattern (undocumented but see /usr/lib/ruby/vendor_ruby/symbiosis/firewall/pattern.rb). So, your pattern, or something like…

wordpress\(.*\)\[\d+\]: Authentication attempt for unknown user .* from __IP__

…seems to fit the bill. The blacklist script will populate /etc/symbiosis/firewall/blacklist.d with .auto files showing the offending IP address (and port list when not “all”). For more detail you can inspect counts and log movements by viewing the SQLite files in /var/lib/symbiosis/firewall-blacklist-*.db.


#15

Ah nice, thanks for that. I’ve had it working for a week now using:

but is it worth adding a $ in case the undocumented feature gets removed one day?

And for pointing sqlite database location! Was very useful indeed (once I figured out to read sqlite files :-D)


#16

How are you getting WordPress to log failed attempts into auth.log in the first place?


#17

There is a plugin called WP fail2ban By Charles Lecklider that I use …