Blocking unwanted web access with 'ipset'


#1

I’m no expert, but I had what feels like a minor victory this morning.

My BigV server seemed to be getting a lot of sustained traffic from a particular IP address in Israel. The system load repeatedly went through the roof. It went on for too long and I needed a neat way of blocking it, so I used ipset.

My main firewall is standard iptables. There is a feature that allows the use of ipset as an extension to your usual rules.

  1. ipset create hackers hash:ip
    This creates a set named ‘hackers’ which can be populated with IP addresses to be blocked.
  2. ipset add hackers xxx.xxx.xxx.xxx
    Add one or more IP addresses to the set.
  3. iptables -I INPUT 1 -m set --match-set hackers src -j DROP
    Add a rule to iptables to use the set defined above and ‘drop’ the traffic.

That fixed it for me.

I don’t know how this scales, but for a few dozen sites with moderate traffic it seems to work just fine.

I hope that you may find this useful.


#2

Oh, that’s nice! We have a (very) long list of IP addresses we block (Russia, Nigeria, China mainly plus miscellaneous ne’er do wells) and, as a result, the output from iptables -L is long and also takes a long time to run if I forget to include the -n switch.

Switching to doing it the way you’re describing will make iptables -L a lot shorter and easier to check so thanks for that.


#3

Yes, ipset is good. We should try and make use of it in Symbiosis too :slight_smile:


#4

The one gotcha I’ve discovered so far is that ipset add expands IP addresses when you give it a range, so this means you fill the hash table up very quickly if you’re blocking say 92.1.0.0/14 (as we are) so thats 65,536 entries for a start!

Apparently you can mitigate this by making the table bigger but I’m still trying to work out how big we need to make ours.

The other “fun” is that once a set has been created it can’t be destroyed for a while once you’ve added entries, even if flushed, as the server reports it’s still in use. It releases it after a while but it’s frustrating when you’re trying to find the right size.


#5

I used to use ipset, but these days I’m “just” blacklisting via routing tables.

This doesn’t seem to suffer from the same size/speed download, and you can easily block whole AS-assignments via public lists.

I documented this here, if it is useful to others:

http://debian-administration.org/article/715/Preventing_SPAM_connections_with_bird.


Out of memory after reboot
Clamd eating CPU/Memory
#6

Hi poldham, hash:ip, and some of the other set types, has a netmask parameter at create time that means it holds netmasks instead of host addresses, but the latter are still used for lookup, being found if they match a netmask’d entry. That could stop the expansion you refer to? http://manned.org/ipset.8