Fwalldb: a tool to inspect the Symbiosis blacklist database


#1

I’ve found that it’s useful to have a tool which tells me what is happening in the Symbiosis firewall, and what’s in the Symbiosis firewall database. I’ve gone on to extend it somewhat so I can delete items in the database - and have used that to reduce the number that are known by the DNSBL sites, because I am now blocking these when they connect (see Exim4: using the connect ACL to improve security).

By default, fwalldb looks at the files in /etc/symbiosis/firewall/blacklist.db, and then looks up the ips in the symbiosis database. It prints a table of the ips in reverse date order, along with how many database entries there are and what the ‘badness’ count is. There are several selection and display options, I include the help text at the bottom of this posting.

Fwalldb sorts the output in reverse date format, where date is the latest time that something has happened for that ip address, which normally comes from the database timestamp. If you are using my script which updates the firewall files from the iptables database (Improving the Symbiosis firewall), then the files in the firewall database will be updated from iptables for active ips and the time on the file is used as the latest date. A ‘*’ is shown in the Report count on the line to show that the latest date is taken from the time on the file and not from the database.

fwalldb is written in Python 2.7. It’s one file, which means that you don’t need to install modules anywhere. Python purists will probably denounce me for heresy.

You can get the command (and a script to lookup blacklisted ips, see below) on https://cloudy.hillside.co.uk/firewall/fwalldb.tar.gz.

To install, fwalldb uses a couple of Python packages that were needed to be installed on my Symbiosis system.

So, first do

sudo apt install python-prettytable python-dateutil

Then you can install fwalldb anywhere that makes sense to you, perhaps /usr/local/bin. The file needs to be made executable too.

sudo cp fwalldb /usr/local/bin
sudo chmod +x /usr/local/bin/fwalldb

Now you can type:

fwalldb

and get a table of data. If you have no firewall entries, nothing is shown. Using the -a flag will then just dump the database for you.

Looking up blacklisted ips

I’ve include a python script which can look up ips, and print out the ones that are known to the three DNSBL sites that I use. I’ve used this to reduce the number of entries in my Symbiosis database, since I am now blocking these sites when they connect. Here’s the shell script sequence I used to do this:

fwalldb -b --before 20190101 | awk '{print $1}' | grep -v ':' > tocheck
python bllookup.py $(cat tocheck) > candelete
for ip in $(cat candelete)
do
	sudo fwalldb -d $ip
done

The first line uses fwalldb with no table lines or headers (-b) to generate a list of ips whose latest time in the database is before 20190101. Awk then picks off the ips in column 1, and grep removes any IPv6 lines (we don’t have the full original IP address stored for IPv6 addresses). The second line uses bllookup.py to generate a list of ips that are in blacklists. Finally there is a loop which deletes the ips from the Symbiosis database. The sudo command may not find the fwalldb command, and you may need to give it the full path to the file.

Fwalldb help information

Here’s what you get when you use fwalldb -h. You need to scroll in the box below to see it all.

Inspect the database of stored information
created by the symbiosis ip blacklist system

Usage: fwalldb [-h|--help] [-s ip|reports|count|date] [-r] [-a] [-b] [-i ip] [-d ip]

By default, examine current firewall directory and print the information based 
on what is there. Information is printed in order of the most recent timestamp
date. Duration is the time from the first to the latest report, and is shown as
days:hours:minutes:seconds.

If the file in the firewall directory was modified later than the latest time in
the database, this time is used and a '*' is shown next to the report count. If
you are updating these files from a firewall lookup script, the '*' indicates
activity in iptables for the ip.

-h | --help	    Print usage

Sorting arguments - can be abbreviated to shortest unique string
-s ip           Sort by ip. Ips are sorted into ascending order 
                as strings, and not numbers
-s reports      Sort by number of incident reports
-s count        Sort by total count
-s date         Sort by date of latest update - ascending order
-r              Reverse the sense of sorting, all fields are sorted by
                default into descending order, apart from ip.

Selection
-a              Show all data in database, sorted by latest (descending)
                Doesn't examine the firewall directory.
-i ip           Lookup the ip in the database, and show matching data.
                The value of ip is matched with the start of the ip string
                in the database, so for example 104. will find all the addresses
                starting with 104.
--before date   Select data before midnight on the nominated date
                (format yyyymmdd).
--after date    Select data after midnight on the nominated date
                (format yyyymmdd).
                --before and --after can be combined

Display
-b              Remove border and heading from the table, helpful for using
                the script with shell scripts

Editing
-d              Delete a specific ip from the database. The address provided
                must match stored address completely. You need root access to
                do this, so use sudo on the command. The deletion process
                refuses to delete IPs that are in the firewall directory. So
                it's safe to use this in shell scripts.

Written in python2.7
Peter Collinson 8/2/2019
$Id: fwalldb.py,v 1.7 2019/02/11 16:28:34 pc Exp $