Downloading IP Lists

You need to have FireHOL's update-ipsets installed for downloading IP Lists directly from their maintainers. Follow Installing update-ipsets to install it.

---

Using update-ipsets

Once the latest development version of FireHOL is installed on your computer, you can run update-ipsets to download all the IP lists you need.

For example to download dshield you should run:

sudo update-ipsets enable dshield

You can enable multiple IP lists at once:

sudo update-ipsets enable dshield blocklist_de spamhaus_drop spamhaus_edrop

Then, to download all the enabled lists, run:

sudo update-ipsets

You can run this command repeatedly. It will not harm. update-ipsets is smart enough to download IP lists only when they are expected to be updated.

For example, it knows that dshield is updated every 10 minutes, so it will not download it again unless there have been 10 minutes since the date/time of the last download (it does not care when you downloaded it, but what date/time the downloaded file had when downloaded). It will attempt to download it again, only if 10 minutes have passed since the modification date/time of the downloaded file.

---

Periodic Updates - The Cron Job

This is how I run update-ipsets via cron:

# sudo crontab -e   or   sudo nano /etc/crontab

*/9 * * * * root update-ipsets >/dev/null 2>&1

I run it every 9 minutes. You can choose 8, 11, 12, etc, it does not matter. Avoid using 5, 10, 15, 20, etc. If all of us choose, let's say 10, the maintainers' sites will get all the requests concurrently. I suggest to pick a random number between 5 and 20.

update-ipsets keeps its files in /etc/firehol/ipsets. In this directory you will have, for each IP list:

• NAME.source - the file downloaded from the IP list maintainer
• NAME.ipset (or .netset depending on the IP list type) - the final processed and normalized file

---

Download all IP Lists

To download all the IP lists known by update-ipsets, use this:

sudo update-ipsets --enable-all

Then, sit back and watch it download all IP lists (it will take some time).

Keep in mind that a few lists (like hphosts_*) publish hostnames, not IPs. These lists are converted to IPs using your name server. update-ipsets is optimized to do parallel DNS queries to minimize the time needed for this conversion, still though it will need some time to finish.

You can set the number of parallel DNS queries using this:

export PARALLEL_DNS_QUERIES=50
sudo update-ipsets

The default is PARALLEL_DNS_QUERIES=10.

You can also put options in its configuration file. The default configuration is /etc/firehol/update-ipsets.conf. In our example, the configuration file should contain:

PARALLEL_DNS_QUERIES=50

---

Updating the ipsets in kernel

update-ipsets will do this by itself, automatically!

If it finds an ipset named with the name of an IP list, it will update it automatically when the IP List is updated.

update-ipsets goes through the following procedure for each IP list:

1. download the IP list from the maintainer

the downloaded file is saved in `/etc/firehol/ipsets` as `NAME.source` - it has the modification date/time of the same file at the maintainer's site

2. process it to extract / normalize its content

the result is saved in `/etc/firehol/ipsets` as `NAME.ipset` or `NAME.netset` depending on its contents

3. if an ipset with the same NAME already exists in kernel, then do all the following:

   1. create a temporary ipset in kernel

   2. load the contents of NAME.ipset or NAME.netset to the temporary ipset

   3. swap the temporary and the production ipsets in kernel

   this swap is an atomic operation that is handled by the kernel itself - either it succeeds (and the new ipset is used) or it fails (and the old ipset is left untouched)

If at any step there is a failure, the operation is aborted (all temporary resources used are released) and a message is sent to syslog.

The whole operation has been designed so that there will be no disruption at all at your firewall.

Keep in mind that the above are not FireHOL specific. update-ipsets will update the ipsets in kernel even if they are created by hand or another netfilter based firewall solution. The only condition is to have the same NAME and of course ipset type.

Since update-ipsets uses iprange, it also optimizes hash:net ipsets for optimal kernel performance. For more information about this optimization check iprange: optimizing ipsets for iptables.