fail2ban installation and configuration notes

A couple of days ago one web site I volunteer to manage was under DDOS attack. I installed and configured fail2ban to protect us from future similar attacks. Here are some notes. The server is the RedHat/Fedora/CentOS variety, as you can tell from commands listed below. Please translate them to your distro’s corresponding commands as needed.

  • Installation is easy:
    sudo yum install fail2ban

    To make fail2ban starts automatically after a reboot, run this:

    sudo systemctl enable fail2ban

  • Configuring is relatively easy. It’s recommended that you create your own jail configuration file, using the jail.conf from the installation as a starting point. Three things are noteworthy from my experience:
    1. Make sure that you provide the correct log file. For web server, there are typically one access log file and one error log file. Ensure that you feed the right log file when using a particular filter;
    2. On this server, fail2ban didn’t properly expand the log and file names when I put wildcard characters in them. I got around that by listing them one by one.
    3. In the jail.conf file, no default banaction was defined. I added the following line:
    banaction = iptables-multiport
  • To write your own custom filter, make sure you put a sample log entry inside the filter file as a comment. Use the following command to debug your filter:
    sudo fail2ban-regex /path2testLogfile/test.log /etc/fail2ban/filter.d/my-filter.conf
    Here is a filter that I wrote:
    [Definition]

    failregex = ^ -.*”POST \/component\/mailto\/\?tmpl=component\&link=aHR0cHM6.*”$

    ignoreregex =

  • After getting your jail.local ready, run the following command to debug any potential issues. I’ve found that if you have issues with your jail or filter files, “sudo systemctl start fail2ban” doesn’t always give you a good enough error message. Use this instead:

    sudo /usr/bin/fail2ban-client -x start

    You may need to start/stop a couple of times. To stop, run

    sudo /usr/bin/fail2ban-client -x stop

  • After debugging, before you finally start fail2ban service, it’s better to search the current access/error log and see if there is a match to the filter you defined. If yes, then take a note of its IP address and the last time it appears in the log file. Then start fail2ban by running
    sudo systemctl enable fail2ban
  • To verify that it works, run iptables -S and if it catches one offender and puts it in jail, you should see it in the output. Now go back to the access/error log and ensure there is no entry from that IP address since the last timestamp.

Good luck in protecting your servers!

Java regex Matcher’s first group is the whole pattern

I didn’t realize that Java’s regex class, Matcher, uses m.group(0) to denote the entire pattern. I spent some time debugging it. Hence this note.

As is stated in the documentation, “Capturing groups are indexed from left to right, starting at one. Group zero denotes the entire pattern, so the expression m.group(0) is equivalent to m.group().”

Here is a sample code to pick out twitter user names out of a string using Java. A set is returned, therefore if a user name is mentioned more than once, it’ll only be stored once in the set. All user names are returned back in lowercase. This function has gone through pretty through testing and works pretty well.

In addition, this is also a pretty good sample of negative lookbehind regex usage: we are not looking for pattern where @ is proceeded by any valid Twitter user name character.

Update: Angle brackets in Java code caused my code formatter to add some junk inside the code. Be aware! I need to look for a good code formatter for WordPress…

package com.haidongji.java;

import java.util.HashSet;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class Tweet {

/**
* Get usernames mentioned in a string.
*
* @param s
* string, a tweet
* @return the set of usernames mentioned in the text of the tweet.
* A username-mention is “@” followed by a Twitter username
* A Twitter username is composed of:
* English letters, digits, dash, and underscore
* The username-mention cannot be immediately preceded or followed
* by any character valid in a Twitter username. Therefore:
* user@example.com does NOT contain a mention of the username example.
* Twitter usernames are case-insensitive
*/
public static Set getMentionedUsersFromString(String s) {
Set
set = new HashSet();
String pattern = “(?< ![a-zA-Z0-9_-])@([a-zA-Z0-9_-]+)"; // see spec above Matcher m = Pattern.compile(pattern).matcher(s); while (m.find()) { set.add(m.group(1).toLowerCase()); } return set; } } [/code]