Web front for my code

Awhile ago I set up my own git server. I’ve been hacking happily using that and Eclipse. In my spare time, I’ve been taking UCSD’s wonderful algorithm course: Algorithmic Design and Techniques. The course provides plenty of programming challenges! I chose the paid version so my code can be evaluated against all tests in the grading server. I solved the programming challenges in 3 languages: Java, Python, and C++. It has been so much fun!

Anyway, I thought it’d be nice to enable a web front end to my own code hosted on my own server. There are a lot of choices: gitweb, gitlab, gitea, etc. A couple of days ago I came across cgit. It’s pretty lightweight, written in C. That’s what I chose for that task. It’s setup now already, please go to https://code.haidongji.com to check it out!

It takes a bit of work to get the clean subdomain working. My environment is Debian 9 running apache2. Below were steps taken to make my environment work. Hope it helps somebody!

  • sudo apt install cgit
    The install creates /etc/cgitrc file and /etc/apache2/conf-available/cgit.conf
  • Here is the content of my apache2 site conf file
    <VirtualHost *:80>
            ServerName code.haidongji.com
            ServerAdmin emailAddress
            DocumentRoot /usr/share/cgit/
    	<Directory "/usr/share/cgit/">
    		AllowOverride None
    		Options ExecCGI
    		Order allow,deny
    		Allow from all
    	</Directory>
    	Alias /cgit-css/cgit.css /usr/share/cgit/cgit.css
    	Alias /cgit-css/cgit.png /usr/share/cgit/cgit.png
    	ScriptAlias / /usr/lib/cgit/cgit.cgi/
    
            ErrorLog ${APACHE_LOG_DIR}/codehaidongji.log
            CustomLog ${APACHE_LOG_DIR}/codehaidongji.log combined
    
    </VirtualHost>
  • sudo apt install python3-pygments for syntax highlighting
  • Here is the content of the /etc/cgitrc file
    css=/cgit-css/cgit.css
    logo=/cgit-css/cgit.png
    
    # enable Pygments syntax highlighting. Must be above scan-path, otherwise it would not work!
    source-filter=/usr/lib/cgit/filters/syntax-highlighting.py
    scan-path=/srv/git
  • sudo a2enmod cgid
    sudo a2enconf cgit
  • sudo systemctl restart apache2

Note that in /etc/cgitrc, syntax highlighting line MUST BE ABOVE the scan-path line, otherwise syntax highlighting will not work!

Cheers and happy coding!

Automatically mount USB 3 attached to my wifi router

As discussed here, I have Seagate Ultra Slim USB 3 attached to my wifi router as a NAS (Network-attached storage). It provides a good backup/sharing storage for my home network that’s accessible from all my home computing devices, which was nice.

Accessing it from Windows is easy to set up, all I needed to do was to map a drive. Accessing it from Linux needs some additional work, but it turned out to be not too bad either.

Initially I created an entry in /etc/fstab. It uses the cifs protocol. Some cursory research shows that cifs is old and oboselete, but I’m not sure of other ways of accessing it. As I mentioned in that post, I had to put in the ver=1.0 option for it to work, something I don’t like.

Anyway, the drawback of /etc/fstab entry is that my Linux system would try to mount it before establishing network access, which doesn’t work for obvious reasons. I had to “sudo mount -a” afterwards. That is manual and gets tedious real fast.

So I did some research. One solution was to use _netdev option in the fstab entry, which didn’t work for me. The other was to use the auto option, which didn’t work either.

In the end, what worked for me was using systemd. Here are two articles that I relied to get mine going.

Tomáš Tomeček, Automatic mounts with systemd
Michael Albert, Systemd: (Auto-) Mount cifs shares This article has good examples for cifs options. Yes, I had to use the ver=1.0.

Thanks guys!

fail2ban installation and configuration notes

A couple of days ago one web site I volunteer to manage was under DDOS attack. I installed and configured fail2ban to protect us from future similar attacks. Here are some notes. The server is the RedHat/Fedora/CentOS variety, as you can tell from commands listed below. Please translate them to your distro’s corresponding commands as needed.

  • Installation is easy:
    sudo yum install fail2ban

    To make fail2ban starts automatically after a reboot, run this:

    sudo systemctl enable fail2ban

  • Configuring is relatively easy. It’s recommended that you create your own jail configuration file, using the jail.conf from the installation as a starting point. Three things are noteworthy from my experience:
    1. Make sure that you provide the correct log file. For web server, there are typically one access log file and one error log file. Ensure that you feed the right log file when using a particular filter;
    2. On this server, fail2ban didn’t properly expand the log and file names when I put wildcard characters in them. I got around that by listing them one by one.
    3. In the jail.conf file, no default banaction was defined. I added the following line:
    banaction = iptables-multiport
  • To write your own custom filter, make sure you put a sample log entry inside the filter file as a comment. Use the following command to debug your filter:
    sudo fail2ban-regex /path2testLogfile/test.log /etc/fail2ban/filter.d/my-filter.conf
    Here is a filter that I wrote:
    [Definition]

    failregex = ^ -.*”POST \/component\/mailto\/\?tmpl=component\&link=aHR0cHM6.*”$

    ignoreregex =

  • After getting your jail.local ready, run the following command to debug any potential issues. I’ve found that if you have issues with your jail or filter files, “sudo systemctl start fail2ban” doesn’t always give you a good enough error message. Use this instead:

    sudo /usr/bin/fail2ban-client -x start

    You may need to start/stop a couple of times. To stop, run

    sudo /usr/bin/fail2ban-client -x stop

  • After debugging, before you finally start fail2ban service, it’s better to search the current access/error log and see if there is a match to the filter you defined. If yes, then take a note of its IP address and the last time it appears in the log file. Then start fail2ban by running
    sudo systemctl enable fail2ban
  • To verify that it works, run iptables -S and if it catches one offender and puts it in jail, you should see it in the output. Now go back to the access/error log and ensure there is no entry from that IP address since the last timestamp.

Good luck in protecting your servers!

More Manjaro config: Chinese input and clock setting

I’m enjoying running Manjaro Arch Linux on my workstation so far. Some additional notes below.

1. I had trouble getting Fcitx Chinese input working, so I used IBus instead. Here is what I installed using Manjaro’s package GUI tool: IBus Preferences and ibus-pinyin;

2. IBus Pinyin works fine on this system except for Emacs. Using the system’s IME for Chinese input into Emacs is a fairly common problem. There are various hacks to get it work (mostly dealing with playing with LC_CTYPE=zh_CN.utf8), none to my liking. So I installed an Emacs package from MELPA, pyim. I put the following into my .emacs.d/init.el:

   (require 'pyim)
   (require 'pyim-basedict) ; 拼音词库设置,五笔用户 *不需要* 此行设置
   (pyim-basedict-enable)   ; 拼音词库,五笔用户 *不需要* 此行设置
   (setq default-input-method "pyim")

Ctrl-backslash invokes this input method. Ctrl-n and Ctrl-b pages down and up for character selections. It’s not as sophisticated as a full-blown IME, but it’ll do in a pinch.

3. I customized the format of the clock in system tray to %Y-%m-%d %A %R

Manjaro Linux usage notes

I’ve been using Linux Mint Cinnamon for many years. For the most part, I love it. It’s based on Ubuntu, very similar to Debian, therefore there are plenty of resources available online. But its GUI bothers me a bit occasionally. Let me give a few examples here:

  • The Eclipse IDE’s tooltip sometimes shows up in black background, which is annoying. I think it’s related to GTK bugs;
  • I use ownCloud. Its Linux client, when running, is supposed to have an icon in the system tray in the bottom left corner. I like that because that icon changes depending on if it’s in sync, and more importantly, tells me if my web server is down. That’s my “poor man’s web site monitor”. With Cinnamon, sometime the icon doesn’t show up!
  • I have to reinstall Mint every so often to keep up with its releases.

I’m not bashing Linux Mint here. I think it’s a great distro and the easy availability of resources online makes it a compelling distro, especially if you are a beginner. But I’ve always been curious about Arch Linux. So when I discovered Manjaro, a distro based on Arch with a great installer, I decided to give it a try. I’ve been using it for about a month. So far I’m loving it. Here are some notes for myself and I hope it helps you too!

  • I love the mhwd utility. I have nvidia GeForce GTX 1070 GPU hardware. Properly configuring it has always been a hassle. With mhwd, it’s so easy: sudo mhwd -a pci nonfree 0300
  • Use the Add/Remove Software utility to install GVim. It’ll install the proper command line version of vim. I discovered that if you just install the command line version of vim, the left-button-select-middle-button-paste mouse action doesn’t work properly! I couldn’t live without it!
  • Software packages to install: Opera (uBlock Origin, Install Chrome Extensions, Vimium), Firefox (uBlock Origin, Vim Vixen), Chromium (uBlock Origin, Vimium), KeePassXC, VirtualBox, Eclipse, IntelliJ IDEA, ownCloud client, Emacs (org-mode, Evil), flash plugin (both types), Terminator, ack, Android Studio
  • For wireless, I have ASUS Dual-Band Wireless-AC1900 PCI-E Adapter (PCE-AC68). I needed to have a wired cable connection before I was able to install wireless driver. I picked linux414-broadcom-wl. After that, the system also picked linux49-broadcom-wl. The driver works fine, and I can use 5G capabilities, in that it sees the 5G hotspot.
  • I have a router based NAS. Here is how I mount it.