Getting Eclipse’s EGit to work with my own git server

I’ve been thinking about setting up my own git server for a while, and finally got it up and running last week. Since I do a lot of hacking with Eclipse, I naturally want Eclipse’s EGit to work with my own git server. Here are a couple of noteworthy points:

  • As of this writing, if you want ssh authentication with EGit, ecdsa type key is NOT going to work. rsa ssh key works. I found this out the hard way, since I’m trying to transition from rsa to ecdsa. Therefore I initially didn’t generate rsa keys on my computers. I had to run “ssh-keygen -t rsa -b 4096” and made sure that Eclipse – Preferences – General – Network connections – SSH2 – General tab – Private keys has id_rsa entered;
  • I followed instructions from here to get a few bare repo that I need initialized. Initially I had the misunderstanding that once I push code to the server, I should be able to see the files on the remote server’s repo directory. That turns out to be NOT the case. There is some magic going on under the objects directory that make things happen, such that although the code directories and files are not visible to the naked eye, I could push/fetch/pull to the remote server without problems. When I have time, I need to dig deeper and have a better idea how it all works. For now, I’m happy that I have it working that I can work with the same code base on different dev computers.
  • I followed exercise here for testing and found it helpful.

Happy coding y’all.

Mathematical modeling of fake news

I took Mathematical Modelling Basics course during the last couple of months. It was produced by Delft University of Technology, offered for free on edX. Thanks TUDelft and edX!

It is a great course introducing mathematical modeling. I like the fact in this short course, 3 important areas are covered with good practices: mathematics, computer programming, and technical writing.

  • The math involved is modeling with system of ordinary differential equations. Both analytical and numeric solutions to said model are introduced and practiced with good exercises;
  • The programming part is using Python to solve the system of ordinary differential equations numerically, with NumPy, using Euler’s method. Plotting is also introduced and practiced with Matplotlib;
  • Finally the course asks students to write a technical report using LaTex.

I also like the fact that the course encourages students to form a team and work together. I was very fortunate to be able to work with Zeus Garyulo, an Argentine currently working in Finland. Zeus is a wonderful teammate, very smart and has a much better grasp of math involved than I have. He broke down problems into actionable items quickly and provided the majority of the modeling, analysis, and validation work. Thanks Zeus!

The problem we chose to tackle is the spread of fake news. Without further ado, below is our report.
FakeNewsMathModel_Z

Automatically mount USB 3 attached to my wifi router

As discussed here, I have Seagate Ultra Slim USB 3 attached to my wifi router as a NAS (Network-attached storage). It provides a good backup/sharing storage for my home network that’s accessible from all my home computing devices, which was nice.

Accessing it from Windows is easy to set up, all I needed to do was to map a drive. Accessing it from Linux needs some additional work, but it turned out to be not too bad either.

Initially I created an entry in /etc/fstab. It uses the cifs protocol. Some cursory research shows that cifs is old and oboselete, but I’m not sure of other ways of accessing it. As I mentioned in that post, I had to put in the ver=1.0 option for it to work, something I don’t like.

Anyway, the drawback of /etc/fstab entry is that my Linux system would try to mount it before establishing network access, which doesn’t work for obvious reasons. I had to “sudo mount -a” afterwards. That is manual and gets tedious real fast.

So I did some research. One solution was to use _netdev option in the fstab entry, which didn’t work for me. The other was to use the auto option, which didn’t work either.

In the end, what worked for me was using systemd. Here are two articles that I relied to get mine going.

Tomáš Tomeček, Automatic mounts with systemd
Michael Albert, Systemd: (Auto-) Mount cifs shares This article has good examples for cifs options. Yes, I had to use the ver=1.0.

Thanks guys!

fail2ban installation and configuration notes

A couple of days ago one web site I volunteer to manage was under DDOS attack. I installed and configured fail2ban to protect us from future similar attacks. Here are some notes. The server is the RedHat/Fedora/CentOS variety, as you can tell from commands listed below. Please translate them to your distro’s corresponding commands as needed.

  • Installation is easy:
    sudo yum install fail2ban

    To make fail2ban starts automatically after a reboot, run this:

    sudo systemctl enable fail2ban

  • Configuring is relatively easy. It’s recommended that you create your own jail configuration file, using the jail.conf from the installation as a starting point. Three things are noteworthy from my experience:
    1. Make sure that you provide the correct log file. For web server, there are typically one access log file and one error log file. Ensure that you feed the right log file when using a particular filter;
    2. On this server, fail2ban didn’t properly expand the log and file names when I put wildcard characters in them. I got around that by listing them one by one.
    3. In the jail.conf file, no default banaction was defined. I added the following line:
    banaction = iptables-multiport
  • To write your own custom filter, make sure you put a sample log entry inside the filter file as a comment. Use the following command to debug your filter:
    sudo fail2ban-regex /path2testLogfile/test.log /etc/fail2ban/filter.d/my-filter.conf
    Here is a filter that I wrote:
    [Definition]

    failregex = ^ -.*”POST \/component\/mailto\/\?tmpl=component\&link=aHR0cHM6.*”$

    ignoreregex =

  • After getting your jail.local ready, run the following command to debug any potential issues. I’ve found that if you have issues with your jail or filter files, “sudo systemctl start fail2ban” doesn’t always give you a good enough error message. Use this instead:

    sudo /usr/bin/fail2ban-client -x start

    You may need to start/stop a couple of times. To stop, run

    sudo /usr/bin/fail2ban-client -x stop

  • After debugging, before you finally start fail2ban service, it’s better to search the current access/error log and see if there is a match to the filter you defined. If yes, then take a note of its IP address and the last time it appears in the log file. Then start fail2ban by running
    sudo systemctl enable fail2ban
  • To verify that it works, run iptables -S and if it catches one offender and puts it in jail, you should see it in the output. Now go back to the access/error log and ensure there is no entry from that IP address since the last timestamp.

Good luck in protecting your servers!

More Manjaro config: Chinese input and clock setting

I’m enjoying running Manjaro Arch Linux on my workstation so far. Some additional notes below.

1. I had trouble getting Fcitx Chinese input working, so I used IBus instead. Here is what I installed using Manjaro’s package GUI tool: IBus Preferences and ibus-pinyin;

2. IBus Pinyin works fine on this system except for Emacs. Using the system’s IME for Chinese input into Emacs is a fairly common problem. There are various hacks to get it work (mostly dealing with playing with LC_CTYPE=zh_CN.utf8), none to my liking. So I installed an Emacs package from MELPA, pyim. I put the following into my .emacs.d/init.el:

   (require 'pyim)
   (require 'pyim-basedict) ; 拼音词库设置,五笔用户 *不需要* 此行设置
   (pyim-basedict-enable)   ; 拼音词库,五笔用户 *不需要* 此行设置
   (setq default-input-method "pyim")

Ctrl-backslash invokes this input method. Ctrl-n and Ctrl-b pages down and up for character selections. It’s not as sophisticated as a full-blown IME, but it’ll do in a pinch.

3. I customized the format of the clock in system tray to %Y-%m-%d %A %R