The Ji Village News

季庄新闻–Haidong Ji’s Blog

I’ve been working in my spare time on a database testing tool, AnyDbTest, with my friend Wade. The program is written in C#, based on the .Net Framework 3.5.

It can be used in many scenarios: business analysts and QA can use it to confirm and validate data, recordsets data comparison; database developers can use it for code refactoring, unit testing, and when run all test cases together, it can also help with regression testing, etc. It currently supports SQL Server, Oracle, and MySQL.

Here are some highlights:

• Writes test case with XML, rather than Java/C++/C#/VB test case code
• Many kinds of standard assertion supported, such as StrictEqual, SetEqual, IsSupersetOf, Overlaps, RecordCountEqual etc
• Allows using Excel spreadsheet/XML as the source of the data for the tests
• Supports Sandbox test model, if test will be done in sandbox, all database operations will be rolled back, meaning any changes will be undone
• Unique cross database testing, which means target and reference result set can come from two databases, even if one is SQL Server, another is Oracle.

Currently we are working on developing a command line version of the application, which will greatly aid automation. With a console application, it is possible to interact with code and test cases stored in source code repositories, and make complete integration and regression testing possible with a simple batch file.

Here is the link to the trial download. I’d appreciate it if you could let me know your feedbacks and suggestions if you decide to try.

今天早上读Glenn GreenWald的博客，读到“They (the Chinese) don’t want to buy a pig in a poke”，虽然从行文里猜得出意思，但我还是用鼠标左键选择，然后右键股沟搜索，来查查这个成语的出处。我到现在还搞不明白为什么IE里没有这个功能，该功能已是我学习的利器之一。

原来欧洲中世纪后期流行的商家坑人手段是挂猪头，卖猫肉：把猫肉放在袋子里当猪肉卖，因猪肉稀罕但猫多。所以“pig in a poke”是忽悠人的意思。这和我们的”挂羊头，卖狗肉”有异曲同工之妙。

引申开来，”Let the cat out of the bag”就是揭秘，还真相于天下的意思。我明白它的意思，并且也用过这个俚语，但今天是第一次搞明白出处，挺有意思的。

感谢宋以郎的链接，让我了解到Glenn Greenwald的博客。个人以为Glenn Greenwald的文笔犀利，逻辑清晰，其揭露媒体之无良、政府之腐败、权威智库之装逼的深刻，令我击掌叫绝。

Update below
Reports in blogsphere about some WordPress blogs being hacked got me alarmed. So I decided to upgrade mine from whatever version it was at (2.3, perhaps, whatever the current version was a year ago) to 2.8, the latest, but certainly not the greatest.

The upgrade process wasn’t hard, just a bit tedious. After posting a new entry and it not showing up in Google Reader two days after the fact, I felt something was not right.

So I clicked the feed link directly, http://www.haidongji.com/feed, and got the error below:

Warning: include_once(/home/xxx/public_html/wp-includes/pomo/mo.php) [function.include-once]: failed to open stream: No such file or directory in /home/xxx/public_html/wp-settings.php on line 307

Fatal error: Class ‘Translations’ not found in /home/xxx/public_html/wp-includes/l10n.php on line 407

The strange thing is that I did have mo.php in the right place, under wp-include. So I went to Feedburner’s site, now part of Google, and it didn’t have anything useful there. My sixth sense told me there might be something fishy with the whole Google feed integration business, so I decided to restore the .htaccess file to the state prior to Feedburner integration, and see what happens. Lo, it worked! That was last night.

Today I saw the comment page link was broken. After clicking the recent comments link on the right side, there was a 404 page not found error. Once again, I invoked my psychic debugging skills and decided to revert .htaccess to the one generated by Feedburner a few years back, the one I ditched the previous night. Behold, it worked!

Now some additional gripe regarding WordPress 2.8: why do we need to make the dashboard admin page more complicated than necessary? The 2.8 dashboard looks busy and noisy, and it made it considerably harder to go to places I wanted to go. After finally locating the settings page I am after, I need to scroll all the way to the bottom to view it! Keep it simple, please.

Update: It turned out I claimed success too early. After I reverted to .htaccess from Feedburner, I noticed the feed was not refreshed with the latest entry. So I went back to the original .htaccess file, where comment page was broken. I googled around and realized the permalinks was the culprit. So I did chmod 666 .htaccess, went to my WordPress admin page, saved permalinks changes, then chmod 644 .htaccess, and now I am back in business.

Now my make-shift working area is no longer the dining table after the move, I am eager to set up a proper environment for learning and developing software at home.

My thought is to have one decent computer that powers a few virtual machines. I like the idea of virtual machines as opposed to physical ones, which are more costly and messy. And I was pretty determined to run a Linux distro as the host, because I want to live and breathe in it for a while, to bring my Linux skill to a similar level of my Windows knowledge.

I’ve learned a few things during my quest for a better computer configuration. At times, it was really frustrating.

• SSD (Solid State Disk) is nice. It provided a tangible, fairly obvious performance boost to my system. I am no longer afraid of the previously daunting prospect of long running install and uninstall, such as SQL Server. Better yet, when I want to test something that can potentially have a negative side effect, I now would do it in a virtual machine. Before such test starts, I take a snapshot. Prior to SSD, it would take a long time to do a snapshot of the VM. With SSD, it is much quicker now. If the test didn’t go well, I would roll back the virtual machine to the state when snapshot was taken;

  Begin rant

• So far I am disappointed with all Linux distro that I’ve tried. Part of it is understandable, as vendors will make their devices working on Windows as a priority. I remember clearly the frustration I had when trying to get dial-up modem work properly on Red Hat, and my struggle later with wireless card. But things like these are inexcusable:

  1. Mouse freezes up for no apparent reason. When that happened, I always had to do a hard shutdown. Yes, I’ve tried various ways to restart X, but a) it didn’t work; b) Even if it worked, I would not take it. Please spare me the lecture of always using keyboard all the time. This is the deal breaker for me;
  2. Sound card stopped working after some system update from Ubuntu’s repository;
  3. Wireless card is flaky. It does not work on a consistent basis, in other words;
  4. System updates broke my display driver. I was forced to use a lower resolution display than what the monitor is capable of. I think if I recompile, I could fix it, but I was so pissed off that I didn’t bother;
  5. I couldn’t enable file sharing. It told me to resolve some dependency issues, but there were none!
  6. Ubuntu comes with Firefox 3.0, and there is no decent way of upgrading it to Firefox 3.5, other than Ubunzilla, but I cannot install Ubunzilla because the aforementioned dependency issue! And if I use apt-get install firefox-3.5, why deliver Firefox 3.0, 3.1, and a beta version of 3.5?
  7. Could we please stop using code names like Hardy Heron, Intrepid Ibex, and Jaunty Jackalope? Use that internally is fine. Version number 8.04, 8.10, and 9.04 will do for the general public.

• After my frustration with Ubuntu, I decided to try Fedora and suseLinux (both gnome and kde). Maybe I didn’t give them enough time, but I encountered mouse and windows freeze issue as well. Ditto for Kunbuntu.

  End Rant

So I came back to Windows. Now the host runs Windows 7 Enterprise with VMWare virtual machines running Windows XP and Linux Mint 7, which looks anesthetically pleasing to me, for now.

For virtual machine software, I am using VMWare Workstation 6.5. I’ve looked at and tested a few other choices: Xen, Microsoft hyper-v, Virtual PC, Virtual Server. And VMWare Workstation came out on top, in my personal opinion. I plan to give VMWare 180 bucks for a license.

PS. I tested VirtualBox before and was reasonably impressed. I really hoped I could use it at home. But it will not make the cut, I am afraid. Read Emilian’s great critique here. I just want to add, in addition to snapshot issues, I had trouble getting the shared folder working.

Should I try FreeBSD as host?

I’ve been frustrated with anti-virus software for a while. At a client last year, I had to fight with Symantec to get a proper ASP development environment up and running. At home, I’ve used AVG, avast, McAfee, and others. Far from the feeling of being protected from the “scary world out there”, I started to view the scanning, listening, warning, and even “calling home” “features” of anti-virus software as hindrance to my daily digital life. Granted, anti-virus software probably needs to do those kind of things, but it needs to get out of the way!

After reading similar complaints on Stackoverflow.com, I decided to follow a few others and started running Windows XP and Windows 7 without any anti-virus software. I’ve run a virus-free and anti-virus-free Windows XP machine for close to a year now, in addition to a few virtual XP machines. Also, the home Windows 7 machine has also been running naked for a few weeks now without issues.

Here are a few things I do or don’t do:
1. Browser: I use Firefox and Chrome. When running Firefox, I use adblock plus plug-in;
2. When installing software, I always pick custom installation, and I uncheck all un-necessary features;
3. Exercise good judgement.

Today I purchased a HP Pavilion p6130y desktop as my main development workstation at home. I pulled the Intel X-25M SSD out of my laptop, and put it into this new box as the primary disk, and loaded Ubuntu 9.04 64-bit on it. So far, I am impressed. Things are fast and the machine is also quite.

However, I did notice a display problem. Out of the box, the default resolution was perhaps 1024×768, certainly below 1920×1080 that this Dell display can handle. So I went to System -> Administration -> Hardware Drivers to get the latest nvidia driver, because nvidia graphics card (NVIDIA GeForce 9100 GS) comes with this HP on the motherboard. It looked nice initially, but whenever I try to maximize a window, the system would freeze, forcing me to shut down the machine ungracefully by holding down the power button. Opening Firefox would cause the same thing, because my Firefox window opens in full / maximized mode.

I then spent the next few hours trying to figure this out. I opened the display setting window and tweaked the values here and there, hoping that would resolve the issue. I also uninstalled and reinstalled the driver a few more times. I ened up rebooting this machine so many times that I lost count.

Then it occured to me that instead of googling and scaning information that looked pretty irrelevant to my issue, I should try nvidia’s site directly. Sure enough, I could download Linux 64-bit display driver on its site! The download file was a .run file. I followed directions here, but the driver wouldn’t install because X is already running. So I booted into recovery mode, ignored the “telinit 3″ (which starts X) prompt, and just followed the directions in the terminal. The process tried to download some kernel files without success, so it compiled. One step also asked for 32-bit compatibility, which I answered “yes”, whether that step was successful was not confirmed. After the driver was successfully installed, I ran “telinit 3″ and the display issue went away.

Hope this helps somebody out there.

Update: to install the same driver on Fedora 11, get into the screen where you can edit grub loader entry, press a, space key, 3, then enter. That will start Fedora without X running. Then run “sh NVidiaDriverFileName.run”, follow instruction on screen and you will be good to go.

In the past, when I read technical books, I tend to skim through them, looking for keywords and only read the part that is relevant at the moment, and move on. Sometimes I would make an attempt at finishing a whole book, but a few months or even years later, I haven’t even finished the first 3 chapters!

I took a different approach recently. Now I’ve set up daily goals to read 50 or more pages or a key section of a technical book, and follow through. I am reading two technical books at the moment: Itzik Ben-Gan et al’s Inside Microsoft SQL Server 2005: T-SQL Querying, and Baron Schwartz et al’s High Performance MySQL. It’s nice to read database books that focus on different vendor implementations (SQL Server, MySQL, Oracle, etc.), because each one explains certain things from a slightly different angle, with slightly different language, and at times this gives you a better feel of the overall picture and clarity to certain key concepts.

Here I am talking about quality technical books, though, because the industry churns out way too much junk. It certainly is a waste of time and money on poorly thought out and written books.

Anyway, today I went through Itzik Ben-Gan’s performance tuning chapter. I used SQL Server 2008’s Management Studio to do tests against a SQL Server 2005 instance. I noticed an interesting change in terminologies: in SQL Server 2008, a bookmark lookup on a table with clustered index is now called Key Lookup, on a table without clustered index is (still) called RID lookup. Here are some screen shots:

Bookmark lookup in SQL Server 2005 on a table with cluster index

Bookmark lookup in SQL Server 2005 on a table without cluster index

Bookmark lookup in SQL Server 2008 on a table with cluster index

Bookmark lookup in SQL Server 2008 on a table without cluster index

To recap, here are the terms used for bookmark lookups in the 3 most recent SQL Server releases:

SQL Server 2000: bookmark lookup
SQL Server 2005: RID lookup on a heap, Clustered Index Seek on a table with clustered index
SQL Server 2008: RID lookup on a heap, Key Lookup on a table with clustered index

SQL Server’s clustered index implementation went through some interesting changes. Prior to SQL Server 7, all non-clustered index contains a pointer to the actual row(s) that has the value of the indexed keys. This pointer (RID, Row ID) physically points to the position on which page in which internal file that row is at. Starting from SQL Server 7, for tables without a clustered index (heap), the implementation stays the same. However, for tables with clustered index, the pointer is the clustered index.

This can potentially have a big impact for bookmark lookup on tables with clustered index. Here is why: to do lookups, SQL Server needs to traverse through clustered index, thus more reads. The number of additional reads this causes depends on the level of clustered index and how many rows the query touches. Suppose the clustered index has 3 levels (root, leaf, and one intermediate level), then a single bookmark lookup will incur 3 additional logical reads. If the query touches 2000 rows, then bookmark lookup will cause 6000 additional reads.

Note I am not bashing against clustered index though. Overall, in my opinion, the benefits of clustered index definitely outweighs its drawbacks. Now this post is getting long and I want to go back to my books, so I will stop here.