The Ji Village News

季庄新闻–Haidong Ji’s Blog

I’ve done this in the past, but thought this time I’ve got to take notes. It can be used as a crude check list in the future. Don’t underestimate the power of a practical, down-to-earth check list! Perhaps documents like this should be kept in a wiki page, for easy updating to avoid being stale, a proeblem with blog entries, it seems.

P in LAMP here stands for php, not Python or Perl. L is CentOS (I used CentOS 5.5) or Red Hat Linux. I am not covering moving all databases in a MySQL instance, just a select few or just one.

I’d appreciate your comments or suggestions.

Software install and configuration
MySQL:
MariaDB or Percona.

MariaDB can be had here.

For Percona server and client tools, it’s best to have direct access to Percona’s repository:

yum install gpg
rpm -Uhv http://www.percona.com/downloads/percona-release/percona-release-0.0-1.x86_64.rpm

Do

yum search Percona

to find Percona packages you are interested in.

Percona MySQL Server package does not create /etc/my.cnf for you. In today’s server, I think you should use /usr/share/doc/Percona-Server-server-51-5.1.54/my-huge.cnf as default /etc/my.cnf if you are not sure. Remember to turn on the slow-query-log. Also, remember to set interactive-timeout=600. You may also need to increase the max_connections setting.

Open up port 3306:
Edit /etc/sysconfig/iptables: add this line somewhere above COMMIT

-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 3306 -j ACCEPT

Restart the firewall:

service iptables restart

Note: Here is how I came up with this configuration. I found the line that has 22 in it, which is the default port for ssh, yanked and pasted it, replacing 22 with 3306. This works, but I am not sure if it is done correctly. Comments welcome.

On source MySQL server:
1. FLUSH TABLES WITH READ LOCK
2. The backup can be done many ways. This time I used xtrabackup, but I am leaning toward using mysqldump or mysqlhotcopy exclusively next time I do it, because I was not satisfied with innobackupex-1.5.1′s –stream and –apply-log functions. Perhaps my bad experience was due to working with a particularly messed up MySQL instance and/or a under-powered server. Percona did mention that they are not satisfied with innobackupex so hopefully a better version will come out.

In any case, here is what I did with innobackupex-1.5.1 for database backup. It did get all MyISAM tables correctly, though, just not the InnoDB tables:

innobackupex-1.5.1 --databases=myDB --user=samIam --password=IamSam --stream=tar ./ | gzip -> /home/sam/myDb.tar.gz

For InnoDB tables, I did:

mysqldump -usamIam -pIamSam myDb innoDbTable > innoDbTable.sql

For stored routines, I did:
[sourcecode language="text"}
mysqldump --routines --no-create-info --no-data --no-create-db --skip-opt myDb > routine.sql
[/sourcecode]

3. Move the backup file to the new server. Percona mentioned a few creative ways of moving files here.

4. Use the excellent mk-show-grants, part of Maatkit, to get the login/user/permission generation script. Modify that as necessary.

On destination MySQL server:

1. Unpack myDB.tar.gz and move all myDb directory and its contents to MySQL’s data-dir, perhaps /var/lib/mysql or /usr/local/mysql/data. While inside MySQL data-dir on the shell, do

chgrp -R mysql myDB/
chown -R mysql myDB/

If you took a dump earlier, pun not intended, run this on the shell

mysql myDB < innoDbTable.sql

For stored routines, move over the dumped file, and run this:

mysql myDb < routine.sql

2. Do a select count(*) on all myDB tables. I don’t have a clever way of doing this, I did “show tables”, grabbed the results, and hacked with Vim’s macro to generate a bunch of select count(*) statements and ran it. This should let you know if there are any issues with your newly restored myDB. mysqldump -d myDB is also a good thing to run to find out problematic tables, I think.

3. Run the login creation script generated earlier with mk-show-grants

Web server:

If a brand new web server, these are needed, at a minimum

yum install php
yum install php-mysql
yum install httpd

It might be necessary to edit /etc/php.ini, so there is a line
extension=myql.so
under Dynamic Extensions

By my sonson
Legislators, you have come here to hear the cons of Saturday school. I come before you today with compelling evidence that Saturday school is a bad thing from the start. First, finance will decrease all over America. Second, companies will fail. And third, the average kid will get over stressed. Eventually, with Saturday school all America will fall.

My first reason is in regard to finance. More electricity will be used to power schools due to extra time spent in school. To pay a teacher for a year will cost a lot more money than now from all the extra work they will do. Even families will have to pay more due to an increase in taxes to pay the school. This will lead to a drop in the economy. The consequence would be America going bankrupt from all the money needed to pay for schools. I suggest that if kids need extra help a tutor can be used instead and would be cheaper than Saturday school.

My second reason is that companies will fail. Families won’t have enough time to buy things because their time is wasted in school. Kids won’t have time to play with toys so they won’t buy them because their time is wasted in school. Eventually, companies will close down with no profit money. With no companies there are no jobs to supply civilians with money. Within months the economy will fall. Then people will emigrate to a different country to avoid the bad economy.

My third reason is about health. Kids will get extremely stressed from all the homework and school time. Then kids will get sick and depressed from too much school. Then from stress people won’t be happy and become bullies. With so many bullies the schools will go crazy. And when bullies grow up they will do horrible things to America.

America is lucky as it stands right now. With Saturday school America will fall. Bills will increase, companies will fail, and people will become stressed. This is not the America we know. This is a free country but with Saturday school it is not.

胡哥访美结束，我对双方做出的关于加强两国人民的交流的共识很有兴趣。要消除隔阂和偏见，没有别的好办法，只能通过人们之间的交流和互动。政客和记者们想放大人民之间的不同，吸引眼球，构建“外人”的怪异、卑鄙、和有威胁感的可怕形象。但从我自身的体会，只要人们的交流和互动增多，会发现虽然我们有种族、文化、语言、外表、风俗和其他方面的等等不同，但我们的相同远远大于不同：我们都是人，我们都追求尊重、理解、和宽容，还有那个叫幸福的东西。有了这个切身体会，这种同理心，或者说empathy，人与人之间的交流就会更顺畅，就会有眼界和心胸来打开局面，解决我们之间的问题。

我希望两国政府都能把这个共识落到实处，给两国人民的签证带来方便。我本人以前就深受签证难之苦。去年我给我州的参议员迪克-得斌（Dick Durbin）写过信，希望他能问询和促进这个问题的解决。他的工作人员后给我回信，很多套话，我非常不满意。晒出来给大家看看。

我同时也希望中国政府给美国人（实际上任何一个国家的人士）签证提供方便。我去过中国驻芝加哥领馆多次，感到那里的工作人员的效率和态度很不错。但我也听到了一些美国朋友在获取工作签证的难处。希望政府能研究和改善这个流程，能欢迎各国的人士来到中国。

电邮，2010年7月28日
Dear Senator Durbin,

My name is Haidong Ji. I am a naturalized US citizen, residing in (township omitted here), Illinois. I became a US citizen on July 5th, 2008, in time for the 2008 general election, where I cast my ballots for Democratic, Republican, and Green Party candidates, which was nice. With the mid-term election coming up, I am collecting info regarding candidates and their positions to make an informed decision when I vote.

One thing that is of particular importance to me is the US-China relationship. Specifically, I’d like to see us making changes with regards to visa and entry permits to Chinese citizens with valid H-1B (and H-4 for spouse and dependents), J-1 (and J2 for spouse and dependents), and F1 (and F2 for spouse and dependents) visas.

Here is what I know of the current policy. Up until middle of July, 2010, for a valid H-1B visa holder with Chinese citizenship, from the day that H-1B/H-4 status is granted, that person is allowed 2 entries to the US for the first 3 months. If said Chinese person leaves the US for whatever reason more than 2 times within those first 3 months or after the first 3 months, the individual will have to apply for permission to enter the US, even if the H-1B/H-4 is still valid for a total duration of 3 years. I’ve learned recently that the 3-month 2 entries arrangement has been expanded to 12-month multiple entries. In my opinion that is not enough.

F1 and F2 visa holders does not fare any better. During university summer and winter breaks, a valid F1/F2 Chinese student will have to apply for permission to enter the US if s/he visits home and family.

I have experienced all of the above. I came to the US in 1995 on F1 visa, and didn’t go back to China to visit my family during graduate school years. The visa and reentry restriction was a significant factor. After school, I started working on H-1B status, got engaged, and visited my in-laws in Sweden. When it was time to come back to the US, I went to the American Embassy in Stockholm to apply for entry permit. I don’t recall all the details now, but suffice it to say that I couldn’t get it in time, and I did remember the rude and brash attitude displayed by the Embassy staff. Therefore I had to adjust my flight and applied for extension to stay in Sweden longer.

Then my son was born and grandma in Sweden was dying of cancer. My wife flew back with a less than 3-month old to say goodbye and for grandma to see the grandchild. I didn’t go along, due to the fear of being rejected of an entry permit, even when I have a perfectly valid visa!

Every single year we have ten of thousands of Chinese students studying in the US on F-1 or J-1, and also a lot of Chinese nationals working in high tech and R&D on H-1B. We also see a lot of Americans working and studying in China. This is all fantastic, if you ask me. However, the entry restrictions on these Chinese nationals means long period of not seeing family and friends, 8 years, in my case. Combining that with the necessary adjustment of language, culture, and social norm, the entry restrictions imposes a real hardship on them, as my personal experience attests.

We are a country of immigrants. And I firmly believe our diversity is the greatest strength of America. To let some bureaucratic process impose undue hardship on people with perfect legal status, I think we can do better than that. Not to mention that China is such an important partner to us that we should do everything within reason to encourage the travel, study, business, and non-governmental exchanges between our two people.

So I’d really appreciate it if you could inquire this matter with relevant governmental agencies, and see if they can remove/lessen the entry restrictions on Chinese citizens with F-1, F-2, H-1B, H-4, J-1, and J-2 visas.

Thank you very much.

Sincerely,

Haidong Ji

收据回复，2010年8月5日
Dear Friend:

Thank you for your email. I appreciate hearing from you.

We are experiencing a high volume of incoming emails, but every message is
read by my staff and If you live in Illinois we will provide a response
as soon as possible.

I appreciate your comments. Thank you again for contacting me.

Sincerely,

Richard J. Durbin
United States Senator

空话连篇的最终回复，2010年11月3日，特别提到他如何“保护”本土美国人的就业。玩H-1B猫腻的当然有，但人家就是政客，悄无声息地就把概念偷换了，最后还能倒打一耙。

November 3, 2010

Dear Mr. Ji:

Thank you for contacting me to express your support for the H-1B and L-1 visa programs. I appreciate hearing from you.

The H-1B visa program provides for the temporary admission of foreign professional workers, including highly skilled technology workers, to the United States. The program is extremely popular with businesses in the technology industry. The L-1 visa program permits companies to transfer existing foreign employees to U.S. offices for a period of up to seven years.

In considering visa programs, we must promote measures that cultivate a strong, well-educated American workforce and enhance the ability of the United States to compete in the global economy. For many years, the United States has both nurtured a highly skilled domestic workforce and attracted the best talent from countries around the world. Countless immigrants have made significant contributions to America’s scientific, literary, artistic, cultural, and economic resources. That is why I have supported reasonable levels of legal permanent employment-based and family-based immigration.

The H-1B visa program provides American employers with thousands of well-educated temporary workers. We must ensure that the H-1B visa program is not adversely affecting the ability of Americans to find work.

On April 23rd, I introduced the H-1B and L-1 Visa Reform Act (S. 887), which would add protections for qualified American workers to the H-1B and L-1 visa programs while also permitting needed foreign workers to be employed in the United States.

The bill would require that an employer make a good faith effort to recruit American workers before giving jobs to visa holders. Before an employer could submit an H-1B visa application, the employer would be required to advertise the job for 30 days on the Department of Labor (DOL) website. The bill also would give the DOL the ability to conduct random audits to ensure that any company that uses the H-1B program and any company with 100 or more workers, with 15 percent or more of these workers with H-1B visas, do not abuse the program.

To ensure that companies are not undercutting American workers by paying substandard wages to H-1B and L-1 workers, the proposed legislation would require employers to pay foreign employees the prevailing U.S. wage. The bill also would prohibit companies that employ more than 50 people from hiring or outsourcing additional H-1B employees if more than 50 percent of existing employees are H-1B visa holders.

My legislation would protect American workers from being placed at a competitive disadvantage in the job market by H-1B and L-1 visa workers. Major portions of the bill were included in the comprehensive immigration reform bill in 2007. Unfortunately, that legislation was not passed by the Senate.

I also have helped craft a proposal that was incorporated into the competitiveness bill that became law to provide grants to domestic students who pursue a degree in science, mathematics, engineering, or a foreign language.

In the context of comprehensive reform, I am open to a discussion about the size of the visa program if the program is reformed to protect American workers and prevent abuses by employers.

Thank you again for contacting me. Please feel free to keep in touch.
Sincerely,

Richard J. Durbin
United States Senator

RJD/ab

One item discussed during President Hu’s state visit to the US is the HR situation in China. I first watched the video clip on msnbc.com. And then watched the whole news conference in its entirety on Youtube, which, by the way, is blocked in China by the firewall, a glaring censorship issue the Chinese government has to face.

But what’s curious is what MSNBC chose to leave out in its version of the HR QA video. The edition on MSNBC’s web site gave the impression that Hu tried to dodge the HR question and refused to answer, which appeared to be the opinion of the questioner, a Bloomberg guy. In fact, that was what I thought Hu did after watching it on MSNBC first. Before you know it, there was a meme spreading on the web alleging that Hu did just that.

I then watched the whole thing on Youtube, and came away with the opposite conclusion. In fact, Hu addressed that head-on, saying he was perfectly willing to answer it, explaining he wasn’t trying to avoid questions but wasn’t able to get to it because of translation/technical reasons, acknowledging China has a long way to go on the HR front. After watching the entire news conference, I do believe Hu’s explanation because of the rambling nature of this give and take in a news conference, the fact that 2 translators were working the room, one from each country, and the fact that Hu does not know much of English.

I am not proud of China’s HR record, and indeed I think there are many areas in China’s HR situation that needs reform and improvements. But for MSNBC to deliberately cut that part of Hu’s explanation, let the impression of him dodging question stand, and mislead readers/viewers, that is wrong! Combining this with the toxic mid-term election campaign that features xenophobic TV ads exploiting the irrational fears of the Muslims, Mexicans, Chinese, and Indians, one can tell what a sorry state our public discourse has become. And this is from the allegedly liberal-leaning MSNBC!

Here are the videos. First one is from MSNBC.

Visit msnbc.com for breaking news, world news, and news about the economy

Here is the complete video. Bloomberg guy started around the 40 minutes mark.

真得很讽刺，我在2007年曾写过美国的太子党和政府、华尔街、说客、国会的根深蒂固的裙带关系，提到过戴笠兄弟。这不，中期选举刚结束，按照优良的民主传统，现任总统（如果是第一届）又要开始投入到火热的下一轮的选战民主生活之中。非常善于演讲修辞的小奥同志，摇动三寸不烂之舌，在巩固、发扬了小布什的酷刑、侵犯个人自由、恐吓民众、给华尔街撒钱、搞垮经济、在全球变暖问题上耍嘴皮子的政策之后，已经开始准备。为此，他提拔了比尔-戴利为白宫大内总管，相当于李莲英的地位。

而上一届白宫大内总管，以色列兼美国公民和以色列陆军志愿者，前众议员，克林顿时期白宫助理，同样是芝加哥政客出身的大坏蛋拉姆-伊曼纽尔先生退下后已经回到芝加哥。拉姆窥视芝加哥市长一职已久：这个职务也是个肥差。自1955年以来至今的56年的时间里，经过民主选举，有43年这个市长姓戴笠：老戴利做了21年左右，他家老大做了22年左右。瞧这家族政治搞得，没治了。老大的任期到今年底结束。真巧啊，老大今年不混了（不参选），伊曼纽尔从大内总管退下后决定竞选芝城市长接这个茬，而戴利家老二比尔呢，顶替伊曼纽尔当李莲英去了。这安排真够紧凑的。我个人对伊曼纽尔的选情不看好。呕，戴利家老三也在芝城供职，好像管财务。

小奥的对手有可能是美国版江青，莎拉-佩林女士。佩林女士所属的右翼也有意思，有不少像张春桥、姚文元之类的大拿。他们不贴大字报，但有人喜欢玩儿黑板报，像那个声色俱厉、有张春桥遗风的格兰-贝克先生（链接里有这孩子做的黑板报）。这家伙做电视时，喜欢把名词在黑板上写出来，画圈儿、打勾、连线的解释，让不少人觉得挺受用。可以保证，接下来的总统选举，和以往一样，热闹着喃，大家有好戏看。制作戏剧和鳌头，是我们的强项，这一点，绝不让人失望！

——

胡哥和小奥要会谈了。这个维基揭秘泄漏出来的美大使馆的关于中美关系的建议比较有意思，我感觉有好建议也有坏建议。我这儿有个关于反贪污的想法。

2010年底的中国国务院发表的反贪污白皮书。我看了。这是个进步，但还是有很多空话和套话。白皮书里谈到反腐的国际合作问题。我就想，这个可不可以作为两位总统的一个议题呢。腐败的官员往美国转钱和转人，中美在这方面可以有作为呀。

顺便说一句，为什么各大报纸和网站的文章里没有这个白皮书的链接？有资料，就给我链起来，我要看原始资料，不要给我添油加醋！东西媒体，都要这样做！

Command line utility to search for objects based on names. Search results will be displayed in well formatted tabular format, left-justified.

1. pyodbc installed;
2. Use trusted authentication by default. Find the relevant code to adjust to login/password as needed;
3. -w for wild card search. Default is exact name match, case insensitive in almost all cases, depending on your SQL Server configuration;
4. If -d is not specified, all databases in instance will be searched, minus the ones that one not accessible at the moment. To specify database(s) you want to search into, use -d followed by database names. Separate them with comma if more than one;
5. Use -S to specify instance;
6. Good sample code for command line parameters processing and pyodbc usage;
7. Won’t work with SQL Server 2000 or lower.

Example 1: python objectSearch.py myobject -S instance1
Result: Database objects that are named myobject in instance1 will be displayed

Example 2: python objectSearch.py -S server1 -w -d db1,db2 myobject
Result: Database objects that has “myobject” in db1 and db2 in server1 will be displayed

import pyodbc, argparse
def pp(cursor, data=None, check_row_lengths=True):
    if not data:
        data = cursor.fetchall( )
    names = [  ]
    lengths = [  ]
    rules = [  ]
    for col, field_description in enumerate(cursor.description):
        field_name = field_description[0]
        names.append(field_name)
        field_length = field_description[2] or 12
        field_length = max(field_length, len(field_name))
        if check_row_lengths:
            # double-check field length, if it's unreliable
            data_length = max([ len(str(row[col])) for row in data ])
            field_length = max(field_length, data_length)
        lengths.append(field_length)
        rules.append('-' * field_length)
    format = " ".join(["%%-%ss" % l for l in lengths])
    result = [ format % tuple(names), format % tuple(rules) ]
    for row in data:
        result.append(format % tuple(row))
    return "\n".join(result)
def objectMatch(wildcardSearch, objectNamePattern, dbName, cn):
	if wildcardSearch:
		sql = "select '%s' as dbName, s.name schema_name, o.name, o.type from %s.sys.objects o inner join %s.sys.schemas s on o.schema_id = s.schema_id where o.name like '%%%s%%\' order by dbname, schema_name, o.name" % (dbName, dbName, dbName, objectNamePattern)
		cursor = cn.cursor()
		cursor.execute(sql)
		rows = cursor.fetchall()
		if len(rows) > 0:
			print pp(cursor, rows, 1)
	else:
		sql = "select '%s' as dbName, s.name schema_name, o.name, o.type from %s.sys.objects o inner join %s.sys.schemas s on o.schema_id = s.schema_id where o.name = '%s' order by dbname, schema_name, o.name" % (dbName, dbName, dbName, objectNamePattern)
		cursor = cn.cursor()
		cursor.execute(sql)
		rows = cursor.fetchall()
		if len(rows) > 0:
			print pp(cursor, rows, 1)
parser = argparse.ArgumentParser(description='SQL Server object search. Case insensitive')
parser.add_argument('-S', '--server', help='Instance you wish to connect to. myInstance is the default if not specified', dest='instance', default='myInstance')
parser.add_argument('-w', help='Wildcard search indicator. If specified, LIKE clause will be used for pattern matching. Otherwise it will look for an exact match. Exact match is default', action="store_true", dest='wild', default=False)
parser.add_argument('-d', '--database', help='Database(s) you want to search in. If more than one, separate them by comma only. It will search all databases by default', action="store", dest="dbs")
parser.add_argument('objectNamePattern', help='Object name pattern you want to search for', action="store")
argList = parser.parse_args()
try:
	cn = pyodbc.connect("DRIVER={SQL Server};SERVER=%s;DATABASE=master;Trusted_Connection=yes" % argList.instance)
except:
	print "Couldn't connect to %s. It is down or you might have had a typo." % argList.instance
if argList.dbs is None:
	cursor = cn.cursor()
	cursor.execute("select name from sys.databases where state = 0")
	rows = cursor.fetchall()
	for row in rows:
		objectMatch(argList.wild, argList.objectNamePattern, row.name, cn)
else:
	dbs = argList.dbs.split(',')
	for db in dbs:
		objectMatch(argList.wild, argList.objectNamePattern, db, cn)