Archive for May, 2009

Perl script to split sql script into separate files

Below is a quick script I put together to split one single file that contains many stored procedures into many files that contain just a single stored procedure. It can be easily modified for views, user-defined functions, etc..

Notes:
1. It does put BEGIN and END before and after the procedure creation. Because some script has grant statements in it, and I want to put END right after the procedure creation, therefore there is a variable called $FirstGrant;
2. The new files created will be named like StoredProcedureName.sql.

#!/usr/bin/perl
use strict;

open(MYDATA, "c:\\junk\\MyDb\\AllProc.sql") or
  die("Error: cannot open file 'AllProc.sql'\n");

    my $FirstGrant = 0;
while( my $line = <mydata> ){
if ($line =~ /^create\s+proc/i) {
   close (FILE);
   $FirstGrant = 0;
   (my $filename) = $line =~ /create\s+proce?d?u?r?e?\s+\[?d?b?o?]?\.?\[?(\w+)/i;
   open (FILE, ">", "c:\\junk\\MyDb\\".$filename.".sql");
   print FILE "IF NOT EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[dbo].[$filename]') AND OBJECTPROPERTY(id,N'IsProcedure') = 1)\n";
   print FILE "BEGIN\n";
   print FILE $line;
}
else {
if (($line =~ /^grant\s+execute/i) && ($FirstGrant == 0)) {
   print FILE "END\n";
   $FirstGrant = 1;
}
   print FILE $line;
}

}
close MYDATA;

Comments

推荐李老师的博客

我的网站右侧的博客链接一直有李老师的“水边树木笔记”。真得很喜欢他的博客,因为他的文笔好,摄影图片耐看,是火腿无线电爱好者。但更重要的是,李老师奋斗在教育战线多年,为厦门本地人以及外来人口如民工等人的孩子的教育做出了实实在在的贡献。不信请看这些链接。

阻止超载车上路
回答一个咨询邮件
关于小学入学的邮件

这样的人才真正值得佩服!咋咋呼呼,人五人六,上纲上线,一知半解,大放厥词,主观臆断的傻逼算老几?

Comments

Calling MySQL fans in the near west suburb of Chicago

I’ve been to the Chicago MySQL meetups a few times at Uncommon Ground near Wriggly Field. Honestly I don’t think I gained much out of it. The few times I was there, the group was small, which is not necessarily bad. (In fact, a small group can even be a good thing.) But the thing got me was that there was too much talk, not enough action. I haven’t been to its meetings for a few years so I cannot comment on its state after I left.

When I think a user group, I think about fans getting together hacking stuff: demonstrating cool tips and techniques, dissecting a new technology such as MySQL proxy by a user who have done that and learned some hard lessons along the way with no marketing talk, discussing solutions to problems a user has, providing pointers and efficient scripts, etc. In such a setting, each member of the group takes turns providing his/her expertise. The presenter does not necessarily needs talking slides, but must prepare beforehand about the topic s/he will present. The key points for a night’s session can be distilled into a few points on an index card, and demonstrated with simple, yet elegant steps. I’ve watched so many demos that needs elaborate setup, that by the time of the supposed highlight the audience have already tuned out or lost interest. When a question arises that can be answered via live demo in a short time, answer the question that way, instead of just giving an verbal response.

Another thing that I’ve been thinking is that for a big metropolitan area such as Chicago, more than one user group is necessary. The biggest reason is traffic. After a hard day’s work, going out of one’s way to attend a meeting that may or may not be productive is a big turn off, especially for married guys/gals with kid(s). Another reason is that big cities usually have a big talent pool and they may spread out around the suburbs.

Now that the arrangement between MySQL and meetup.com is over, there are a number of suggestions as to how to keep MySQL user group going. One suggestion was to use Facebook. I have created a Facebook group called MySQL OPRF. Here is the blurb I put out for this group: “MySQL group for Oak Park, River Forest and surrounding area like Forest Park, Berwyn, Elmwood Park, etc. The primary purpose is to swap ideas, hack software, and help each other to learn by sharing experiences, providing pointers, and keeping us motivated.” So, if that is something you are interested and you want to participate, by all means, join the club! I will arrange a meetup if there are enough interest. I’d also like to take this opportunity to solicit help in the form of technical sessions, books, money, and other resources that are of interest to a group like this.

I understand that the MySQL meetup on meetup.com in Chicago will continue. So by all means, join that group if it works for you.

Comments

Generate insert scripts for Sql Server tables with Perl

Here is a quick hack with Perl to generate insert statements to reproduce data already exist in a given table. It is similar to results mysqldump generates.

A few things worth mentioning here:
1. This script uses a text file, TableList, that contains a list of tables that you want insert statements created. Each line in the list contains the database name and table name, separated by space(s). This is a quick hack, one could easily pass the file name as an input;
2. Obviously the connection information needs to be modified for your own needs;
3. The script does escape single quote in character fields properly;
4. Use
perl ScriptName.pl > c:\MyFolder\MyTableInsert.sql
to collect the results into c:\MyFolder\MyTableInsert.sql file.
5. Thanks to Summit Amar and people who commented on this article for sql script ideas.
6. Update: modified the script so it handles identity field if it is present in a table.

By the way, Microsoft’s new scripting language, PowerShell, borrowed a lot of good stuff from Perl and Unix shell scripting, which is pretty powerful. My friends Yan Pan and Ananthakumar Muthusamy wrote a book, Microsoft SQL Server 2008 Administration with Windows PowerShell, which I helped to tech-edit. If you are looking to learn more automation and scripting tricks with PowerShell, definitely check out that book, which will come out next month.

#!/usr/bin/perl
use strict;
use Win32::SqlServer;

# Get server name from the command line
# my $ServerName = shift or die "Please enter server name as the first only parameter needed";
# my $TableName = "authors";

open(MYDATA, "RefList.txt") or
  die("Error: cannot open file 'RefList.txt'\n");

while( my $line = <mydata> ){
  $line =~ /(\w+)\s+(\w+)/;
  my $db;
  my $table;

  if ($db ne $1){
	print "-----Tables in $1-----\n";
  }
  $db = $1;
  my $sqlsrv = sql_init(".", "sa", "s3cr3t", $1);

  my $HasIdentityColumn = $sqlsrv->sql_one("select objectproperty(object_id('$2'), 'TableHasIdentity')");
  if ($HasIdentityColumn == 1) {
	print "SET IDENTITY_INSERT $2 ON\n";
  }
# Our SQL statement.
  my $stmnt = < <SQLEND;
--Declare a cursor to retrieve column specific information for the specified table
DECLARE cursCol CURSOR FAST_FORWARD FOR
SELECT column_name,data_type FROM information_schema.columns WHERE table_name = '$2'
OPEN cursCol
DECLARE \@string nvarchar(3000) --for storing the first half of INSERT statement
DECLARE \@stringData nvarchar(3000) --for storing the data (VALUES) related statement
DECLARE \@dataType nvarchar(1000) --data types returned for respective columns
SET \@string='INSERT '+ '$2' +'('
SET \@stringData=''

DECLARE \@colName nvarchar(50)

FETCH NEXT FROM cursCol INTO \@colName,\@dataType

IF \@\@fetch_status<>0
	begin
	--print 'Table ' + $2 + ' not found, processing skipped.'
	close curscol
	deallocate curscol
	return
END

WHILE \@\@FETCH_STATUS=0
BEGIN
IF \@dataType in ('varchar','char','nchar','nvarchar')
BEGIN
        SET \@stringData=\@stringData+''''+'''+isnull('''''+'''''+REPLACE('+\@colName+','''''+''''','''''+'''''+'''''+''''')+'''''+''''',''NULL'')+'',''+'
END
ELSE
if \@dataType in ('text','ntext') --if the datatype is text or something else
BEGIN
        SET \@stringData=\@stringData+'''''''''+isnull(REPLACE(cast('+\@colName+' as varchar(2000)),'''''+''''','''''+'''''+'''''+'''''),'''')+'''''',''+'
END
ELSE
IF \@dataType = 'money' --because money doesn't get converted from varchar implicitly
BEGIN
	SET \@stringData=\@stringData+'''convert(money,''''''+isnull(cast('+\@colName+' as varchar(200)),''0.0000'')+''''''),''+'
END
ELSE
IF \@dataType='datetime'
BEGIN
	SET \@stringData=\@stringData+'''convert(datetime,'+'''+isnull('''''+'''''+convert(varchar(200),'+\@colName+',121)+'''''+''''',''NULL'')+'',121),''+'
END
ELSE
IF \@dataType='image'
BEGIN
	SET \@stringData=\@stringData+'''''''''+isnull(cast(convert(varbinary,'+\@colName+') as varchar(6)),''0'')+'''''',''+'
END
ELSE --presuming the data type is int,bit,numeric,decimal
BEGIN
	--SET \@stringData=\@stringData+'''''''''+isnull(cast('+\@colName+' as varchar(200)),''0'')+'''''',''+'
	--SET \@stringData=\@stringData+'''convert(datetime,'+'''+isnull('''''+'''''+convert(varchar(200),'+\@colName+',121)+'''''+''''',''NULL'')+'',121),''+'
	SET \@stringData=\@stringData+''''+'''+isnull('''''+'''''+convert(varchar(200),'+\@colName+')+'''''+''''',''NULL'')+'',''+'
END

SET \@string=\@string+\@colName+','

FETCH NEXT FROM cursCol INTO \@colName,\@dataType
END
DECLARE \@Query nvarchar(4000)

SET \@query ='SELECT '''+substring(\@string,0,len(\@string)) + ') VALUES(''+ ' + substring(\@stringData,0,len(\@stringData)-2)+'''+'')'' Col1 FROM '+ '$2'
exec sp_executesql \@query

CLOSE cursCol
DEALLOCATE cursCol
SQLEND

# Run query. The return value is a reference to an array.
  my $result = $sqlsrv->sql($stmnt);
# Print results. Each array entry is a hash reference.
  foreach my $row (@$result) {
	print "$$row{Col1}\n";
  }

  if ($HasIdentityColumn == 1) {
	print "SET IDENTITY_INSERT $2 OFF\n";
  }
}
close MYDATA;

Comments

ActiveState Perl 5.10, Windows XP, and DBD-mysql

A few months ago I installed ActivePerl 5.10 on a Windows XP Pro workstation. Next I tried to install DBD::mysql using CPAN, it failed. When I browsed modules via ppm, a GUI in this version of ActiveState Perl on Windows, DBD::mysql was not listed as an available module.

I then downloaded the source code, manually modified the MAKE file, fiddled with Visual Studio NMAKE, compiled it a few times, without success.

Google revealed the existence of Strawberry Perl. So I removed ActiveState Perl, installed Strawberry, ran ppm install DBI, ppm install DBD-mysql. And it worked.

Fast-forward a few weeks, I started playing with EPIC, a plugin for Eclipse that supposedly provides a nice IDE for Perl development and debugging. As a result, I started fiddling with PadWalker, a prerequisites for EPIC. I couldn’t remember all the details now due to frustration, but suffice it to say that I wrestled with EPIC, Eclipse, and Strawberry so much that I thought maybe I should gave ActiveState another look. Plus, I always wanted to verify for myself what the deal is with ActiveState Perl and DBD-mysql.

So I un-installed Strawberry Perl 5.10, reinstalled ActiveState Perl 5.10. And just like last time, DBD-mysql wouldn’t install. After un-installing ActiveState, reinstalling Strawberry, running ppm install DBD-mysql, I am back in business.

So, if you want to do Perl scripting with MySQL on Windows using DBD-mysql, pick Strawberry Perl, not ActiveState.

Comments (8)