Marshall's Weblog

Embiggen Your Mind

Marshall's Weblog header image 2

Unix Awk Saved My Life!

November 15th, 2004 · No Comments

Well not really… So I was presented with a very tedious task today. Convert a bunch of database schema files into an xml format which I created. The XML file will then be used to help read in the data from a fixed length file into a C# app which will then import certain columns and records into a SQL database for further processing. The schema file looks something like this:

...
0Q  ARPP019B-STAFF-ID VAL Q(06).
...

I do not know what the first column stands for and I do not really care. The only columns I need are the second one which describes the column namd and part of the forth column which gives me the fixed length of the column. Now I could have written a parser for this format myself, but I am lazy. Plus, unix has the awk command. I fired up my favorite Unix-On-Windows shell, Cygwin and typed in the following line:

 awk '{ print $2 " " substr($4,3,2) }' schema.txt

and I got a nice list of column names and lengths as an output. Now what I really want is to output the XML nodes directly from the schema file. I can now do something like this:

 awk '{ print "<field len="\" substr($4,3,2) "\">" $2 "</field>"  }' comp.txt

and I will get the output

...
<field len="06">ARPP019B-STAFF-ID </field>
...

This will save me a lot of time writing these XML files as some of the schema files have upwards of 100 columns in them.

Tags: Programming