Wednesday, March 7, 2012

DTS - import flat textfile into two separate tables

I have a report that's created each day as a flat textfile.

Because I came from the Access world, I created a macro that imports
it with a schema that gives meaningful names to the various columns,
and then uses a query to massage some of the data for me (deletes the
first blank row and does a couple of calculations)

Then I use DTS to import the Access query as a table.

the textfile has a column called "File_num", and among several others,
a column called "Serial_num". (the file numbers represent shipments,
and sometimes there are more than one serial number in the shipment,
etc., so there is a separate line for every serial number)

Naturally, I would like to split this info into two tables..one that
does not contain the serial numbers and has a primary key on the
"File_num" column, and another table that would contain just the
"File_num" and "Serial_num" columns. That way I could relate them
later...but most importantly, it will give me a table where I can use
the "File_num" as my primary key.

What would be the best way to import these two tables from one source
textfile? The other thing that gives me problems is that the text
file has no column names, and the first row is always blank.

I'm very new to SQL and DTS and would appreciate any direction.

Thanks,

Larry

- - - - - - - - - - - - - - - - - -
"Forget it, Jake. It's Chinatown.""Larry Rekow" <larry@.netgeexdotcom> wrote in message
news:3h1cj0976dtpkliussk5c1nr7jmubvt4rp@.4ax.com...
>I have a report that's created each day as a flat textfile.
> Because I came from the Access world, I created a macro that imports
> it with a schema that gives meaningful names to the various columns,
> and then uses a query to massage some of the data for me (deletes the
> first blank row and does a couple of calculations)
> Then I use DTS to import the Access query as a table.
> the textfile has a column called "File_num", and among several others,
> a column called "Serial_num". (the file numbers represent shipments,
> and sometimes there are more than one serial number in the shipment,
> etc., so there is a separate line for every serial number)
> Naturally, I would like to split this info into two tables..one that
> does not contain the serial numbers and has a primary key on the
> "File_num" column, and another table that would contain just the
> "File_num" and "Serial_num" columns. That way I could relate them
> later...but most importantly, it will give me a table where I can use
> the "File_num" as my primary key.
> What would be the best way to import these two tables from one source
> textfile? The other thing that gives me problems is that the text
> file has no column names, and the first row is always blank.
> I'm very new to SQL and DTS and would appreciate any direction.
> Thanks,
> Larry
> - - - - - - - - - - - - - - - - - -
> "Forget it, Jake. It's Chinatown."

A common technique is to import the source file directly into a staging
table, then transform the data using SQL. This is often easier than trying
to implement complex transformations in DTS itself. In your case, you could
do something like this:

insert into dbo.FileTable
(file_num, col1, col2, ...)
select file_num, col1, col2, ...
from dbo.StagingTable

insert into dbo.FileSerialTable
(file_num, serial_num)
select file_num, serial_num
from dbo.StagingTable

Simon

No comments:

Post a Comment