I still haven't found in all the docs I read if there is some "built-in" DTS
task or function or wathever, to do a mixed "insert/update" import from a
source, giving a unique field as primary key.
I'll try to be more specific. The problem I would like to solve is this:
I have a source file (csv) but it could be any source. I must check this
file for all the records and compare them with the ones in the db (giving a
unique field as a way of checking), so that all the records that already
exist, are UPDATED, and the others are INSERTED.
I guess this is one most common task to accomplish, when you have a local
based application that you regularly update and then you want to export the
data to another "slave" application without using replication. But even if
this sounds to me like a common task, I found no "buil-in" function for that
in DTS. I read something about "lookups" but it don't know if it's related
somehow.. it wasn't very clean.
Thanks in advance for suggestions.
--
:: Massimiliano Mattei
:: Project Leader
:: E.xtranet V.irtual A.pplication
:: www.evagroup.net :: www.commy.bizMassimiliano,
Yes, that is a very common task. The best way to handle this is to
import directly into a temp table and then run a procedure to
check/insert/update your data into a production table.
If you need a specific example to follow, we have one on our website at
www.TechnicalVideos.net. $19.99 buys all of our videos for 3 months. Our
DTS videos will have an expert show you step by step how to do this.
Hope that helps,
Best Regards,
Chuck Conover
www.TechnicalVideos.net
"xMANIGHTx" <manight@.ufologia.net> wrote in message
news:c2aav5$noi$1@.lacerta.tiscalinet.it...
> I'm new to DTS. I read some docs before adventuring into this matter.
> I still haven't found in all the docs I read if there is some "built-in"
DTS
> task or function or wathever, to do a mixed "insert/update" import from a
> source, giving a unique field as primary key.
> I'll try to be more specific. The problem I would like to solve is this:
> I have a source file (csv) but it could be any source. I must check this
> file for all the records and compare them with the ones in the db (giving
a
> unique field as a way of checking), so that all the records that already
> exist, are UPDATED, and the others are INSERTED.
> I guess this is one most common task to accomplish, when you have a local
> based application that you regularly update and then you want to export
the
> data to another "slave" application without using replication. But even if
> this sounds to me like a common task, I found no "buil-in" function for
that
> in DTS. I read something about "lookups" but it don't know if it's related
> somehow.. it wasn't very clean.
> Thanks in advance for suggestions.
> --
> :: Massimiliano Mattei
> :: Project Leader
> :: E.xtranet V.irtual A.pplication
> :: www.evagroup.net :: www.commy.biz|||Thanks Chuck!
Your videos are really a good idea!! I checked the demo one and they are
kool... maybe I'm going to subscribe, even if I'm italian.
I must say that the language is well understandable even if a bit "American"
:)
You say that the best pratice is to import first all teh data in a temp
table (maybe to use the bulk inserta advantages) but what if the records are
really too much? Say I have to import 100.000 records or more.. Would you
use a #temp_table, just a "memory" table or a real table?
I don't know if SQL Server writes down #temp_tables just like it does with
real ones, otherwise the server would have to mantain 100.000 records in
memory...
Wich type of table would you suggest? Maybe SQL Server has memory
optimization routines wich write to disk data of #tables not to hold them
only in the RAM?
Do you think this is the best method? DTS is for data trasformation so it's
ODD it doesn't have something that lets you
import/check/transform/insert-update data without writing a specific
procedure for that.
--
:: Massimiliano Mattei
:: Project Leader
:: E.xtranet V.irtual A.pplication
:: www.evagroup.net :: www.commy.biz|||Massimiliano,
Sorry. I know we are too American. We really need to get out more.
My explanation was not very good. By "temp" table, I mean a table that
is not a production table. We can call this a "working" table. So, I would
use a real table to temporarily store the data for your bulk upload, not an
actual temp table that gets stored in memory. I would do it this way even
if you only have a few records. Then, my DTS package would look like this:
- delete all records from the working table
- upload my data file to my working table
- run stored procedure to check/insert/update data row by row from the
working table to my production table
This way, we can execute each step one at a time, and browse the working
table to make sure the data went into each field correctly.
Hope this helps,
Chuck Conover
www.TechnicalVideos.net
"xMANIGHTx" <manight@.ufologia.net> wrote in message
news:c2f4ku$ccj$1@.lacerta.tiscalinet.it...
> Thanks Chuck!
> Your videos are really a good idea!! I checked the demo one and they are
> kool... maybe I'm going to subscribe, even if I'm italian.
> I must say that the language is well understandable even if a bit
"American"
> :)
> You say that the best pratice is to import first all teh data in a temp
> table (maybe to use the bulk inserta advantages) but what if the records
are
> really too much? Say I have to import 100.000 records or more.. Would you
> use a #temp_table, just a "memory" table or a real table?
> I don't know if SQL Server writes down #temp_tables just like it does with
> real ones, otherwise the server would have to mantain 100.000 records in
> memory...
> Wich type of table would you suggest? Maybe SQL Server has memory
> optimization routines wich write to disk data of #tables not to hold them
> only in the RAM?
> Do you think this is the best method? DTS is for data trasformation so
it's
> ODD it doesn't have something that lets you
> import/check/transform/insert-update data without writing a specific
> procedure for that.
> --
> :: Massimiliano Mattei
> :: Project Leader
> :: E.xtranet V.irtual A.pplication
> :: www.evagroup.net :: www.commy.bizsql
No comments:
Post a Comment