Hi All,
Trying to copy data from DB-A to DB-B. The logic should be -
1. check the max value for a given column at destination
2. if there are rows at the source with a value greater than what we got in the previous step, return a chunk of data and write it to destination
3. repeat until step 2 returns 0 (no more rows to copy).
I thought to do it in a transformation, but I see many articles advising against loops in transformations.
Alternately, I thought to put a loop in a job, but I don't find a way to query source and destination DBs. What to do? TIA.
Trying to copy data from DB-A to DB-B. The logic should be -
1. check the max value for a given column at destination
2. if there are rows at the source with a value greater than what we got in the previous step, return a chunk of data and write it to destination
3. repeat until step 2 returns 0 (no more rows to copy).
I thought to do it in a transformation, but I see many articles advising against loops in transformations.
Alternately, I thought to put a loop in a job, but I don't find a way to query source and destination DBs. What to do? TIA.