Using Iterate action with skip action

1

Hi,

I’m using iterate action to export multiple files with my module but each time it loads the source data(1m rows) again from the database and it takes a lot of time for each iteration. One time loading is enough for me so I was trying to use “skip action on condition” action before loading the data so it will skip the data loading for each iteration but apparently not working.

Is there any other way of doing it or am I missing something?

image

image

2

Hi Tahsin and welcome to the Community!

The “Iterate table” action is intended exactly for the cases when each iteration has to deal with a large table. It passes the table to the iterated module right in memory, thus avoiding the need to read the table in each iteration.

3

Thanks I will use that action. There are multiple derived tables from the source data so, how the “iterate table” action inputs the data to that tables?

One more thing, If I want to add one more data source, is it possible to do multiple iterate table?

4

The “Iterate table” passes one table into the iterated module. If you need to pass more tables, there are two options:

  1. Append them into one dataset in the calling module, and then split them back in the called module
  2. Save the datasets into the .dset format using the “Export dataset” action before iteration. Then read the .dset files in the iterated module. The .dset format is native to EasyMorph and therefore EasyMorph reads and writes .dset files very quickly.