Hello,
At the moment I am using EM to iterate through a list of documents from a folder. Each of these files must be transformed in bulks and I have already made this work using the iteration function. However, the main issue that I have is that within each iteration I do a Lookup into another table and retrieve some values. I believe each time I run the process the importing of the CSV is ran again (it is a large dataset of 5 million rows). I was wondering if there is a way where I can deem this table as constant through iterations (only running it once). In such way the module that is iterating does not necessarily has to keep repeating the same importing task over and over and hopefully result in less runtime. Initially I was querying from a Database but I decided to use the CSV example
Hello Leopoldo, and welcome to the Community!
In the case you described, instead of the regular “Iterate” action, use the “Iterate table” action. The latter is very similar to “Iterate” and allows passing a table (dataset) into the iterated module. It is intended exactly for the cases such as the one you described - when a large dataset should be used in an iteration. Therefore you can read your lookup table once before iteration and then use the “Iterate table” to run another module and pass the lookup table into the module without rereading it.
The iterated module should have the “Input” action that will receive the table. Note that the output of the action appears empty when designing a workflow. It’s is, however, gets populated with data during run time. To populate it with a sample dataset press “Populate automatically” in the action’s properties.
Hello Dmitry,
I just tested the solution, it was a bit hard at the beginning but then I understood perfectly and the runtime has decreased from around 20 minutes to only 6. Thank you so much!!
You’re welcome 