I have a module that has one Parameter (FilePath) which is receives from the main module via a ITERATE transform.
I am finding that even though I see from the results that it processes two files, I am only getting one file added to the Database.
However, if I do one file at a time, then all works. I am wondering if there is a race condition in my file processor since it reads two sheets from each excel book.
TD0001.xlsx (159.5 KB) TD0002.xlsx (117.5 KB) TestDataUpdate.morph (19.6 KB)
I have attached the project.
There is no race condition in your project.
The iterated module deletes and re-creates database table in every iteration which discards the result of previous iteration. That’s why you always have only 1 file in the database - the last one, no matter how many files you processed.
Any bad factoring or other tips you see wrong with the project that could be improved?
One thing I can think of is removing maintenance tasks (one shot activities like rebuild DB from scratch) from the daily project runs.
Nothing in particular have struck me - everything else looks reasonable.
I typically put one-off operations (maintenance tasks) in a sandbox. Actions in a sandbox are ignored when the sandbox is empty, which is convenient. If I need to perform a maintenance task I just populate the sandbox with data and disable/enable necessary actions in it.
