I made a generated list of serial numbers (900,000 of them). It starts from 100000 to 999999.
And I have another list of items that will be assigned these new serial numbers. About 70 of them.
I need to save a file containing the “Generated ID” and the “Supplier ID”.
Sounds simple enough, but the thing is, when I iterate item by item in my module and save the newly assigned item ID and serial number, it goes slow. These 70 items took 9 minutes if I recall.
Saving a CSV file every time I add a new item is not the way. I have over 35,000 items by several suppliers that need a generated serial number assigned. With checks that items do not repeat themselves in the list. They need to fill up for every suppliers items I add.
Is there a way I could assign a bulk of ID’s to that generated list and then save, instead of saving per row.
It’s hard to say what takes long in your workflow without seeing or understanding your assignment logic. I assume you’re trying to assign a general ID to products of different suppliers with their own IDs. For that, you generated a lookup table that forms the relation between Generated ID and Supplier ID.
Technically, once you save this table in a .dset file with the “Export dataset” action, it can be quickly loaded inside iterations.
Also, consider joining data using the “Merge” action whenever possible, instead iterating row by row.
You definitely don’t need iterations in this case. There are many ways how to accomplish the task. For instance, you can use filters to keep only rows without supplier ID, then use the “Append table” in the “Append columns” mode.
Or, you can reverse the logic. Instead of using a pre-generated sequence of generated IDs, you can build a list of all supplier IDs without a generated ID and then simply generate IDs for them with the “Enumerate rows” action. Add a number that is the previous max generated ID (i.e. the biggest ID from the previously generated).
Last night I came to the same conclusion. I started from the wrong end. As I just need to increment the Generated ID and have to check if Supplier ID is not in the list. Thanks.