Calling API in parallel tracks

Hi,

We have to match a lot of addresses. It is possible to call that API in parallel in EasyMorph?

How can we split our dataset in EasyMorph (in memory, not to file) and then call an API for the different tracks (= datasets) and merge all that data back in 1 table ?

One option I can think of would be:

  1. Create N derived tables to split the dataset.
  2. Call a module that iterates over a dataset and calls the API (module “iterateAPI”
  3. Merge the output of all N-derived tables together using append actions.
  • I do not know if calling a module at the same time from different tables is possible in EasyMorph? Is it really performed in parallel in the background ?

  • Maybe this could be feature that could be implemented at some point?

Thanks !
Nikolaas

Yes, it's possible. A module is immutable and can be called concurrently by any number of other modules.

You may also want to check if the API you're accessing can accept batches of addresses for verification.

Hi Dmitry,

Thanks.

So the workflow I described above is the one to follow currently if we want to doe it in parallel ?

I have no experience with API’s that accept batches. How does that work ? Do you send a number of rows at once to the API ? How should that be implemented in EasyMorph is this was possible for this specific API ?

Thanks !
Nikolaas

Yes.

There is no standard on that. EasyMorph queries would depend on the API specification, but on average they would include multiple addresses per request.