Getting data from a web api which limits the number of records per request

Support
1

Hi,

I am calling a web api where there are thousands of records but the service I am calling limits the number of records to download per request to 200. The web service does not give me any method to find out how many records are present as well.

Any ideas on how to handle this?

Thanks

Shaheed

2

There is a not very elegant but working solution described in this topic: How to do recursive loops in EasyMorph. You would have to provide offset (e.g. 0, 200, 400, etc) for each new iteration and use the request in the loop.

In the next release we will be introducing the “Repeat” action for such cases as you described (it’s called paging).

3

OK so when you say paging, will EM be able to automatically figure out how many pages to keep requesting or will we need to define the number of pages?

4

Automatic paging is on our roadmap, but we’re still thinking how to do it right. The problem is that there is no standard for paging - every web-service do it in their own way. The “Repeat” action will simplify dealing with paging because it will be possible to handle paging of any logic with it. So it won’t be automatic, but sufficiently easy to handle.

Eventually we will be adding a few typical patterns for paging to the “Web request” action. However, since there is no standard for paging, they won’t be able to cover all real-life cases.