API Pagination - Next Page

[adambeltz]

Hi - I'm still having a tough time with this concept (Using sample pokemon data)

We make the initial request:

then on the iteration I can never seem to get it to populate back for the repeat.
Pokemon - Sample API Requests.morph (8.2 KB)

Thoughts on what I'm doing wrong?

[Twitch]

Hi Adam,

You should definitely override my reply with anything that Dmitry happens to send your way on this. But I had a API pagination item somewhat similar to this recently.

I added an action to your table that contains the loop to append the first 20 results at the end. You can definitely do other cleanup.. and I don't really know the content of this. I removed your action to write out to the csv file etc. I am just concentrating on attempting to get all the records back.

The append will bring all the records from your initial table back to the repeat table without interrupting the loop. I saw you were writing to a file in the "Get Data" module so you may not need to do this.

I think your main issue is the "Results" table. It is set to always run. And you should have it only run on condition so that the table being empty or a column being empty can signal to the loop action from the main module to exit. Take a look at my adjustment to the derive table action in image 3.

Image 2

Image 3

This configuration when not always set to run, will only run if you have at least one row with a value in it. If the upstream table is empty it skips, and this signals the loop to stop.

Here is what the first iteration through looks like (image 4)

Image 4

Here are the next 20 records coming in (image 5)

Use the export to sandbox option on the last action of your table to see the next loop iteration (image 6).

Here is the next iteration (image 7).

Here is the last iteration and there are no more values coming back. There is probably a way to handle this but I don't have the time at the moment to dig deeper (image 8).

Now if we try one more iteration, the "Input" table is empty and the "Results" table skips all the actions signaling the loop to stop in your main module (image 9).

And now looking back at your main module, all your records are there (image 10).

[adambeltz]

Hey @Twitch - Thanks for the reply. This gave me some clarity. "most" of the API's I deal with give me total count and from that I've derived the process without to repeat action.

I've cleaned it up and attaching a final example should anyone else need to see how this works as I'm believe Dmitry's example public API has been shut down

Pokemon - Sample API RequestsV2.morph (13.5 KB)