Repeat action for API - passing max date per batch

[mattf]

Hi Alistair

You're logic is pretty much correct in that you need to use the Repeat action. The steps needed would be as follows:

You need a main module which has 2 actions:

1. Firstly, create a list with the starting date. The field should be called whatever the field is called in the API and for which you want to get the max each time so you can use it in the API request.
   
   
2. Followed by the repeat action to repeatedly call another module (API Call). Set it to repeat until the returned result is empty and to append and return all results.
   
   

Then in another module which the repeat action calls (API Call) you need to do as follows:

1. Firstly use the Input action to receive the table of data from the current iteration of the repeat action. Note that on the first call you will receive the starting date you set in the list on the main module. For all subsequent calls, it will receive the full set of data from the initial list plus the previous iterations.
   
   
2. Use the Aggregate action to get the max date in the field. Don't group by any fields so we just get a single total maximum
   
   
3. Make the web request using the web request action, setting the parameter or header field as needed by the API to "First value of column" and selecting the data column.
   
   
4. Do whatever you need to do to parse the result (e.g. if its JSON then parse the JSON as needed).
5. If no data was returned then you need to have a way of detecting that and returning an empty table. In the below, because the API I used returns an empty body if no data is available from the date passed, the Parse JSON action causes errors. So I use the Filter by type action set to filter out errors which results in a table with no rows. What you'll need to do specifically will depend on what the API you are using passes back if no records are available.
   
   

The other actions in this example are just handling different data types (dates as number or dates as text) and so might not be needed for your API.

Hope that makes sense! If not then shout and we can have a call to walk through it.

This post has a working example which is very similar, it just misses out the max date part. API Pagination - Next Page

Regards
Matt