Iterate Mailchimp API data

[nbegbaaji]

I would really like to get a handle on iterating API data requests.

I've looked at a few examples here but they don't quite match my use case because they all use oauth, while I'm using API auth.

Here is my use case:

The events team send out a Fundraising Army newsletter every quarter. I want to ask Mailchimp to display the recipients of each of these newsletters. There is a limit of (I think) 1000 records per request but the number of recipients will vary each time.

I would also like an alternative to offsetting the list of campaigns, which currently looks like this:

Please can someone help?

Thanks in advance.

Nike

[dgudkov]

Check out this article:

API Pagination - Next Page

Below is a method to design workflows that query APIs with pagination using the "Repeat" action. This is a simplified and improved version of what I suggested previously. It still has a few non-obvious gotchas, so I tried to cover all of them. The method also benefits additionally from recently added features. The example is using PokéAPI (pokeapi.co) as suggested by @adambeltz. Here is the project. It contains just 7 actions in two modules. Its logic and the design process are explained below…

[nbegbaaji]

Thank you very much for this.

I have come back to this today but I am still struggling with this example.

I can successfully run the Get Page module, alone (via Run Action), pull data through and parse my desired JSON fields but I can't get it to repeat.

The reason could be:

1. because I 'bury' all my code in connectors and parameters which that I can call them from anywhere in my modules. My base URL is in a connector and my URL parameters are often called from stored Easymorph parameters. With this example, I'm needing to unfold and deconstruct my stored Easymorph parameters, to get to the API parts
2. because I'm completely unfamiliar with Pokemon (without needing to take a crash course). In my use case, I am interrogating our digital marketing platform - Mailchimp, so my references are campaign names, campaign date, list name, lsist_id, campaign_subject, email_id, opt_in_date, opt_out etc

Screenshots of my errors:

Do you have any other marketing based examples?

[dgudkov]

Hi Nike,

Sorry for the late reply. We will prepare an example of pulling contacts from Mailchimp and publish it here.

[nbegbaaji]

That's very much appreciated.

Thank you.

[andrew.rybka]

Hi Nike,

Here is the example project:
MailchimpPagingExample.morph (12.6 KB)

To make it work, you must set your API key as a password in the connector settings and specify your campaign ID as a parameter.

The tricky parts of this task are:

• generating a correct offset value for each call
• detecting the first "empty" response

The former is achieved by passing the [offset] column between the executions of the Query API module and increasing that number by the number of requested recipients after each execution. The [offset] column is removed from the final result in the Main module.

The latter is achieved by checking the existence of the sent_to{*} in the "Parse JSON" action results. That column will appear only when the response doesn't have any recipients. There might be better alternatives to this method, but I decided to go with this one to avoid overcomplicating the example.

When the "Skip actions on condition" action detects the presence of the sent_to{*} column, it will return an empty dataset. That will make the "Derive table" action in the "Result" table disable all the actions and return an empty dataset, which will make the "Repeat" action in the main project stop executing the "Query API" module and return the combined result. An empty dataset will be returned if the specified campaign has no recipients.

I hope this helps.