From time to time, now more often, we have this error when we try to read Google Worksheets through Google Drive connector: Exception of type 'Morph.DataDrivers.GoogleApi.GoogleRequestException' was thrown.
After a small debug this is happening if exists transformations that load data from Google Worksheets that are running in parallels. It is not a issue about the file that we want to load because the error are manifest are to several files.
The project are old and are running without any error for long time ago. We use EM Desktop 5.8.0.13.
The temporary solution was to synchronize loading data from Google Worksheets transformation (put in serial) but it is not a practical solution when we have dozens of loading transformation.
Does this error happen when you import the same file or different files in parallel?
Please:
post a screenshot of the error
click the right mouse button on the error message, choose “Copy diagnostics information to Clipboard” and post the copied text here or send it to support@easymorph.com
Getting tons of this error in the past week too. It doesn't fail everytime. It appears some sort of rate limit possibly google introduced? Could we get a delay in the action or a try retry functionality?
It seems that Google has started returning the 502 Bad Gateway error in case of a large number of requests. We are limiting the number of requests to the Google Sheets API within a single project according to the official API limits. However, these limits may be exceeded when multiple projects run in parallel. Typically, when the limits are exceeded, the API returns the 429 error, and we have a try-retry algorithm that is supposed to help in such cases. However, it is not designed to handle the 502 Bad Gateway error.
We have improved the handling of Google API error messages, and now the message should mention the Bad Gateway error explicitly.
We recommend not running actions that work with Google Sheets in parallel as a workaround for this error. Within a single project, this can be done using the "Synchronize" action. If that won't help, consider adding the "Wait" between Google Sheets actions. If multiple projects are scheduled to run on EasyMorph Server, it's better to stagger their start times by 5-10 minutes.
@Justin_Grewe, how many projects that work with Google Sheets do you have? Do they run in parallel?
Do those projects work with the same Google Drive connector?
Same Google Drive connector. There are a number of actions that import/export in a single morph that aren't currently synchronized. I guess I will start that work. There is a lot of them we have. So it will be some work. These all were working fine for years. Maybe a few of these run at the same time on the server. I am not sure on that. I wish the import/export Google Sheet action or just the connector could have logic built in - that would save me immense time. Something like a try catch or a sleep random milliseconds that Google recommends.
We've published an updated version with improved handling of Google API errors. That should resolve the problem with Google Drive/Sheets. Please update EasyMorph to the most recent version from our website.