Azure Blob Storage Command

Traveling. Getting error message for large parquet files: "Insufficient system resources exist to complete the requested service."

Is this due to ISP bandwidth limitations or laptop memory limitations. My laptop has 128GB RAM so I doubt it's the laptop

Hi Rick,

To clarify, the error is produced by the "Azure Blob Storage command" action when you perform the "Upload file" command, correct?

Yes exactly.

I am able to write parquet files to a local folder. The error message only occurs when I upload the file through the Azure Blob storage command. I haven't had any issues prior to today. I am in a new location. I assume it's upload speed constraints, but just making sure.

A few more questions:

  • What's the approximate size of the Parquet file?
  • How much time passes from the upload start until the error appears (approximately)?

There are approx 20 export files. Only 2 of them are large. One is 2gb and the other is 5.1gb. Not sure about the time of failure. I will run again this am. I am for now assuming it's upload speed. I am in a vacation spot and the internet speed is sluggish.

OK error happened at 37 mins in and only at the upload of the 5.8gb parquet file. I will break it up and append downstream

Update. Breaking it up worked ...

EasyMorph uses the standard Azure SDK under the hood. I'd assume that extended uploads may trigger some internal timeouts in it. Although, it's hard to say for sure.

If splitting the files worked, great!

I also thought about delegating the work to the EasyMorph Server and triggering it remotely. Do the upload to Azure not from your laptop, but from EasyMorph Server that is plugged into a fast stable internet. For instance, make a task on your EasyMorph Server in the office that takes the files from some file storage and uploads them to Azure (given that your office internet is fast). You can trigger the task remotely (e.g. when travelling) in many ways, for instance:

  • Using Web Tasks
  • Using webhooks
  • By sending an email with a specific subject
  • By making the task to periodically check the folder to which the files are uploaded
  • By asking someone in the office to trigger the task manually :slight_smile:

When such a task is set up, all you will need is to upload these files to that storage using any utility that works well over unstable and slow connections, and then trigger the Server task remotely - it will do the rest.

Although, I might be overengineering here.