Error on "Iterate Web Requests" when script runs on server but not when using Desktop

Alex · September 9, 2022, 8:12am

Dear,

We are experiencing an issue with the Iterate Webrequest function.

The easymorph project works when used on the Desktop application.

However it doesn’t work when it runs on the server. We tested using the endpoint via PostMan and via Browser on the server and it works.

We added a log on the project to get the error, here it is :

09/09/2022 09:00:20 AM 000106            ERROR     Source: action "Iterate Web request", module "Send SMS", table "Make Requests to Message API"
09/09/2022 09:00:20 AM 000107            ERROR     
09/09/2022 09:00:20 AM 000108            ERROR     Exception: 
09/09/2022 09:00:20 AM 000109            ERROR     HttpRequestException
09/09/2022 09:00:20 AM 000110            ERROR     An error occurred while sending the request.
09/09/2022 09:00:20 AM 000111            ERROR     Call stack:
09/09/2022 09:00:20 AM 000112            ERROR        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
09/09/2022 09:00:20 AM 000113            ERROR        at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
09/09/2022 09:00:20 AM 000114            ERROR        at Morph.DataDrivers.RestApi.ConfigurableHttpHandler.
________________________________________
.MoveNext()
09/09/2022 09:00:20 AM 000115            ERROR     --- End of stack trace from previous location where exception was thrown ---
09/09/2022 09:00:20 AM 000116            ERROR        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
09/09/2022 09:00:20 AM 000117            ERROR        at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
09/09/2022 09:00:20 AM 000118            ERROR        at Morph.DataDrivers.WebRequests.WebRequestDriver..MoveNext()
09/09/2022 09:00:20 AM 000119            ERROR     --- End of stack trace from previous location where exception was thrown ---
09/09/2022 09:00:20 AM 000120            ERROR        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
09/09/2022 09:00:20 AM 000121            ERROR        at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
09/09/2022 09:00:20 AM 000122            ERROR        at Morph.DataDrivers.WebRequests.WebRequestDriver..MoveNext()
09/09/2022 09:00:20 AM 000123            ERROR     --- End of stack trace from previous location where exception was thrown ---
09/09/2022 09:00:20 AM 000124            ERROR        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
09/09/2022 09:00:20 AM 000125            ERROR        at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
09/09/2022 09:00:20 AM 000126            ERROR        at Morph.Engine.WebRequest.Iterate@61-6.Invoke(Unit [1])
09/09/2022 09:00:20 AM 000127            ERROR        at Ply.TplPrimitives.ContinuationStateMachine`1.System-Runtime-CompilerServices-IAsyncStateMachine-MoveNext()
09/09/2022 09:00:20 AM 000128            ERROR     --- End of stack trace from previous location where exception was thrown ---
09/09/2022 09:00:20 AM 000129            ERROR        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
09/09/2022 09:00:20 AM 000130            ERROR        at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
09/09/2022 09:00:20 AM 000131            ERROR        at Morph.DataServices.DatasetModule.appendManyAsync@385-1.Invoke(Unit [1])
09/09/2022 09:00:20 AM 000132            ERROR        at Ply.TplPrimitives.ContinuationStateMachine`1.System-Runtime-CompilerServices-IAsyncStateMachine-MoveNext()
09/09/2022 09:00:20 AM 000133            ERROR     --- End of stack trace from previous location where exception was thrown ---
09/09/2022 09:00:20 AM 000134            ERROR        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
09/09/2022 09:00:20 AM 000135            ERROR        at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
09/09/2022 09:00:20 AM 000136            ERROR        at Morph.Engine.WebRequest.Iterate@53-7.Invoke(Unit [1])
09/09/2022 09:00:20 AM 000137            ERROR        at Ply.TplPrimitives.AwaitableContinuation`3.Invoke(Unit r)
09/09/2022 09:00:20 AM 000138            ERROR     --- End of stack trace from previous location where exception was thrown ---
09/09/2022 09:00:20 AM 000139            ERROR        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
09/09/2022 09:00:20 AM 000140            ERROR        at Ply.TplPrimitives.tryFinally@257.Invoke(AwaitableContinuation`3 this)
09/09/2022 09:00:20 AM 000141            ERROR        at Ply.TplPrimitives.ContinuationStateMachine`1.System-Runtime-CompilerServices-IAsyncStateMachine-MoveNext()
09/09/2022 09:00:20 AM 000142            ERROR     --- End of stack trace from previous location where exception was thrown ---
09/09/2022 09:00:20 AM 000143            ERROR        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
09/09/2022 09:00:20 AM 000144            ERROR        at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
09/09/2022 09:00:20 AM 000145            ERROR        at <StartupCode$Morph-Engine>.$Engine.clo@236-44.Invoke(Unit [1])
09/09/2022 09:00:20 AM 000146            ERROR        at Ply.TplPrimitives.ContinuationStateMachine`1.System-Runtime-CompilerServices-IAsyncStateMachine-MoveNext()
09/09/2022 09:00:20 AM 000147            ERROR     --- End of stack trace from previous location where exception was thrown ---
09/09/2022 09:00:20 AM 000148            ERROR        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
09/09/2022 09:00:20 AM 000149            ERROR        at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
09/09/2022 09:00:20 AM 000150            ERROR        at <StartupCode$Morph-Engine>.$Engine.clo@494-52.Invoke(Unit [1])
09/09/2022 09:00:20 AM 000151            ERROR        at Ply.TplPrimitives.AwaitableContinuation`3.Invoke(Unit r)
09/09/2022 09:00:20 AM 000152            ERROR     --- End of stack trace from previous location where exception was thrown ---
09/09/2022 09:00:20 AM 000153            ERROR        at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
09/09/2022 09:00:20 AM 000154            ERROR        at Ply.TplPrimitives.tryFinally@257.Invoke(AwaitableContinuation`3 this)
09/09/2022 09:00:20 AM 000155            ERROR        at Ply.TplPrimitives.AwaitableContinuation`3.Invoke(Unit r)
09/09/2022 09:00:20 AM 000156            ERROR     Inner exception:
09/09/2022 09:00:20 AM 000157            ERROR     WebException
09/09/2022 09:00:20 AM 000158            ERROR     The underlying connection was closed: An unexpected error occurred on a send.
09/09/2022 09:00:20 AM 000159            ERROR     Call stack:
09/09/2022 09:00:20 AM 000160            ERROR        at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, TransportContext& context)
09/09/2022 09:00:20 AM 000161            ERROR        at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar)
09/09/2022 09:00:20 AM 000162            ERROR     Inner exception:
09/09/2022 09:00:20 AM 000163            ERROR     PSInvalidOperationException

Do you know what could be the cause of this issue ? Thank you for your help.

ckononenko · September 9, 2022, 10:51am

Hi @Alex
Possibly, it could be a TLS error handshake. EasyMorph uses Windows ciphers and TLS configuration for HTTPS connections. So it entirely depends on OS features. And if your API server uses a more modern cipher than OS Windows supports, the software will throw an error.

Another possibility is the API server throttling. Too many requests per second cause the API server to break the connection. Or the request body is too large or wrong formatted (e.g., missing headers, URL parameters, etc.).

Which OS version do you use for the EasyMorph Server and the EasyMorph Desktop?
Do you have the same issues using an IterateWebRequest when a table contains only a single row (single request)?
Do you have the same issues using an IterateWebRequest with another API server provider (e.g., does GET request to google com work)?

Alex · September 9, 2022, 2:37pm

Hello @ckononenko , we will make those different tests and we will let you know.
We might have found the source of the issue but have to perform another test to be sure. I’ll update the ticket here with the solution if the source is what we thought.

Alex · September 12, 2022, 7:56am

Hello,

This was linked to Powershell error: There is no Runspace available to run scripts in this thread. – AgilePoint NX Unleashed

We had another project using that specific line before and it wasn’t reset. We added the reset and it works now.

ckononenko · September 12, 2022, 1:19pm

Hi, thanks for sharing this info.
Could you please tell us more about why you need to set ServerCertificateValidationCallback via PowerShell?

Alex · September 12, 2022, 1:33pm

We needed to upload a file on the FTP of a customer who is using FTPS (not SFTP). Therefore SSH commands were not possible and we had to rely on Powershell to upload the file.

As our customer certificate didn’t match hostname, it failed the verification, therefore we used it to force the validation since we trust the destination.