XML project and Locked File

Hi. This may be a bit long winded so I will try to be a succinct as I can. Up until ‘recently’ this process as worked fine.

Senario

Server Version: 1.2.0.12

PARENT PROJECT

  1. Retrieve list of files from a PENDING folder that have an extensio nof .xml
  2. Iterate over the list and call a sub-project

CHILD PROJECT
3. Extract the required data from the XML file and derive a new table that is passed back to the parent project
4. Return the derived table to PARENT project

PARENT PROJECT
5. TRUNCATE SQL Server Staging Table
6. Push the data to staging table
7. Execute Stored Procedure to production table populating foreign keys etc
8. Call a Windows batch script to MOVE the original .xml file to a PROCESSED Folder
9. REPEAT for each File in Parent Table

We have exactly the same process in different projects based on file types (CSV, XLS etc)

All Good so far…

Today, we find that the XML processing fails at step 8 and the reports that the file cannot be MOVED as it is in use. After some forensics I have discovered that it is EasyMorph Server that has it still open and the Open Files console on Windows shows the same file opened and never released every time we re-run the scheduled server project.

I have included a screen shot showing this.

It seems to me that the XML read process is not releasing the file handle when it returns from the sub-project at step 4. This creates a problem later for the windows batch command to MOVE the file as it considers the file to be locked and a copy/rename and delete (aka a MOVE) is not possible.

As I have said we have different projects to handle CSV and XLS files from different trading partners but the flow of each is identical. It is just the sub-project that changes and in this case we do some pivoting to extract the data we need out of the XML and then push that back to the parent project for a downstream push to the database.

Any ideas?

It is difficult for us to provide a sample as we are dealing with sensitive data.

thanks
Bruce

Bruce,

we will try to deduct what could keep the file handlers from releasing. Meanwhile, can you please check if the files are released when the service is restarted?

Also, have you installed recently any major Windows updates?

Sorry. Forgot to mention that there have been no recent major Windows Upgrades. Current version is Windows Server 2012 R2 Version 6.3 (Build 9600). The .Net Framework appears to be v4.6.01055

Seems, one of my replies has gone AWOL.

I can confirm

  • that forcing the execution of the project via the server UI results in the file in question being added to the ‘Open File Shares’ on the Open Files/Computer Management panel.

  • that shutting down the EM Server Service releases the files

  • It appears to be limited to the processing of XML files as we have the same process flow for XLS and CSV files from different partners

Thank you for the details, Bruce. We’re investigating it.

Hi Bruce,

We were able to reproduce this issue. And we have found a possible source of it.

We are going to run some tests and then we will send you an updated version of the Sever.

Thanks,

Andrew

Thanks Andrew. Really appreciate the quick turn around. Lokk forward to the update.
Cheers

@bruce.gibbins the bug has been fixed. The download link has been emailed to you.

Thanks Dmitry. Will give it a go asap