Professional license using the EasyLaucher. The project reads from a webservice and the output is a CSV. The job runs on a schedule of every 15 minutes. The process works great.
I have noticed that the file is locked in that it can not be deleted until the next cycle.
The workflow has an issue in that once the downstream process ingests the csv, it needs to move the file to an archive folder. It processes the backup files for the process is set to create new file.
Is there a workaround this issue. Looking to output to csv and then close the file for deletion/moving.
Most likely you're writing your output files to a Dropbox-synced directory and Dropbox locks the files. Consider writing your output files to a regular folder to avoid potential issues with Dropbox-synced directorie, to prevent any other processes from interfering with subsequent deletion.
Can you please test the export with a folder that's not synchronized with Dropbox or a similar service and is located on a local disk? Will it lock the file?
@jaatsolutions, I tested the "Export to into delimited text file" action. And I was able to delete the created file right away. Both manually and with the "File command" action.
It seems that something else is locking the file on your side.
Not sure what it can be, hence my ask. I will keep digging. The only thing in that folder is the easymorph output. We have a job that watches the folder and when it sees it then processes it. It will NOT process open or locked files.
I ran a test with robocopy to see how it responded. It too is unable to move the file until easylauncher writes the next file, which then "releases" the first file to create the second file which means I am ONE cycle out. release is just a term for the process of easymorph project "renaming" the original one and then creating a new one for output.
It is consistent and repeatable. If I shut down easy launcher the file is processed.
It was odd, so I thought I would just ask.
Thank you for looking into it.
I have built a workaround using robocopy and every minute it moves the backup files to the main folder for processing and leaves the main file there for processing.
Not pretty but it works.
I am enjoying the platform/toolset and cutting my teeth on it. Still so much to learn.
If I understand correctly, you've set up an EasyMorph Launcher that executes a scheduled task. This task periodically retrieves data from a particular service, processes it, and exports the results as a CSV file to a Dropbox folder. Subsequently, another, external non-EasyMorph process monitors changes in that folder. Upon detection of a new file, this process attempts to transfer it to another location. The issue seems to arise when this second process fails to delete the original file generated by the Launcher.
In theory, this setup could function smoothly if the second process initiates its task after the Launcher has finished processing the file. However, it also introduces scope for potential errors. For instance, EasyMorph typically employs a temporary file for exports: it writes data to a {random-name}.temp file before renaming it to the final version upon completion. In this case, your directory watchdog might be triggered prematurely by the intermediate .temp file.
Additionally, you mentioned that 'It will NOT process open or locked files.' However, determining the presence of a Read, Write, or Delete lock accurately might not be possible all the time. It is then likely that the second process attempts to manipulate the file while the Launcher maintains a Delete lock, even though there may not be Read or Write locks.
To fix your workflow and circumvent this synchronization issue altogether, would you consider modifying it so that the Launcher and not that second process assumes responsibility for moving the file to the target folder?
You are absolutely correct in how the process should work. Again, dropbox is just what they call the windows folder. I completely concur the secondary process which has been in play for many years monitors many directories for many processes. EasyMorph is the new player in town and is slowly migrating the processes it can.
So for now, I will use the work around till the larger work is done, then I will sweep back to have the stored procedure work that ingests that CSV can be acheived via easymorph.
The process is to get a series of rows from the side source. In this case GPS update records. It then runs each row through a stored procedure that has lots of business rules against it and manages several tables.
The next phase for me will be to have easymorph send the rows to an actual database table and then run the stored procedure against that.
Crawl, walk, run. For now the customer is getting their data and their dispatchers have visibility to this new source of GPS records.
The easymorph platform does a lot, I just have to bring it into the fold slowly as to to NOT break processes already in play.
Thanks for your excellent assessment. For that is what I believed was happening.
Can easymorph actually move files from one folder to another?
Just found the file operations. Nut behind the wheel. This is the solution. Thank you. I will have easymory build the file is an alternate folder, then add the action to move the file. I will also adjust the output function to create the file if not there.
It is now working perfectly in easymorph and easylauncher due to your insight.
step 1 : move file from working area to live area in the event we have some left over. Should never see a file
step 2: fetch data from outside source
Step3: keep the columns you want
Step4: rename the columns for downstream process
Step5: create output
Step6: move file action identical to first step.
