we actually use easymorph to put data from different sources into a PostgreSQL database for reporting reasons.
We have flat files, different databases and a RabbitMQ bus as sources. We pick up the data from RabbitMQ with a windows service, which gets the messages immediately and saves them in a file directory. Our easymorph job is scheduled every 5 mintes to get the files and process them into the target database. So far, everything works fine.
Now, some of our colleages would like to process the data into the target database immediately. The question now is, whether it is planned (or even possible now) to have a listener on the server that starts processing as soon as a new file arrives instead of a periodically running job.
the use case is clear and makes sense. At this point, EasyMorph Server is not ready to trigger tasks by events, because currently a task can’t be triggered multiple times in parallel. If a task has started, then attempting to start it one more time will fail.
We’re planning to modify Server to allow a task be triggered as many times as needed regardless if a previous task launch has finished or not. We call it multi-session tasks.
Once Server moves to multi-session tasks, more advanced use cases, such as event-based task triggering will become possible.
@Jochen_Marquardt
Currently EasyMorph Server doesn’t support event-based task triggering
But you could try some workarounds:
ems-cmd utility allows to trigger any task and upload/download files via EasyMorph Server API. This utility comes together with EasyMorph Server. More details could be found here.
If the task is already running, ems-cmd will fail with non-zero exit code.
Another approach is to decrease scheduled task interval.
We use the following workaround: create a dropzone folder which may contain only 1 incoming file. We schedule the task at the lowest possible interval on EM-server. In EasyMorph we check if the folder is empty. If it’s not empty, we start processing the file, else we skip the other actions.
We would love to have the triggers automatically built into EasyMorph since it is a use case that often reappears.
Is this still on the roadmap ? When could we expect to have this feature as standard ?
Another feature that we would love to have is a GANTT chart on EM-server to draw out the order in which tasks have to be executed. Now we use an excel worksheet which contains the names of the morph files in the correct order and then we trigger the corresponding tasks on EM-server using ems-cmd but that solution is a bit cumbersome. If we could have that GANTT chart on EM-server, we could easily see which tasks could run in parallel and which could not.
Any plans to incorporate such a feature ? Or any suggestions to deal with a more complex scheduling in which the order of execution of tasks matters (datawarehouse environment) ?
