Could anyone explain me how to work with EasyMorph server ?
If you develop data flows on a local machine in EasyMorph desktop and you want to run this on the server wich is installed on another machine, will it run automatically or do you need an exact copy of the ETL-flow ?
If you have multiple users, won’t you run into problems with incompatible folder structures between local machines and the server ?
Is it better to just have a windows server with EasyMorph desktop and EasyMorph server installed and then with multiple users work on that machine to avoid conflicts in folder structures?
It is generally recommended to have the same folder structure for data files on developer desktops and the Server. In this case, all file locations can be calculated from a parameter pointing to a root folder. When a project is published to Server, task parameters reassign the parameter with a root folder on the Server.
While it is possible to do development remotely on the Server using EasyMorph Desktop installed on the Server, it’s not necessary and can be inconvenient. You can design projects on local machines, then publish projects on the Server. When data folder structures are identical, and file locations are calculated from a root folder, publishing and updating projects is simple and requires minimal effort.
**** moved to #server category
Hi,
I have one additional question:
Can I publish to the EasyMorph Server only via EasyMorph Desktop professional or is EasyMorph Desktop free sufficient? (The actions and loops restriction for the Desktop free aside).
KR
Markus
Publishing is just uploading files to the Server. You can publish to EasyMorph Server using the free edition of EasyMorph Desktop as well, or just by dragging and dropping files to Servers’s file manager (tab Files).
1 Like
Hi @dgudkov,
Hope you are well.
Until recently, I was using a separate development machine and a separate server. However, I did find it highly inconvenient to have separate machines because of the following issues:
- I did not have the same folder structure on Server and Dev
- Sometimes projects were failing and it was more difficult to troubleshoot because I only had the free version of EM Client on my server.
I have now made the decision to merge the server and client machines. Since I am the only developer at our company it is manageable for now.
Having said that, if you have any tips for how to structure things properly in this scenario, I would be grateful to receive them. So far I have the following ideas:
- Inside main EM folder, have a separate folder for each space.
- Within each space have a separate folder per project even if each project folder only contains one file.
- Rather than using Git, just do a version number for each file.
I am unsure on what else to do. Here are some other ideas I was thinking of:
a. Separate folder for each module
b. Separate folder for data import sources
c. Separate folder for data export sources
Any thoughts?
Thanks
Shaheed
The question is a bit vague, although I will try to suggest some ideas. Maybe, some of them will be helpful.
That's typically a good idea. Depending on how your users (if any) access Server you may want to use a root folder for the admin space, and subfolder(s) for the user space(s). This will make kind of nested spaces with the "outer" space having more privileges and wider access then the "inner" space(s).
This can be cumbersome to manage. I typically group projects in folders by purpose, function, source, or target.
The folder structure in a space could be as follows:
/Root public folder
./Source data
./Source A
./Source B
./Output data
./Target A
./Target B
./Projects
./Folder A
./Folder B
etc.
The key idea is to use relative paths wherever possible. The folder structure described above makes it possible. It's also convenient for backup/restore.
That's one way of doing version control. Note that modules in projects can also be used for version control. For instance, before modifying a module, you can clone it and keep the original version in the same project. If something doesn't work right in the edited version of module, you can easily switch to using the older version. A project may contain modules that are not used and only serve for version control, that's OK.
Git for version control is also a good idea. Since EM projects are regular XML files, it's relatively easy to do a diff between two versions and see the changes.
Other suggestions:
- Use Server-hosted repositories when possible - it's more robust and secure
- Keep the same folder structure in DEV and PROD. This works well when you widely use relative paths in projects
- If you use different spaces for DEV and PROD (which is always a good idea) use the same connector names in DEV and PROD repositories, however make them point to DEV and PROD instances of databases and systems. For instance, you can have a connector named "main database" in both DEV and PROD spaces/repositories. However in DEV space/repository the connector would point to a DEV database, while in the PROD space/repository it would point to a PROD database. In this case, promoting a project from DEV to PROD won't require any changes. Especially, if it also uses relative paths.
1 Like
Thanks for the reply @dgudkov. Apologies for being vague. I appreciate your advice. I am having difficulty following the rationale for your folder structure:
/Root public folder
./Source data
./Source A
./Source B
./Output data
./Target A
./Target B
./Projects
./Folder A
./Folder B
etc.
The structure you propose has a separate folder for each project when you said that it would be cumbersome to manage a separate folder for each one. Kindly elaborate.
Regarding having the parent folders for source and output, is your rationale that different projects within a space may need to access the same data?
In this case Folder A and Folder B are not project names but rather names of source or target systems, or functional areas. Each folder would contain multiple projects that have a similar function or purpose, or source/target system.
Yes, the folder structure assumes many-to-many relationship between sources and projects and targets, i.e. various projects may access various sources.
Also targets may be sources for other projects. In this case, you may want to introduce another subfolder Intermedate data on the same level as folder Source data and Output data.
Thanks @dgudkov. These suggestions were really valuable. I add a couple of other folders based on my experience. So far in testing out the new approach it works. Here’s what I ended up with:
1 Like