This is very useful as at the moment I have to filter the data, export to an excel file, edit the file, and then re-import and update, so this would be massively useful.
It would be useful to have a filter and/or search option to quickly find particular data to edit - I typically have millions of rows but just need to edit a dozen or so cells!
Can there be a way where if I'm looking at the data and it is in a derived table (which could be many derived tables down the process) that I can right click or similar to edit that cell and it takes me to the source, or allows me to edit in situ and updates the source? I often see errors towards the end of the process is the point I'm trying to make!
This is really interesting @dgudkov. I have been pondering on dsets, the API gateway and now this feature. Does this mean one could use EasyMorph as a backend for a fully fledged application? I need to make an app internally and was thinking of using FlutterFlow. FlutterFlow integrates with Firebase and Supabase but I am wondering whether we can get away without them. What are the pros and cons of using EM?
We're making two improvements to enable the use of EasyMorph as a simple data store for external apps:
The Dataset Editor described above.
The "Append" mode in the "Export to dataset" action.
With these two improvements, EasyMorph gets "database-like" functionality that can be used in simple low-workload, low data volume scenarios.
Easy to use, built-in data editor, suitable for non-technical users
Native data format for EasyMorph
No need for an external database
Simple backup/restore since it's just .dset files
Not suitable for high intensity workfloads. E.g. apps that that perform multiple reads/writes per second
Not suitable for high data volumes. Unlike in relational databases, the time required for appending to a .dset file grows proportionally to the file size (for databases it's constant). There are techniques to get around it (e.g. append to a new file every day), but it still limits the scope of use. Also, while technically possible, Dataset Editor might require a lot of RAM when editing large .dset files (e.g. >1 mln rows)
Not suitable for highly parallel use. While there is a mechanism of optimistic locking when appending to a Server .dset file, it's not as good as the mechanisms provided by databases. Also, when you open a dataset in Dataset Editor, edit it, and save back, then if any records were appended to it while it was edited will be lost (overwritten). While EasyMorph will warn you if the source .dset file has changed since opening, it still might not be convenient.