We’re working on a new feature in v4.5 which is high-level workflow diagrams. Such a diagram would show a high-level overview of the workflow of a module. The diagram shows tables, groups, and dependencies between them.
Note that what you know as tabs will be now called groups (initially we wanted to call them blocks but that term would be easy to confuse with modules). In a regular tabbed view that is currently available the groups (think of it as “logical groups of tables and charts”) are shown in tabs - one group per tab. A diagram shows groups as boxes with tables. So basically, one box in a diagram is equivalent to one tab. For instance, in the screenshot below, two groups (tabs) “Transform” and “Report” are shown. If you switch to the regular tabbed view, that would be two tabs with same names - “Transform” and “Report”.
Actions are not shown in diagrams, only tables. Adding/editing actions will remain in the regular (tabbed) view mode and in the maximized table view.
When a table is selected a list of actions is shown in the sidebar. Clicking an action will switch to the tabbed view mode with the action properties open. Also it’s possible to jump to the maximized table view and back right from the diagram.
Click to zoom:
What we learned while working on high-level workflow diagrams:
- When tables have descriptive names, such a diagram gives a good high-level insight into workflow logic and dependencies. The diagram forces naming tables semantically, instead of generic “Table 1” or “Table 2”.
- Showing actions in a diagram is a bad idea so we ditched it. It’s more or less OK on small projects, but with 300-400 actions readability becomes extremely poor. Instead, focusing on the tables and their dependencies provides a nice, high-level, semantically meaningful picture of workflow.
- Groups must not have cyclical dependencies. This will also enable new, very efficient error handling similar to the try/catch pattern in mainstream programming languages. The new error handling will become available later in 2021.
- The diagrams are good for exploring dependencies, especially when external dependencies will be added (more on that below).
The initial version of diagrams released in v4.5 will be rather basic. In the future releases we will add a few more features to it:
- Highlighting for the tables that the selected table depends upon, and the tables that depend upon the selected ones (i.e. highlighting table dependencies)
- A rich, on-hover popup with additional table metadata - annotations, input/output dimensions.
- External dependencies - files, connectors, called modules, external applications, etc. All dependencies will be clickable for easy navigation/exploration.
I’m posting this while this feature is still in the works because we would like to hear your opinion on it before it’s finalized and released. How does it look to you? Any feedback / questions / suggestions / critics?