Announcement: Server journal

In version 5 we will be introducing a new feature - Server journal. The journal records various Server events, such as:

  • Workflow runs completion (successful or not). The event record includes parameters, errors, notifications, status messages, run duration, initiator, and other metadata.
  • User events
    • Logged in
    • Task triggered / changed / deleted
    • File uploaded / downloaded / deleted
    • Folder created / renamed / deleted
    • Repository connector created / edited / deleted

These events will move from the Server log and task logs into the journal. The Server log will only contain technical Server events related to the functioning of the Server itself (e.g. adding a space).

The task logs will be discontinued and replaced with the task journal, basically a user-facing journal with a task-related subset of events.

The journal under the hood is a database table. Two types of database connections are supported: embedded and ODBC:

Connection Description Editions
Embedded (Default) Embedded SQLite database, comes with Server installation, requires no configuration. No access restrictions. No failover. Team / Enterprise
External ODBC (MS SQL Server or Postgres). Doesn’t come with Server installation and must be configured separately. Allows restricting access. Automatic failover switching to the embedded journal database if the external database is not available. Enterprise only

Regardless of whether an embedded or external ODBC database connection is used, the journal format is the same.

The journal UI in EasyMorph Server allows simple filtering:

  • By date
  • By project
  • By initiator (user, scheduler, API request, etc.)
  • By status (success, failed, canceled, etc.)

Filters can be combined. For instance, you can filter by date1 OR date2 AND failed status.

The journal database table can also be queried using the Query Editor in EasyMorph, or any SQL-compatible data visualization tool such as Tableau or Power BI.

Besides completed events, the journal displays in real time:

  • Currently running workflows (administrators can cancel any workflow in any space, users can cancel workflows in their spaces only)
  • Currently logged in users (administrators can log out any user)

Benefits of the journal:

  • Much better visibility into Server activity than with text logs
  • Easy to query and analyze using EasyMorph or 3rd party tools
  • Easy to set up various alerts using EasyMorph or 3rd party tools
  • Suitable for compliance audit (for every event the initiator is recorded)
  • Access restriction (in case of external database) to protect sensitive data (e.g. parameter values)
  • Enables different views (task journal currently, and user journal and space journals in the future)

Potential downsides of the journal compared to the text logs in previous versions:

  • Only the last 20 status messages are recorded
  • Only the last 20 warnings are recorded

1 Like

The journal works in the “always append” mode and therefore the number of records in the journal always increases. While the embedded journal database can handle billions of records, over time with extensive Server use the journal database may grow very large.

EasyMorph Server doesn’t remove old records from the journal database automatically. The journal data retention policy is left to the Server admins. As a suggested solution, design and schedule an EasyMorph project that would automatically remove old journal records.

Hello,

Is it possible to see the Status Log and the parameters of a running workflow? I don’t see the “information” icon for Running workflows, which appears in Completed Workflows to the far right.

Thanks,

Roberto

No, that’s not currently possible.