đź’ˇ Announcement: Custom API Server

Server
1

In version 5.4 (planned for release in November) we will be introducing a new major capability of EasyMorph Server - custom API endpoints. You will be able to configure a custom API server and use EasyMorph workflows to process incoming HTTP requests from your internal (no-code or not) applications. Here is how it’s going to work:

  • New Server tab “API endpoints” is used to configure custom API endpoints - endpoint URL paths and EasyMorph projects that will process HTTP requests sent to the endpoints. Different modules of the projects handle different HTTP methods, one module per method. URL path segments can be automatically assigned to module parameters.

  • The modules that handle requests, must have the new “Incoming HTTP request” action. Request URL, query parameters, headers, and body will be returned by the action as columns. Request body (e.g. JSON) can be parsed and processed using regular EasyMorph actions.

  • An HTTP response is configured by another new action “HTTP response”. With the action, you can configure a response’s headers (including cookies) and body. Also, there will be response options for downloading a file or a URL redirect. Response body (e.g. JSON) can be constructed using regular EasyMorph actions.

  • Endpoints can employ authentication either via an API key or using the HTTP Basic authentication.

With this new capability, you will be able to rapidly design API endpoints for serving internal applications – various automations, no-code apps, system-to-system real-time integrations, etc.

We did preliminary tests and even before optimizations, the Server can process tens of HTTP requests per minute which should be sufficient for most internal purposes. In the future, we’re planning to optimize project execution startup time and increase request throughput by an order of magnitude.

UI mockups

List of endpoints

image
image1350Ă—754 94.7 KB

Endpoint properties

image
image1410Ă—899 188 KB

Questions? Suggestions?

UPDATE
Changed the version and estimated release date to 5.4 / November.

2 Likes
2

Dmitry,

A great addition to the platform! Our shop has many custom internal apps and you are helping us expand the realm of automation possibilities. This will give us the ability to hand off complex data engineering tasks to EM and simplify the code base oof our apps.

Good stuff!

3

Will space users have control over this or only server admins have control over it? The reason is that the easyMorph server in our organization is managed by IT team.

4

Yes, space users will be able to control it. Custom API endpoints are a space feature.

1 Like
5

Great feature!
It’ll help us to implement EM in many new ways!

6

Environments in Custom API Server

Serving API requests is different from running ETL tasks because it’s real-time and this implies a new kind of technical requirements. One of the requirements is “hot” updating of endpoint configurations, i.e. replacing projects or changing parameter settings without downtime.

To address that requirement, we will be using environments (such as production, test, dev, etc.). One space will be able to have multiple environments. Each environment will have its own:

  • Base URL path
  • Endpoint configurations
  • API keys
  • Request sampling configuration
  • Error message configuration

It will be possible to promote an endpoint configuration from one environment (e.g. “Test”) to another environment (e.g. “Production”) with a single button click (also with a command of the “EasyMorph Server command” action).

An endpoint configuration is promoted as is, i.e. without changes. Basically, it overwrites the endpoint configuration in the target environment. After promotion, only the endpoint’s public URL changes (because the target environment has a different base URL path than the source one). The project path and the HTTP method handlers and their parameter settings don’t change. It means that if you tested an endpoint in one environment and it worked as expected, it’s guaranteed to work after promotion. Also, promotion is done in a way that guarantees zero downtime. As soon as pending requests are complete, new requests will be served using the new endpoint configuration.

An endpoint configuration in an environment can be disabled. For instance, after you promoted an endpoint configuration from “Test” to “Production” you can disable it in the test environment (to not expose unnecessary API methods) while it will continue working in production. Configurations of multiple endpoints can be disabled at once.

Here are UI mockups for managing environments. The mockups in the opening post have been updated as well.

List of environments:

image
image954Ă—711 49.2 KB

Environment settings:

image
image933Ă—847 84.3 KB

When designing workflows, it will be possible to retrieve sample requests from any environment. Therefore, for instance, you can retrieve a failed request from the production environment, debug/fix the workflow in the test environment, and then promote the bugfix into production.

In the initial release, only promoting endpoint configurations from one environment to another in the same space will be possible. Such a promotion guarantees zero downtime.

In future releases, we will add the ability to promote endpoint configurations from one space to another space. Such promotion will not guarantee zero downtime, but it will make it possible to develop/test endpoints in a different space (e.g. with a different set of connectors).

7

Even better!
Using multiple environment allow us to follow a test-production pattern, avoiding annoying problems like bugged project in production

8

UPDATE

We’ve simplified things a bit and removed environments as an additional abstraction layer. Instead, spaces can be configured as environments. Environments can be configured for various scenarios – shared connector repositories or separated, shared endpoint configurations or separated, full access permission to all environments or configured individually per environment, etc.

Also, we’ve settled on the difference in Server editions:

Feature Team Edition Enterprise Edition
Endpoints per space Unlimited Unlimited
Authentication API Key, HTTP Basic API Key, HTTP Basic
Spaces with API feature enabled 1 space Unlimited spaces
HTTP Methods GET, POST Any

The table above indicates that the Team edition can have API enabled in only 1 space (environment) and accept only GET and POST requests. This should be sufficient for most cases. More complex scenarios that require multiple environments or support for other HTTP methods such as PUT or DELETE will require the Enterprise edition.

Other remarks:

  • The way endpoints are configured and versioned will be compatible with git and other version control systems so it will be possible to promote changes via git.
  • It will be possible to promote endpoint configurations between two Servers (not just between two spaces of a Server) using git or folder copying.
  • Folder copying will be added as a standard operation in the “Files” tab) as it will be frequently used for endpoint versioning.
  • It will be possible to lock endpoints (disable editing) similarly to locking tasks. This can help reduce (but not eliminate) the chance of accidental changes in production environments.
  • Two spaces will be able to share the same set of endpoints but have different base URLs or connector repositories.
  • We will add new commands to the “Server command” action to enable the automation of complex deployment scenarios with automated testing, promotion between environments, file/folder management, etc. It may work especially well after the “Git command” action is introduced (it’s on the roadmap).
  • The “HEAD” HTTP method will be treated as “GET” but the response body will be removed so only headers remain.
  • Setting or modifying cookies will not be possible in the initial release but can be added later

Beta-testing of the API endpoints feature is expected to start in mid-November.

1 Like
9

The API Server feature is nearing the beta-testing stage. Below is a quick overview of the feature. It demonstrates how to create an API endpoint to serve incoming GET requests. The endpoint in the demo simply returns two parameters of an incoming web request as a JSON object. Not much practical value, but it demonstrates the process of creating an API endpoint.

10

Hi @dgudkov

I am very curious about the API Server functionality’s capability to be used as a backend for No Code applications. Are you therefore positioning EM Server as a self hosted and perhaps ever more user friendly alternative to something like Xano?

Regards,

Shaheed

11

Hi Shaheed,

I’m not familiar with Xano, can’t compare EasyMorph to it. However, our goal is to make the API Server a backend for no-code application building platforms. What app-building platform are you interested in? If you prefer it, we can have a call and talk about it in more detail.

12

Everyone,

Beta-testing of the API Server is now open. Please feel free to send me a direct message for the download links (EM Server beta, EM Desktop beta).

What can be done with the beta version:

  • Create and configure API endpoints
    • Assign module parameters with request properties (URL parameters, headers, body, IP address, etc)
    • Any HTTP method - GET, POST, PUT, DELETE, PATCH, etc
  • The “Construct HTTP response” action (only the Success and Redirect types of responses)
  • The optional “Incoming HTTP request” action (not required in many cases)
  • Request sampling (last 10 requests for each HTTP method for each endpoint)
  • API versioning
  • Endpoint journal

What’s not available yet in the beta (this list is updated every few days):

  • Authentication (by API key or HTTP Basic)
  • The “File download” and “Error” types of responses in the “Construct HTTP response” action
  • CORS handling
13

Thanks for the reply. We are in between platforms at the moment. Previously we were using Zoho Creator but found that sharing objects and logic across applications was very siloed. Therefore, we have opted to use Salesforce for our CRM and therefore a low code application builder.

We use Coda.io. Coda is nice and simple but does not support huge data sets. This is why were were exploring Xano for one use case.

I think it would be useful to have a call to get your take on things. Kindly let me know via email about your availability.

1 Like
14

Thanks for the details, Shaheed. I sent you an email.

15

Hello @dgudkov I would like to try it out, however what’s the best way to test it out while maintaining our current server configuration at the same time?

Thanks,

Roberto

16

It’s best to have a separate Server installation for the beta. Get a temporary license key at sales@easymorph.com.

I’ll send you a direct message with the download link.

1 Like
17

Hello @dgudkov,

Regarding the “Up to 100 queries per second” limitation mentioned in the API Endpoint demo video, does this mean that if more than 100 request come in one second, not all of them are going to be processed? At the moment we are using the “Reload Webhook” functionality to trigger a task via a GET request, and we have “Simultaneous Runs” set to “Unlimited”. With the API Server, would we have a limitation?

Thanks

18

Currently, there is no limit on the number of requests processed by an endpoint simultaneously. It's equivalent to "Simultaneous runs" set to "Unlimited" permanently for all endpoints. All requests are served in parallel, independently from each other.

The 100qps rate mentioned in the post is empirical, almost arbitrary. It shouldn't be taken literally. There is no hard limit. The rate is mostly an indicator that the API Server is not intended for very high request rates and gives an idea of what kind of request rates it can handle. Of course, for heavy workflows, the max request rate will be lower.

19

One of the possible use cases for the API Server is to pair it with an Excel add-on to provide custom Excel functions for easy lookup in Excel spreadsheets. Data can come from any system supported by EasyMorph - a database, an ERP or CRM system, a web API, etc.

For instance, if you have a list of product SKUs in a spreadsheet in column A, then obtaining product prices would be as easy as entering in Excel the formula below and stretching it down the list:

=PRODUCT.PRICE(A1)

Would that make life easier for people in your organization?

20

The release of the API Server (v5.4) has been delayed for a week. We’re adding two more features:

  • Windows Identity authentication for requests (required Active Directory, so only available in the Enterprise Edition)
  • The ability to assign user identity (Windows Identity or API key name) to a module parameter
  • The ability to assign a custom value to a module parameter
2 Likes