💡 Announcement: Custom API Server

Server
#5

Great feature!
It’ll help us to implement EM in many new ways!

#6

Environments in Custom API Server

Serving API requests is different from running ETL tasks because it’s real-time and this implies a new kind of technical requirements. One of the requirements is “hot” updating of endpoint configurations, i.e. replacing projects or changing parameter settings without downtime.

To address that requirement, we will be using environments (such as production, test, dev, etc.). One space will be able to have multiple environments. Each environment will have its own:

  • Base URL path
  • Endpoint configurations
  • API keys
  • Request sampling configuration
  • Error message configuration

It will be possible to promote an endpoint configuration from one environment (e.g. “Test”) to another environment (e.g. “Production”) with a single button click (also with a command of the “EasyMorph Server command” action).

An endpoint configuration is promoted as is, i.e. without changes. Basically, it overwrites the endpoint configuration in the target environment. After promotion, only the endpoint’s public URL changes (because the target environment has a different base URL path than the source one). The project path and the HTTP method handlers and their parameter settings don’t change. It means that if you tested an endpoint in one environment and it worked as expected, it’s guaranteed to work after promotion. Also, promotion is done in a way that guarantees zero downtime. As soon as pending requests are complete, new requests will be served using the new endpoint configuration.

An endpoint configuration in an environment can be disabled. For instance, after you promoted an endpoint configuration from “Test” to “Production” you can disable it in the test environment (to not expose unnecessary API methods) while it will continue working in production. Configurations of multiple endpoints can be disabled at once.

Here are UI mockups for managing environments. The mockups in the opening post have been updated as well.

List of environments:

image
image954×711 49.2 KB

Environment settings:

image
image933×847 84.3 KB

When designing workflows, it will be possible to retrieve sample requests from any environment. Therefore, for instance, you can retrieve a failed request from the production environment, debug/fix the workflow in the test environment, and then promote the bugfix into production.

In the initial release, only promoting endpoint configurations from one environment to another in the same space will be possible. Such a promotion guarantees zero downtime.

In future releases, we will add the ability to promote endpoint configurations from one space to another space. Such promotion will not guarantee zero downtime, but it will make it possible to develop/test endpoints in a different space (e.g. with a different set of connectors).

#7

Even better!
Using multiple environment allow us to follow a test-production pattern, avoiding annoying problems like bugged project in production

#8

UPDATE

We’ve simplified things a bit and removed environments as an additional abstraction layer. Instead, spaces can be configured as environments. Environments can be configured for various scenarios – shared connector repositories or separated, shared endpoint configurations or separated, full access permission to all environments or configured individually per environment, etc.

Also, we’ve settled on the difference in Server editions:

Feature Team Edition Enterprise Edition
Endpoints per space Unlimited Unlimited
Authentication API Key, HTTP Basic API Key, HTTP Basic
Spaces with API feature enabled 1 space Unlimited spaces
HTTP Methods GET, POST Any

The table above indicates that the Team edition can have API enabled in only 1 space (environment) and accept only GET and POST requests. This should be sufficient for most cases. More complex scenarios that require multiple environments or support for other HTTP methods such as PUT or DELETE will require the Enterprise edition.

Other remarks:

  • The way endpoints are configured and versioned will be compatible with git and other version control systems so it will be possible to promote changes via git.
  • It will be possible to promote endpoint configurations between two Servers (not just between two spaces of a Server) using git or folder copying.
  • Folder copying will be added as a standard operation in the “Files” tab) as it will be frequently used for endpoint versioning.
  • It will be possible to lock endpoints (disable editing) similarly to locking tasks. This can help reduce (but not eliminate) the chance of accidental changes in production environments.
  • Two spaces will be able to share the same set of endpoints but have different base URLs or connector repositories.
  • We will add new commands to the “Server command” action to enable the automation of complex deployment scenarios with automated testing, promotion between environments, file/folder management, etc. It may work especially well after the “Git command” action is introduced (it’s on the roadmap).
  • The “HEAD” HTTP method will be treated as “GET” but the response body will be removed so only headers remain.
  • Setting or modifying cookies will not be possible in the initial release but can be added later

Beta-testing of the API endpoints feature is expected to start in mid-November.

1 Like
#9

The API Server feature is nearing the beta-testing stage. Below is a quick overview of the feature. It demonstrates how to create an API endpoint to serve incoming GET requests. The endpoint in the demo simply returns two parameters of an incoming web request as a JSON object. Not much practical value, but it demonstrates the process of creating an API endpoint.

#10

Hi @dgudkov

I am very curious about the API Server functionality’s capability to be used as a backend for No Code applications. Are you therefore positioning EM Server as a self hosted and perhaps ever more user friendly alternative to something like Xano?

Regards,

Shaheed

#11

Hi Shaheed,

I’m not familiar with Xano, can’t compare EasyMorph to it. However, our goal is to make the API Server a backend for no-code application building platforms. What app-building platform are you interested in? If you prefer it, we can have a call and talk about it in more detail.

#12

Everyone,

Beta-testing of the API Server is now open. Please feel free to send me a direct message for the download links (EM Server beta, EM Desktop beta).

What can be done with the beta version:

  • Create and configure API endpoints
    • Assign module parameters with request properties (URL parameters, headers, body, IP address, etc)
    • Any HTTP method - GET, POST, PUT, DELETE, PATCH, etc
  • The “Construct HTTP response” action (only the Success and Redirect types of responses)
  • The optional “Incoming HTTP request” action (not required in many cases)
  • Request sampling (last 10 requests for each HTTP method for each endpoint)
  • API versioning
  • Endpoint journal

What’s not available yet in the beta (this list is updated every few days):

  • Authentication (by API key or HTTP Basic)
  • The “File download” and “Error” types of responses in the “Construct HTTP response” action
  • CORS handling
#13

Thanks for the reply. We are in between platforms at the moment. Previously we were using Zoho Creator but found that sharing objects and logic across applications was very siloed. Therefore, we have opted to use Salesforce for our CRM and therefore a low code application builder.

We use Coda.io. Coda is nice and simple but does not support huge data sets. This is why were were exploring Xano for one use case.

I think it would be useful to have a call to get your take on things. Kindly let me know via email about your availability.

1 Like
#14

Thanks for the details, Shaheed. I sent you an email.

#15

Hello @dgudkov I would like to try it out, however what’s the best way to test it out while maintaining our current server configuration at the same time?

Thanks,

Roberto

#16

It’s best to have a separate Server installation for the beta. Get a temporary license key at sales@easymorph.com.

I’ll send you a direct message with the download link.

1 Like
#17

Hello @dgudkov,

Regarding the “Up to 100 queries per second” limitation mentioned in the API Endpoint demo video, does this mean that if more than 100 request come in one second, not all of them are going to be processed? At the moment we are using the “Reload Webhook” functionality to trigger a task via a GET request, and we have “Simultaneous Runs” set to “Unlimited”. With the API Server, would we have a limitation?

Thanks

#18

Currently, there is no limit on the number of requests processed by an endpoint simultaneously. It’s equivalent to “Simultaneous runs” set to “Unlimited” permanently for all endpoints. All requests are served in parallel, independently from each other.

The 100qps rate mentioned in the post is empirical, almost arbitrary. It shouldn’t be taken literally. There is no hard limit. The rate is mostly an indicator that the API Server is not intended for very high request rates and gives an idea of what kind of request rates it can handle. Of course, for heavy workflows, the max request rate will be lower.

#19

One of the possible use cases for the API Server is to pair it with an Excel add-on to provide custom Excel functions for easy lookup in Excel spreadsheets. Data can come from any system supported by EasyMorph - a database, an ERP or CRM system, a web API, etc.

For instance, if you have a list of product SKUs in a spreadsheet in column A, then obtaining product prices would be as easy as entering in Excel the formula below and stretching it down the list:

=PRODUCT.PRICE(A1)

Would that make life easier for people in your organization?

#20

The release of the API Server (v5.4) has been delayed for a week. We’re adding two more features:

  • Windows Identity authentication for requests (required Active Directory, so only available in the Enterprise Edition)
  • The ability to assign user identity (Windows Identity or API key name) to a module parameter
  • The ability to assign a custom value to a module parameter
2 Likes
#21

What is the release date of 5.4

#22

Hopefully, by the end of this week.

1 Like
#23

Version 5.4 has been released. This topic is now closed. The discussion continues here: How to create API endpoints without coding

#24