Opinions needed: Python integration

We're working on tight Python integration in EasyMorph - EasyMorph workflows will be able to call Python scripts as if they were another workflow. This will be achieved with a native Python package (plug-in) that will allow the following:

  • Obtaining workflow parameters in Python
  • Obtaining the input dataset in Python
  • Reading .dset files in Python
  • Building in Python an output dataset that will be returned to the calling EasyMorph workflow or saved into a .dset file
  • Accessing Shared Memory from Python
  • Retrieving dataset assets from EasyMorph's catalog in Python

A Python script for use in EasyMorph might look something like that (the syntax is approximate, just to illustrate the concept):

import easymorph

# somewhere down the code

# get a parameter
date = easymorph.parameters["start date"].as_date()
output_dset = easymorph.Dataset()

# process the input dataset row by row
for row in easymorph.input.rows:
    description = row["description"]
    print(description)
    output_row = {
        "timestamp": datetime.now()
        "description": description.upper()
    }
    # build the output dataset
    output.add(output_row)

easymorph.output = output_dset

Thoughts? Suggestions? Questions?

1 Like

Combining the power of EasyMorph workflows for extracting, manipulating, cleansing, transforming and automating data, with the capabilities of python for performing advanced data science or creating machine learning models. Isn't this what all of us data nerds dream of? :pray:

Good move from a product standpoint for sure!

3 Likes

We have been using EasyMorph for about two years now, and there are very few things we haven't been able to achieve without Python. However, for the few cases where we do use it (e.g., interacting with the Qlik Sense API server) and to open up a new range of possibilities—while keeping a good balance between tech and non-tech users—it's, in my opinion, a great enhancement.

3 Likes

Wonderfull !
Question : what is exactly the python plugin : a venv with a special package easymorph but able to use whatever whl I want (pip install xxx)? Or is it something with more restrictions ?

1 Like

Hey @Samuel_Flandrin,

This is still an early announcement, so we have not yet settled on the distribution method (for now, in development, it's just a .whl; no venv). Let us know any specific use cases or constraints you have.

This is fantastic news!

At the moment I have workflows where I output data, parse it through Python, then pull the parsed back in, so it would be amazing to be able to do that within Easymorph.

It would also be good to have the same AI feature as you have rolled out for writing functions to write Python code, so for non-coders they could write a description of what they want the Python to do and have it generated for them. The alternative is that they do it in Gemini, ChatGPT etc. but without the benefit of parameter integration, so they would then have to work this out.

2 Likes