Best approach to process a list of files with different schema's into EM Datasets with unique names

Here is my scenario:

I have a directory that is populated once per day with a set of files with unique names but the format for that file for that day may be xlsx or csv (working on getting the team to produce a consistent output format). Based upon the file name I know how to process that file. What I am attempting to do is the following:

  1. Using the “List of Files” action, generate the file list with path, names, and extension.
  2. Use “Iterate” to process each row
    a) create a column to set the “UseModuleName” based upon file name and extension
    b) Use “Call” using “UseModuleName” parameter to set the called module and return a dataset.

Here is my question - How do I rename to the dataset to the correct name? There is no way to use a parameter to set the returning table name. And maybe I am missing something, but I don’t see an action to “Rename Table”.

Any help would be greatly appreciated,

Keith

I believe, your workflow should be arranged a bit differently. For instance, you can have a module with a file name and file format specified as parameters. In this module, have two import actions, one for CSV and one for XLSX executed conditionally, depending on the format. In this case, either one import action will work, or the other. The result can be combined using the “Either” action.

This module can be iterated across a single list with XLSX and CSV files.

See the example below.
multiformat-import.morph (5.2 KB)

image

Alternatively, you can split the original list of files into two derived tables and keep only one file format in each table. Then iterate each list separately with a separate module.

Hi Dmitri - I got that part, my issue was how do I rename the table that would result from the iteration call? I need each file in it’s own table when the iterations are complete.

Workflow structure can’t be changed during project execution. Tables, actions, and modules can’t be created by a workflow.

I suspect that you need to change the approach. Or maybe I just don’t understand your question.