Bug with column format

Hello,

Initial state :

Change format on field "Champ externe date" :

Another field change with same type.
These fields derive from same field. But in this field I have several types.

I use format because I must create json to send to REST API and number must be number (not string) and date must have another format.

It's like a bug.
Have a nice day.

EasyMorph version 5.6.2.7

The format you select in the bar above doesn't change the cell value. It only changes how that value is displayed in the grid in EasyMorph. Values remain the same.

To change cell values, use actions and expressions. For instance, if you need a text date made from a number date, use the format() function.

No. You don't have understand. I have text from csv and want to convert to number or date.
I try to create json to call web service.

I read some csv and try to send these files to web service with mapping made by user with an Excel.
I read metadata from webservice to find expected forma.

To create a json with EasyMorph, we must have 2 columns (name and value).
We don't have any option like a third column to specify format.
Some forma are inside "" and others with direct value.
So it's mandatory to have data in right forma.

On this point EasyMorph is not stable.
With same code and same mapping, sometimes data is in date format and sometimes in number format in final json.

So I don't understand why, when a format is applied to a field that has undergone transformations, a derived table (even after the field has been renamed) undergoes the same format.

Maybe it's not so important if the other actions don't take it into account. But in that case, we need an option to specify format in other actions such as json creation.

When I open the morph file to process a new csv, sometimes everything's fine, sometimes result is just dates and sometimes just numbers (in final json).

Do you construct the JSON using the "Construct JSON" action or somehow else? It's not clear from the screenshots.

Also, keep in mind that column formatting is passed from action to action if it's semantically the same column. E.g. renaming a column inherits (preserves) the column forma, but creating a new column doesn't.

Screenshot shows only convert.

Now I have all rewrite.
I manage text to date in text. But now when user has to modify the morph file to integrate a new file, it will be more complicated for him.

I use "Construct JSON".

But sometimes number appear as date in result (json : Mapping line). For some or all fields (field with number).
I don't know what's causing it. By relaunching Easymorph, results are different.
I don't have any input files to test at the moment.

How does Easymorph select format of number field in generated json?

See the attached project, as an example. In the project, the same value (40 000) is converted to JSON as a number in one table and as a date in another table, deterministically.

json.morph (3.9 KB)

The difference is in column formatting. Since in EasyMorph dates are numbers, the "Construct JSON" action looks at the column formatting in order to determine whether a numeric value should be a date or a number. The selected date format is not important, it's just the fact that there is a date format applied is sufficient for the action to format the value as a JSON date (in the standard ISO format, as per the JSON specification).

What's obscure is that if the user didn't specify the column format explicitly and left the column format as "No format" then EasyMorph tries to auto-detect the format and this behavior is not deterministic. However, if the user sets a column format explicitly (using the droplist in the column bar above), it will remain permanently and no auto-detection will take place. Make sure you save the project, though.

In the example above, I explicitly set the column format to a number in one table and a date in another. Therefore, it now behaves deterministically.

Perhaps, we need to add an indication when a date format was auto-detected, because now it's unclear if it was auto-detected or set explicitly and this causes confusion.

I understand.
My issue is that all data is in the same column and that, on transformation, formatting also applies upstream and in parallel branches.

So to solve this issue, we need to add a new column that copies value directly. And on this new column, I apply format and then, after merge, I put data from this new column back into the common column.

I think it's disturbing that column format is not only applied downstream, but also upstream.
Easymorph works a bit like Excel, and several formats can exist in the same column.
Applying forma upstream complicates process when it comes to processing each type of data in the same column.

Especially as we often transform within the same column. When we indicate column format, it often wasn't the case before we asked to apply it.

I would recommend avoiding such situations because it's not possible to control format of data if it's of different kind but all in one column. Use pivoting/unpivoting to transpose data into columns and then you will be able to control format in each column. The "Construct JSON" action is also intended for cases where all data is in multiple columns, not in one column.

I understand better now.