Breaking change in v4.4: Aggregate with Concatenate and grouping

[dgudkov]

Hi Nikolaas,

@reynsnivea

Could it be so that there will be an impact because the concatenated values will be in the reversed order in v4.4 when compared with the same dataset in older versions ?

Yes, in this case v4.4 will generate a different hash on the same data, compared to previous versions.

There are 2 possible solutions: either workflow needs to be altered, or data needs to be altered. Specifically:

A) Alter workflow: add actions to reverse the row order inside groups before aggregation, or
B) Alter data: re-run hash generation in v4.4 on all old data and don't change workflow

Altering workflow would be as follows:

1. Enumerate rows inside the same groups as used for aggregation.
2. Sort rows by columns used for grouping in aggregation PLUS the column with row numbers in descending order. Sorting by row numbers in descending order should come last in the list.
3. Aggregate as usually. The column with row numbers will be removed automatically because it's not used in the aggregation.

Below is a sample project that demonstrates how to reverse row order.
aggregate-concatenate-test.morph (5.5 KB)

@reynsnivea

Wouldn’t it be better to convert the old morph files automatically when they are opened in v4.4 ? So if grouping was applied in the aggregate action, keep the sort order how it was in the old version automatically ?

We thought about this but it doesn't seem reliable. Inserting two actions automatically may interfere with the project workflow if there are other enumeration or sort actions used prior to affected aggregation. As a result, another errors can potentially be introduced without any indication or warning.

The only automatic thing happens is that EasyMorph shows a migration warning when an existing project (i.e. saved in an earlier version) with aggregation with concatenation and grouping is opened in v4.4:

@reynsnivea

Maybe you could provide a radio button with the type of order: straight order or reverse order with the correct radio button enabled depending on the specific morph project ?

We thought about this too, but such option wouldn't have much value beyond one-time migration to v4.4. Supporting until the end of time an option that was only needed once doesn't seem reasonable either.

I realize that the breaking change can be painful. We're supporting you with tools, notifications, and advice how to handle it in a least painful way. It's a one-time effort. Let's just get through it and forget about it.