Import delimited text file - auto-check encoding and fail if not expected encoding

Hello,

In many cases in the data world, there are problems due to different encoding of CSV files than expected. For example, expecting a file in ANSI but receiving it in UTF-8, or viceversa, can give issues with unreadable characters when opening the file.

For EasyMorph processes that have a text file as an input file, it would be great for EasyMorph to auto-check the encoding of the text file. This is something that Notepad++ does for example, telling you the current encoding of the CSV file:

image

Let’s say I just want to run the process if I’m sure that the CSV file I have as input is encoded as UTF-8, because I know that it won’t give me any problems. If I receive a file in ANSI, I would want t the process to fail and ask the user to upload the CSV file with UTF-8 encoding.

Is this something that could be viable? Or is there something I’m missing from the current EasyMorph version that could do the trick?

Thanks a lot!

Roberto

Hello,

For your information, Notepad++ try to detect encoding and open by default with encoding detected.
But you can’t know if it’s correct. I use Dynamics Nav and all files use OEM-US (ASCII CP437) by default.
Notepad++ always detect it as ANSI while there are special characters. The only thing Notepad++ can do is to detect if any characters seem to encode multiple bytes to differentiate ANSI from UTF8.

There is no standard for detecting file’s encoding.

You can make one morph to check if an unknown character is present.

1 Like