Export to parquet

1

Hi Dmitry,

I would like to iterate through the database (currently SQL Server) to retrieve some tables (the table name comes as a parameter in an Excel file) and save each one into Parquet files. Iteration works perfectly with QVD files, but since I have to manually check each field and set the datatype to export into Parquet, EM raises an error and the process stops.

Additionally, there are issues with datatypes. When exporting, I receive an error: "Error: Column [xxx] contains cells that do not match export type. Source: action 'Export to Apache Parquet', table 'yyy'."

Is there a way to achieve this, and is there a way to force the text datatype on error? I know I can force to text and check the 'Write null option', but doing this removes the error without showing what's wrong, and it requires quite a number of clicks if I have many columns (4 clicks per column).

Or maybe something that could check the datatype errors and filter the involved fields (because right now it's a trial-and-error method field by field).

Cheers and regards,
Chris

2

We will add a mode that exports all columns with auto-detecting their formats.

is there a way to force the text datatype on error?

You can convert errors to text using the "Convert data types" action or use the "Halt data type mismatch" action to make sure errors don't end up in the ouput file.