13GB CSV File won't load

1

I have a 13GB CSV file. It loads but tells me there are thousands of errors. So it seems it hasn’t loaded all the records. I don’t know how many records are in the file as I can’t open it.

One of the columns looks like some HTML data and it looks to me that this data is somehow “corrupting” the CSV format.

Any ideas?

2

Hi,

that’s quite a huge file.
Maybe split it in several pieces with the split delimited file feature.
Then try to import the resulting files with a loop calling an import sub module.
it will help to check the structure of the files.

Regards

3

Hi Gavin,

What kind of errors are you getting? You can see the actual errors by checking the “Show parsing errors” option in the “Advanced options” pop-up.

Also, please check the “Add new columns when rows have more values than headers” option in the same pop-up. This will make sure that the action will parse all the values from the CSV file.

If the HTML-like values are in the same column in all the imported rows, most likely, those values don’t “corrupt” the CSV format.

Also, you can use the preview pane (“Show preview” button at the top right corner of the action properties) to check if the first 100 rows are parsed correctly. If they are, it is highly probable that all the other rows are parsed correctly too.

4

Thanks guys. This morning I downloaded the latest version of EasyMorph and it has managed to load the whole file without issues!