Unable to read json file

1

Even though it's a geojson file format I'm not looking for any spatial functionality - just the ability to read/load the file for manipulating various fields. I receive the following error - any ideas

Error: Can't parse JSON. Wrong JSON content: 'Additional text encountered after finished reading JSON content: {. Path '', line 2, position 0.'
Source: action "Import JSON from file", table "brantford-addresses-city.geojson"

{"type": "Feature", "properties": {"hash": "02cfb33e81085b24", "number": "5", "street": "NEWCASTLE ST", "unit": "", "city": "", "district": "", "region": "", "postcode": "N3R 1Y5", "id": ""}, "geometry": {"type": "Point", "coordinates": [-80.2667418, 43.1541329]}}
{"type": "Feature", "properties": {"hash": "532db368b5a3da9f", "number": "12", "street": "NIAGARA ST", "unit": "", "city": "", "district": "", "region": "", "postcode": "N3T 3X7", "id": ""}, "geometry": {"type": "Point", "coordinates": [-80.2726338, 43.1482615]}}
{"type": "Feature", "properties": {"hash": "b26467652010a2ce", "number": "22", "street": "NIAGARA ST", "unit": "", "city": "", "district": "", "region": "", "postcode": "N3R 4E2", "id": ""}, "geometry": {"type": "Point", "coordinates": [-80.272167, 43.1489121]}}

2

I tried parquet and that generated an error - unsupported data type. Only uncompressed CSV file format seemed to work. Am looking for a better way to read/write data files for automated transfers.

image
image2092×622 169 KB

3

The "Import JSON from file" action expects the JSON file to contain a valid JSON node, such as an object or an array, and not several objects separated by a line break.

As a workaround, you can use the "Import plain text" action followed by the "Parse JSON" action.

4

For clarification - the problem I'm trying to solve is that all of the source files I'm accessing are stored in S3 using the .geojson format. I need a way to download these files, load them into easymorph to transform them and then output the file in whatever format makes sense - ideally parquet when finished. I understand write to parquet is not support yet so in the interim I'm using duckdb with all of it's connectors to manipulate the files but would like to somehow replace that process using easymorph as the automated workflow engine.

5

thank you for your quick reply. Will give it a try and let you know how it goes. Do you have an ETA (+/-) on export to parquet format?

6

Export to .parquet was added in v5.7.3. You can already use it.

7

It worked thanks. I used Extract JSON properties but then had to sanitize the field to remove what appeared to be some embedded hidden characters

image
image3024×1710 803 KB

8

Confirming that the export to parquet worked like a charm.

1 Like