Bulk loader postgres

Hi,

Any plans when a bulk loader will be available for postgres.

The problem that we have today is that Amazon RDS only suppoorts bulk load from S3.
Will the bulk loader that will be released in EasyMorph in the future also work with cloud based databases like AWS RDS ?

Thanks
Nikolaas

Bulk export to PostgreSQL/Amazon is next in line after SQL Server, so we’re working on it right now. If we manage to get good performance from the vendor-provided driver, we’ll include bulk export for PostgreSQL/Redshift in the next release.

Hi Dmitry,

Ok thanks. So it will only work with redshift or also with Amazon RDS ?

Currently we use the method below but it’s a bit cumbersome. I have created an EasyMorph module for this so that it is a bit easier to use it but it is not ideal because columns have to be in the right order, a csv has to be created, then uploaded to S3 for import.
So if all that could be put into 1 bulk load action that works with AWS RDS, this would be great.

https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/PostgreSQL.Procedural.Importing.html#USER_PostgreSQL.S3Import

Nikolaas

I don’t know at this point what’s going to work and what isn’t.

Update:

Bulk export for PostgreSQL will be available in native connectors only, but not in ODBC connectors or Redshift.

Hi Dmitry,

Thanks for the update. Currently we use the native postgres connector for connections to an Amazon RDS postgres instance. I think it’s an aurora database with postgres flavour. Would this mean that we could use this bulk loader for that type of connection ?

Thanks !
Nikolaas

I don’t know. At this point we test with PostgreSQL only. Once it becomes available, you will be able to test it with AWS Aurora.

Hi Dmitry,

We use AWS RDS postgres (I think it’s aurora) and bulk load seems to work.
I have test it with a table of 10 mln rows (1 column with testdata) and it worked.

We will do some further testing in the near future but it will greatly simplify our work.

Kind regards
Nikolaas

Thank you for the update, Nikolaas.