DuckDb connector

Feature requests
1

Hi,

Would it be possible to add a DuckDB connector to EasyMorph?

DuckDB is an embedded analytical database that's becoming quite popular, and I think it would be a great addition for several reasons:

  1. Performance - DuckDB is extremely fast for analytical queries, often many times faster than traditional databases for aggregations and complex joins.

  2. No setup required - It's an embedded database like SQLite, so no server installation or configuration needed.

  3. Handles large datasets - DuckDB can process files larger than RAM efficiently through smart buffer management and columnar compression, which would really help when working with very large files.

  4. Modern format support - It has excellent native support for Parquet files and other modern data lake formats, which would be useful as more organizations move to cloud data platforms. DuckDB can also query CSV and Parquet files directly without importing them first, which is incredibly convenient.

  5. SQL flexibility - For users who want to mix visual transformations with SQL for specific analytical operations, DuckDB would provide a nice bridge within the same workflow.

I think this would complement EasyMorph's strengths really well and open up some new use cases.

Best regards, Knut P

2 Likes
2

Hi Knut,

Can you please elaborate more on what kind of connector you need - for a local DuckDB database, or for the Motherduck cloud platform?

3

Hi Dmitry,

I was primarily thinking about a connector for local DuckDB databases, with support for both:
Connecting to a DuckDB file (e.g., ~/my_database.duckdb)
In-memory mode (using :memory:)
Read-only mode option for local DuckDB files

However, it would be valuable to also include optional MotherDuck support in the same connector, which would require:

A token authentication field
The ability to use the md: prefix for cloud databases (e.g., md:my_database)

Since the same DuckDB ODBC driver handles both local and MotherDuck connections (the difference is only in connection parameters), having a unified connector would provide maximum flexibility.

I've created a temporary solution by downloading the DuckDB ODBC driver and defining a custom connector using EasyMorph's ODBC (SQL:2003) driver option. This works, but a native DuckDB connector would be more user-friendly and eliminate the manual ODBC setup step.

It would be great it you can look further into this.

1 Like