Update to latest DuckDB packages

[ebardelli]

Description

The DuckDB packages have been updated to their latest versions:

• @duckdb/node-api to version 1.4.2
• @duckdb/duckdb-wasm to version 1.31.1-dev1.0

duckdb-async has been replaced with duckdb/node-api. This led to some changes in how queries
are run and how results are processed. The new duckdb/node-api package offers improved performance
and follows the same release schedule as the main DuckDB project. duckdb-async will be discontinued in
about 6 months.

Additional tests have been added to ensure that semicolons inside block comments do not split SQL
statements incorrectly.

The new packages work on my production site. There might be remaining bugs with the duckdb/node-api implementation, but I haven't run into them yet.

The duckdb-wasm update is a drop-in replacement for the old one.

Checklist

• For UI or styling changes, I have added a screenshot or gif showing before & after
• I have added a changeset
• I have added to the docs where applicable
• I have added to the VS Code extension where applicable

[archiewood]

huge PR, thank you!

[archiewood]

This is kind of two changes

• Relatively low risk: Upgrade the DuckDB connector to use the node-api. This only affects the DuckDB connector and the writing of parquet files using that.

• Relatively high risk: Upgrade WASM. This impacts the SQL runtime for the whole of Evidence and merits some careful checking

[ebardelli]

This is kind of two changes

• Relatively low risk: Upgrade the DuckDB connector to use the node-api. This only affects the DuckDB connector and the writing of parquet files using that.
• Relatively high risk: Upgrade WASM. This impacts the SQL runtime for the whole of Evidence and merits some careful checking

I think that the risk is kind of flipped.

The WASM update is a drop-in replacement and stuff just works (as far as I can tell). I have been running the new WASM packages for some time now and nothing has broken.

The switch to node-api was more of a pain to make it work. Beside the small change from Database to DuckDBInstance, it broke a bunch of other stuff because of how results are returned. The new tests address all the issues I ran into in my deployment. There might be other hedge cases that I haven't experienced, but I'm doing some weird stuff with my duckdb connector, so I'm feeling pretty confident that I'm catching most of the issues. The only thing that comes to mind that I haven't tried if using TEMP TABLES in a query.

[ebardelli]

I pushed a new commit that uses the stable releases as of today.

• @duckdb/node-api uses version 1.4.2-r.1. I think they are doing .patch now?
• @duckdb/duckdb-wasm to version 1.30.0. This uses an older version of duckdb, but that's what it's tagged as current on npmjs, even if they released 1.31.0 without a dev tag. 🤷

[archiewood]

looks like you need to regenerate the lockfile

version 1.30.0. This uses an older version of duckdb

How old we talking? WASM sometimes lags duckdb

I think that the risk is kind of flipped.

I guess I was thinking more about the risk to the product as a whole. Impacting the duckdb connector vs impacting the runtime. I take your point around the complexity

[ebardelli]

looks like you need to regenerate the lockfile

Done and forced pushed.

version 1.30.0. This uses an older version of duckdb

How old we talking? WASM sometimes lags duckdb

https://www.npmjs.com/package/@duckdb/duckdb-wasm?activeTab=versions

It looks like 1.30.0 was released 2 months ago, 1.31.0 1 month ago, and 1.31.1-dev31.0 yesterday (probably to line up with 1.4.2).

[archiewood]

seems like some legitimate tests failing in universal-sql to do with parquet writing

https://github.com/evidence-dev/evidence/actions/runs/19345379304/job/55344745970

[ebardelli]

I thought I did run format and added a changeset. I might have removed them when I forced pushed yesterday.

[ebardelli]

Yeah. These fail also on main. I fixed them in #3243.

Would you rather just merge the two pull requests into one?

[adam-homeboost]

thank you for this!