What's Changed
- fix: instructions for installing v2 package by @polastre in #8
- feat: Optimize memory use and configure postgres for maximum performance by @polastre in #9
There are significant changes in this release, including critical performance optimizations and upgrading to go 1.25. Specifically:
- Creates a reusable buffer pool for gzip compression so that each tile generated doesn't allocate a new buffer. This significantly reduces garbage collector pressure
- Reuse the gzip writer instance, since each allocation of the gzip writer incurs a 1.4MB allocation
- Use the underlying
pgtype.DriverBytesfrom pgx for postgres queries. Instead of allocating a buffer to write the tile data into from postgres, reuse the underlying pgx buffer which results in no allocations. - Pre-generate the sql queries for each zoom and store them in a shared map. Since the sql query at each zoom is the same, there's no need to generate the query string on each and every tile.
- Refactor the query strings to use coordinate placeholders
($1, $2, $3)so that a single statement per zoom can be prepared and reused by pgx. Before, every tile query had the coordinate embedded in the query, so each query was a new prepared statement and didn't benefit from the reuse of previous queries. - Turn JIT off in each connection session. When running many repeated queries, JIT adds 40ms per query that is completely unnecessary.
- Add a custom
--initcommand that accepts a sql statement to run on connection session start. This can be used to further optimize the performance of postgres queries based on the behavior of your tileset.json. - Refactor the exporter and worker into its own package to make it easier to decompose and improve, rather than in main.go
- Add memory and gc stats to progress reporter to verify memory allocations
- Upgrade to go 1.25, and upgrade dependencies
Full Changelog: v2.0.2...v2.1.0