From e258937a2317b10b93ced1e7c58550119946f970 Mon Sep 17 00:00:00 2001 From: atovpeko Date: Wed, 12 Nov 2025 12:44:39 +0200 Subject: [PATCH 01/11] update for jsonb compression --- .../hypercore/compression-methods.md | 26 +++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/use-timescale/hypercore/compression-methods.md b/use-timescale/hypercore/compression-methods.md index 87f2ab13a7..9cd95cab09 100644 --- a/use-timescale/hypercore/compression-methods.md +++ b/use-timescale/hypercore/compression-methods.md @@ -279,6 +279,32 @@ have a lot of repeated values, then the dictionary is the same size as the original data. TimescaleDB automatically detects this case, and falls back to not using a dictionary in that scenario. +#### JSONB compression + +For JSONB columns, TimescaleDB uses a two-layer compression approach: + +1. **TimescaleDB dictionary compression**: First, TimescaleDB attempts to apply + its own dictionary compression to the JSONB data. This works well when JSONB + values have high repetition. + +2. **PostgreSQL TOAST compression**: If dictionary compression doesn't perform + well (for example, when each JSONB value is unique or nearly unique), + TimescaleDB skips compression and allows PostgreSQL to handle the data using + its TOAST (The Oversized-Attribute Storage Technique) compression. By default, + PostgreSQL uses `pglz` compression, but this can be configured to use more + modern compression algorithms like `lz4` by setting the `default_toast_compression` + configuration parameter. + +You can check your current TOAST compression setting: + +```sql +SHOW default_toast_compression; +``` + +This two-layer approach ensures that JSONB data is compressed efficiently +regardless of its characteristics, using dictionary compression when beneficial +and falling back to LZ-based compression when dictionary compression is not effective. + [decompress-chunks]: /use-timescale/:currentVersion:/compression/decompress-chunks [manual-compression]: /use-timescale/:currentVersion:/compression/manual-compression/ [delta]: /use-timescale/:currentVersion:/hypercore/compression-methods/#delta-encoding From 4b38ee618655995b6aebc0bba020a3e9673bfd6f Mon Sep 17 00:00:00 2001 From: Anastasiia Tovpeko <114177030+atovpeko@users.noreply.github.com> Date: Fri, 14 Nov 2025 14:07:22 +0200 Subject: [PATCH 02/11] Update use-timescale/hypercore/compression-methods.md Co-authored-by: Iain Cox Signed-off-by: Anastasiia Tovpeko <114177030+atovpeko@users.noreply.github.com> --- use-timescale/hypercore/compression-methods.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/use-timescale/hypercore/compression-methods.md b/use-timescale/hypercore/compression-methods.md index 9cd95cab09..8a0d0fd8a5 100644 --- a/use-timescale/hypercore/compression-methods.md +++ b/use-timescale/hypercore/compression-methods.md @@ -281,7 +281,7 @@ not using a dictionary in that scenario. #### JSONB compression -For JSONB columns, TimescaleDB uses a two-layer compression approach: +For JSONB columns, $TIMESCALE_DB uses a two-layer compression approach: 1. **TimescaleDB dictionary compression**: First, TimescaleDB attempts to apply its own dictionary compression to the JSONB data. This works well when JSONB From 3066024fece82336713f5331f4550bf78958b6cf Mon Sep 17 00:00:00 2001 From: Anastasiia Tovpeko <114177030+atovpeko@users.noreply.github.com> Date: Fri, 14 Nov 2025 14:07:31 +0200 Subject: [PATCH 03/11] Update use-timescale/hypercore/compression-methods.md Co-authored-by: Iain Cox Signed-off-by: Anastasiia Tovpeko <114177030+atovpeko@users.noreply.github.com> --- use-timescale/hypercore/compression-methods.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/use-timescale/hypercore/compression-methods.md b/use-timescale/hypercore/compression-methods.md index 8a0d0fd8a5..3a89b2ceac 100644 --- a/use-timescale/hypercore/compression-methods.md +++ b/use-timescale/hypercore/compression-methods.md @@ -283,7 +283,7 @@ not using a dictionary in that scenario. For JSONB columns, $TIMESCALE_DB uses a two-layer compression approach: -1. **TimescaleDB dictionary compression**: First, TimescaleDB attempts to apply +1. **$TIMESCALE_DB dictionary compression**: $TIMESCALE_DB attempts to apply its own dictionary compression to the JSONB data. This works well when JSONB values have high repetition. From 8bad7e52d831ef1100b0a9c20c51270506fe9fc6 Mon Sep 17 00:00:00 2001 From: Anastasiia Tovpeko <114177030+atovpeko@users.noreply.github.com> Date: Fri, 14 Nov 2025 14:07:48 +0200 Subject: [PATCH 04/11] Update use-timescale/hypercore/compression-methods.md Co-authored-by: Iain Cox Signed-off-by: Anastasiia Tovpeko <114177030+atovpeko@users.noreply.github.com> --- use-timescale/hypercore/compression-methods.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/use-timescale/hypercore/compression-methods.md b/use-timescale/hypercore/compression-methods.md index 3a89b2ceac..6536c10a29 100644 --- a/use-timescale/hypercore/compression-methods.md +++ b/use-timescale/hypercore/compression-methods.md @@ -285,7 +285,7 @@ For JSONB columns, $TIMESCALE_DB uses a two-layer compression approach: 1. **$TIMESCALE_DB dictionary compression**: $TIMESCALE_DB attempts to apply its own dictionary compression to the JSONB data. This works well when JSONB - values have high repetition. + values have high repetition. It does not perform when each JSONB value is unique or nearly unique. 2. **PostgreSQL TOAST compression**: If dictionary compression doesn't perform well (for example, when each JSONB value is unique or nearly unique), From 2ea921327b572964ff21e9c29b41dbcfba0fb5f1 Mon Sep 17 00:00:00 2001 From: Anastasiia Tovpeko <114177030+atovpeko@users.noreply.github.com> Date: Fri, 14 Nov 2025 14:08:53 +0200 Subject: [PATCH 05/11] Update use-timescale/hypercore/compression-methods.md Co-authored-by: Iain Cox Signed-off-by: Anastasiia Tovpeko <114177030+atovpeko@users.noreply.github.com> --- use-timescale/hypercore/compression-methods.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/use-timescale/hypercore/compression-methods.md b/use-timescale/hypercore/compression-methods.md index 6536c10a29..3bcd065da6 100644 --- a/use-timescale/hypercore/compression-methods.md +++ b/use-timescale/hypercore/compression-methods.md @@ -291,7 +291,7 @@ For JSONB columns, $TIMESCALE_DB uses a two-layer compression approach: well (for example, when each JSONB value is unique or nearly unique), TimescaleDB skips compression and allows PostgreSQL to handle the data using its TOAST (The Oversized-Attribute Storage Technique) compression. By default, - PostgreSQL uses `pglz` compression, but this can be configured to use more + PostgreSQL uses `pglz` compression. To configure $PG to use more modern compression algorithms like `lz4` by setting the `default_toast_compression` configuration parameter. From b2bc3af89dbfc4cff98a79d3031f42ab93932439 Mon Sep 17 00:00:00 2001 From: Anastasiia Tovpeko <114177030+atovpeko@users.noreply.github.com> Date: Fri, 14 Nov 2025 14:09:01 +0200 Subject: [PATCH 06/11] Update use-timescale/hypercore/compression-methods.md Co-authored-by: Iain Cox Signed-off-by: Anastasiia Tovpeko <114177030+atovpeko@users.noreply.github.com> --- use-timescale/hypercore/compression-methods.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/use-timescale/hypercore/compression-methods.md b/use-timescale/hypercore/compression-methods.md index 3bcd065da6..70df204d22 100644 --- a/use-timescale/hypercore/compression-methods.md +++ b/use-timescale/hypercore/compression-methods.md @@ -295,7 +295,7 @@ For JSONB columns, $TIMESCALE_DB uses a two-layer compression approach: modern compression algorithms like `lz4` by setting the `default_toast_compression` configuration parameter. -You can check your current TOAST compression setting: +To check your current TOAST compression setting: ```sql SHOW default_toast_compression; From 6547d6ae5543a6585241ad4b7bbde6384f4c595c Mon Sep 17 00:00:00 2001 From: Anastasiia Tovpeko <114177030+atovpeko@users.noreply.github.com> Date: Fri, 14 Nov 2025 14:09:19 +0200 Subject: [PATCH 07/11] Update use-timescale/hypercore/compression-methods.md Co-authored-by: Iain Cox Signed-off-by: Anastasiia Tovpeko <114177030+atovpeko@users.noreply.github.com> --- use-timescale/hypercore/compression-methods.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/use-timescale/hypercore/compression-methods.md b/use-timescale/hypercore/compression-methods.md index 70df204d22..8a0cf1470e 100644 --- a/use-timescale/hypercore/compression-methods.md +++ b/use-timescale/hypercore/compression-methods.md @@ -287,7 +287,7 @@ For JSONB columns, $TIMESCALE_DB uses a two-layer compression approach: its own dictionary compression to the JSONB data. This works well when JSONB values have high repetition. It does not perform when each JSONB value is unique or nearly unique. -2. **PostgreSQL TOAST compression**: If dictionary compression doesn't perform +2. **$PS TOAST compression**: If dictionary compression does not perform, $TiimescaleDB skips to enable well (for example, when each JSONB value is unique or nearly unique), TimescaleDB skips compression and allows PostgreSQL to handle the data using its TOAST (The Oversized-Attribute Storage Technique) compression. By default, From 0ff5c47809af2f63a8c8695c69cbda21811ed16c Mon Sep 17 00:00:00 2001 From: Anastasiia Tovpeko <114177030+atovpeko@users.noreply.github.com> Date: Fri, 14 Nov 2025 14:09:27 +0200 Subject: [PATCH 08/11] Update use-timescale/hypercore/compression-methods.md Co-authored-by: Iain Cox Signed-off-by: Anastasiia Tovpeko <114177030+atovpeko@users.noreply.github.com> --- use-timescale/hypercore/compression-methods.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/use-timescale/hypercore/compression-methods.md b/use-timescale/hypercore/compression-methods.md index 8a0cf1470e..26a6267a91 100644 --- a/use-timescale/hypercore/compression-methods.md +++ b/use-timescale/hypercore/compression-methods.md @@ -292,7 +292,7 @@ For JSONB columns, $TIMESCALE_DB uses a two-layer compression approach: TimescaleDB skips compression and allows PostgreSQL to handle the data using its TOAST (The Oversized-Attribute Storage Technique) compression. By default, PostgreSQL uses `pglz` compression. To configure $PG to use more - modern compression algorithms like `lz4` by setting the `default_toast_compression` + modern compression algorithms like `lz4`, set the `default_toast_compression` configuration parameter. To check your current TOAST compression setting: From cbf21063dfa6cd66d6e290cedc0e3aa081d42259 Mon Sep 17 00:00:00 2001 From: Anastasiia Tovpeko <114177030+atovpeko@users.noreply.github.com> Date: Fri, 14 Nov 2025 14:09:36 +0200 Subject: [PATCH 09/11] Update use-timescale/hypercore/compression-methods.md Co-authored-by: Iain Cox Signed-off-by: Anastasiia Tovpeko <114177030+atovpeko@users.noreply.github.com> --- use-timescale/hypercore/compression-methods.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/use-timescale/hypercore/compression-methods.md b/use-timescale/hypercore/compression-methods.md index 26a6267a91..d60966085f 100644 --- a/use-timescale/hypercore/compression-methods.md +++ b/use-timescale/hypercore/compression-methods.md @@ -302,7 +302,7 @@ SHOW default_toast_compression; ``` This two-layer approach ensures that JSONB data is compressed efficiently -regardless of its characteristics, using dictionary compression when beneficial +regardless of its characteristics. and falling back to LZ-based compression when dictionary compression is not effective. [decompress-chunks]: /use-timescale/:currentVersion:/compression/decompress-chunks From 12efab9e0c7e2d7acf02faf4ed72757c8aa0a8d1 Mon Sep 17 00:00:00 2001 From: atovpeko Date: Fri, 14 Nov 2025 14:13:06 +0200 Subject: [PATCH 10/11] update on review --- use-timescale/hypercore/compression-methods.md | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/use-timescale/hypercore/compression-methods.md b/use-timescale/hypercore/compression-methods.md index d60966085f..a4619b98e6 100644 --- a/use-timescale/hypercore/compression-methods.md +++ b/use-timescale/hypercore/compression-methods.md @@ -287,11 +287,9 @@ For JSONB columns, $TIMESCALE_DB uses a two-layer compression approach: its own dictionary compression to the JSONB data. This works well when JSONB values have high repetition. It does not perform when each JSONB value is unique or nearly unique. -2. **$PS TOAST compression**: If dictionary compression does not perform, $TiimescaleDB skips to enable - well (for example, when each JSONB value is unique or nearly unique), - TimescaleDB skips compression and allows PostgreSQL to handle the data using +2. **$PG TOAST compression**: if dictionary compression does not perform, $TIMESCALE_DB skips compression and allows $PG to handle the data using its TOAST (The Oversized-Attribute Storage Technique) compression. By default, - PostgreSQL uses `pglz` compression. To configure $PG to use more + $PG uses `pglz` compression. To configure $PG to use more modern compression algorithms like `lz4`, set the `default_toast_compression` configuration parameter. @@ -301,10 +299,6 @@ To check your current TOAST compression setting: SHOW default_toast_compression; ``` -This two-layer approach ensures that JSONB data is compressed efficiently -regardless of its characteristics. -and falling back to LZ-based compression when dictionary compression is not effective. - [decompress-chunks]: /use-timescale/:currentVersion:/compression/decompress-chunks [manual-compression]: /use-timescale/:currentVersion:/compression/manual-compression/ [delta]: /use-timescale/:currentVersion:/hypercore/compression-methods/#delta-encoding From 9cda77e38983a4ea809e27b3ac7fc620e1591bfe Mon Sep 17 00:00:00 2001 From: Iain Cox Date: Tue, 18 Nov 2025 09:39:28 +0100 Subject: [PATCH 11/11] Apply suggestion from @billy-the-fish Signed-off-by: Iain Cox --- use-timescale/hypercore/compression-methods.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/use-timescale/hypercore/compression-methods.md b/use-timescale/hypercore/compression-methods.md index a4619b98e6..c5593e904c 100644 --- a/use-timescale/hypercore/compression-methods.md +++ b/use-timescale/hypercore/compression-methods.md @@ -288,7 +288,7 @@ For JSONB columns, $TIMESCALE_DB uses a two-layer compression approach: values have high repetition. It does not perform when each JSONB value is unique or nearly unique. 2. **$PG TOAST compression**: if dictionary compression does not perform, $TIMESCALE_DB skips compression and allows $PG to handle the data using - its TOAST (The Oversized-Attribute Storage Technique) compression. By default, + Oversized-Attribute Storage Technique (TOAST) compression. By default, $PG uses `pglz` compression. To configure $PG to use more modern compression algorithms like `lz4`, set the `default_toast_compression` configuration parameter.