From 2f9df31fc18ae55a6199f25124f180ac544ac374 Mon Sep 17 00:00:00 2001 From: billy-the-fish Date: Tue, 18 Nov 2025 09:46:45 +0100 Subject: [PATCH 1/3] chore: remove enable_cagg_window_functions --- _partials/_timescaledb-gucs.md | 1 - .../alter_materialized_view.md | 1 - .../create-a-continuous-aggregate.md | 98 ------------------- 3 files changed, 100 deletions(-) diff --git a/_partials/_timescaledb-gucs.md b/_partials/_timescaledb-gucs.md index 8eb327637f..9b623b9988 100644 --- a/_partials/_timescaledb-gucs.md +++ b/_partials/_timescaledb-gucs.md @@ -25,7 +25,6 @@ | `enable_cagg_sort_pushdown` | `BOOLEAN` | `true` | Enable pushdown of ORDER BY clause for continuous aggregates | | `enable_cagg_wal_based_invalidation` | `BOOLEAN` | `false` | Use WAL to track changes to hypertables for continuous aggregates. This feature is early access from TimescaleDB v2.23.0 | | `enable_cagg_watermark_constify` | `BOOLEAN` | `true` | Enable constifying cagg watermark for real-time caggs | -| `enable_cagg_window_functions` | `BOOLEAN` | `false` | Allow window functions in continuous aggregate views | | `enable_chunk_append` | `BOOLEAN` | `true` | Enable using chunk append node | | `enable_chunk_skipping` | `BOOLEAN` | `false` | Enable using chunk column stats to filter chunks based on column filters | | `enable_chunkwise_aggregation` | `BOOLEAN` | `true` | Enable the pushdown of aggregations to the chunk level | diff --git a/api/continuous-aggregates/alter_materialized_view.md b/api/continuous-aggregates/alter_materialized_view.md index d12a2a4e5d..b7d2fa6226 100644 --- a/api/continuous-aggregates/alter_materialized_view.md +++ b/api/continuous-aggregates/alter_materialized_view.md @@ -70,7 +70,6 @@ ALTER MATERIALIZED VIEW SET ( timescaledb. = [, . | `timescaledb.compress_segmentby` | TEXT | No segementation by column. | ✖ | Set the list of columns used to segment the compressed data. An identifier representing the source of the data such as `device_id` or `tags_id` is usually a good candidate. | | `column_name` | TEXT | - | ✖ | Set the name of the column to order by or segment by. | | `timescaledb.compress_chunk_time_interval` | TEXT | - | ✖ | Reduce the total number of compressed/columnstore chunks for `table`. If you set `compress_chunk_time_interval`, compressed/columnstore chunks are merged with the previous adjacent chunk within `chunk_time_interval` whenever possible. These chunks are irreversibly merged. If you call to [decompress][decompress]/[convert_to_rowstore][convert_to_rowstore], merged chunks are not split up. You can call `compress_chunk_time_interval` independently of other compression settings; `timescaledb.compress`/`timescaledb.enable_columnstore` is not required. | -| `timescaledb.enable_cagg_window_functions` | BOOLEAN | `false` | ✖ | EXPERIMENTAL: enable window functions on continuous aggregates. Support is experimental, as there is a risk of data inconsistency. For example, in backfill scenarios, buckets could be missed. | | `timescaledb.chunk_interval` (formerly `timescaledb.chunk_time_interval`) | INTERVAL | 10x the original hypertable. | ✖ | Set the chunk interval. Renamed in $TIMESCALE_DB V2.20. | diff --git a/use-timescale/continuous-aggregates/create-a-continuous-aggregate.md b/use-timescale/continuous-aggregates/create-a-continuous-aggregate.md index 69b5f93c97..29d1c9e27b 100644 --- a/use-timescale/continuous-aggregates/create-a-continuous-aggregate.md +++ b/use-timescale/continuous-aggregates/create-a-continuous-aggregate.md @@ -216,101 +216,3 @@ by default. However, if you use them in a materialized query a warning is return When using non-immutable functions you have to ensure these functions produce consistent results across continuous aggregate refresh runs. For example, if a function depends on the current time zone you have to ensure all your $CAGG refreshes run with a consistent setting for this. - -## Use $CAGGs with window functions: experimental - - - -Window functions have experimental supported in the $CAGG query definition. Window functions are disabled - by default. To enable them, set `timescaledb.enable_cagg_window_functions` to `true`. - - - -Support is experimental, there is a risk of data inconsistency. For example, in backfill scenarios, buckets could be missed. - - - -### Create a window function - -To use a window function in a $CAGG: - -1. Create a simple table with to store a value at a specific time: - - ```sql - CREATE TABLE example ( - time TIMESTAMPZ NOT NULL, - value TEXT NOT NULL, - ); - ``` - -1. Enable window functions. - - As window functions are experimental, in order to create continuous aggregates with window functions. - you have to `enable_cagg_window_functions`. - - ```sql - SET timescaledb.enable_cagg_window_functions TO TRUE; - ``` - -1. Bucket your data by `time` and calculate the delta between time buckets using the `lag` window function: - - Window functions must stay within the time bucket. Any query that tries to look beyond the current - time bucket will produce incorrect results around the refresh boundaries. - ```sql - CREATE MATERIALIZED VIEW example_aggregate - WITH (timescaledb.continuous) AS - SELECT - time_bucket('1d', time), - customer_id, - sum(amount) AS amount, - sum(amount) - LAG(sum(amount),1,NULL) OVER (PARTITION BY time_bucket('1d', time) ORDER BY sum(amount) DESC) AS amount_diff, - ROW_NUMBER() OVER (PARTITION BY time_bucket('1d', time) ORDER BY sum(amount) DESC) - FROM sales GROUP BY 1,2; - ``` - Window functions that partition by time_bucket should be safe even with LAG()/LEAD() - - -### Window function workaround for older versions of $TIMESCALE_DB - -For $TIMESCALE_DB v2.19.3 and below, $CAGGs do not support window functions. To work around this: - -1. Create a simple table with to store a value at a specific time: - - ```sql - CREATE TABLE example ( - time TIMESTAMPZ NOT NULL, - value TEXT NOT NULL, - ); - ``` - -1. Create a continuous aggregate that does not use a window function: - - ```sql - CREATE MATERIALIZED VIEW example_aggregate - WITH (timescaledb.continuous) AS - SELECT - time_bucket('10 minutes', time) AS bucket, - first(value, time) AS value - FROM example GROUP BY bucket; - ``` - -1. Use the `lag` window function on your continuous aggregate at query time: - - This speeds up your query by calculating the aggregation ahead of time. The - delta is calculated at query time. - - ```sql - SELECT - bucket, - value - lag(value, 1) OVER (ORDER BY bucket) AS delta - FROM example_aggregate; - ``` - -[api-time-bucket-gapfill]: /api/:currentVersion:/hyperfunctions/gapfilling/time_bucket_gapfill/ -[api-time-bucket]: /api/:currentVersion:/hyperfunctions/time_bucket/ -[cagg-function-support]: /use-timescale/:currentVersion:/continuous-aggregates/about-continuous-aggregates/#function-support -[postgres-immutable]: -[postgres-rls]: -[postgres-security-barrier]: -[with-no-data]: /use-timescale/:currentVersion:/continuous-aggregates/create-a-continuous-aggregate/#using-the-with-no-data-option -[create_materialized_view]: /api/:currentVersion:/continuous-aggregates/create_materialized_view/#parameters \ No newline at end of file From f5d30c120d9e8da107ec4d3c502e0325014afc8b Mon Sep 17 00:00:00 2001 From: billy-the-fish Date: Tue, 18 Nov 2025 12:55:10 +0100 Subject: [PATCH 2/3] chore: put back in. --- _partials/_timescaledb-gucs.md | 2 + .../create-a-continuous-aggregate.md | 98 +++++++++++++++++++ 2 files changed, 100 insertions(+) diff --git a/_partials/_timescaledb-gucs.md b/_partials/_timescaledb-gucs.md index 9b623b9988..1ea2fd545c 100644 --- a/_partials/_timescaledb-gucs.md +++ b/_partials/_timescaledb-gucs.md @@ -7,6 +7,7 @@ | `cagg_processing_wal_batch_size` | `INTEGER` | `10000` | Number of entries processed from the WAL at a go. Larger values take more memory but might be more efficient.
min: `1000`, max: `10000000` | | `compress_truncate_behaviour` | `ENUM` | `COMPRESS_TRUNCATE_ONLY` | Defines how truncate behaves at the end of compression. 'truncate_only' forces truncation. 'truncate_disabled' deletes rows instead of truncate. 'truncate_or_delete' allows falling back to deletion. | | `compression_batch_size_limit` | `INTEGER` | `1000` | Setting this option to a number between 1 and 999 will force compression to limit the size of compressed batches to that amount of uncompressed tuples.Setting this to 0 defaults to the max batch size of 1000.
min: `1`, max: `1000` | +| `compression_batch_size_limit` | `INTEGER` | `1000` | Setting this option to a number between 1 and 999 will force compression to limit the size of compressed batches to that amount of uncompressed tuples.Setting this to 0 defaults to the max batch size of 1000.
min: `1`, max: `1000` | | `compression_orderby_default_function` | `STRING` | `"_timescaledb_functions.get_orderby_defaults"` | Function to use for calculating default order_by setting for compression | | `compression_segmentby_default_function` | `STRING` | `"_timescaledb_functions.get_segmentby_defaults"` | Function to use for calculating default segment_by setting for compression | | `current_timestamp_mock` | `STRING` | `NULL` | this is for debugging purposes | @@ -25,6 +26,7 @@ | `enable_cagg_sort_pushdown` | `BOOLEAN` | `true` | Enable pushdown of ORDER BY clause for continuous aggregates | | `enable_cagg_wal_based_invalidation` | `BOOLEAN` | `false` | Use WAL to track changes to hypertables for continuous aggregates. This feature is early access from TimescaleDB v2.23.0 | | `enable_cagg_watermark_constify` | `BOOLEAN` | `true` | Enable constifying cagg watermark for real-time caggs | +| `enable_cagg_window_functions` | `BOOLEAN` | `false` | Allow window functions in continuous aggregate views | | `enable_chunk_append` | `BOOLEAN` | `true` | Enable using chunk append node | | `enable_chunk_skipping` | `BOOLEAN` | `false` | Enable using chunk column stats to filter chunks based on column filters | | `enable_chunkwise_aggregation` | `BOOLEAN` | `true` | Enable the pushdown of aggregations to the chunk level | diff --git a/use-timescale/continuous-aggregates/create-a-continuous-aggregate.md b/use-timescale/continuous-aggregates/create-a-continuous-aggregate.md index 29d1c9e27b..69b5f93c97 100644 --- a/use-timescale/continuous-aggregates/create-a-continuous-aggregate.md +++ b/use-timescale/continuous-aggregates/create-a-continuous-aggregate.md @@ -216,3 +216,101 @@ by default. However, if you use them in a materialized query a warning is return When using non-immutable functions you have to ensure these functions produce consistent results across continuous aggregate refresh runs. For example, if a function depends on the current time zone you have to ensure all your $CAGG refreshes run with a consistent setting for this. + +## Use $CAGGs with window functions: experimental + + + +Window functions have experimental supported in the $CAGG query definition. Window functions are disabled + by default. To enable them, set `timescaledb.enable_cagg_window_functions` to `true`. + + + +Support is experimental, there is a risk of data inconsistency. For example, in backfill scenarios, buckets could be missed. + + + +### Create a window function + +To use a window function in a $CAGG: + +1. Create a simple table with to store a value at a specific time: + + ```sql + CREATE TABLE example ( + time TIMESTAMPZ NOT NULL, + value TEXT NOT NULL, + ); + ``` + +1. Enable window functions. + + As window functions are experimental, in order to create continuous aggregates with window functions. + you have to `enable_cagg_window_functions`. + + ```sql + SET timescaledb.enable_cagg_window_functions TO TRUE; + ``` + +1. Bucket your data by `time` and calculate the delta between time buckets using the `lag` window function: + + Window functions must stay within the time bucket. Any query that tries to look beyond the current + time bucket will produce incorrect results around the refresh boundaries. + ```sql + CREATE MATERIALIZED VIEW example_aggregate + WITH (timescaledb.continuous) AS + SELECT + time_bucket('1d', time), + customer_id, + sum(amount) AS amount, + sum(amount) - LAG(sum(amount),1,NULL) OVER (PARTITION BY time_bucket('1d', time) ORDER BY sum(amount) DESC) AS amount_diff, + ROW_NUMBER() OVER (PARTITION BY time_bucket('1d', time) ORDER BY sum(amount) DESC) + FROM sales GROUP BY 1,2; + ``` + Window functions that partition by time_bucket should be safe even with LAG()/LEAD() + + +### Window function workaround for older versions of $TIMESCALE_DB + +For $TIMESCALE_DB v2.19.3 and below, $CAGGs do not support window functions. To work around this: + +1. Create a simple table with to store a value at a specific time: + + ```sql + CREATE TABLE example ( + time TIMESTAMPZ NOT NULL, + value TEXT NOT NULL, + ); + ``` + +1. Create a continuous aggregate that does not use a window function: + + ```sql + CREATE MATERIALIZED VIEW example_aggregate + WITH (timescaledb.continuous) AS + SELECT + time_bucket('10 minutes', time) AS bucket, + first(value, time) AS value + FROM example GROUP BY bucket; + ``` + +1. Use the `lag` window function on your continuous aggregate at query time: + + This speeds up your query by calculating the aggregation ahead of time. The + delta is calculated at query time. + + ```sql + SELECT + bucket, + value - lag(value, 1) OVER (ORDER BY bucket) AS delta + FROM example_aggregate; + ``` + +[api-time-bucket-gapfill]: /api/:currentVersion:/hyperfunctions/gapfilling/time_bucket_gapfill/ +[api-time-bucket]: /api/:currentVersion:/hyperfunctions/time_bucket/ +[cagg-function-support]: /use-timescale/:currentVersion:/continuous-aggregates/about-continuous-aggregates/#function-support +[postgres-immutable]: +[postgres-rls]: +[postgres-security-barrier]: +[with-no-data]: /use-timescale/:currentVersion:/continuous-aggregates/create-a-continuous-aggregate/#using-the-with-no-data-option +[create_materialized_view]: /api/:currentVersion:/continuous-aggregates/create_materialized_view/#parameters \ No newline at end of file From 2a4ee5933a15b11e1e76aab46c6b4864784859b6 Mon Sep 17 00:00:00 2001 From: billy-the-fish Date: Tue, 18 Nov 2025 12:58:05 +0100 Subject: [PATCH 3/3] chore: put back in. --- _partials/_timescaledb-gucs.md | 1 - 1 file changed, 1 deletion(-) diff --git a/_partials/_timescaledb-gucs.md b/_partials/_timescaledb-gucs.md index 1ea2fd545c..8eb327637f 100644 --- a/_partials/_timescaledb-gucs.md +++ b/_partials/_timescaledb-gucs.md @@ -7,7 +7,6 @@ | `cagg_processing_wal_batch_size` | `INTEGER` | `10000` | Number of entries processed from the WAL at a go. Larger values take more memory but might be more efficient.
min: `1000`, max: `10000000` | | `compress_truncate_behaviour` | `ENUM` | `COMPRESS_TRUNCATE_ONLY` | Defines how truncate behaves at the end of compression. 'truncate_only' forces truncation. 'truncate_disabled' deletes rows instead of truncate. 'truncate_or_delete' allows falling back to deletion. | | `compression_batch_size_limit` | `INTEGER` | `1000` | Setting this option to a number between 1 and 999 will force compression to limit the size of compressed batches to that amount of uncompressed tuples.Setting this to 0 defaults to the max batch size of 1000.
min: `1`, max: `1000` | -| `compression_batch_size_limit` | `INTEGER` | `1000` | Setting this option to a number between 1 and 999 will force compression to limit the size of compressed batches to that amount of uncompressed tuples.Setting this to 0 defaults to the max batch size of 1000.
min: `1`, max: `1000` | | `compression_orderby_default_function` | `STRING` | `"_timescaledb_functions.get_orderby_defaults"` | Function to use for calculating default order_by setting for compression | | `compression_segmentby_default_function` | `STRING` | `"_timescaledb_functions.get_segmentby_defaults"` | Function to use for calculating default segment_by setting for compression | | `current_timestamp_mock` | `STRING` | `NULL` | this is for debugging purposes |