Skip to content

Latest commit

Β 

History

History
167 lines (128 loc) Β· 11.9 KB

File metadata and controls

167 lines (128 loc) Β· 11.9 KB
api_name excerpt topics keywords tags products api
add_columnstore_policy()
Set a policy to automatically move chunks in a hypertable to the columnstore when they reach a given age.
hypercore
columnstore
jobs
columnstore
hypercore
policies
scheduled jobs
background jobs
automation framework
cloud
mst
self_hosted
license type
community
procedure

import Since2180 from "versionContent/_partials/_since_2_18_0.mdx"; import OldCreateHypertable from "versionContent/_partials/_old-api-create-hypertable.mdx"; import CreateHypertablePolicyNote from "versionContent/_partials/_create-hypertable-columnstore-policy-note.mdx";

add_columnstore_policy()

Create a job that automatically moves chunks in a hypertable to the $COLUMNSTORE after a specific time interval.

  • $CAGG_CAPs:

    You first call ALTER MATERIALIZED VIEW to enable the $COLUMNSTORE on a $CAGG, then create the job that converts your data to the $COLUMNSTORE with a call to add_columnstore_policy.

  • $HYPERTABLE_CAPs:

When $COLUMNSTORE is enabled, bloom filters are enabled by default, and every new chunk has a bloom index. Bloom indexes are not retrofitted, existing chunks need to be fully recompressed to have the bloom indexes present. If you converted chunks to $COLUMNSTORE using $TIMESCALE_DB v2.19.3 or below, to enable bloom filters on that data you have to convert those chunks to the $ROWSTORE, then convert them back to the $COLUMNSTORE.

To view the policies that you set or the policies that already exist, see informational views.

A $COLUMNSTORE policy is applied on a per-chunk basis. If you remove an existing policy and then add a new one, the new policy applies only to the chunks that have not yet been converted to $COLUMNSTORE. The existing chunks in the $COLUMNSTORE remain unchanged. This means that chunks with different $COLUMNSTORE settings can co-exist in the same $HYPERTABLE.

Samples

To create a $COLUMNSTORE job:

  • Enable $COLUMNSTORE

    For efficient queries on data in the columnstore, remember to segmentby the column you will use most often to filter your data.

    • Use ALTER MATERIALIZED VIEW for a continuous aggregate

      ALTER MATERIALIZED VIEW assets_candlestick_daily SET (
         timescaledb.enable_columnstore = true, 
         timescaledb.segmentby = 'symbol');
    • Use CREATE TABLE for a $HYPERTABLE. The columnstore policy is created automatically.

      CREATE TABLE crypto_ticks (
         "time" TIMESTAMPTZ,
         symbol TEXT,
         price DOUBLE PRECISION,
         day_volume NUMERIC
      ) WITH (
        tsdb.hypertable,
        tsdb.segmentby='symbol', 
        tsdb.orderby='time DESC'
      );
  • Add a policy to move chunks to the $COLUMNSTORE at a specific time interval

    For example:

    • 60 days after the data was added to the table:

      CALL add_columnstore_policy('crypto_ticks', after => INTERVAL '60d');
    • 3 months prior to the moment you run the query:

      CALL add_columnstore_policy('crypto_ticks', created_before => INTERVAL '3 months');
    • With an integer-based time column:

      CALL add_columnstore_policy('table_with_bigint_time', BIGINT '600000');
    • Older than eight weeks:

      CALL add_columnstore_policy('cpu_weekly', INTERVAL '8 weeks');
    • Control the time your policy runs:

      When you use a policy with a fixed schedule, $TIMESCALE_DB uses the initial_start time to compute the next start time. When $TIMESCALE_DB finishes executing a policy, it picks the next available time on the schedule, skipping any candidate start times that have already passed.

      When you set the next_start time, it only changes the start time of the next immediate execution. It does not change the computation of the next scheduled execution after that next execution. To change the schedule so a policy starts at a specific time, you need to set initial_start. To change the next immediate execution, you need to set next_start. For example, to modify a policy to execute on a fixed schedule 15 minutes past the hour, and every hour, you need to set both initial_start and next_start using alter_job:

      select * from alter_job(1000, fixed_schedule => true, initial_start => '2025-07-11 10:15:00', next_start => 
      '2025-07-11 11:15:00');
  • View the policies that you set or the policies that already exist

    SELECT * FROM timescaledb_information.jobs
    WHERE proc_name='policy_compression';

    See timescaledb_information.jobs.

Arguments

Calls to add_columnstore_policy require either after or created_before, but cannot have both.

Name Type Default Required Description
hypertable REGCLASS - βœ” Name of the hypertable or continuous aggregate to run this job on.
after INTERVAL or INTEGER - βœ– Add chunks containing data older than now - {after}::interval to the $COLUMNSTORE.
Use an object type that matchs the time column type in hypertable:
  • TIMESTAMP, TIMESTAMPTZ, or DATE: use an INTERVAL type.
  • Integer-based timestamps : set an integer type using the integer_now_func.
after is mutually exclusive with created_before.
created_before INTERVAL NULL βœ– Add chunks with a creation time of now() - created_before to the $COLUMNSTORE.
created_before is
  • Not supported for continuous aggregates.
  • Mutually exclusive with after.
schedule_interval INTERVAL 12 hours when chunk_time_interval >= 1 day for hypertable. Otherwise chunk_time_interval / 2. βœ– Set the interval between the finish time of the last execution of this policy and the next start.
initial_start TIMESTAMPTZ The interval from the finish time of the last execution to the next_start. βœ– Set the time this job is first run. This is also the time that next_start is calculated from.
next_start TIMESTAMPTZ - βœ– Set the start time of the next immediate execution. It does not change the computation of the next scheduled time after the next execution.
timezone TEXT UTC. However, daylight savings time(DST) changes may shift this alignment. βœ– Set to a valid time zone to mitigate DST shifting. If initial_start is set, subsequent executions of this policy are aligned on initial_start.
if_not_exists BOOLEAN false βœ– Set to true so this job fails with a warning rather than an error if a $COLUMNSTORE policy already exists on hypertable