Skip to content

Commit

Permalink
Doc changes and cron job schedule
Browse files Browse the repository at this point in the history
  • Loading branch information
shhnwz committed Jun 30, 2024
1 parent ea70065 commit ce24936
Show file tree
Hide file tree
Showing 3 changed files with 51 additions and 7 deletions.
28 changes: 28 additions & 0 deletions doc/reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,34 @@ Clears the compression policy for a table (so its data will never be dropped).

`interval`, the previous policy for this table, or `NULL` if none was set.

## Object Store Tier (AWS S3)

### `set_ts_tier_policy`

Sets the tier policy for a time-series table. Lazily applied every fortnightly.

#### Arguments

* `target_table_id` (`regclass`), **required** — the time-series enhanced table whose tier schedule is to be modified
* `new_compression` (`interval`), **required** — the new tier duration for the time-series table

#### Returns

`interval`, the previous policy for this table, or `NULL` if none was set.

### `clear_ts_tier_policy`

Clears the tier policy for a table.

#### Arguments

* `target_table_id` (`regclass`), **required** — the time-series enhanced table whose tier schedule is to be cleared

#### Returns

`interval`, the previous policy for this table, or `NULL` if none was set.


## Analytics Functions

These functions are not related to the maintenance of time-series tables, but do sometimes rely on the related metadata to function. They are intended to make time-series queries easier to read and maintain.
Expand Down
15 changes: 15 additions & 0 deletions doc/timeseries.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,21 @@ Sometimes you know older data isn't queried very often, but still don't want to

By calling `set_ts_compression_policy` on a time-series table with an appropriate interval (perhaps`'1 month'`), this extension will take care of compressing partitions (using a columnar storage method) older than the specified interval, once an hour. As with the retention policy functionality, a function is also provided for clearing any existing policy (existing partitions will not be decompressed, however).

### Object Store Tier

Integration with pg_tier enables extended data retention and low cost data retrieval as compare to traditional archival and retrieval solutions. If partition is aged enough to be ignored from mainstream queries, then it can be tiered to AWS S3 object store. DML queries, Analytical queries and Backfilling INSERT or UPDATE is supported on the tiered partition.

**Setting up AWS S3 bucket and credential**.

```sql
select tier.set_tier_config('my-storage-bucket','AWS_ACCESS_KEY', 'AWS_SECRET_KEY','AWS_REGION');
```
**Tier policy creation**

```sql
select set_ts_tier_policy('TIMESERIES_TABLE', 'DURATION_IN_INTERVAL');
```

### Analytics Helpers

This extension includes several functions intended to make writing correct time-series queries easier. Certain concepts can be difficult to express in standard SQL and helper functions can aid in readability and maintainability.
Expand Down
15 changes: 8 additions & 7 deletions sql/timeseries.sql
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ CREATE TABLE @[email protected]_config(
retention_duration interval,
tier_duration interval,
compression_duration interval,
CONSTRAINT policy_duration_order_check
CHECK(compression_duration < tier_duration
CONSTRAINT policy_duration_order_check
CHECK(compression_duration < tier_duration
AND tier_duration < retention_duration
AND compression_duration < retention_duration));

Expand Down Expand Up @@ -669,9 +669,9 @@ BEGIN
END;
$function$;

-- This function will call tier function for the qualified partitions
-- This function will call tier function for the qualified partitions
-- (which must be time-series enabled) and a compression offset, all partitions falling
-- entirely behind the offset (from the present time).
-- entirely behind the offset (from the present time).
CREATE OR REPLACE FUNCTION @[email protected]_tier_policy(target_table_id regclass, comp_offset interval)
RETURNS void
LANGUAGE plpgsql
Expand Down Expand Up @@ -707,12 +707,13 @@ BEGIN
part_row.partition_tablename);

IF part_end < (now() - comp_offset) THEN
EXECUTE 'SELECT tier.table(' ||
chr(39)|| part_row.partition_schemaname || '.' || part_row.partition_tablename ||
EXECUTE 'SELECT tier.table(' ||
chr(39)|| part_row.partition_schemaname || '.' || part_row.partition_tablename ||
chr(39)|| ')';
END IF;
END LOOP;
END;
$function$;


--Tier data every fortnight (15 days).
SELECT cron.schedule('timeseries-tiering', '0 23 15 * *', $$SELECT @[email protected]_tier_policy(table_id, tier_duration) FROM ts_config;$$);

0 comments on commit ce24936

Please sign in to comment.