Hypercore API ref #3699

billy-the-fish · 2025-01-09T10:38:45Z

No description provided.

github-actions · 2025-01-09T10:38:57Z

Allow 10 minutes from last push for the staging site to build. If the link doesn't work, try using incognito mode instead. For internal reviewers, check web-documentation repo actions for staging build status. Link to build for this PR: http://docs-dev.timescale.com/docs-235-docs-rfc-update-the-api-ref-for-hyperstore-and-compression-for-any-changes-due-to-the-hyperstore

github-actions · 2025-01-09T13:29:24Z

Oh no, the docs build failed! To see why visit the failing build (link only accessible to Timescalers).

github-actions · 2025-01-09T13:43:36Z

Oh no, the docs build failed! To see why visit the failing build (link only accessible to Timescalers).

…tore-and-compression-for-any-changes-due-to-the-hyperstore

philkra · 2025-01-15T07:46:35Z

@billy-the-fish the changes look good to me. One thing, I would look like to add is a reference from the deprecated functions / parameters to the new ones. For example:
compress_chunk is deprecated in favour of convert_to_columnstore. As user, I would love to know if there is a replacement for the deprecated function, having a reference to the replacement would be handy.

…l these docs go live.

…tore-and-compression-for-any-changes-due-to-the-hyperstore

erimatnor

Made a quick pass and added some suggestions.

_partials/_hypercore_policy_workflow.md

api/hypercore/add_columnstore_policy.md

api/hypercore/page-index/page-index.js

api/hypercore/remove_columnstore_policy.md

Awesome review, thank you. Co-authored-by: Erik Nordström <[email protected]> Signed-off-by: Iain Cox <[email protected]>

jonatas · 2025-01-16T16:02:35Z

api/compression/index.md

@@ -30,6 +31,8 @@ Compressed chunks have the following limitations:
    after constraint creation.
 *   [Timescale SkipScan][skipscan] does not currently work on compressed chunks.

+<Deprecated2180 /> Replaced by <a href="https://docs.timescale.com/api/latest/hypercore/">Hypercore</a>.


We may emphasize that compression works in the new version.

…tore-and-compression-for-any-changes-due-to-the-hyperstore

github-actions · 2025-01-17T11:43:38Z

Oh no, the docs build failed! To see why visit the failing build (link only accessible to Timescalers).

github-actions · 2025-01-17T12:26:32Z

Oh no, the docs build failed! To see why visit the failing build (link only accessible to Timescalers).

mkindahl

I noted that many entries are noted as required while they are not.

mkindahl · 2025-01-17T12:40:57Z

api/hypercore/convert_to_columnstore.md

+|Column| Type                                            | Description                                     |
+|-|-------------------------------------------------|-------------------------------------------------|
+| `convert_to_columnstore` | REGCLASS | The name of the chunk added to the columnstore. |


The result set here does not have a name, so not sure how you want to handle this. We have two different cases: 1) return a table-like result set with zero or more rows, and 2) return a single value of a specific type. This one just returns a regclass with no name.

mkindahl · 2025-01-17T12:42:26Z

api/hypercore/convert_to_columnstore.md

+|----------------------|--|---------|--|----------------------------------------------------------------------------------------------------------------|
+| `chunk`         | REGCLASS | -       |✔| Name of the chunk to add to the columnstore.                                                                   |
+| `if_not_columnstore` | BOOLEAN | `true`  |✖| Set to `false` so this job fails with an error rather than a warning if `chunk` is already in the columnstore. |
+| `recompress`         | BOOLEAN | `false` |✖| Set to `true` to add a chunk that had more data inserted after being added to the columnstore.                 |


This parameter requires a more elaborate description. How do we add that to the reference documentation? Do we have it after the table with the short description?

mkindahl · 2025-01-17T14:34:19Z

api/hypercore/add_columnstore_policy.md

+| `schedule_interval`       |INTERVAL| 12 hours when [chunk_time_interval][chunk_time_interval] >= `1 day` for `hypertable`. Otherwise `chunk_time_interval` / `2`. | ✖        | Set the interval between the finish time of the last execution of this policy and the next start.|
+| `initial_start`     |TIMESTAMPTZ| The interval from the finish time of the last execution to the [next_start][next-start].| ✖| Set the time this job is first run. This is also the time that `next_start` is calculated from.|
+| `timezone`          |TEXT| UTC. However, daylight savings time(DST) changes may shift this alignment. | ✖ | Set to a valid time zone to mitigate DST shifting. If `initial_start` is set, subsequent executions of this policy are aligned on `initial_start`.|
+| `if_not_exists`     |BOOLEAN| `false` | ✖ | Set to `true` so this job fails with a warning rather than an error if a columnstore policy already exists on `hypertable` |


mkindahl · 2025-01-17T14:38:55Z

api/hypercore/add_columnstore_policy.md

+| `created_before` |INTERVAL| NULL | ✖ | Add chunks with a creation time of `now() - created_before` to the columnstore. <br/> `created_before` is <ul><li>Not supported for continuous aggregates.</li><li>Mutually exclusive with `after`.</li></ul> |
+| `schedule_interval`       |INTERVAL| 12 hours when [chunk_time_interval][chunk_time_interval] >= `1 day` for `hypertable`. Otherwise `chunk_time_interval` / `2`. | ✖        | Set the interval between the finish time of the last execution of this policy and the next start.|
+| `initial_start`     |TIMESTAMPTZ| The interval from the finish time of the last execution to the [next_start][next-start].| ✖| Set the time this job is first run. This is also the time that `next_start` is calculated from.|
+| `timezone`          |TEXT| UTC. However, daylight savings time(DST) changes may shift this alignment. | ✖ | Set to a valid time zone to mitigate DST shifting. If `initial_start` is set, subsequent executions of this policy are aligned on `initial_start`.|


This is not required:

Suggested change

| `timezone` |TEXT| UTC. However, daylight savings time(DST) changes may shift this alignment. | ✖ | Set to a valid time zone to mitigate DST shifting. If `initial_start` is set, subsequent executions of this policy are aligned on `initial_start`.|

| `timezone` |TEXT| UTC. However, daylight savings time(DST) changes may shift this alignment. | | Set to a valid time zone to mitigate DST shifting. If `initial_start` is set, subsequent executions of this policy are aligned on `initial_start`.|

mkindahl · 2025-01-17T14:39:30Z

api/hypercore/add_columnstore_policy.md

+| `hypertable`              |REGCLASS| - | ✔ | Name of the hypertable or continuous aggregate to run this [job][job] on.|
+| `after`          |INTERVAL or INTEGER|- | ✖ | Add chunks containing data older than `now - {after}::interval` to the columnstore. <br/> Use an object type that matchs the time column type in `hypertable`: <ul><li><b><code>TIMESTAMP</code>, <code>TIMESTAMPTZ</code>, or <code>DATE</code></b>: use an <code>INTERVAL</code> type.</li><li><b> Integer-based timestamps </b>: set an integer type using the [integer_now_func][set_integer_now_func].</li></ul> `after` is mutually exclusive with `created_before`. |
+| `created_before` |INTERVAL| NULL | ✖ | Add chunks with a creation time of `now() - created_before` to the columnstore. <br/> `created_before` is <ul><li>Not supported for continuous aggregates.</li><li>Mutually exclusive with `after`.</li></ul> |
+| `schedule_interval`       |INTERVAL| 12 hours when [chunk_time_interval][chunk_time_interval] >= `1 day` for `hypertable`. Otherwise `chunk_time_interval` / `2`. | ✖        | Set the interval between the finish time of the last execution of this policy and the next start.|


This is not required:

Suggested change

| `schedule_interval` |INTERVAL| 12 hours when [chunk_time_interval][chunk_time_interval] >= `1 day` for `hypertable`. Otherwise `chunk_time_interval` / `2`. | ✖ | Set the interval between the finish time of the last execution of this policy and the next start.|

| `schedule_interval` |INTERVAL| 12 hours when [chunk_time_interval][chunk_time_interval] >= `1 day` for `hypertable`. Otherwise `chunk_time_interval` / `2`. | | Set the interval between the finish time of the last execution of this policy and the next start.|

chore: space to validate initial build.

7bc034a

billy-the-fish added 5 commits January 9, 2025 12:12

chore: add first hypercore files.

b7e966e

chore: add api ref files and alter_table to the build.

3899a6e

chore: add remove_columnstore_policy to the build.

bd9946c

chore: add convert_to_columnstore to the build.

5ad76e2

chore: add convert_to_rowstore to the build.

993cf07

chore: updates for build.

d1fde35

billy-the-fish added 5 commits January 9, 2025 14:49

chore: updates for build.

f727241

chore: remove links from steps in a list.

67c638f

chore: remove links from steps in a list.

39c8f8c

Merge branch 'latest' into 235-docs-rfc-update-the-api-ref-for-hypers…

193f0d6

…tore-and-compression-for-any-changes-due-to-the-hyperstore

chore: hopefully this will work now.

af34323

billy-the-fish marked this pull request as draft January 9, 2025 15:07

billy-the-fish added 5 commits January 9, 2025 16:27

chore: add deprecation for compression.

a80f250

chore: simplify workflow, keep code for main docs.

c03cae9

chore: updates on review. .

e58a7c9

Merge branch 'latest' into 235-docs-rfc-update-the-api-ref-for-hypers…

338bac1

…tore-and-compression-for-any-changes-due-to-the-hyperstore

chore: updates on review.

1409dfd

billy-the-fish requested review from mkindahl, esclymeTS, philkra and LeeHampton and removed request for esclymeTS January 10, 2025 13:22

billy-the-fish added 2 commits January 13, 2025 09:44

Merge branch 'latest' into 235-docs-rfc-update-the-api-ref-for-hypers…

d8008e0

…tore-and-compression-for-any-changes-due-to-the-hyperstore

Merge branch 'latest' into 235-docs-rfc-update-the-api-ref-for-hypers…

a4bf7f1

…tore-and-compression-for-any-changes-due-to-the-hyperstore

feat: update structure for hypercore

de1c436

billy-the-fish added 2 commits January 15, 2025 15:51

chore: add replacedby link. Has to be hardcoded so will not work unti…

834a406

…l these docs go live.

Merge branch 'latest' into 235-docs-rfc-update-the-api-ref-for-hypers…

e969ec0

…tore-and-compression-for-any-changes-due-to-the-hyperstore

erimatnor reviewed Jan 15, 2025

View reviewed changes

billy-the-fish and others added 4 commits January 15, 2025 16:42

Apply suggestions from code review

fe6d956

Awesome review, thank you. Co-authored-by: Erik Nordström <[email protected]> Signed-off-by: Iain Cox <[email protected]>

chore: update on review.

8c87b3c

chore: add enable_segmentwise_recompression.

25a2bcc

chore: add enable_segmentwise_recompression.

c849517

jonatas reviewed Jan 16, 2025

View reviewed changes

billy-the-fish added 3 commits January 16, 2025 17:09

Merge branch 'latest' into 235-docs-rfc-update-the-api-ref-for-hypers…

be89feb

…tore-and-compression-for-any-changes-due-to-the-hyperstore

Merge branch 'latest' into 235-docs-rfc-update-the-api-ref-for-hypers…

93ebdde

…tore-and-compression-for-any-changes-due-to-the-hyperstore

chore: update deprecation notices.

e4a4a57

chore: update deprecation notices.

8025d11

chore: remove stuff breaking the build.

ce662f7

mkindahl reviewed Jan 17, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hypercore API ref #3699

Hypercore API ref #3699

billy-the-fish commented Jan 9, 2025

github-actions bot commented Jan 9, 2025

github-actions bot commented Jan 9, 2025

github-actions bot commented Jan 9, 2025

philkra commented Jan 15, 2025

erimatnor left a comment

jonatas Jan 16, 2025

github-actions bot commented Jan 17, 2025

github-actions bot commented Jan 17, 2025

mkindahl left a comment

mkindahl Jan 17, 2025

mkindahl Jan 17, 2025

mkindahl Jan 17, 2025

mkindahl Jan 17, 2025

mkindahl Jan 17, 2025

	\| `if_not_exists` \|BOOLEAN\| `false` \| ✖ \| Set to `true` so this job fails with a warning rather than an error if a columnstore policy already exists on `hypertable` \|
	\| `if_not_exists` \|BOOLEAN\| `false` \| \| Set to `true` so this job fails with a warning rather than an error if a columnstore policy already exists on `hypertable` \|

	\| `timezone` \|TEXT\| UTC. However, daylight savings time(DST) changes may shift this alignment. \| ✖ \| Set to a valid time zone to mitigate DST shifting. If `initial_start` is set, subsequent executions of this policy are aligned on `initial_start`.\|
	\| `timezone` \|TEXT\| UTC. However, daylight savings time(DST) changes may shift this alignment. \| \| Set to a valid time zone to mitigate DST shifting. If `initial_start` is set, subsequent executions of this policy are aligned on `initial_start`.\|

	\| `schedule_interval` \|INTERVAL\| 12 hours when [chunk_time_interval][chunk_time_interval] >= `1 day` for `hypertable`. Otherwise `chunk_time_interval` / `2`. \| ✖ \| Set the interval between the finish time of the last execution of this policy and the next start.\|
	\| `schedule_interval` \|INTERVAL\| 12 hours when [chunk_time_interval][chunk_time_interval] >= `1 day` for `hypertable`. Otherwise `chunk_time_interval` / `2`. \| \| Set the interval between the finish time of the last execution of this policy and the next start.\|

Hypercore API ref #3699

Are you sure you want to change the base?

Hypercore API ref #3699

Conversation

billy-the-fish commented Jan 9, 2025

github-actions bot commented Jan 9, 2025

github-actions bot commented Jan 9, 2025

github-actions bot commented Jan 9, 2025

philkra commented Jan 15, 2025

erimatnor left a comment

Choose a reason for hiding this comment

jonatas Jan 16, 2025

Choose a reason for hiding this comment

github-actions bot commented Jan 17, 2025

github-actions bot commented Jan 17, 2025

mkindahl left a comment

Choose a reason for hiding this comment

mkindahl Jan 17, 2025

Choose a reason for hiding this comment

mkindahl Jan 17, 2025

Choose a reason for hiding this comment

mkindahl Jan 17, 2025

Choose a reason for hiding this comment

mkindahl Jan 17, 2025

Choose a reason for hiding this comment

mkindahl Jan 17, 2025

Choose a reason for hiding this comment