Star Tree Search changes related to new Aggregations supported

_field-types/supported-field-types/star-tree.md

@@ -37,7 +37,8 @@ PUT logs
   "settings": {
     "index.number_of_shards": 1,
     "index.number_of_replicas": 0,
-    "index.composite_index": true
+    "index.composite_index": true,
+    "index.append_only.enabled": true
   },
   "mappings": {
     "composite": {
@@ -54,6 +55,9 @@ PUT logs
             },
             {
               "name": "port"
+            },
+            {
+              "name": "method"
             }
           ],
           "metrics": [
@@ -89,6 +93,9 @@ PUT logs
       "request_size": {
         "type": "integer"
       },
+      "method" : {
+        "type": "keyword"
+      },
       "latency": {
         "type": "scaled_float",
         "scaling_factor": 10
@@ -118,9 +125,12 @@ When using the `ordered_dimesions` parameter, follow these best practices:
 
 - The order of dimensions matters. You can define the dimensions ordered from the highest cardinality to the lowest cardinality for efficient storage and query pruning. 
 - Avoid using high-cardinality fields as dimensions. High-cardinality fields adversely affect storage space, indexing throughput, and query performance.
-- Currently, fields supported by the `ordered_dimensions` parameter are all [numeric field types]({{site.url}}{{site.baseurl}}/field-types/supported-field-types/numeric/), with the exception of `unsigned_long`. For more information, see [GitHub issue #15231](https://github.com/opensearch-project/OpenSearch/issues/15231). 
-- Support for other field types, such as `keyword` and `ip`, will be added in future versions. For more information, see [GitHub issue #16232](https://github.com/opensearch-project/OpenSearch/issues/16232).
 - A minimum of `2` and a maximum of `10` dimensions are supported per star-tree index.
+- The `ordered_dimensions` parameter supports the following field types:
+  - All numeric field types excluding `unsigned_long` and `scaled_float`.
+  - `keyword` 
+  - `object`
+- Support for other field types, such as `date` and `ip`, will be added in future versions. For more information, see [GitHub issue #13875](https://github.com/opensearch-project/OpenSearch/issues/13875).
 
 The `ordered_dimensions` parameter supports the following property.
 

_search-plugins/star-tree-index.md

@@ -26,7 +26,7 @@
 
 Star-tree indexes have the following limitations:
 
-- A star-tree index should only be enabled on indexes whose data is not updated or deleted because updates and deletions are not accounted for in a star-tree index.
+- A star-tree index should only be enabled on indexes whose data is not updated or deleted because updates and deletions are not accounted for in a star-tree index. To enforce this policy and use star-tree indexes, set the `index.append_only.enabled` setting to true.
 - A star-tree index can be used for aggregation queries only if the queried fields are a subset of the star-tree's dimensions and the aggregated fields are a subset of the star-tree's metrics.
 - After a star-tree index is enabled, it cannot be disabled. In order to disable a star-tree index, the data in the index must be reindexed without the star-tree mapping. Furthermore, changing a star-tree configuration will also require a reindex operation.
 - [Multi-values/array values]({{site.url}}{{site.baseurl}}/field-types/supported-field-types/index/#arrays) are not supported.
@@ -68,6 +68,7 @@
 - Set the feature flag `opensearch.experimental.feature.composite_index.star_tree.enabled` to `true`. For more information about enabling and disabling feature flags, see [Enabling experimental features]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/experimental/).
 - Set the `indices.composite_index.star_tree.enabled` setting to `true`. For instructions on how to configure OpenSearch, see [Configuring settings]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/#static-settings).
 - Set the `index.composite_index` index setting to `true` during index creation.
+- Set the `index.append_only.enabled` index setting to `true` during index creation.
 - Ensure that the `doc_values` parameter is enabled for the `dimensions` and `metrics` fields used in your star-tree mapping.
 
 
@@ -81,7 +82,8 @@
   "settings": {
     "index.number_of_shards": 1,
     "index.number_of_replicas": 0,
-    "index.composite_index": true
+    "index.composite_index": true,
+    "index.append_only.enabled": true
   },
   "mappings": {
     "composite": {
@@ -94,6 +96,9 @@
             },
             {
               "name": "port"
+            },
+            {
+              "name": "method"
             }
           ],
           "metrics": [
@@ -123,6 +128,9 @@
       "size": {
         "type": "integer"
       },
+      "method" : {
+        "type": "keyword"
+      },
       "latency": {
         "type": "scaled_float",
         "scaling_factor": 10
@@ -140,14 +148,18 @@
 
 ### Supported queries
 
-The following queries are supported as of OpenSearch 2.18:
+The following queries are supported as of OpenSearch 2.19:
 
 - [Term query]({{site.url}}{{site.baseurl}}/query-dsl/term/term/)
+- [Terms query]({{site.url}}{{site.baseurl}}/query-dsl/term/terms/)
 - [Match all docs query]({{site.url}}{{site.baseurl}}/query-dsl/match-all/)
+- [Range query]({{site.url}}{{site.baseurl}}/query-dsl/term/range/)
 
-To use a query with a star-tree index, the query's fields must be present in the `ordered_dimensions` section of the star-tree configuration. Queries must also be paired with a supported aggregation. 
+To use a query in supported aggregations with a star-tree index, the query's fields must be present in the `ordered_dimensions` section of the star-tree configuration. Queries without aggregaions are not supported, they must be paired with a supported aggregation. 
 
 ### Supported aggregations
+
+#### Metric aggregations
 
 The following metric aggregations are supported as of OpenSearch 2.18:
 - [Sum]({{site.url}}{{site.baseurl}}/aggregations/metric/sum/)
@@ -156,12 +168,12 @@
 - [Value count]({{site.url}}{{site.baseurl}}/aggregations/metric/value-count/)
 - [Average]({{site.url}}{{site.baseurl}}/aggregations/metric/average/)
 
-To use aggregations:
+To use aggregations searchable via star-tree:
 
 - The fields must be present in the `metrics` section of the star-tree configuration.
 - The metric aggregation type must be part of the `stats` parameter.
 
-### Aggregation example
+##### Example
 
 The following example gets the sum of all the values in the `size` field for all error logs with `status=500`, using the [example mapping](#example-mapping):
 
@@ -185,6 +197,48 @@
 
 Using a star-tree index, the result will be retrieved from a single aggregated document as it traverses the `status=500` node, as opposed to scanning through all of the matching documents. This results in lower query latency.
 
+### Date histogram with metric aggregations
+
+You can use [date histograms]({{site.url}}{{site.baseurl}}/aggregations/bucket/date-histogram/) on calendar intervals with metric sub-aggregations.
+
+To use date histogram aggregations and make then searchable in the star-tree index, use the following steps:
+
+- The calendar intervals in star-tree mapping configuration should have either the request calendar field or a lower granularity calendar field. For example, `month` can be resolved by the star-tree from `day` field as well if present in star-tree mapping.
+- A metric sub-aggregation must be part of the aggregation request.
+
+#### Example
+
+The following example gets the sum of all the values in the `size` field aggregated for each calendar month, for all error logs with `status=500`:
+
+```json
+POST /logs/_search
+{
+  "query": {
+    "term": {
+      "status": "500"
+    }
+  },
+  "size": 0,
+  "aggs": {
+    "by_hour": {
+      "date_histogram": {
+        "field": "@timestamp",
+        "calendar_interval": "month"
+      },
+      "aggs": {
+        "sum_size": {
+          "sum": {
+            "field": "size"
+          }
+        }
+      }
+    }
+  }
+}
+```
+
+
+
 ## Using queries without a star-tree index
 
 Set the `indices.composite_index.star_tree.enabled` setting to `false` to run queries without using a star-tree index.