Evolve "Features" and "Application Domains" sections

.gitignore

@@ -1,3 +1,4 @@
+__pycache__
 _out/
 .build
 .clone

backlog.md

@@ -7,6 +7,7 @@
 - Rework container/index
 - Update HTTP links, use Sphinx references instead
 - Update remaining links from crate.io to cratedb.com
+- Gallery: https://python.arviz.org/en/stable/examples/
 
 ## Iteration +2
 - Render Jupyter Notebooks?

docs/_include/card/timeseries-datashader.md

@@ -0,0 +1,28 @@
+::::{info-card}
+
+:::{grid-item}
+:columns: auto 9 9 9
+**Display millions of data points using hvPlot, Datashader, and CrateDB**
+
+[HoloViews] and [Datashader] frameworks enable channeling millions of data
+points from your backend systems to the browser's glass.
+
+This notebook plots the venerable NYC Taxi dataset after importing it
+into a CrateDB Cloud database cluster.
+
+🚧 _Please note this notebook is a work in progress._ 🚧
+
+{{ '{}[cloud-datashader-github]'.format(nb_github) }} {{ '{}[cloud-datashader-colab]'.format(nb_colab) }}
+:::
+
+:::{grid-item}
+:columns: 3
+{tags-primary}`Time series visualization`
+
+{tags-secondary}`Python`
+{tags-secondary}`HoloViews`
+{tags-secondary}`hvPlot`
+{tags-secondary}`Datashader`
+:::
+
+::::

docs/_include/card/timeseries-explore.md

@@ -0,0 +1,27 @@
+::::{info-card}
+
+:::{grid-item}
+:columns: auto 9 9 9
+**CrateDB for Time Series Modeling, Exploration, and Visualization**
+
+Access time series data from CrateDB via SQL, load it into pandas DataFrames,
+and visualize it using Plotly.
+
+About advanced time series operations in SQL, like aggregations, window
+functions, interpolation of missing data, common table expressions, moving
+averages, relational JOINs, and the handling of JSON data.
+
+{{ '{}[timeseries-queries-and-visualization-github]'.format(nb_github) }} {{ '{}[timeseries-queries-and-visualization-colab]'.format(nb_colab) }}
+:::
+
+:::{grid-item}
+:columns: 3
+{tags-primary}`Time series visualization`
+
+{tags-secondary}`Python`
+{tags-secondary}`pandas`
+{tags-secondary}`Plotly`
+{tags-secondary}`Dash`
+:::
+
+::::

docs/_include/links.md

@@ -0,0 +1,24 @@
+[cloud-datashader-colab]: https://colab.research.google.com/github/crate/cratedb-examples/blob/amo/cloud-datashader/topic/timeseries/explore/cloud-datashader.ipynb
+[cloud-datashader-github]: https://github.com/crate/cratedb-examples/blob/amo/cloud-datashader/topic/timeseries/explore/cloud-datashader.ipynb
+[Datashader]: https://datashader.org/
+[Dynamic Database Schemas]: https://cratedb.com/product/features/dynamic-schemas
+[Geospatial Data Model]: https://cratedb.com/data-model/geospatial
+[Geospatial Database]: https://cratedb.com/geospatial-spatial-database
+[HoloViews]: https://www.holoviews.org/
+[Indexing, Columnar Storage, and Aggregations]: https://cratedb.com/product/features/indexing-columnar-storage-aggregations
+[JSON Database]: https://cratedb.com/solutions/json-database
+[LangChain and CrateDB: Code Examples]: https://github.com/crate/cratedb-examples/tree/main/topic/machine-learning/llm-langchain
+[langchain-similarity-binder]: https://mybinder.org/v2/gh/crate/cratedb-examples/main?labpath=topic%2Fmachine-learning%2Fllm-langchain%2Fvector_search.ipynb
+[langchain-similarity-colab]: https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/machine-learning/llm-langchain/vector_search.ipynb
+[langchain-similarity-github]: https://github.com/crate/cratedb-examples/blob/main/topic/machine-learning/llm-langchain/vector_search.ipynb
+[langchain-rag-sql-binder]: https://mybinder.org/v2/gh/crate/cratedb-examples/main?labpath=topic%2Fmachine-learning%2Fllm-langchain%2Fcratedb-vectorstore-rag-openai-sql.ipynb
+[langchain-rag-sql-colab]: https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/machine-learning/llm-langchain/cratedb-vectorstore-rag-openai-sql.ipynb
+[langchain-rag-sql-github]: https://github.com/crate/cratedb-examples/blob/main/topic/machine-learning/llm-langchain/cratedb-vectorstore-rag-openai-sql.ipynb
+[Multi-model Database]: https://cratedb.com/solutions/multi-model-database
+[Nested Data Structure]: https://cratedb.com/product/features/nested-data-structure
+[query DSL based on JSON]: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html
+[Relational Database]: https://cratedb.com/solutions/relational-database
+[timeseries-queries-and-visualization-colab]: https://colab.research.google.com/github/crate/cratedb-examples/blob/main/topic/timeseries/timeseries-queries-and-visualization.ipynb
+[timeseries-queries-and-visualization-github]: https://github.com/crate/cratedb-examples/blob/main/topic/timeseries/timeseries-queries-and-visualization.ipynb
+[Vector Database (Product)]: https://cratedb.com/solutions/vector-database
+[Vector Database]: https://en.wikipedia.org/wiki/Vector_database

docs/_include/styles.html

@@ -0,0 +1,49 @@
+<!--
+Custom styles for efficient tile layouts using card elements.
+
+TODO: Upstream to crate-docs-theme.
+-->
+<style>
+/* General */
+/*
+.sd-card-body {
+    line-height: 1.1em;
+}
+*/
+.sd-card-footer {
+    font-size: small;
+}
+
+/* No margins for images in tight layouts, e.g. badges  */
+/* Needed for domain/timeseries/advanced.md */
+.wrapper-content-right img {
+  margin-bottom: 0 !important;
+}
+
+
+/* Document Store */
+.wrapper-content-right ul {
+  margin-left: 0;
+}
+.rubric-slimmer p.rubric {
+  margin-bottom: 0.25em;
+}
+.rubric-slim p.rubric {
+  margin-bottom: 0;
+}
+.title-slim .sd-col > * {
+  margin-top: 0;
+  margin-bottom: 0;
+}
+.no-margin > * {
+  margin-top: 0 !important;
+  margin-bottom: 0 !important;;
+}
+
+
+/* Cards with Links */
+.sd-hide-link-text {
+  height: 0;
+}
+
+</style>

docs/admin/clustering/index.rst

@@ -1,4 +1,4 @@
-.. _clustering:
+.. _admin-clustering:
 
 ==========
 Clustering

docs/admin/sharding-partitioning.rst

@@ -64,7 +64,7 @@ partition as a set of shards. For each partition, the number of shards defined
 by ``CLUSTERED INTO x SHARDS`` are created, when a first record with a specific 
 ``partition key`` is inserted.
 
-In the following example - which represents a very simple time-series use-case 
+In the following example - which represents a very simple time series use-case
 - we added another column ``part`` that automatically generates the current 
 month upon insertion from the ``ts`` column. The ``part`` column is further used 
 as the ``partition key``.
@@ -111,7 +111,7 @@ cluster.
     Over-sharding and over-partitioning are common flaws leading to an overall 
     poor performance.
 
-    **As a rule of thumb, a single shard should hold somewhere between 5 - 100 
+    **As a rule of thumb, a single shard should hold somewhere between 5 - 50
     GB of data.**
 
     To avoid oversharding, CrateDB by default limits the number of shards per 
@@ -129,15 +129,15 @@ benchmarks across various strategies. The following steps provide a general guid
 - Calculate the throughput
 
 Then, to calculate the number of shards, you should consider that the size of each
-shard should roughly be between 5 - 100 GB, and that each node can only manage
+shard should roughly be between 5 - 50 GB, and that each node can only manage
 up to 1000 shards.
 
-Time-series example
+Time series example
 -------------------
 
 To illustrate the steps above, let's use them on behalf of an example. Imagine
 you want to create a *partitioned table* on a *three-node cluster* to store
-time-series data with the following assumptions:
+time series data with the following assumptions:
 
 - Inserts: 1.000 records/s
 - Record size: 128 byte/record
@@ -146,12 +146,12 @@ time-series data with the following assumptions:
 Given the daily throughput is around 10 GB/day, the monthly throughput is 30 times
 that (~ 300 GB). The partition column can be day, week, month, quarter, etc. So,
 assuming a monthly partition, the next step is to calculate the number of shards
-with the **shard size recommendation** (5 - 100 GB) and the **number of nodes** in
+with the **shard size recommendation** (5 - 50 GB) and the **number of nodes** in
 the cluster in mind.
 
-With three shards, each shard will hold 100 GB (300 GB / 3 shards), which is too
-close to the upper limit. With six shards, each shard will manage 50 GB
-(300 GB / 6 shards) of data, which is closer to the recommended size range (5 - 100 GB).
+With three shards, each shard would hold 100 GB (300 GB / 3 shards), which is above
+the upper limit. With six shards, each shard will manage 50 GB (300 GB / 6 shards)
+of data, which is right on the spot.
 
 .. code-block:: psql
 

docs/admin/troubleshooting/index.md

@@ -72,6 +72,12 @@ infrastructure operations. For example:
 - Clean up stale node data.
 :::
 
+:::{card} {material-outlined}`wysiwyg;1.6em` About `crash`
+:link: https://cratedb.com/docs/crate/crash/en/latest/troubleshooting.html
+Troubleshooting the CLI program `crash`.
+:::
+
+
 :::{note}
 You can find a lot of troubleshooting guides that explain how to perform
 diagnostics on Java applications.

docs/admin/upgrade/index.md

@@ -1,3 +1,4 @@
+(upgrading)=
 # Upgrading
 
 Guidelines about upgrading CrateDB clusters.

docs/conf.py

@@ -34,9 +34,30 @@
 ]
 
 # Configure intersphinx.
+if "sphinx.ext.intersphinx" not in extensions:
+    extensions += ["sphinx.ext.intersphinx"]
+
+if "intersphinx_mapping" not in globals():
+    intersphinx_mapping = {}
+
 intersphinx_mapping.update({
     'ctk': ('https://cratedb-toolkit.readthedocs.io/', None),
     'matplotlib': ('https://matplotlib.org/stable/', None),
     'pandas': ('https://pandas.pydata.org/pandas-docs/stable/', None),
     'numpy': ('https://numpy.org/doc/stable/', None),
     })
+
+
+# Configure substitutions.
+if "myst_substitutions" not in globals():
+    myst_substitutions = {}
+
+myst_substitutions.update({
+    "nb_colab": "[![Notebook on Colab](https://img.shields.io/badge/Open-Notebook%20on%20Colab-blue?logo=Google%20Colab)]",
+    "nb_binder": "[![Notebook on Binder](https://img.shields.io/badge/Open-Notebook%20on%20Binder-lightblue?logo=binder)]",
+    "nb_github": "[![Notebook on GitHub](https://img.shields.io/badge/Open-Notebook%20on%20GitHub-darkgreen?logo=GitHub)]",
+    "readme_github": "[![README](https://img.shields.io/badge/Open-README-darkblue?logo=GitHub)]",
+    "blog": "[![Blog](https://img.shields.io/badge/Open-Blog-darkblue?logo=Markdown)]",
+    "tutorial": "[![Navigate to Tutorial](https://img.shields.io/badge/Navigate%20to-Tutorial-darkcyan?logo=Markdown)]",
+    "readmore": "[![Read More](https://img.shields.io/badge/Read-More-darkyellow?logo=Markdown)]",
+})

docs/domain/analytics/index.md

@@ -0,0 +1,97 @@
+(analytics)=
+# Raw-Data Analytics
+
+**CrateDB provides real-time analytics on raw data stored for the long term**
+
+In all domains of real-time analytics where you absolutely must have access to all
+the records, and can't live with any down-sampled variants, because records are
+unique, and need to be accounted for within your analytics queries.
+
+If you find yourself in such a situation, you need a storage system which
+manages all the high-volume data in its hot zone, to be available right on
+your fingertips, for live querying. Batch jobs to roll up raw data into
+analytical results are not an option, because users' queries are too
+individual, so you need to run them on real data in real time.
+
+With CrateDB, compatible to PostgreSQL, you can do all of that using plain SQL.
+Other than integrating well with commodity systems using standard database
+access interfaces like ODBC or JDBC, it provides a proprietary HTTP interface
+on top.
+
+:Tags:
+  {tags-primary}`Analytics`
+  {tags-primary}`Long Term Storage`
+
+:Related:
+  [](#timeseries) •
+  [](#timeseries-longterm) •
+  [](#machine-learning)
+
+:Product:
+  [Real-time Analytics Database]
+
+
+(bitmovin)=
+## Bitmovin Insights
+
+Multi tenant data analytics on top of billions of records.
+
+> CrateDB enables use cases we couldn't satisfy with other
+database systems, also with databases which are even stronger
+focused on the time series domain.
+>
+> CrateDB is not your normal database!
+>
+> <small>-- Daniel Hölbling-Inzko, Director of Engineering Analytics, Bitmovin</small>
+
+:Industry:
+  {tags-secondary}`Broadcasting`
+  {tags-secondary}`Media Transcoding`
+  {tags-secondary}`Streaming Media`
+
+:Tags:
+  {tags-primary}`Event Tracking`
+  {tags-primary}`Real-Time Analytics`
+  {tags-primary}`Multi Tenancy`
+  {tags-primary}`SaaS`
+
+:Related:
+  [CrateDB provides the backbone of Bitmovin's real-time video analytics platform] \
+  [How Bitmovin uses CrateDB to monitor the biggest live video events]
+
+
+::::{info-card}
+
+:::{grid-item}
+:columns: 8
+
+{material-outlined}`analytics;2em` &nbsp; **Bitmovin: Real-Time Analytics**
+
+Bitmovin, as a leader in video codec algorithms and as a web-based video
+stream broadcasting provider, produces billions of rows of data and stores
+them in CrateDB, allowing their customers to do analytics on it.
+
+One of their product's subsystems, a video analytics component, required to
+serve real-time analytics on very large and fast-moving data, so they needed
+to find a performing database at the right cost.
+
+- [Bitmovin: Improving the Streaming Experience with Real-Time Analytics]
+
+The use-case of Bitmovin illustrates why traditional databases weren't capable
+to deal with so many data records and keep them all available for querying in
+real time.
+:::
+
+:::{grid-item}
+:columns: 4
+
+<iframe width="240" src="https://www.youtube-nocookie.com/embed/4BPApD0Piyc?si=J0w5yG56Ld4fIXfm" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
+:::
+
+::::
+
+
+[Bitmovin: Improving the Streaming Experience with Real-Time Analytics]: https://youtu.be/4BPApD0Piyc?feature=shared
+[CrateDB provides the backbone of Bitmovin's real-time video analytics platform]: https://cratedb.com/customers/bitmovin
+[How Bitmovin uses CrateDB to monitor the biggest live video events]: https://youtu.be/IR6hokaYv5g?feature=shared
+[Real-time Analytics Database]: https://cratedb.com/solutions/real-time-analytics-database