Skip to content

Commit

Permalink
Introduce release tracks. Rename Versionless to Latest
Browse files Browse the repository at this point in the history
  • Loading branch information
jtcohen6 committed Dec 2, 2024
1 parent 5343a8a commit bb6224e
Show file tree
Hide file tree
Showing 62 changed files with 209 additions and 143 deletions.
2 changes: 1 addition & 1 deletion website/blog/2021-11-23-how-to-upgrade-dbt-versions.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ It's been a few years since dbt-core turned 1.0! Since then, we've committed to

In 2024, we're taking this promise further by:
- Stabilizing interfaces for everyone — adapter maintainers, metadata consumers, and (of course) people writing dbt code everywhere — as discussed in [our November 2023 roadmap update](https://github.com/dbt-labs/dbt-core/blob/main/docs/roadmap/2023-11-dbt-tng.md).
- Introducing **Versionless** in dbt Cloud. No more manual upgrades and no more need for _a second sandbox project_ just to try out new features in development. For more details, refer to [Upgrade Core version in Cloud](/docs/dbt-versions/upgrade-dbt-version-in-cloud).
- Introducing **Latest** release track in dbt Cloud. No more manual upgrades and no more need for _a second sandbox project_ just to try out new features in development. For more details, refer to [Upgrade Core version in Cloud](/docs/dbt-versions/upgrade-dbt-version-in-cloud).

We're leaving the rest of this post as is, so we can all remember how it used to be. Enjoy a stroll down memory lane.

Expand Down
2 changes: 1 addition & 1 deletion website/blog/2024-04-22-extended-attributes.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ All you need to do is configure an environment as staging and enable the **Defer

## Upgrading on a curve

Lastly, let’s consider a more specialized use case. Imagine we have a "tiger team" (consisting of a lone analytics engineer named Dave) tasked with upgrading from dbt version 1.6 to the new **Versionless** setting, to take advantage of added stability and feature access. We want to keep the rest of the data team being productive in dbt 1.6 for the time being, while enabling Dave to upgrade and do his work in the new versionless mode.
Lastly, let’s consider a more specialized use case. Imagine we have a "tiger team" (consisting of a lone analytics engineer named Dave) tasked with upgrading from dbt version 1.6 to the new **Latest** setting, to take advantage of added stability and feature access. We want to keep the rest of the data team being productive in dbt 1.6 for the time being, while enabling Dave to upgrade and do his work in the new Latest mode.

### Development environment

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "How we're making sure you can confidently go \"Versionless\" in dbt Cloud"
title: "How we're making sure you can confidently go \"Latest\" in dbt Cloud"
description: "Over the past 6 months, we've laid a stable foundation for continuously improving dbt."
slug: latest-dbt-stability

Expand All @@ -16,19 +16,19 @@ As long as dbt Cloud has existed, it has required users to select a version of d

However, this came at a cost. While bumping a project's dbt version *appeared* as simple as selecting from a dropdown, there was real effort required to test the compatibility of the new version against existing projects, package dependencies, and adapters. On the other hand, putting this off meant foregoing access to new features and bug fixes in dbt.

But no more. Today, we're ready to announce the general availability of a new option in dbt Cloud: [**"Versionless."**](https://docs.getdbt.com/docs/dbt-versions/upgrade-dbt-version-in-cloud#versionless)
But no more. Today, we're ready to announce the general availability of a new option in dbt Cloud: [**"Latest."**](https://docs.getdbt.com/docs/dbt-versions/upgrade-dbt-version-in-cloud#latest)

<!--truncate-->

For customers, this means less maintenance overhead, faster access to bug fixes and features, and more time to focus on what matters most: building trusted data products. This will be our stable foundation for improvement and innovation in dbt Cloud.

But we wanted to go a step beyond just making this option available to you. In this blog post, we aim to shed a little light on the extensive work we've done to ensure that using "Versionless" is a stable, reliable experience for the thousands of customers who rely daily on dbt Cloud.
But we wanted to go a step beyond just making this option available to you. In this blog post, we aim to shed a little light on the extensive work we've done to ensure that using "Latest" is a stable, reliable experience for the thousands of customers who rely daily on dbt Cloud.

## How we safely deploy dbt upgrades to Cloud

We've put in place a rigorous, best-in-class suite of tests and control mechanisms to ensure that all changes to dbt under the hood are fully vetted before they're deployed to customers of dbt Cloud.

This pipeline has in fact been in place since January! It's how we've already been shipping continuous changes to the hundreds of customers who've selected "Versionless" while it's been in Beta and Preview. In that time, this process has enabled us to prevent multiple regressions before they were rolled out to any customers.
This pipeline has in fact been in place since January! It's how we've already been shipping continuous changes to the hundreds of customers who've selected "Latest" while it's been in Beta and Preview. In that time, this process has enabled us to prevent multiple regressions before they were rolled out to any customers.

We're very confident in the robustness of this process**. We also know that we'll need to continue building trust with time.** We're sharing details about this work in the spirit of transparency and to build that trust.

Expand Down Expand Up @@ -82,9 +82,9 @@ All incidents are retrospected to make sure we not only identify and fix the roo

:::

The outcome of this process is that, when you select "Versionless" in dbt Cloud, the time between an improvement being made to dbt Core and you *safely* getting access to it in your projects is a matter of days — rather than months of waiting for the next dbt Core release, on top of any additional time it may have taken to actually carry out the upgrade.
The outcome of this process is that, when you select "Latest" in dbt Cloud, the time between an improvement being made to dbt Core and you *safely* getting access to it in your projects is a matter of days — rather than months of waiting for the next dbt Core release, on top of any additional time it may have taken to actually carry out the upgrade.

We’re pleased to say that since the beta launch of “Versionless” in dbt Cloud in March, **we have not had any functional regressions reach customers**, while we’ve also been shipping multiple improvements to dbt functionality every day. This is a foundation that we aim to build on for the foreseeable future.
We’re pleased to say that since the beta launch of "Latest" in dbt Cloud in March, **we have not had any functional regressions reach customers**, while we’ve also been shipping multiple improvements to dbt functionality every day. This is a foundation that we aim to build on for the foreseeable future.

## Stability as a feature

Expand All @@ -98,7 +98,7 @@ The adapter interface — i.e. how dbt Core actually connects to a third-party d

To solve that, we've released a new set of interfaces that are entirely independent of the `dbt-core` library: [`dbt-adapters==1.0.0`](https://github.com/dbt-labs/dbt-adapters). From now on, any changes to `dbt-adapters` will be backward and forward-compatible. This also decouples adapter maintenance from the regular release cadence of dbt Core — meaning maintainers get full control over when they ship implementations of new adapter-powered features.

Note that adapters running in dbt Cloud **must** be [migrated to the new decoupled architecture](https://github.com/dbt-labs/dbt-adapters/discussions/87) as a baseline in order to support the new "Versionless" option.
Note that adapters running in dbt Cloud **must** be [migrated to the new decoupled architecture](https://github.com/dbt-labs/dbt-adapters/discussions/87) as a baseline in order to support the new "Latest" option.

### Managing behavior changes: stability as a feature

Expand All @@ -118,7 +118,7 @@ We’ve now [formalized our development best practices](https://github.com/dbt-l

In conclusion, we’re putting a lot of new muscle behind our commitments to dbt Cloud customers, the dbt Community, and the broader ecosystem:

- **Continuous updates**: "Versionless" dbt Cloud simplifies the update process, ensuring you always have the latest features and bug fixes without the maintenance overhead.
- **Continuous updates**: "Latest" dbt Cloud simplifies the update process, ensuring you always have the latest features and bug fixes without the maintenance overhead.
- **A rigorous new testing and deployment process**: Our new testing pipeline ensures that every update is carefully vetted against documented interfaces, Cloud-supported adapters, and popular packages before it reaches you. This process minimizes the risk of regressions — and has now been successful at entirely preventing them for hundreds of customers over multiple months.
- **A commitment to stability**: We’ve reworked our approaches to adapter interfaces, behaviour change management, and metadata artifacts to give you more stability and control.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,6 @@ Traditionally dbt is the T in ELT (dbt overview [here](https://docs.getdbt.com/t

In order to get this functionality shipped quickly, EQT opened a pull request, Snowflake helped with some problems we had with CI and a member of dbt Labs helped write the tests and merge the code in!

dbt now features this functionality in dbt 1.8+ or the “Versionless” option of dbt Cloud (dbt overview [here](/docs/dbt-versions/upgrade-dbt-version-in-cloud#versionless)).
dbt now features this functionality in dbt 1.8+ or the "Latest" option of dbt Cloud (dbt overview [here](/docs/dbt-versions/upgrade-dbt-version-in-cloud#latest)).

dbt Labs staff and community members would love to chat more about it in the [#db-snowflake](https://getdbt.slack.com/archives/CJN7XRF1B) slack channel.
2 changes: 1 addition & 1 deletion website/dbt-versions.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
exports.versions = [
{
version: "1.10",
customDisplay: "Cloud (Versionless)",
customDisplay: "Cloud (Latest)",
},
{
version: "1.9",
Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/build/incremental-microbatch.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ id: "incremental-microbatch"

:::info Microbatch

The new `microbatch` strategy is available in beta for [dbt Cloud Versionless](/docs/dbt-versions/upgrade-dbt-version-in-cloud#versionless) and dbt Core v1.9.
The new `microbatch` strategy is available in beta for [dbt Cloud "Latest"](/docs/dbt-versions/cloud-release-tracks) and dbt Core v1.9.

Check warning on line 11 in website/docs/docs/build/incremental-microbatch.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/docs/build/incremental-microbatch.md#L11

[custom.Typos] Oops there's a typo -- did you really mean 'v1.9'?
Raw output
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'v1.9'? ", "location": {"path": "website/docs/docs/build/incremental-microbatch.md", "range": {"start": {"line": 11, "column": 131}}}, "severity": "WARNING"}

If you use a custom microbatch macro, set a [distinct behavior flag](/reference/global-configs/behavior-changes#custom-microbatch-strategy) in your `dbt_project.yml` to enable batched execution. If you don't have a custom microbatch macro, you don't need to set this flag as dbt will handle microbatching automatically for any model using the [microbatch strategy](#how-microbatch-compares-to-other-incremental-strategies).

Expand Down
8 changes: 4 additions & 4 deletions website/docs/docs/build/metricflow-time-spine.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ tags: [Metrics, Semantic Layer]
---
<VersionBlock firstVersion="1.9">

<!-- this whole section is for 1.9 and higher + Versionless -->
<!-- this whole section is for 1.9 and higher + Release Tracks -->

It's common in analytics engineering to have a date dimension or "time spine" table as a base table for different types of time-based joins and aggregations. The structure of this table is typically a base column of daily or hourly dates, with additional columns for other time grains, like fiscal quarters, defined based on the base column. You can join other tables to the time spine on the base column to calculate metrics like revenue at a point in time, or to aggregate to a specific time grain.

Expand Down Expand Up @@ -108,7 +108,7 @@ models:
- It needs to reference a column defined under the `columns` key, in this case, `date_hour` and `date_day`, respectively.
- It sets the granularity at the column-level using the `granularity` key, in this case, `hour` and `day`, respectively.
- MetricFlow will use the `standard_granularity_column` as the join key when joining the time spine table to another source table.
- [The `custom_granularities` field](#custom-calendar), (available in Versionless and dbt v1.9 and higher) lets you specify non-standard time periods like `fiscal_year` or `retail_month` that your organization may use.
- [The `custom_granularities` field](#custom-calendar), (available in dbt Cloud Latest and dbt Core v1.9 and higher) lets you specify non-standard time periods like `fiscal_year` or `retail_month` that your organization may use.

Check warning on line 111 in website/docs/docs/build/metricflow-time-spine.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/docs/build/metricflow-time-spine.md#L111

[custom.Typos] Oops there's a typo -- did you really mean 'v1.9'?
Raw output
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'v1.9'? ", "location": {"path": "website/docs/docs/build/metricflow-time-spine.md", "range": {"start": {"line": 111, "column": 101}}}, "severity": "WARNING"}

For an example project, refer to our [Jaffle shop](https://github.com/dbt-labs/jaffle-sl-template/blob/main/models/marts/_models.yml) example.

Expand Down Expand Up @@ -310,9 +310,9 @@ You only need to include the `date_day` column in the table. MetricFlow can hand

<VersionBlock lastVersion="1.8">

The ability to configure custom calendars, such as a fiscal calendar, is available in [dbt Cloud Versionless](/docs/dbt-versions/versionless-cloud) or dbt Core [v1.9 and higher](/docs/dbt-versions/core).
The ability to configure custom calendars, such as a fiscal calendar, is available in [dbt Cloud Latest](/docs/dbt-versions/cloud-release-tracks) or dbt Core [v1.9 and higher](/docs/dbt-versions/core).

To access this feature, [upgrade to Versionless](/docs/dbt-versions/upgrade-dbt-version-in-cloud#versionless) or your dbt Core version to v1.9 or higher.
To access this feature, [upgrade to dbt Cloud Latest](/docs/dbt-versions/cloud-release-tracks) or your dbt Core version to v1.9 or higher.

Check warning on line 315 in website/docs/docs/build/metricflow-time-spine.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/docs/build/metricflow-time-spine.md#L315

[custom.Typos] Oops there's a typo -- did you really mean 'v1.9'?
Raw output
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'v1.9'? ", "location": {"path": "website/docs/docs/build/metricflow-time-spine.md", "range": {"start": {"line": 315, "column": 124}}}, "severity": "WARNING"}

</VersionBlock>

Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/build/metrics-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ import SLCourses from '/snippets/_sl-course.md';
<VersionBlock lastVersion="1.8">
Default time granularity for metrics is useful if your time dimension has a very fine grain, like second or hour, but you typically query metrics rolled up at a coarser grain.
To set the default time granularity for metrics, you need to be on dbt Cloud Versionless or dbt v1.9 and higher.
Default time granularity for metrics is available now in dbt Cloud Latest, and it will be available in dbt Core v1.9+.

Check warning on line 98 in website/docs/docs/build/metrics-overview.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/docs/build/metrics-overview.md#L98

[custom.Typos] Oops there's a typo -- did you really mean 'v1.9+'?
Raw output
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'v1.9+'? ", "location": {"path": "website/docs/docs/build/metrics-overview.md", "range": {"start": {"line": 98, "column": 113}}}, "severity": "WARNING"}
</VersionBlock>
Expand Down
8 changes: 3 additions & 5 deletions website/docs/docs/build/snapshots.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ This order is now in the "shipped" state, but we've lost the information about w
<VersionBlock lastVersion="1.8" >

- To configure snapshots in versions 1.8 and earlier, refer to [Configure snapshots in versions 1.8 and earlier](#configure-snapshots-in-versions-18-and-earlier). These versions use an older syntax where snapshots are defined within a snapshot block in a `.sql` file, typically located in your `snapshots` directory.
- Note that defining multiple resources in a single file can significantly slow down parsing and compilation. For faster and more efficient management, consider the updated snapshot YAML syntax, [available in Versionless](/docs/dbt-versions/versionless-cloud) or [dbt Core v1.9 and later](/docs/dbt-versions/core).
- Note that defining multiple resources in a single file can significantly slow down parsing and compilation. For faster and more efficient management, consider the updated snapshot YAML syntax, [available now in dbt Cloud Latest](/docs/dbt-versions/cloud-release-tracks) and landing in [dbt Core v1.9](/docs/dbt-versions/core).


</VersionBlock>
Expand Down Expand Up @@ -402,9 +402,7 @@ snapshots:

Snapshot <Term id="table">tables</Term> will be created as a clone of your source dataset, plus some additional meta-fields*.

Starting in 1.9 or with [dbt Cloud Versionless](/docs/dbt-versions/upgrade-dbt-version-in-cloud#versionless):
- These column names can be customized to your team or organizational conventions using the [`snapshot_meta_column_names`](/reference/resource-configs/snapshot_meta_column_names) config.
- Use the [`dbt_valid_to_current` config](/reference/resource-configs/dbt_valid_to_current) to set a custom indicator for the value of `dbt_valid_to` in current snapshot records (like a future date such as `9999-12-31`). By default, this value is `NULL`. When set, dbt will use this specified value instead of `NULL` for `dbt_valid_to` for current records in the snapshot table.
Starting in 1.9 (or available sooner in [dbt Cloud "Latest"](/docs/dbt-versions/cloud-release-tracks)), these column names can be customized to your team or organizational conventions via the [`snapshot_meta_column_names`](/reference/resource-configs/snapshot_meta_column_names) config.

| Field | Meaning | Usage |
| -------------- | ------- | ----- |
Expand Down Expand Up @@ -495,7 +493,7 @@ To configure snapshots in versions 1.9 and later, refer to [Configuring snapshot

- In dbt versions 1.8 and earlier, snapshots are `select` statements, defined within a snapshot block in a `.sql` file (typically in your `snapshots` directory). You'll also need to configure your snapshot to tell dbt how to detect record changes.
- The earlier dbt versions use an older syntax that allows for defining multiple resources in a single file. This syntax can significantly slow down parsing and compilation.
- For faster and more efficient management, consider[ upgrading to Versionless](/docs/dbt-versions/versionless-cloud) or the [latest version of dbt Core](/docs/dbt-versions/core), which introduces an updated snapshot configuration syntax that optimizes performance.
- For faster and more efficient management, consider [upgrading to dbt Cloud Latest](/docs/dbt-versions/cloud-release-tracks) for early access to this functionality, which will be landing in dbt Core v1.9.

Check warning on line 496 in website/docs/docs/build/snapshots.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/docs/build/snapshots.md#L496

[custom.Typos] Oops there's a typo -- did you really mean 'v1.9'?
Raw output
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'v1.9'? ", "location": {"path": "website/docs/docs/build/snapshots.md", "range": {"start": {"line": 496, "column": 201}}}, "severity": "WARNING"}

The following example shows how to configure a snapshot:

Expand Down
4 changes: 2 additions & 2 deletions website/docs/docs/build/unit-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,13 @@ keywords:

:::note

This functionality is only supported in dbt Core v1.8+ or accounts that have opted for a ["Versionless"](/docs/dbt-versions/upgrade-dbt-version-in-cloud#versionless) dbt Cloud experience.
Unit testing functionality is available in [dbt Cloud Release Tracks](/docs/dbt-versions/cloud-release-tracks) or dbt Core v1.8+

Check warning on line 13 in website/docs/docs/build/unit-tests.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/docs/build/unit-tests.md#L13

[custom.Typos] Oops there's a typo -- did you really mean 'v1.8+'?
Raw output
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'v1.8+'? ", "location": {"path": "website/docs/docs/build/unit-tests.md", "range": {"start": {"line": 13, "column": 124}}}, "severity": "WARNING"}

:::

Historically, dbt's test coverage was confined to [“data” tests](/docs/build/data-tests), assessing the quality of input data or resulting datasets' structure. However, these tests could only be executed _after_ building a model.

With dbt Core v1.8 and dbt Cloud environments that have gone versionless by selecting the **Versionless** option, we have introduced an additional type of test to dbt - unit tests. In software programming, unit tests validate small portions of your functional code, and they work much the same way here. Unit tests allow you to validate your SQL modeling logic on a small set of static inputs _before_ you materialize your full model in production. Unit tests enable test-driven development, benefiting developer efficiency and code reliability.
Starting in dbt Core v1.8, we have introduced an additional type of test to dbt - unit tests. In software programming, unit tests validate small portions of your functional code, and they work much the same way here. Unit tests allow you to validate your SQL modeling logic on a small set of static inputs _before_ you materialize your full model in production. Unit tests enable test-driven development, benefiting developer efficiency and code reliability.

Check warning on line 19 in website/docs/docs/build/unit-tests.md

View workflow job for this annotation

GitHub Actions / vale

[vale] website/docs/docs/build/unit-tests.md#L19

[custom.Typos] Oops there's a typo -- did you really mean 'v1.8'?
Raw output
{"message": "[custom.Typos] Oops there's a typo -- did you really mean 'v1.8'? ", "location": {"path": "website/docs/docs/build/unit-tests.md", "range": {"start": {"line": 19, "column": 22}}}, "severity": "WARNING"}

## Before you begin

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Auto-exposures help data teams optimize their efficiency and ensure data quality

To access the features, you should meet the following:

1. Your environment and jobs are on [Versionless](/docs/dbt-versions/versionless-cloud) dbt.
1. Your environment and jobs are on a supported [release track](/docs/dbt-versions/cloud-release-tracks) dbt.
2. You have a dbt Cloud account on the [Enterprise plan](https://www.getdbt.com/pricing/).
3. You have set up a [production](/docs/deploy/deploy-environments#set-as-production-environment) deployment environment for each project you want to explore, with at least one successful job run.
4. You have [admin permissions](/docs/cloud/manage-access/enterprise-permissions) in dbt Cloud to edit project settings or production environment settings.
Expand Down
2 changes: 0 additions & 2 deletions website/docs/docs/cloud/cloud-cli-installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,6 @@ dbt commands are run against dbt Cloud's infrastructure and benefit from:
## Prerequisites
The dbt Cloud CLI is available in all [deployment regions](/docs/cloud/about-cloud/access-regions-ip-addresses) and for both multi-tenant and single-tenant accounts.

- You are on dbt version 1.5 or higher. Alternatively, set it to [**Versionless**](/docs/dbt-versions/upgrade-dbt-version-in-cloud#versionless) to automatically stay up to date.

## Install dbt Cloud CLI

You can install the dbt Cloud CLI on the command line by using one of these methods.
Expand Down
Loading

0 comments on commit bb6224e

Please sign in to comment.