Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Consistent version query semantics #3477

Merged
merged 6 commits into from
Feb 21, 2025

Conversation

jsimnz
Copy link
Member

@jsimnz jsimnz commented Feb 21, 2025

Relevant issue(s)

Resolves #3476

Description

Fixes the consistency between regular queries and time-travel queries with respect to the _version subquery. Before, the regular queries would return all commits from the start to the current, but time-travel queries only used a "latestCommit" query at the target version, which meant only returning a single commit entry.

This PR implements some semantics changes to the overall commits API, but maintains all necessary backwards compatibility (however, it does introduce new errors when using invalid cid/docid combinations).

New semantics:

  • Only doc ID, default max depth, start from "latest" cid
  • doc ID + CID, default 0 depth, start from "target" cid
  • doc ID + CID + depth, use depth, start from "target" cid

Unchanged semantics:

  • doc ID + depth, use depth, start from "latest" cid

Technically, there is also an optimization where we no longer use the HeadFetcher in the commit queries if a CID is specified.

Tasks

  • I made sure the code is well commented, particularly hard-to-understand areas.
  • I made sure the repository-held documentation is changed accordingly.
  • I made sure the pull request title adheres to the conventional commit style (the subset used in the project can be found in tools/configs/chglog/config.yml).
  • I made sure to discuss its limitations such as threats to validity, vulnerability to mistake and misuse, robustness to invalidation of assumptions, resource requirements, ...

How has this been tested?

(replace) Describe the tests performed to verify the changes. Provide instructions to reproduce them.

Specify the platform(s) on which this was tested:

  • (modify the list accordingly)
  • Ubuntu (WSL2)

@jsimnz jsimnz added bug Something isn't working area/query Related to the query component labels Feb 21, 2025
@jsimnz jsimnz added this to the DefraDB v0.16 milestone Feb 21, 2025
@jsimnz jsimnz self-assigned this Feb 21, 2025
@jsimnz jsimnz force-pushed the jsimnz/fix/consistent-version-query branch from 296b841 to e6baea5 Compare February 21, 2025 12:15
Copy link

codecov bot commented Feb 21, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.47%. Comparing base (9302cff) to head (b63d4da).
Report is 1 commits behind head on develop.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #3477      +/-   ##
===========================================
- Coverage    78.48%   78.47%   -0.01%     
===========================================
  Files          397      397              
  Lines        37574    37599      +25     
===========================================
+ Hits         29489    29504      +15     
- Misses        6394     6402       +8     
- Partials      1691     1693       +2     
Flag Coverage Δ
all-tests 78.47% <100.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
internal/planner/commit.go 85.77% <100.00%> (+2.65%) ⬆️
internal/planner/errors.go 57.14% <ø> (ø)
internal/planner/select.go 85.47% <100.00%> (+0.08%) ⬆️

... and 16 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9302cff...b63d4da. Read the comment docs.

Comment on lines -73 to +74
cid: "bafybeica4js2abwqjjrz7dcialbortbz32uxp7ufxu7yljbwvmhjqqxzny"
cid: "bafyreiale6qsjc7qewod3c6h2odwamfwcf7vt4zlqtw7ldcm57xdkgxja4"
Copy link
Member Author

@jsimnz jsimnz Feb 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just highlighting this for reviewers. This isnt required by the new changes to the commit query semantics. Its actually fixing a unconsequential bug which is using the incorrect CID for the goal of the test. The test wanted to test field IDs, but the original CID was a composite CID.

Copy link
Collaborator

@fredcarle fredcarle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Just one minor todo and a typo.

@@ -20,7 +20,7 @@ import (
// desired behaviour (should return all latest commits).
func TestQueryLatestCommits(t *testing.T) {
test := testUtils.TestCase{
Description: "Simple latest commits query",
Description: "Simple latest commits queries",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: why is this plural now? I see only one query request in the test. Also then test name itself has only Query in it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

honestly, fantastic question 🙃. I just saw fred suggestion and went with it. Im thinking now that the s was there accidentally from me trying to press ctrl+s to save at some point and not registering the ctrl

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assumed John was meaning to make it plural because there were multiple queries in the test (Git change was not showing the full test and just took it for granted)

The assumption cascade is strong with this one... lol

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lol 😆 I looked at it three times, thinking am I missing something or tripping.

Copy link
Member

@shahzadlone shahzadlone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! one question

@jsimnz jsimnz force-pushed the jsimnz/fix/consistent-version-query branch from e078848 to b63d4da Compare February 21, 2025 21:03
@@ -0,0 +1,2 @@
# Consistent version query semantics
Updated semantics for commit/version/time-travel queries when using both DocID and CID.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

todo: There is no data breaking change in this PR, if the change detector failed it is because the integration test definitions were changed, not the data on disk. Please change the wording to reflect this.

@jsimnz jsimnz merged commit 11ffc34 into develop Feb 21, 2025
44 of 46 checks passed
@jsimnz jsimnz deleted the jsimnz/fix/consistent-version-query branch February 21, 2025 21:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/query Related to the query component bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Inconsistent _version behavior between time-travel and regular queries
4 participants