Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

verify previous path state before backup #4630

Merged
merged 5 commits into from
Nov 10, 2023

Conversation

ryanfkeepers
Copy link
Contributor

Drive previous paths have a prior bug or condition that allows multiple previousPath metadata folder ID entries within a single drive point to the same previous path string.
In these cases we need to treat the backup as having no prior metadata, and re-run the full delta query to correct the bad state.


Does this PR need a docs update or release note?

  • ✅ Yes, it's included

Type of change

  • 🐛 Bugfix

Test Plan

  • ⚡ Unit test
  • 💚 E2E

@ryanfkeepers ryanfkeepers requested review from ashmrtn and meain November 8, 2023 20:11
@ryanfkeepers ryanfkeepers self-assigned this Nov 8, 2023
Copy link
Contributor

aviator-app bot commented Nov 8, 2023

Current Aviator status

Aviator will automatically update this comment as the status of the PR changes.
Comment /aviator refresh to force Aviator to re-examine your PR (or learn about other /aviator commands).

This PR was merged manually (without Aviator). Merging manually can negatively impact the performance of the queue. Consider using Aviator next time.


See the real-time status of this PR on the Aviator webapp.

Use the Aviator Chrome Extension to see the status of your PR within GitHub.

CHANGELOG.md Outdated Show resolved Hide resolved
src/internal/m365/collection/drive/collections.go Outdated Show resolved Hide resolved
src/internal/m365/collection/drive/collections.go Outdated Show resolved Hide resolved
src/internal/m365/collection/drive/collections.go Outdated Show resolved Hide resolved
src/internal/m365/collection/drive/collections.go Outdated Show resolved Hide resolved
driveID1: {
"root": rootFolderPath1,
"folder": rootFolderPath1 + "/folder",
"folder2": rootFolderPath1 + "/folder",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you please add another folder that doesn't appear in the result set and make sure it's marked as deleted

Copy link
Contributor Author

@ryanfkeepers ryanfkeepers Nov 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, well, it's not marked as deleted. It's not included in the result set either, which I think is correct given the other results and full enumeration. I'll follow up on this in the next PR where we don't force a full enumeration and make sure it's handled the right way there.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if that's correct behavior or not. It would depend on what other data is returned. If we don't return a flag saying the base shouldn't be merged (which we still need to remove at some point) or mark the collection as deleted then it'll be pulled forward

I know this PR train isn't trying to solve that problem anymore, but I'm concerned about losing track of this particular test. It's important to have because we should handle it properly eventually, I just don't know the best way to record it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's tracked on a test in the next PR.

src/internal/m365/collection/drive/collections.go Outdated Show resolved Hide resolved
errCheck: assert.NoError,
},
{
name: "DuplicatePreviousPaths_separateDrives",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you please also add a test where one drive has duplicate paths and the other doesn't? We should only be dropping metadata info for the one that has the duplicate in the ideal case

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going to skip this. Same reason as skipping the per-drive metadata removal; we won't be forcing full enumeration on the next PR, so it won't be handled this way.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's still worth adding a test for this so we can, at the very least, build on it later on. Adding it will help ensure we don't lose track of that case and that alerts act as expected in the next PR

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fwiw, there's already a test for that in the next PR

@ryanfkeepers
Copy link
Contributor Author

@ashmrtn ready for final review

Copy link
Contributor

@ashmrtn ashmrtn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would really like to see a few more tests added so we don't lose track of them

errCheck: assert.NoError,
},
{
name: "DuplicatePreviousPaths_separateDrives",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's still worth adding a test for this so we can, at the very least, build on it later on. Adding it will help ensure we don't lose track of that case and that alerts act as expected in the next PR

driveID1: {
"root": rootFolderPath1,
"folder": rootFolderPath1 + "/folder",
"folder2": rootFolderPath1 + "/folder",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if that's correct behavior or not. It would depend on what other data is returned. If we don't return a flag saying the base shouldn't be merged (which we still need to remove at some point) or mark the collection as deleted then it'll be pulled forward

I know this PR train isn't trying to solve that problem anymore, but I'm concerned about losing track of this particular test. It's important to have because we should handle it properly eventually, I just don't know the best way to record it

Drive previous paths have a prior bug or condition that allows multiple
previousPath metadata folder ID entries within a single drive point
to the same previous path string.
In these cases we need to treat the backup as having no prior
metadata, and re-run the full delta query to correct the bad state.
fail the backup if it produces a previous path collision in drive
metadata.
@ryanfkeepers ryanfkeepers force-pushed the catch-duplicate-prev-paths branch from 4229eac to 4014578 Compare November 10, 2023 02:31
@ryanfkeepers ryanfkeepers merged commit f1dc93c into debug-prev-paths Nov 10, 2023
4 checks passed
@ryanfkeepers ryanfkeepers deleted the catch-duplicate-prev-paths branch November 10, 2023 02:31
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants