Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: simplify rebuildShares #197

Merged
merged 3 commits into from
Jun 28, 2023
Merged

Conversation

rootulp
Copy link
Collaborator

@rootulp rootulp commented Jun 26, 2023

Closes #196

Since Decode returns original + parity shares, there is no need to separately rebuild the parity shares. This PR removes an unnecessary conditional inside rebuildShares which let us also remove a param to that function. After the refactor, a few helper methods were no longer used so they were also removed.

@rootulp rootulp self-assigned this Jun 26, 2023
@codecov
Copy link

codecov bot commented Jun 26, 2023

Codecov Report

Merging #197 (fd86dfb) into master (9df814c) will decrease coverage by 0.57%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #197      +/-   ##
==========================================
- Coverage   81.36%   80.80%   -0.57%     
==========================================
  Files           7        7              
  Lines         526      500      -26     
==========================================
- Hits          428      404      -24     
+ Misses         58       57       -1     
+ Partials       40       39       -1     
Impacted Files Coverage Δ
extendeddatacrossword.go 70.94% <100.00%> (-2.71%) ⬇️

Copy link
Member

@evan-forbes evan-forbes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahh okay so the encode and decode abstractions were (are?) leaky

@@ -249,7 +247,6 @@ func (eds *ExtendedDataSquare) solveCrosswordCol(
// 2. Whether the original shares could be decoded from the shares parameter.
// 3. [Optional] an error.
func (eds *ExtendedDataSquare) rebuildShares(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need this function at all if its just decoding?

Copy link
Collaborator Author

@rootulp rootulp Jun 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good question, I can explore deleting it entirely in a FLUP issue

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH I'm only creating a new issue rather than tackling in this PR b/c I don't want to dismiss existing approvals 😞 #199

Copy link
Contributor

@staheri14 staheri14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! left two questions, but they are non-blocking as long as we are certain about them.

@@ -259,25 +256,6 @@ func (eds *ExtendedDataSquare) rebuildShares(
return nil, false, nil
}

if isExtendedPartIncomplete {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Question] I suppose there have not been any tests associated with this part of the code, as no test file has been updated in this PR, right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -259,25 +256,6 @@ func (eds *ExtendedDataSquare) rebuildShares(
return nil, false, nil
}

if isExtendedPartIncomplete {
// If needed, rebuild the parity shares too.
rebuiltExtendedShares, err := eds.codec.Encode(rebuiltShares[0:eds.originalDataWidth])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Question] The only potential issue could arise if the parity portion is incomplete and the rebuilt extended shares obtained from rebuiltShares, err := eds.codec.Decode(shares) (the second half of rebuiltShares) and rebuiltExtendedShares, err := eds.codec.Encode(rebuiltShares[0:eds.originalDataWidth]) turn out to be different. Have we conducted any tests to verify this?

In the original code, we initially decode the entire row, and if the extended part (before decoding) is incomplete, we discard that part from the decoded version and replace it with the encoding result. However, in the new version, we retain whatever the decode function provides us with. If we are confident that these two have the same result, then everything looks good to me! (I think they should be the same)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only potential issue could arise if the parity portion is incomplete

I don't expect this to happen because Decode calls Reconstruct here and the godoc for Reconstruct claims:

// If there are too few shards to reconstruct the missing
// ones, ErrTooFewShards will be returned.

Ref: https://github.com/klauspost/reedsolomon/blob/523164698be98f1603cf1235f5a1de17728b2091/reedsolomon.go#L56-L57

@rootulp rootulp merged commit 2557620 into celestiaorg:master Jun 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Why rebuild extended shares?
3 participants