Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor: Update Base64 as non-throwing API #11149

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

refactor: Update Base64 as non-throwing API #11149

wants to merge 2 commits into from

Conversation

Joe-Abraham
Copy link
Contributor

@Joe-Abraham Joe-Abraham commented Oct 2, 2024 •
edited
Loading

Follow-up PR: #10371

The following changes are done in this PR.

  1. Refactor the APIs in Base64 as non-throwing APIs.

Changing the char to std::string_view and std::string will be done in subsequent PR.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 2, 2024
@netlify Netlify
Copy link

netlify bot commented Oct 2, 2024 •
edited
Loading

✅ Deploy Preview for meta-velox canceled.

Name Link
🔨 Latest commit 1a864f6
🔍 Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/675bf62a9784840008e2c445

@Joe-Abraham Joe-Abraham force-pushed the status branch 7 times, most recently from f3e0450 to 4b0ca32 Compare October 3, 2024 11:05
@Joe-Abraham Joe-Abraham marked this pull request as ready for review October 3, 2024 11:26
@Joe-Abraham
Copy link
Contributor Author

@majetideepak Can you please review the changes?

@kgpai kgpai requested a review from majetideepak October 11, 2024 06:19
@kgpai
Copy link
Contributor

kgpai commented Oct 11, 2024

Thanks @Joe-Abraham , based on brief glance, it seems its mostly renaming variables etc for readability and consistency (which is welcome change); Want to make sure I am not missing any material change.

@Joe-Abraham
Copy link
Contributor Author

@kgpai I have also made the APIs used in BinaryFunctions.h as non-throwing.

@majetideepak
Copy link
Collaborator

  1. Refactor the APIs in Base64 as non-throwing APIs.

@Joe-Abraham this should be its own PR since its a functional change. We can leave the remaining code improvements this PR.
Can you please open another PR with only the non-throwing API changes? Thanks.

@Joe-Abraham Joe-Abraham force-pushed the status branch 2 times, most recently from c421830 to 62510fb Compare November 13, 2024 15:33
@Joe-Abraham Joe-Abraham changed the title Update Base64 as a non-throwing API Update Base64 as non-throwing API Nov 13, 2024
majetideepak
majetideepak reviewed Nov 18, 2024
View reviewed changes
velox/common/encode/Base64.h Outdated
@@ -139,7 +143,8 @@ class Base64 {
// character.
static uint8_t base64ReverseLookup(
char encodedChar,
const ReverseIndex& reverseIndex);
const ReverseIndex& reverseIndex,
Status& status);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Status should always be the return type. The return value is not valid if Status is not OK. You can combine Status with a return value using folly::Expected.
See #7589
Here I think you can return uint8_t as a return argument.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the code in a similar way.

velox/common/encode/Base64.cpp Outdated
if (reverseLookupValue >= 0x40) {
VELOX_USER_FAIL("decode() - invalid input string: invalid characters");
status = Status::UserError(
"decode() - invalid input string: invalid characters");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably print the invalid character here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the code and added the testcases as well.

velox/common/encode/Base64.cpp Outdated
outputBuffer[0] = (decodedBlock >> 16) & 0xff;
outputBuffer[1] = (decodedBlock >> 8) & 0xff;
outputBuffer[2] = decodedBlock & 0xff;
(base64ReverseLookup(input[0], reverseIndex, lookupStatus) << 18) |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can may-be directly pass outputBuffer[i] to store the return value for each call.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the code.

velox/common/encode/Base64.cpp Outdated
@@ -163,10 +161,10 @@ std::string Base64::encodeImpl(
const T& input,
const Charset& charset,
bool includePadding) {
size_t encodedSize = calculateEncodedSize(input.size(), includePadding);
const size_t encodedSize{calculateEncodedSize(input.size(), includePadding)};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const is not meaningful for scalar values.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated the code

velox/common/encode/Base64.cpp Outdated
}

// static
template <class T>
void Base64::encodeImpl(
Status Base64::encodeImpl(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does encodeImpl throw? If not why use Status?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

base64ReverseLookup call is being made from encodeImpl, Can you have a look into the updated code.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't base64ReverseLookup being called from decodeImpl?
I don't see any function call being made inside encodeImpl.

@Joe-Abraham Joe-Abraham force-pushed the status branch 5 times, most recently from 80aefc1 to c940434 Compare November 24, 2024 11:54
@Joe-Abraham Joe-Abraham changed the title Update Base64 as non-throwing API refactor: Update Base64 as non-throwing API Nov 24, 2024
@Joe-Abraham Joe-Abraham force-pushed the status branch 3 times, most recently from 439959f to 47d0c0d Compare December 9, 2024 05:03
majetideepak
majetideepak reviewed Dec 9, 2024
View reviewed changes
velox/common/encode/Base64.cpp
@@ -163,10 +161,10 @@ std::string Base64::encodeImpl(
const T& input,
const Charset& charset,
bool includePadding) {
size_t encodedSize = calculateEncodedSize(input.size(), includePadding);
size_t encodedSize{calculateEncodedSize(input.size(), includePadding)};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this change?

velox/common/encode/Base64.cpp Outdated
}

// static
template <class T>
void Base64::encodeImpl(
Status Base64::encodeImpl(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't base64ReverseLookup being called from decodeImpl?
I don't see any function call being made inside encodeImpl.

velox/common/encode/Base64.cpp Outdated
@@ -320,29 +322,36 @@ void Base64::decode(
const std::pair<const char*, int32_t>& payload,
std::string& decodedOutput) {
size_t inputSize = payload.second;
decodedOutput.resize(calculateDecodedSize(payload.first, inputSize));
decode(payload.first, inputSize, decodedOutput.data(), decodedOutput.size());
size_t decodedSize;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

initialize the variable to 0.

velox/common/encode/Base64.cpp
decodedOutput.resize(calculateDecodedSize(payload.first, inputSize));
decode(payload.first, inputSize, decodedOutput.data(), decodedOutput.size());
size_t decodedSize;
(void)calculateDecodedSize(payload.first, inputSize, decodedSize);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the return Status from calculateDecodedSize ignored here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@majetideepak majetideepak majetideepak left review comments

@assignUser assignUser Awaiting requested review from assignUser assignUser is a code owner

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Joe-Abraham @kgpai @majetideepak @facebook-github-bot