-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make snapshot load faster #222
Conversation
src/Service/KeeperStore.h
Outdated
@@ -297,6 +301,9 @@ class KeeperStore | |||
/// Build path children after load data from snapshot | |||
void buildPathChildren(bool from_zk_snapshot = false); | |||
|
|||
// Build childrenset for the node in specified block after load data from snapshot. | |||
void buildBlockChildren(const std::vector<BlocksEdges> & all_objects_edges, UInt32 block_idx); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
from_zk_snapshot is omited
@lzydmxy Nice optimization! Please describe your solution. |
Is there any optimization for parsing data from disk? For I found it is faster in your benchmark. |
|
} | ||
catch (Exception & e) | ||
{ | ||
LOG_ERROR(log, "parseObject error {}, {}", it->second, getExceptionMessage(e, true)); | ||
throw; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please throw Exception
and add an error code
@JackyWoo The test failed for apple clang, should we continue support it ?
|
We should not use Apple clang. Let us update the CI env. |
Which issues of this PR fixes:
This PR try to fix #221
For nodes count 58680021, snapshot size 12GB
Change log:
Optimizations: