Improve memory usage by move ZMQ serialize buffer from ZmqProducerStateTable to ZmqClient #955
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
liuh-80
commented
Nov 23, 2024
| @@ -38,8 +38,6 @@ ZmqProducerStateTable::ZmqProducerStateTable(RedisPipeline *pipeline, const stri | |||
|
|
|||
| void ZmqProducerStateTable::initialize(DBConnector *db, const std::string &tableName, bool dbPersistence) | |||
| { | |||
| m_sendbuffer.resize(MQ_RESPONSE_MAX_COUNT); | |||
There was a problem hiding this comment.
This line cause GNMI memory issue, it will allocate 16MB memory for every ZmqProducerStateTable.
However, by add debug log, the dtor of invoked by GNMI. so this memory should be release.
However, the GNMI service memory will keep fast increase until 1.3GB.
Seems it's related with GO memory management or go SWIG wrapper: some memory allocates and released by C++ not free in GO side.
There was a problem hiding this comment.
I don't understand, if dtor is invoked, why is this memory not released?
There was a problem hiding this comment.
I'm not sure why, this can only reproduce on starlab
ganglyu
approved these changes
Nov 25, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why I did it
Every ZmqProducerStateTable will allocate 16MB buffer, this can be improve by share same buffer in ZmqClient.
How I did it
Improve memory usage by move ZMQ serialize buffer from ZmqProducerStateTable to ZmqClient.
Work item tracking
How to verify it
Pass all test cases.
Which release branch to backport (provide reason below if selected)
Description for the changelog
Improve memory usage by move ZMQ serialize buffer from ZmqProducerStateTable to ZmqClient.
Link to config_db schema for YANG module changes
A picture of a cute animal (not mandatory but encouraged)