Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[multiple clients] Do not reference static or long-lived protobuf message objects from transient protobuf messages #28100

Open
dazuma opened this issue Dec 12, 2024 · 0 comments

Comments

@dazuma
Copy link
Member

dazuma commented Dec 12, 2024

There is a known issue in protobuf (protocolbuffers/protobuf#10106) in which if transient message object A references a long-lived message object B, the protobuf data underlying A will not get garbage collected until B is collectable, even if A is not reachable. This is because A and B are forced into the same protobuf arena, which protobuf will not dispose as long as B is live. If many such transient objects A are created, this will present as a memory leak. (Note: this may be a simplification of the exact repro case and explanation; see the original protobuf issue for more details.)

The upshot is, client libraries should not hold onto any protobuf message objects that can be used as field values in other objects that are meant to be transient. A common example could be a "common" value held in a Ruby constant, which is then used as a field value in a client request object—this will cause all such request objects to remain in memory indefinitely, as long as the Ruby constant remains in scope.

Instead, any such common objects should either be created "on the fly" so they can be collected along with the object that references them, or "deep copied" so the copy actually referenced is collectable even if the original is long-lived.

We need to audit all handwritten client libraries and make this change. One known case is in the google-cloud-bigtable client: the Google::Cloud::Bigtable::RowFilter module includes several constants which are proto messages used in filter chains.

References:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant