Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initialize recursive protobuf fields more efficiently. #1548

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

copybara-service[bot]
Copy link
Contributor

@copybara-service copybara-service bot commented Feb 3, 2025

Initialize recursive protobuf fields more efficiently.

Currently, a field (F0) is not initialized if there are subfields of the form F0 -> F1 -> ... -> Fs -> ... -> Fn -> Fs. This means that for a huge protobuf field that has a recursive subfield deep in its definition, the whole field is not initialized. And later, even when F0 is initialized, F1 won't get initialized, etc. This could be very inefficient. To avoid this, we define "recursion breaker fields". For example, "Fs" becomes a recursion breaker. Then, all fields up to Fs are initialized. And later when Fs gets initialized, all Fs -> ... -> Fn get initialized.

This CL consists of the following changes:

  • IsProtoRecursive deals with infinite recursions only.
  • IsFinitelyRecursive is replaced with IsRecursionBreaker which is implemented separately.
  • Having separate strategy for handling of infinite recursion and finite recursion (through recursion breakers), we need to explicitely avoid initialization and mutation of infinitely recursive fields.

@copybara-service copybara-service bot force-pushed the cl/722802824 branch 8 times, most recently from deadd80 to b8405aa Compare February 7, 2025 23:48
Currently, a field (F0) is not initialized if there are subfields of the form F0 -> F1 -> ... -> Fs -> ... -> Fn -> Fs. This means that for a huge protobuf field that has a recursive subfield deep in its definition, the whole field is not initialized. And later, even when F0 is initialized, F1 won't get initialized, etc. This could be very inefficient. To avoid this, we define "recursion breaker fields". For example, "Fs" becomes a recursion breaker. Then, all fields up to Fs are initialized. And later when Fs gets initialized, all Fs -> ... -> Fn get initialized.

This CL consists of the following changes:
- IsProtoRecursive deals with infinite recursions only.
- IsFinitelyRecursive is replaced with IsRecursionBreaker which is implemented separately.
- Having separate strategy for handling of infinite recursion and finite recursion (through recursion breakers), we need to explicitely avoid initialization and mutation of infinitely recursive fields.

PiperOrigin-RevId: 722802824
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant