WIP: the tip of my filterx performance efforts #434
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bazsi
force-pushed
the
filterx-perf-project
branch
4 times, most recently
from
6e3ce03 to
0b045c1
Compare
bazsi
force-pushed
the
filterx-perf-project
branch
from
0b045c1 to
b22eedc
Compare
Signed-off-by: Balazs Scheidler <[email protected]>
bazsi
force-pushed
the
filterx-perf-project
branch
2 times, most recently
from
fbff61c to
5cab36c
Compare
Signed-off-by: Balazs Scheidler <[email protected]>
Instead of coding this individually in all FilterXExpr derivatives. Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
…ction Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
To make it easier to understand stackdumps. Signed-off-by: Balazs Scheidler <[email protected]>
bazsi
force-pushed
the
filterx-perf-project
branch
from
5cab36c to
486d5b6
Compare
…in addition to is_floating Signed-off-by: Balazs Scheidler <[email protected]>
…instances We do not really support message contexts (as does the original filter language and templates), so remove that support. This makes it simpler to initialize FilterXEvalContext as well as we do not have to manage the LogMessage array separately. Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
…s to the front Signed-off-by: Balazs Scheidler <[email protected]>
It does not really free @self, so it only clears up allocations within an existing instance, these are usually called *_clear() functions. Signed-off-by: Balazs Scheidler <[email protected]>
…ayer Previously the type enum was only used during the initialization of FilterXExprVariable which was then translated to a set of booleans at the variable layer. Also clean up naming a bit. Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
bazsi
force-pushed
the
filterx-perf-project
branch
from
486d5b6 to
44a270b
Compare
Signed-off-by: Balazs Scheidler <[email protected]>
If we pull in a variable from the message and we need to unmarshal it (e.g. turn FilterXMessageValue to a more specific type like FilterXString or FilterXJSON), do not consider that a change of that variable. Changing it in-place, or assignment of a new value should be remain to be a change. Signed-off-by: Balazs Scheidler <[email protected]>
# Conflicts: # lib/filterx/expr-compound.c # lib/filterx/expr-condition.c # lib/filterx/expr-done.c # lib/filterx/expr-drop.c # lib/filterx/expr-function.c # lib/filterx/expr-get-subscript.c # lib/filterx/expr-getattr.c # lib/filterx/expr-literal.c # lib/filterx/expr-set-subscript.c # lib/filterx/expr-setattr.c # lib/filterx/expr-template.c # lib/filterx/expr-variable.c # lib/filterx/filterx-expr.c # lib/filterx/filterx-expr.h
… into tip/axodepot # Conflicts: # lib/filterx/expr-variable.c
…' into tip/axodepot # Conflicts: # lib/filterx/filterx-scope.c
Previously a coupling was established between logmsg and FilterXScope, namely whenever the LogMessage was cloned, the logmsg layer called filterx_scope_set_log_msg_has_changes(), which was then subsequently used for invalidating message tied variables in the scope. This was broken for cases where the message was writable, so it changed without being cloned. In those cases the stale variables survived anyway. Another issue was that this produced excessive calls to the expensive filterx_scope_invalidate_log_msg_cache(), as it may be filterx_scope_sync() that causes the LogMessage to be cloned. In those cases we executed both filterx_scope_sync() and an entirely unnecessary filterx_scope_invalidate_log_msg_cache() both iterating on all FilterXVariable instances in the scope. This mechanism is being replaced by the generation counter mechanism, but to make the patches easier to review, this patch just removes the entire log_msg_has_changes() mechanism. Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
…e inline functions Signed-off-by: Balazs Scheidler <[email protected]>
…ocessed I want to delegate the responsibility of tracking LogMessage changes to the scope (just as floating values are tracked by it), and as a preparation add a "msg" member to FilterXScope and make sure it always contains the right message. Signed-off-by: Balazs Scheidler <[email protected]>
Signed-off-by: Balazs Scheidler <[email protected]>
…XScope Previously message-tied variables were managed in part within expr-variable and in part within FilterXScope. Now with the message being available in FilterXScope, we can delegate this in entirety to FilterXScope. This also implements the validation of message-tied values, so if the LogMessage changes independently from FilterXScope, we will notice that too and consider the values of those variables stale. Signed-off-by: Balazs Scheidler <[email protected]>
A scope is only considered dirty if it has message-tied variables that are changed. In any other case it's not dirty, so no sync is needed. Signed-off-by: Balazs Scheidler <[email protected]>
Link scopes together. Signed-off-by: Balazs Scheidler <[email protected]>
…rations Previously generations was reset to 0 in case we ended up doing a clone. Let's retain that instead, so we don't need to adjust the generation value for FilterXValues either. This is a preparation for sharing FilterXVariable descriptors so that we don't have to clone them. Signed-off-by: Balazs Scheidler <[email protected]>
Instead of cloning all variables into subsequent scopes, let's start with an empty array and only clone the ones that are actually used. This improves performance a lot in our use-cases. Signed-off-by: Balazs Scheidler <[email protected]>
bazsi
force-pushed
the
filterx-perf-project
branch
from
44a270b to
8e3b9c3
Compare
Signed-off-by: Balazs Scheidler <[email protected]>
…he stack At the same time get rid of reference counting and clone. Signed-off-by: Balazs Scheidler <[email protected]>
Although this code has a single user, it is relatively complex, relies on arcane mechanics of FilterXScope and FilterXEvalContext. Hide it and delegate it to show where it belongs. These have become macros, as we need to use the caller's stack frame to allocate the scope. Signed-off-by: Balazs Scheidler <[email protected]>
This reverts commit e93c530c12f03547552449c1fb52437efb9b86cb.
bazsi
force-pushed
the
filterx-perf-project
branch
from
8e3b9c3 to
d54d015
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Please ignore this PR as it is not intended to be merged. It contains work-in-progress local patches I am yet to extract into separate pull requests.
It is rebased on top of my local merge of all related PRs, so I can maintain those branches separately as they get merged.