Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add FilterProjectReplayer #11351

Closed
wants to merge 1 commit into from
Closed

Conversation

duanmeng
Copy link
Collaborator

@duanmeng duanmeng commented Oct 25, 2024

Add FilterProjectReplayer. For the plan fragment involving
FilterNode->ProjectNode, users should use the ProjectNode
ID for tracing. This is because the planner will combine these
two operators into a single FilterProject operator. During replay,
the ProjectNode ID will locate the trace data directory.

Part of #9668

@duanmeng duanmeng requested review from xiaoxmeng and removed request for majetideepak and assignUser October 25, 2024 15:32
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 25, 2024
Copy link

netlify bot commented Oct 25, 2024

Deploy Preview for meta-velox canceled.

Name Link
🔨 Latest commit 40b7b0a
🔍 Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/67245bf313c83700088782be

@duanmeng duanmeng force-pushed the trace_fp branch 8 times, most recently from bbcfc21 to 03c8edf Compare October 30, 2024 15:57
Copy link
Contributor

@xiaoxmeng xiaoxmeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@duanmeng LGTM % minors. Could you add a test for project in velox/exec/OperatorTraceTest.cpp? Thanks!

velox/tool/trace/OperatorReplayerBase.h Outdated Show resolved Hide resolved
velox/tool/trace/FilterProjectReplayer.h Outdated Show resolved Hide resolved
velox/tool/trace/FilterProjectReplayer.cpp Outdated Show resolved Hide resolved
velox/tool/trace/FilterProjectReplayer.cpp Outdated Show resolved Hide resolved
VELOX_CHECK(
!isFilterProject(filterNode),
"If the target node is aFilterNode, it must be a standalone "
"FilterNode, without a ProjectNode as its parent.");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

without a ProjectNode as its consumer ?

not sure if it should call parent or consumer. It is basically the downstream node of filter node? thanks!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just use "If the target node is a FilterNode, it must be a standalone FilterNode"?

velox/tool/trace/FilterProjectReplayer.cpp Outdated Show resolved Hide resolved
velox/tool/trace/FilterProjectReplayer.cpp Outdated Show resolved Hide resolved
@duanmeng duanmeng force-pushed the trace_fp branch 3 times, most recently from c577fa0 to 91ae9d8 Compare November 1, 2024 03:16
Copy link
Contributor

@xiaoxmeng xiaoxmeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@duanmeng LGTM. Thanks for the update!

@facebook-github-bot
Copy link
Contributor

@xiaoxmeng has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@xiaoxmeng merged this pull request in ad27145.

Copy link

Conbench analyzed the 1 benchmark run on commit ad27145c.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.

facebook-github-bot pushed a commit that referenced this pull request Nov 4, 2024
Summary:
A follow-up PR for #11351 to cleanup CMake.

`TraceReplayRunner.cpp` already added as a library in #11388.

CC duanmeng

Pull Request resolved: #11406

Reviewed By: Yuhta

Differential Revision: D65420171

Pulled By: mbasmanova

fbshipit-source-id: 4111bee7a238baa1bb87e885d100d35cdb569626
@kgpai
Copy link
Contributor

kgpai commented Nov 5, 2024

cc: #11441 @duanmeng

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants