Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding steal-half functionalities to work-requesting scheduler #6422

Merged
merged 1 commit into from
Jan 29, 2024

Conversation

hkaiser
Copy link
Member

@hkaiser hkaiser commented Jan 20, 2024

  • adding new scheduler: --hpx:queuing=local-workrequesting-mc that relies on the Moody-Camel queue
  • flyby: a lot of cleanup in various related places
  • flyby: fixing config-registry for command line module

working towards #4703

@hkaiser
Copy link
Member Author

hkaiser commented Jan 20, 2024

A couple of local performance numbers (future_overhead_test):

  • default scheuler
invoked 500000, futures  create_thread_hierarchical           latch               none in 0.292718 seconds : 0.585436 us/future, queue              default, numa    0, threads   20, info              no-info
invoked 500000, futures                       apply   limiting-Exec  parallel_executor in 0.347093 seconds : 0.694186 us/future, queue              default, numa    0, threads   20, info              no-info
invoked 500000, futures                      action        WaitEach        no-executor in 5.070678 seconds : 10.141356 us/future, queue              default, numa    0, threads   20, info              no-info
invoked 500000, futures                      action         WaitAll        no-executor in 4.601552 seconds : 9.203105 us/future, queue              default, numa    0, threads   20, info              no-info
invoked 500000, futures                       async        WaitEach  parallel_executor in 4.246172 seconds : 8.492345 us/future, queue              default, numa    0, threads   20, info              no-info
invoked 500000, futures                       async         WaitAll  parallel_executor in 3.786355 seconds : 7.572709 us/future, queue              default, numa    0, threads   20, info              no-info
invoked 500000, futures                       apply     Sliding-Sem  parallel_executor in 4.295582 seconds : 8.591163 us/future, queue              default, numa    0, threads   20, info              no-info
invoked 500000, futures                    for_loop             par  parallel_executor in 0.000702 seconds : 0.001404 us/future, queue              default, numa    0, threads   20, info              no-info
invoked 500000, futures                    for_loop             par scheduler_executor<thread_pool_scheduler> in 0.000669 seconds : 0.001338 us/future, queue              default, numa    0, threads   20, info              no-info
invoked 500000, futures                    for_loop             par parallel_executor_nostack in 0.000535 seconds : 0.001070 us/future, queue              default, numa    0, threads   20, info              no-info
invoked 500000, futures               register_work           latch               none in 1.136111 seconds : 2.272222 us/future, queue              default, numa    0, threads   20, info              no-info
invoked 500000, futures               create_thread           latch               none in 1.328317 seconds : 2.656634 us/future, queue              default, numa    0, threads   20, info              no-info
invoked 500000, futures          apply_hierarchical           latch  parallel_executor in 0.197347 seconds : 0.394695 us/future, queue              default, numa    0, threads   20, info              no-info
  • workrequesting scheduler (fifo)
invoked 500000, futures  create_thread_hierarchical           latch               none in 0.305966 seconds : 0.611931 us/future, queue local-workrequesting-fifo, numa    0, threads   20, info              no-info
invoked 500000, futures                       apply   limiting-Exec  parallel_executor in 0.362330 seconds : 0.724661 us/future, queue local-workrequesting-fifo, numa    0, threads   20, info              no-info
invoked 500000, futures                      action        WaitEach        no-executor in 2.789453 seconds : 5.578905 us/future, queue local-workrequesting-fifo, numa    0, threads   20, info              no-info
invoked 500000, futures                      action         WaitAll        no-executor in 2.409016 seconds : 4.818032 us/future, queue local-workrequesting-fifo, numa    0, threads   20, info              no-info
invoked 500000, futures                       async        WaitEach  parallel_executor in 2.085483 seconds : 4.170967 us/future, queue local-workrequesting-fifo, numa    0, threads   20, info              no-info
invoked 500000, futures                       async         WaitAll  parallel_executor in 1.715133 seconds : 3.430267 us/future, queue local-workrequesting-fifo, numa    0, threads   20, info              no-info
invoked 500000, futures                       apply     Sliding-Sem  parallel_executor in 3.787963 seconds : 7.575926 us/future, queue local-workrequesting-fifo, numa    0, threads   20, info              no-info
invoked 500000, futures                    for_loop             par  parallel_executor in 0.000492 seconds : 0.000984 us/future, queue local-workrequesting-fifo, numa    0, threads   20, info              no-info
invoked 500000, futures                    for_loop             par scheduler_executor<thread_pool_scheduler> in 0.000534 seconds : 0.001067 us/future, queue local-workrequesting-fifo, numa    0, threads   20, info              no-info
invoked 500000, futures                    for_loop             par parallel_executor_nostack in 0.000469 seconds : 0.000937 us/future, queue local-workrequesting-fifo, numa    0, threads   20, info              no-info
invoked 500000, futures               register_work           latch               none in 1.046038 seconds : 2.092075 us/future, queue local-workrequesting-fifo, numa    0, threads   20, info              no-info
invoked 500000, futures               create_thread           latch               none in 1.003888 seconds : 2.007775 us/future, queue local-workrequesting-fifo, numa    0, threads   20, info              no-info
invoked 500000, futures          apply_hierarchical           latch  parallel_executor in 0.250559 seconds : 0.501117 us/future, queue local-workrequesting-fifo, numa    0, threads   20, info              no-info
  • workrequesting scheduler (steal-half, moody-camel)
invoked 500000, futures  create_thread_hierarchical           latch               none in 0.302476 seconds : 0.604953 us/future, queue local-workrequesting-mc, numa    0, threads   20, info              no-info
invoked 500000, futures                       apply   limiting-Exec  parallel_executor in 0.324260 seconds : 0.648520 us/future, queue local-workrequesting-mc, numa    0, threads   20, info              no-info
invoked 500000, futures                      action        WaitEach        no-executor in 2.712715 seconds : 5.425430 us/future, queue local-workrequesting-mc, numa    0, threads   20, info              no-info
invoked 500000, futures                      action         WaitAll        no-executor in 2.324837 seconds : 4.649675 us/future, queue local-workrequesting-mc, numa    0, threads   20, info              no-info
invoked 500000, futures                       async        WaitEach  parallel_executor in 1.933456 seconds : 3.866911 us/future, queue local-workrequesting-mc, numa    0, threads   20, info              no-info
invoked 500000, futures                       async         WaitAll  parallel_executor in 1.540370 seconds : 3.080739 us/future, queue local-workrequesting-mc, numa    0, threads   20, info              no-info
invoked 500000, futures                       apply     Sliding-Sem  parallel_executor in 3.630330 seconds : 7.260659 us/future, queue local-workrequesting-mc, numa    0, threads   20, info              no-info
invoked 500000, futures                    for_loop             par  parallel_executor in 0.000528 seconds : 0.001056 us/future, queue local-workrequesting-mc, numa    0, threads   20, info              no-info
invoked 500000, futures                    for_loop             par scheduler_executor<thread_pool_scheduler> in 0.000551 seconds : 0.001101 us/future, queue local-workrequesting-mc, numa    0, threads   20, info              no-info
invoked 500000, futures                    for_loop             par parallel_executor_nostack in 0.000442 seconds : 0.000884 us/future, queue local-workrequesting-mc, numa    0, threads   20, info              no-info
invoked 500000, futures               register_work           latch               none in 1.049016 seconds : 2.098031 us/future, queue local-workrequesting-mc, numa    0, threads   20, info              no-info
invoked 500000, futures               create_thread           latch               none in 1.010455 seconds : 2.020909 us/future, queue local-workrequesting-mc, numa    0, threads   20, info              no-info
invoked 500000, futures          apply_hierarchical           latch  parallel_executor in 0.179287 seconds : 0.358574 us/future, queue local-workrequesting-mc, numa    0, threads   20, info              no-info

@hkaiser hkaiser added this to the 1.10.0 milestone Jan 20, 2024
Copy link

codacy-production bot commented Jan 20, 2024

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation Diff coverage
-0.07% 71.43%
Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (c0d79c8) 201862 171892 85.15%
Head commit (76f2baf) 191893 (-9969) 163271 (-8621) 85.08% (-0.07%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#6422) 553 395 71.43%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings    Change summary preferences

You may notice some variations in coverage metrics with the latest Coverage engine update. For more details, visit the documentation

@hkaiser hkaiser force-pushed the fixing_4703 branch 6 times, most recently from fef03b6 to 0ada434 Compare January 21, 2024 15:21
- adding new scheduler: --hpx:queuing=local-workrequesting-mc that relies on the Moody-Camel queue
- flyby: a lot of cleanup in various related places
- flyby: fixing config-registry for command line module
@hkaiser
Copy link
Member Author

hkaiser commented Jan 29, 2024

The failures seem to be unrelated, and the changes do not affect existing code (hopefully)... Merging now.

@hkaiser hkaiser merged commit d14a919 into master Jan 29, 2024
61 of 73 checks passed
@hkaiser hkaiser deleted the fixing_4703 branch January 29, 2024 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant