Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

faster version of per_file channels #431

Merged

Conversation

rettigl
Copy link
Member

@rettigl rettigl commented Jun 22, 2024

This PR implements an alternative method to generate the per_file columns, which is much more performant.
Additionally, the function to add offsets is modified for a moderate speedup (removing the map_partitions call, and rather directly apply to the df columns)

@coveralls
Copy link
Collaborator

coveralls commented Jun 22, 2024

Pull Request Test Coverage Report for Build 9628505451

Details

  • 33 of 39 (84.62%) changed or added relevant lines in 1 file are covered.
  • 2 unchanged lines in 1 file lost coverage.
  • Overall coverage increased (+0.008%) to 92.003%

Changes Missing Coverage Covered Lines Changed/Added Lines %
sed/loader/mpes/loader.py 33 39 84.62%
Files with Coverage Reduction New Missed Lines %
sed/loader/mpes/loader.py 2 86.81%
Totals Coverage Status
Change from base Build 9588080384: 0.008%
Covered Lines: 6500
Relevant Lines: 7065

💛 - Coveralls

@coveralls
Copy link
Collaborator

coveralls commented Jun 22, 2024

Pull Request Test Coverage Report for Build 9628850484

Details

  • 42 of 48 (87.5%) changed or added relevant lines in 2 files are covered.
  • 2 unchanged lines in 1 file lost coverage.
  • Overall coverage increased (+0.003%) to 91.998%

Changes Missing Coverage Covered Lines Changed/Added Lines %
sed/loader/mpes/loader.py 33 39 84.62%
Files with Coverage Reduction New Missed Lines %
sed/loader/mpes/loader.py 2 86.81%
Totals Coverage Status
Change from base Build 9588080384: 0.003%
Covered Lines: 6496
Relevant Lines: 7061

💛 - Coveralls

@rettigl
Copy link
Member Author

rettigl commented Jun 22, 2024

@zain-sohail This solution here passes the benchmarks, even with slight improvements on speed (maybe due to different worker specs), whereas the earlier code consistently is a factor 2-3 slower. I merge this here with the other branch, and then have you review it, if you want to see the changes, you can refer to this PR.

@rettigl rettigl merged commit 0004113 into energy_calibration_bias_shift Jun 22, 2024
6 checks passed
@rettigl rettigl deleted the energy_calibration_performance_fix branch June 22, 2024 23:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants