Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes for parallell runs #115

Merged

Conversation

oskbor
Copy link
Contributor

@oskbor oskbor commented May 20, 2024

  • Create markers/{slot_name} early in run_marker_search so other processes can detect that marker search is in progress for this slot.
  • Add try in __get_size for Metadata. This mitigates issues that can happen when another process is currently writing a zarr group to the hierarchy.

scarf/datastore/datastore.py Show resolved Hide resolved
scarf/metadata.py Show resolved Hide resolved
@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 80.00000% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 67.44%. Comparing base (440bab1) to head (c00d268).

Files Patch % Lines
scarf/metadata.py 50.00% 2 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #115      +/-   ##
==========================================
- Coverage   67.46%   67.44%   -0.01%     
==========================================
  Files          22       22              
  Lines        5577     5580       +3     
==========================================
+ Hits         3762     3763       +1     
- Misses       1815     1817       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@parashardhapola parashardhapola merged commit 2a1b6e5 into parashardhapola:master May 22, 2024
4 checks passed
@oskbor oskbor deleted the 05-20-fixes_for_parallell_runs branch May 23, 2024 06:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants