Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

package_checker creates file scaling.json in result directory #278

Open
plvlnisse opened this issue Oct 12, 2022 · 2 comments
Open

package_checker creates file scaling.json in result directory #278

plvlnisse opened this issue Oct 12, 2022 · 2 comments

Comments

@plvlnisse
Copy link

When running the package checker to check for a valid MLPerf training submission, the package_checker leaves a file
scaling.json in the results/.../maskrcnn directory. This file makes the submission directory invalid for submission it seems.

python3 -m mlperf_logging.package_checker Final/Lenovo training 2.1.0 --rcp_bert_train_samples

@maanug-nv
Copy link
Contributor

I think this is because of #271 .
You can certainly get rid of these files before creating submission with something like:
find <submission folder> -type f -name 'scaling.json' -delete .

We can also add a flag to the package checker to prevent creation of these scaling.json files (or disable by default and flag to enable), however the scaling.json files must be present to ensure the results summarizer produces accurate normalized scores.

@pgmpablo157321 @emizan76 what do you think, does it make sense to add such a flag to the package checker ?

@pgmpablo157321
Copy link
Contributor

@maanug-nv I think for now the default value of the flag should be to not create such files. That way it wont mess any submission by accident, we can call the package checker with the flag in the submission UI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants