Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow users to score models without making them public? #610

Open
YudiXie opened this issue Mar 8, 2024 · 1 comment
Open

allow users to score models without making them public? #610

YudiXie opened this issue Mar 8, 2024 · 1 comment

Comments

@YudiXie
Copy link
Contributor

YudiXie commented Mar 8, 2024

Currently, it seems that BrainScore is designed in a way that people who submit the models have to make both the model weight and network definition publicly available before getting a score (because they are on github). Would it be possible for the user to opt for just exposing the model to the BrainScore team but not making it publicly available?

This feature might be desirable for two reasons:

  1. people might want to score a lot of models during the development process so that the repo will eventually contain many many redundant model entries, and everyone who uses brainscore has to download all models ever scored by all other users.
  2. some users might want to make the model public only after publication, or might not want the model to be public.
@mschrimpf
Copy link
Member

related to brain-score/core#26

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants