Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch dataset locations from Google Drive to MLCommons Cloud #680

Closed
wants to merge 1 commit into from

Conversation

nathanw-mlc
Copy link
Member

@nathanw-mlc nathanw-mlc commented Aug 14, 2023

Some datasets residing on Google Drive have moved to MLCommons' Cloud storage solution. This PR updates the instructions for acquiring the datasets to link to the MLCommons Cloud storage location.

@github-actions
Copy link

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@nathanw-mlc nathanw-mlc marked this pull request as ready for review September 12, 2023 20:41
@nathanw-mlc nathanw-mlc requested a review from a team as a code owner September 12, 2023 20:41
@nv-rborkar
Copy link
Contributor

@sgpyc tagging reference owner for review. Thanks Nathan!

@arjunsuresh
Copy link
Contributor

Hi @nathanw-mlc can you please tell if there is a way to do checksum check while downloading files from MLCommons cloud? We do get errors like this due to incomplete downloads which are not getting flagged.

@WarrenSchultz
Copy link

I'm also getting failed downloads for DLRMv2. Other downloads via MLCommons cloud have extremely erratic download rates.

@nathanw-mlc
Copy link
Member Author

Has this been on ongoing problem? There was some server maintenance yesterday that caused unexpected interuptions.

@WarrenSchultz
Copy link

For the past week at least, I think?

@arjunsuresh
Copy link
Contributor

@nathanw-mlc For us the concern is we are not able to validate the downloaded file as the checksum is not constant across repeated downloads. We first noticed this for gptj-6B download which was 2 months back - checksums were different but the downloaded zip file was extracted successfully and worked as expected.

@nathanw-mlc
Copy link
Member Author

Hmm, that's very strange. Thanks for bringing this to my attention. I'm looking into it.

@arjunsuresh
Copy link
Contributor

Thank you @nathanw-mlc . This is the relevant issue.

@github-actions github-actions bot locked and limited conversation to collaborators Feb 23, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants