Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how to transfer raw data between universities #7

Open
lpantano opened this issue Jul 27, 2018 · 5 comments
Open

how to transfer raw data between universities #7

lpantano opened this issue Jul 27, 2018 · 5 comments
Labels
docs issues related to knowledgebase docs
Milestone

Comments

@lpantano
Copy link
Contributor

Harvard is subscribed to globus.org. If the people have their institution with that service we can transfer this way:

talk to @sjhosui to get an idea of how it works.

you log in with your harvard credentials.

@lpantano lpantano added the docs issues related to knowledgebase docs label Jul 27, 2018
@sjhosui
Copy link
Contributor

sjhosui commented Jul 28, 2018

Once a user has set up an endpoint for you to download the data from using Globus, you will receive an email with a URL to access the share. It will be of the form https://www.globus.org/app/transfer?&origin_id=______&origin_path=/&add_identity=______.

Clicking on the link will take you to the Globus Web App. You will need to select "Harvard University" as your institute.

screen shot 2018-07-28 at 2 21 59 pm

When you click "Continue", you may be redirected to an error page. Ignore this and refresh your browser. It will take you to the HarvardKey login page. Use your HarvardKey credentials to login and you will be taken to the "Transfer files" page that lists the files available for download from their endpoint on the left, and your endpoint on the right.

screen shot 2018-07-28 at 2 26 49 pm

To set up your endpoint, click on "Endpoints" in the menu above the file browser. Enter a name for your endpoint, and click "Generate a set up key". Download and install Globus Connect Personal

Note that this is to transfer files to your personal machine. To transfer files directly to O2, instructions are available at https://wiki.rc.hms.harvard.edu/display/O2/Globus+Connection+Personal+from+command+line.

screen shot 2018-07-28 at 2 34 09 pm

Use the set up key to complete the installation. Return to "Transfer files" and select your new endpoint in the right hand panel.

Transfer files as you would for a typical FTP client. Select the directory or file in the left panel, then click the right arrow to move it to your endpoint in the right panel.

screen shot 2018-07-28 at 2 41 18 pm

@sjhosui sjhosui changed the title how to transfer raw data between university how to transfer raw data between universities Jul 28, 2018
@sjhosui
Copy link
Contributor

sjhosui commented Jul 28, 2018

Just to re-iterate in case you missed it, the above instructions allow you to set up your own machine as an endpoint. To transfer data directly to O2, follow the instructions at https://wiki.rc.hms.harvard.edu/display/O2/Globus+Connection+Personal+from+command+line.

@lpantano lpantano added this to the First version milestone Aug 3, 2018
@jnhutchinson
Copy link
Collaborator

jnhutchinson commented Aug 21, 2018

Has anyone gotten the directions for direct data transfer to O2 to actually work? I can't even login into the transfer node they way they outline. Logging into an interactive node also doesn't work, globus isn't installing.

@jnhutchinson
Copy link
Collaborator

I'm following up with HMS RC

@jnhutchinson
Copy link
Collaborator

In the instructions, you can
a) skip the transfer node login (we don't have permissions) and
b) don't use the system packages when setting up the virtual environment (Thanks to Ilya for figuring this out!) as this will try to link to a system file that we also can't access.

So instead of "virtualenv venv --system-site-packages" just use "virtualenv venv"

I emailed HMS RC with this info

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs issues related to knowledgebase docs
Projects
None yet
Development

No branches or pull requests

3 participants