Skip to content

Commit

Permalink
Reorganize connection lesson
Browse files Browse the repository at this point in the history
  • Loading branch information
s-sajid-ali committed Aug 22, 2024
1 parent 2da2383 commit 0da2ac7
Showing 1 changed file with 159 additions and 155 deletions.
314 changes: 159 additions & 155 deletions _episodes/11-connecting.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,165 @@ Even if your cluster does not require it, the next section will guide you
through the use of SSH keys and an SSH agent to both strengthen your security
_and_ make it more convenient to log in to remote systems.

## Log In to the Cluster

Go ahead and open your terminal or graphical SSH client, then log in to the cluster.

Check failure on line 70 in _episodes/11-connecting.md

View workflow job for this annotation

GitHub Actions / lint_common_files

Trailing spaces [Expected: 0 or 2; Actual: 1]

_episodes/11-connecting.md:70:85 MD009/no-trailing-spaces Trailing spaces [Expected: 0 or 2; Actual: 1]
If you are connecting from a remote location that is not on the NYU network (your home
for example), you have two options:
- VPN Option: [set up your computer to use the NYU VPN]

Check failure on line 73 in _episodes/11-connecting.md

View workflow job for this annotation

GitHub Actions / lint_common_files

Unordered list indentation [Expected: 0; Actual: 2]

_episodes/11-connecting.md:73:1 MD007/ul-indent Unordered list indentation [Expected: 0; Actual: 2]

Check failure on line 73 in _episodes/11-connecting.md

View workflow job for this annotation

GitHub Actions / lint_common_files

Lists should be surrounded by blank lines [Context: "- VPN Option: [set up your com..."]

_episodes/11-connecting.md:73 MD032/blanks-around-lists Lists should be surrounded by blank lines [Context: "- VPN Option: [set up your com..."]
(https://www.nyu.edu/life/information-technology/infrastructure/network-services/vpn.html)

Check failure on line 74 in _episodes/11-connecting.md

View workflow job for this annotation

GitHub Actions / lint_common_files

Bare URL used [Context: "https://www.nyu.edu/life/infor..."]

_episodes/11-connecting.md:74:5 MD034/no-bare-urls Bare URL used [Context: "https://www.nyu.edu/life/infor..."]
. Once you've created a VPN connection, you can proceed as if you were connected to the NYU net.
- Gateway Option: go through our gateway servers (example below). Gateways are designed to

Check failure on line 76 in _episodes/11-connecting.md

View workflow job for this annotation

GitHub Actions / lint_common_files

Unordered list indentation [Expected: 0; Actual: 2]

_episodes/11-connecting.md:76:1 MD007/ul-indent Unordered list indentation [Expected: 0; Actual: 2]

Check failure on line 76 in _episodes/11-connecting.md

View workflow job for this annotation

GitHub Actions / lint_common_files

Trailing spaces [Expected: 0 or 2; Actual: 1]

_episodes/11-connecting.md:76:93 MD009/no-trailing-spaces Trailing spaces [Expected: 0 or 2; Actual: 1]
support only a very minimal set of commands and their only purpose is to let users connect
HPC systems without needing to first connect to the VPN.

You do not need to use the NYU VPN or gateways if you are connected to the NYU network (wired

Check failure on line 80 in _episodes/11-connecting.md

View workflow job for this annotation

GitHub Actions / lint_common_files

Trailing spaces [Expected: 0 or 2; Actual: 1]

_episodes/11-connecting.md:80:94 MD009/no-trailing-spaces Trailing spaces [Expected: 0 or 2; Actual: 1]
connection in your office or WiFi) or if you have VPN connection initiated. In this case you
can ssh directly to the clusters.

> ## Before copying these command typing them on your computer
>
> Replace `{{ site.remote.user }}` with your username for all the instructions below!
{: .callout}

* If you choose the Gateway Option, start by hopping onto the gateway nodes first by:

Check failure on line 89 in _episodes/11-connecting.md

View workflow job for this annotation

GitHub Actions / lint_common_files

Unordered list style [Expected: dash; Actual: asterisk]

_episodes/11-connecting.md:89:1 MD004/ul-style Unordered list style [Expected: dash; Actual: asterisk]
```
{{ site.local.prompt }} ssh {{ site.remote.user }}@gw.hpc.nyu.edu
```
{: .language-bash}


Check failure on line 95 in _episodes/11-connecting.md

View workflow job for this annotation

GitHub Actions / lint_common_files

Multiple consecutive blank lines [Expected: 1; Actual: 2]

_episodes/11-connecting.md:95 MD012/no-multiple-blanks Multiple consecutive blank lines [Expected: 1; Actual: 2]
```
{{ site.local.prompt }} ssh {{ site.remote.user }}@{{ site.remote.login }}
```
{: .language-bash}

You may be asked for your password. Watch out: the characters you type after
the password prompt are not displayed on the screen. Normal output will resume
once you press `Enter`.

You may have noticed that the prompt changed when you logged into the remote
system using the terminal (if you logged in using PuTTY this will not apply
because it does not offer a local terminal). This change is important because
it can help you distinguish on which system the commands you type will be run
when you pass them into the terminal. This change is also a small complication
that we will need to navigate throughout the workshop. Exactly what is displayed
as the prompt (which conventionally ends in `$`) in the terminal when it is
connected to the local system and the remote system will typically be different
for every user. We still need to indicate which system we are entering commands
on though so we will adopt the following convention:

* `{{ site.local.prompt }}` when the command is to be entered on a terminal

Check failure on line 116 in _episodes/11-connecting.md

View workflow job for this annotation

GitHub Actions / lint_common_files

Unordered list style [Expected: dash; Actual: asterisk]

_episodes/11-connecting.md:116:1 MD004/ul-style Unordered list style [Expected: dash; Actual: asterisk]
connected to your local computer
* `{{ site.remote.prompt }}` when the command is to be entered on a
terminal connected to the remote system
* `$` when it really doesn't matter which system the terminal is connected to.

## Looking Around Your Remote Home

Very often, many users are tempted to think of a high-performance computing
installation as one giant, magical machine. Sometimes, people will assume that
the computer they've logged onto is the entire computing cluster. So what's
really happening? What computer have we logged on to? The name of the current
computer we are logged onto can be checked with the `hostname` command. (You
may also notice that the current hostname is also part of our prompt!)

```
{{ site.remote.prompt }} hostname
```
{: .language-bash}

```
{{ site.remote.host }}
```
{: .output}

So, we're definitely on the remote machine. Next, let's find out where we are
by running `pwd` to **p**rint the **w**orking **d**irectory.

```
{{ site.remote.prompt }} pwd
```
{: .language-bash}

```
{{ site.remote.homedir }}/{{ site.remote.user }}
```
{: .output}

Great, we know where we are! Let's see what's in our current directory:

```
{{ site.remote.prompt }} ls
```
{: .language-bash}
```
id_ed25519.pub
```
{: .output}

The system administrators may have configured your home directory with some
helpful files, folders, and links (shortcuts) to space reserved for you on
other filesystems. If they did not, your home directory may appear empty. To
double-check, include hidden files in your directory listing:

```
{{ site.remote.prompt }} ls -a
```
{: .language-bash}
```
. .bashrc id_ed25519.pub
.. .ssh
```
{: .output}

In the first column, `.` is a reference to the current directory and `..` a
reference to its parent (`{{ site.remote.homedir }}`). You may or may not see
the other files, or files like them: `.bashrc` is a shell configuration file,
which you can edit with your preferences; and `.ssh` is a directory storing SSH
keys and a record of authorized connections.

{% unless site.remote.portal %}

### Install Your SSH Key

If you transferred your SSH public key with `scp`, you should see
`id_ed25519.pub` in your home directory. To "install" this key, it must be
listed in a file named `authorized_keys` under the `.ssh` folder.

If the `.ssh` folder was not listed above, then it does not yet
exist: create it.

```
{{ site.remote.prompt }} mkdir ~/.ssh
```
{: .language-bash}

Now, use `cat` to print your public key, but redirect the output, appending it
to the `authorized_keys` file:

```
{{ site.remote.prompt }} cat ~/id_ed25519.pub >> ~/.ssh/authorized_keys
```
{: .language-bash}

That's all! Disconnect, then try to log back into the remote: if your key and
agent have been configured correctly, you should not be prompted for the
password for your SSH key.

```
{{ site.remote.prompt }} logout
```
{: .language-bash}

```
{{ site.local.prompt }} ssh {{ site.remote.user }}@{{ site.remote.login }}
```
{: .language-bash}




### Better Security With SSH Keys

The [Lesson Setup]({{ page.root }}/setup) provides instructions for installing
Expand Down Expand Up @@ -312,161 +471,6 @@ Use the **s**ecure **c**o**p**y tool to send your public key to the cluster.
{: .language-bash}
{% endif %}
## Log In to the Cluster
Go ahead and open your terminal or graphical SSH client, then log in to the cluster.
If you are connecting from a remote location that is not on the NYU network (your home
for example), you have two options:
- VPN Option: [set up your computer to use the NYU VPN]
(https://www.nyu.edu/life/information-technology/infrastructure/network-services/vpn.html)
. Once you've created a VPN connection, you can proceed as if you were connected to the NYU net.
- Gateway Option: go through our gateway servers (example below). Gateways are designed to
support only a very minimal set of commands and their only purpose is to let users connect
HPC systems without needing to first connect to the VPN.
You do not need to use the NYU VPN or gateways if you are connected to the NYU network (wired
connection in your office or WiFi) or if you have VPN connection initiated. In this case you
can ssh directly to the clusters.
> ## Before copying these command typing them on your computer
>
> Replace `{{ site.remote.user }}` with your username for all the instructions below!
{: .callout}
* If you choose the Gateway Option, start by hopping onto the gateway nodes first by:
```
{{ site.local.prompt }} ssh {{ site.remote.user }}@gw.hpc.nyu.edu
```
{: .language-bash}
```
{{ site.local.prompt }} ssh {{ site.remote.user }}@{{ site.remote.login }}
```
{: .language-bash}
You may be asked for your password. Watch out: the characters you type after
the password prompt are not displayed on the screen. Normal output will resume
once you press `Enter`.
You may have noticed that the prompt changed when you logged into the remote
system using the terminal (if you logged in using PuTTY this will not apply
because it does not offer a local terminal). This change is important because
it can help you distinguish on which system the commands you type will be run
when you pass them into the terminal. This change is also a small complication
that we will need to navigate throughout the workshop. Exactly what is displayed
as the prompt (which conventionally ends in `$`) in the terminal when it is
connected to the local system and the remote system will typically be different
for every user. We still need to indicate which system we are entering commands
on though so we will adopt the following convention:
* `{{ site.local.prompt }}` when the command is to be entered on a terminal
connected to your local computer
* `{{ site.remote.prompt }}` when the command is to be entered on a
terminal connected to the remote system
* `$` when it really doesn't matter which system the terminal is connected to.
## Looking Around Your Remote Home
Very often, many users are tempted to think of a high-performance computing
installation as one giant, magical machine. Sometimes, people will assume that
the computer they've logged onto is the entire computing cluster. So what's
really happening? What computer have we logged on to? The name of the current
computer we are logged onto can be checked with the `hostname` command. (You
may also notice that the current hostname is also part of our prompt!)
```
{{ site.remote.prompt }} hostname
```
{: .language-bash}
```
{{ site.remote.host }}
```
{: .output}
So, we're definitely on the remote machine. Next, let's find out where we are
by running `pwd` to **p**rint the **w**orking **d**irectory.
```
{{ site.remote.prompt }} pwd
```
{: .language-bash}
```
{{ site.remote.homedir }}/{{ site.remote.user }}
```
{: .output}
Great, we know where we are! Let's see what's in our current directory:
```
{{ site.remote.prompt }} ls
```
{: .language-bash}
```
id_ed25519.pub
```
{: .output}
The system administrators may have configured your home directory with some
helpful files, folders, and links (shortcuts) to space reserved for you on
other filesystems. If they did not, your home directory may appear empty. To
double-check, include hidden files in your directory listing:
```
{{ site.remote.prompt }} ls -a
```
{: .language-bash}
```
. .bashrc id_ed25519.pub
.. .ssh
```
{: .output}
In the first column, `.` is a reference to the current directory and `..` a
reference to its parent (`{{ site.remote.homedir }}`). You may or may not see
the other files, or files like them: `.bashrc` is a shell configuration file,
which you can edit with your preferences; and `.ssh` is a directory storing SSH
keys and a record of authorized connections.
{% unless site.remote.portal %}
### Install Your SSH Key
If you transferred your SSH public key with `scp`, you should see
`id_ed25519.pub` in your home directory. To "install" this key, it must be
listed in a file named `authorized_keys` under the `.ssh` folder.
If the `.ssh` folder was not listed above, then it does not yet
exist: create it.
```
{{ site.remote.prompt }} mkdir ~/.ssh
```
{: .language-bash}
Now, use `cat` to print your public key, but redirect the output, appending it
to the `authorized_keys` file:
```
{{ site.remote.prompt }} cat ~/id_ed25519.pub >> ~/.ssh/authorized_keys
```
{: .language-bash}
That's all! Disconnect, then try to log back into the remote: if your key and
agent have been configured correctly, you should not be prompted for the
password for your SSH key.
```
{{ site.remote.prompt }} logout
```
{: .language-bash}
```
{{ site.local.prompt }} ssh {{ site.remote.user }}@{{ site.remote.login }}
```
{: .language-bash}
{% endunless %}
{% include links.md %}
Expand Down

0 comments on commit 0da2ac7

Please sign in to comment.