Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

draft HPC instruction with brown instructions #14

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 108 additions & 0 deletions onboarding/hpc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
# HPC Access

## Lab computers

- medium compute in the lab computers
- request access on slack for lab computers
- there is a default password there

## URI HPC


- request URI access by e-mailing with Dr. Brown in cc
- [HPC at URI]()


## Unity
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Unity




## HPC Interactive Sessions with Jupyter Notebooks

Using Notebooks with CCV (Oscar) at Brown
prereq: Get an account on Oscar from CCV

log in to oscar:

```
ssh <username>@ssh.ccv.brown.edu
```
To use a notebook on oscar, you will need an interactive session. Get one with the following: see the above documentation

```
interact -t 01:00:00 -m 100g
```
that gives a time (-t)limit of 1 hour and (100Gb) of ram on one (default number) node

1. load python 3.6. You can also add this to your auto load as described in the help on software

```
module load python/3.6.6
```
1. (optional, recommended, cd to project folder) launch a notebook to be used remotely

```
jupyter notebook --no-browser --ip=$(hostname -i) --port=8888
```
the notebook will launch and the response will include the ip address and the token.
```
http://172.20.204.32:8888/?token=a40dd28fc49f415b9e2f5c867c653ee9d458bb8c74432575
http://<ip-address>:<remote_port>/?token=<token>
```
1. in a new terminal window locally, ssh again to map the notebook to a local port

```
ssh -N -L 8000:172.20.204.32:8888 <user>@ssh6.ccv.brown.edu
```
enter your password and nothing will happen, but it will be working.
1. point browser to localhost:8000 and enter the token to access the notebook server
1. moving files to work locally if neeeded with:

```
scp /path/to/source/file <username>@transfer.ccv.brown.edu:/path/to/destination/file
```



# Log into Unity account and run commands:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Log into Unity account and run commands:
## Unity


- https://unity.uri.edu/panel/account.php
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you expand this step a little?

- Either use Jupyter notebook for a more user interface-based experience. If preferred consol/CMD, add the SSH key to the local computer by following the following steps.
- Generate the SSH key from the account. The key should begin with ssh-rsa… … ….
- Use the key to setup a configuration file to the local system. To do this, save the ssh key as a .key file. Notepad++ can be used for this.
- Save the .key file to a folder. Such as: C:\Users\pujag\.ssh\unity-privkey.key.
- Then generate a configuration file to connect to Unity.
Comment on lines +70 to +74
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's fix the formatting her so that the steps are gouped under some heading 3 ### subheadings



## The configuration file should be like the following:

- “Host unity
- HostName unity.rc.umass.edu
- User puja_ghosh_uri_edu
- IdentityFile C:\Users\pujag\.ssh\unity-privkey.key”
Comment on lines +79 to +82
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this the content of the file? can you make it a verbatim/ code block like this:

```
this is an example
```


- Save the file to C:\Users\pujag\.ssh\ as config file. This file should have no file extension. The file name should be only config.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you make this general? like "save it to your home directory"


- Now Unity can be launched by opening CMD and running the code “ssh unity”

- The details are documented here https://docs.unity.uri.edu/connecting/ssh.html


## Add Git to Unity with SSH:

- To pull or clone any Git repository, Git does not use password anymore. It uses SSH key. To generate the key, go to account>settings>SSH..>generate a new key.
- Save the key to a secure location. This key replaces the password. Whenever the console asks for password, use this key instead.

## To add Git account to the system, run the following command in the console:

- ssh-keygen -t ed25519 -C "your_email@example.com"
- Note: If you are using a legacy system that doesn't support the Ed25519 algorithm, use:
ssh-keygen -t rsa -b 4096 -C “your_email@example.com”
- when the command asks for a file name click enter to set it to default. The command should look like this “> Enter a file in which to save the key (/c/Users/YOU/.ssh/id_ALGORITHM):[Press enter]”
- This will generate a .ssh folder in the home directory. This cannot be directly seen in the file browser as it is a hidden folder. To show hidden folders turn on show dot folders in https://ood.unity.rc.umass.edu/pun/sys/dashboard/files/fs//home/<user_name>
- The folder now contains two key files privet and public. The id_rsa is the privet key and id_rsa.pub is the public key.
- Open the public key file, copy the key (the key starts with ssh-rsa) and add it to your git profile. Profile>settings>SSH…>add new key.
- This will add the remote server to your account.
- Now run “ssh -T git@github.com” to check if your username shows up in the console. This will confirm that the account has been added.