3-Shell-and-Git.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=images/ucsc_banner.png width=\"500\">\n",
    "\n",
    "# Command Line and Git"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Command line interface\n",
    "\n",
    "The command line interface (CLI) is a powerful way to interact with your computer's operating system. In contrast a graphical user interface (GUI) may be more convenient to use, but you have less control in how you use the software. Also, most cutting-edge research tools are first developed for command line interfaces. \n",
    "\n",
    "<img src=images/gvng.jpg height=\"75%\">\n",
    "*http://www.datacarpentry.org/shell-genomics/lessons/01_the_filesystem.html\n",
    "\n",
    "The shell takes commands from the command line and converts them into instructions for your computer to execute. There are many shells out there. The most popular shell is called bash, but other popular shells include zsh, csh, and tcsh. We are going to use bash here, but I recommend trying out zsh. The shell is started in a terminal emulator and how you start your terminal session depends on your operating system. Macs and linux machines usually have a terminal program pre-installed, but windows users will need to download a terminal emulator.\n",
    "\n",
    "### Mac\n",
    "Applications -> Utilities -> Terminal\n",
    "\n",
    "### Windows\n",
    "Download and install a terminal emulator like PuTTY or MobaXterm\n",
    "\n",
    "### Linux \n",
    "On Ubuntu there is a shortcut: ctrl-alt-t"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Commands\n",
    "\n",
    "We're going to be working with data on a remote linux server. The interface is entirely in text, so we will need to use the command line. Log onto your server using SSH and then we are going to explore the filesystem. The first command we are going to look at is **ls** - list directory contents\n",
    "\n",
    "<img src=images/command_line.jpg>\n",
    "\n",
    "\n",
    "Try these different flags:\n",
    "* ls --help\n",
    "* ls --version\n",
    "* ls -lha\n",
    "\n",
    "The manual page for **ls** can be found using this command:\n",
    "```\n",
    "man ls\n",
    "```\n",
    "You can search the manual page by typing / and then a search term. Try searching for what -h does by typing /-h and hitting enter. You can move the cursor to the next instance of the search term by typing n. Shift-N searches in the reverse direction.\n",
    "\n",
    "Another important command is **cd** - change directory. First, let's go to the root of the filesystem. This can be done using:\n",
    "```\n",
    "cd /\n",
    "```\n",
    "Use ls to list all of the files in your root directory. These folders have all of the programs and configuration files that make up your operating system. \n",
    "\n",
    "### Pick a program in /bin and look at it's manual page\n",
    "\n",
    "<img src=images/filesystem.jpg>\n",
    "*http://www.blackmoreops.com/wp-content/uploads/2015/02/Linux-file-system-hierarchy-Linux-file-structure-optimized.jpg\n",
    "\n",
    "\n",
    "To go back to your home directory type:\n",
    "```\n",
    "cd ~\n",
    "```\n",
    "Another important utility is **mkdir** - make directory. Let's make a directory in our home directory called **bin**.\n",
    "```\n",
    "mkdir bin \n",
    "```\n",
    "Your ~/bin directory is a good place to store executables."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## Pro Practice\n",
    "\n",
    "1. Count the number of lines in data/titles.csv (look at **wc**)\n",
    "\n",
    "2. Use the **find** command to locate a file called cpuinfo and report the number of cpus on the machine.  \n",
    "\n",
    "3. Pipe several commands together to sort the titles in data/titles.csv. Try to remove the comma and the date at the end of the line without modifying the file (use **cat**, **sed** or **awk** and **sort**)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Git\n",
    "\n",
    "A great way to merge code from several collaborators and to fix broken code while also developing new features is to use a version control system called **git**. Git keeps a history of code changes in a source repository that you can share with other programmers.\n",
    "\n",
    "<img src=images/git_fig.png width=\"500\">\n",
    "*https://www.atlassian.com/git/tutorials/why-git/git-for-developers\n",
    "\n",
    "## Install git\n",
    "Many computer do not come with git by default. To install git on our remote machine type in this command: \n",
    "\n",
    "```\n",
    "sudo apt-get install git\n",
    "```\n",
    "\n",
    "This is a command to install many different kinds of software. It will show you all of the software to be installed and ask you if you want to continue with the installation. Type Y and enter. The sudo command gives you special permission to modify important files and directories. With great power comes great responsibility, so be careful whenever you use the sudo command!\n",
    "\n",
    "Now enter **git** into the command line and you should see all of the git commands you can run.\n",
    "\n",
    "## Make a github account\n",
    "\n",
    "Go to [github](https://github.com/) and click the sign up button. This is a great place to save your code in case your laptop breaks or gets stolen. It's also a great place to collaborate on projects and to share your code with others. Git and github have special vocabulary you will need to learn to get the most out of this software. For instance, you store your code in what is called a github repository or repo.\n",
    "\n",
    "Once you have your github account set up, go to the [workshop repository](https://github.com/jvivian/BD2K-Summer-Workshop). Here is all of the code we prepared for the workshop. To make your own working copy of the repository, **fork** it by clicking the fork button. Now you own a copy of the repo you can download and make changes to. To download your workshop repo go to the terminal session that is connected to the remote server and type in this command:\n",
    "```\n",
    "git clone https://url/to/your/repo\n",
    "```\n",
    "Let's **cd** into the repo and list the files using **ls**. Let's check the status of the repo using this command:\n",
    "```\n",
    "git status\n",
    "```\n",
    "Now let's make a change to the repo. We will add a file that lists the contributors to the repo.\n",
    "```\n",
    "echo John Smith > contributors.txt \n",
    "```\n",
    "Run ```git status``` again and you should see that git recognizes a new file in your workspace. Now we need to add the file to the git index.\n",
    "```\n",
    "git add contributors.txt\n",
    "```\n",
    "\n",
    "Now contributors.txt is in the index and next we want to commit the file to your local repository. At this point, we are going to add a message to our commit so that we can keep track of all the commits in the repository.\n",
    "```\n",
    "git commit -m 'Add a contributors file'\n",
    "```\n",
    "Now that you have committed changes to your local repository, let's copy these changes to your github repository. To do this, we **push** changes to the remote repository. The remote repository is called the **origin** and **master** is the branch.\n",
    "```\n",
    "git push origin master\n",
    "```\n",
    "Git will prompt you for a github username and password. If someone else makes changes to your github repository, you will need to update your local repository using the **pull** command.\n",
    "```\n",
    "git pull origin master\n",
    "```\n",
    "When you start working on a new feature in a git repo, it's a good idea to work from a separate branch. This way, if you break something you can just delete the branch and switch back to the master branch. If you want to add your changes to master you can **rebase** your branch onto the master branch. To make a new branch, use the **branch** command.\n",
    "```\n",
    "git branch\n",
    "``` \n",
    "```\n",
    "git branch bd2k-workshop-2016\n",
    "```\n",
    "Use the **checkout** command to switch to your new branch.\n",
    "```\n",
    "git checkout bd2k-workshop-2016\n",
    "```\n",
    "\n",
    "## Pro practice\n",
    "\n",
    "* Commit a new change to your branch and do an interactive rebase onto the master branch\n",
    "* Check out this article about writing informative [commit messages](http://chris.beams.io/posts/git-commit/). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NIH BD2K Center for Big Data in Translational Genomics, UCSC Genomics Institute"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}