GIN-client-tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook I give an example of how you can manage your GIN repository using command line tools. First step is to download and [install the GIN client](https://gin.g-node.org/G-Node/Info/wiki/GIN+CLI+Setup) on your computer. GIN client can be installed on Windows, Mac, and Linux machines. Also make sure that you install git-annex (the Windows installer can be found [here](https://downloads.kitenet.net/git-annex/windows/current/)). Once the GIN client is installed, register your account with [the web GIN](https://gin.g-node.org/user/sign_up). Registration is easy and when it is done, you can login to your GIN account via the command line (just specify your own username; if on Windows make sure you are using the cmd terminal and not the powerShell):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "powershell"
    }
   },
   "outputs": [],
   "source": [
    "gin login dervinism"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we intend to create a repository. We go to the folder where the repository is located. In my instance it is in C:\\Users\\44079\\Work\\Bristol\\mockRepositoryCLI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "powershell"
    }
   },
   "outputs": [],
   "source": [
    "cd C:\\Users\\44079\\Work\\Bristol\\mockRepositoryCLI"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This mock repository contains mock Neuropixels data acquired during a single recording session with two silicon probes. It has the following folder structure:\n",
    "mock_repository\\Jack_Mellor_group\\MD\\mock_project\\animal_ID\\session1_2022-05-16\\probe1 (all data pertaining to the recording probe 1)\n",
    "mock_repository\\Jack_Mellor_group\\MD\\mock_project\\animal_ID\\session1_2022-05-16\\probe2 (all data pertaining to the recording probe 2)\n",
    "mock_repository\\Jack_Mellor_group\\MD\\mock_project\\animal_ID\\session1_2022-05-16 (data that is related to both recording probes like pupil video, for example).\n",
    "First, we initialise the mock_repository folder as a local GIN repository by typing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "powershell"
    }
   },
   "outputs": [],
   "source": [
    "gin init"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What this does is creating .git folder which keeps track of changes we make to our local repository. There is no gitignore file, so all files in the local repository are tracked. If you want to make exceptions to that, please add a [gitignore file](https://git-scm.com/docs/gitignore) specifying which files should not be synchronised with the remote repository. We can now create the remote repository on the remote GIN server by typing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "powershell"
    }
   },
   "outputs": [],
   "source": [
    "gin create"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will be prompted to enter the name of the newly created repository. Let's name it mockRepositoryCLI to indicate that we are using the command line interface to create this repository and so it has a different name than the one we might have created earlier using the web interface. If you login to your account using the web interface, you will be able to see that a new empty repository with this name has been created. Next, we associate our local GIN server with the remote GIN server by typing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "powershell"
    }
   },
   "outputs": [],
   "source": [
    "gin add-remote mockRepositoryCLI gin:dervinism/mockRepositoryCLI"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "gin: indicates that the server resides on the remote GIN server. If the remote server was residing localy, you could specify it in the form dir:path. Next we upload the contents of our local GIN repository to the remote GIN repository by typing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "powershell"
    }
   },
   "outputs": [],
   "source": [
    "gin upload ."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now If you want to transfer the ownership of mockRepositoryCLI to the University of Bristol this could be done using the web interface. We create a team and associate it with a particular repository using web interface too, as gin command line tools are very limited. Alternatively, one can use a [DataLad service](http://handbook.datalad.org/en/latest/basics/101-139-gin.html) or Git command line tools as GIN is integrated with these much more powerful tools. However, there is a steep learning curve associated with mastering the use of DataLad.\n",
    "\n",
    "Let's say your collaborator wants to download (clone) the remote repository on the local computer. This could simply be done by typing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "powershell"
    }
   },
   "outputs": [],
   "source": [
    "gin get dervinism/mockRepositoryCLI"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The remote repository will be downloaded with its .git folder which has the info associating it with its remote instance. If you want to make sure that your local repository is up to date with its remote instance, you can just simply run:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "powershell"
    }
   },
   "outputs": [],
   "source": [
    "gin sync"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This command updates the local repository. However, any new files on the remote server are downloaded only as pointers. If you want to download full files, you should specify that file content is required:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "powershell"
    }
   },
   "outputs": [],
   "source": [
    "gin sync --content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "powershell"
    }
   },
   "outputs": [],
   "source": [
    "gin sync\n",
    "gin get-content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After working with files in the repository, submit any changes to the local repository by typing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "powershell"
    }
   },
   "outputs": [],
   "source": [
    "gin commit --message \"updated spike times\" \\\n",
    "C:\\Users\\44079\\Work\\Bristol\\mockRepositoryCLI\\Jack_Mellor_group\\MD\\mock_project\\animal_ID\\session1_2022-05-16\\probe1\\spike_times.npy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And push those changes from the local to the remote repository by typing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "powershell"
    }
   },
   "outputs": [],
   "source": [
    "gin upload ."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A useful action that you can perform on a remote repository is to revert certain unwanted recent changes and commit new changes. This action can only be performed using the Git command line tools as web interface does not permit this action (same true of Github). We revert the remote repository to its previous state by typing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "powershell"
    }
   },
   "outputs": [],
   "source": [
    "git log --oneline\n",
    "git reset --hard <commit-hash>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now make the changes and commit them to the remote repository"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "powershell"
    }
   },
   "outputs": [],
   "source": [
    "gin commit --message \"updated spike times again\" \\\n",
    "C:\\Users\\44079\\Work\\Bristol\\mockRepositoryCLI\\Jack_Mellor_group\\MD\\mock_project\\animal_ID\\session1_2022-05-16\\probe1\\spike_times.npy\n",
    "gin upload ."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can find out more about all available GIN commands by typing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "powershell"
    }
   },
   "outputs": [],
   "source": [
    "gin help"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This was a basic demonstration of the GIN command line tools. A more extensive description is available at [Local GIN Client Usage Guide](https://gin.g-node.org/G-Node/Info/wiki/GIN+CLI+Usage+Tutorial).\n",
    "\n",
    "GIN command line tools can also be called as system commands from Python and Matlab. However, as GIN client is a git wrapper, GIN repositories can be managed directly using git commands. In Python one also has a package available for that purpose, called [GitPython](https://gitpython.readthedocs.io/en/stable/). Matlab has git version control integrated within its user interface, so basic GIN functionality should also be available within Matlab (has to be tested though). GIN is also compatible with [DataLad](http://handbook.datalad.org/en/latest/basics/101-139-gin.html) which provides command line tools and Python API for managing your data repositories."
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}