Exercises.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercises\n",
    "\n",
    "Learning a language requires _using_ it. This is why I can no longer speak French. \n",
    "\n",
    "This page gives gives a series of exercises for basic analyses of data in the Datajoint pipeline. \n",
    "\n",
    "If you have not yet reconfigured your Datajoint client to access the latest version of the pipeline, you can find details [on the kavli wiki here](https://www.ntnu.no/wiki/display/kavli/DataJoint%3A+Electrophysiology+Pipeline). You will require your NTNU username, database password, and the access and secret keys to the object storage. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import scipy.ndimage\n",
    "import datetime\n",
    "import numpy as np\n",
    "import datajoint as dj\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "SECRET_KEY = \"\"\n",
    "ACCESS_KEY = \"\"\n",
    "USER = \"\"\n",
    "PASS = \"\"\n",
    "\n",
    "dj.config['database.host'] = 'datajoint.it.ntnu.no'\n",
    "dj.config['database.user'] = USER\n",
    "dj.config['database.password'] = PASS\n",
    "dj.config[\"enable_python_native_blobs\"] = True\n",
    "dj.config[\"stores\"] = {   'ephys_store': {   'access_key': ACCESS_KEY,\n",
    "                                     'bucket': 'ephys-store-computed',\n",
    "                                     'endpoint': 's3.stack.it.ntnu.no:443',\n",
    "                                     'secure': True,\n",
    "                                     'location': '',\n",
    "                                     'protocol': 's3',\n",
    "                                     'secret_key': SECRET_KEY},\n",
    "                  'ephys_store_manual': {   'access_key': ACCESS_KEY,\n",
    "                                            'bucket': 'ephys-store-manual',\n",
    "                                            'endpoint': 's3.stack.it.ntnu.no:443',\n",
    "                                            'secure': True,\n",
    "                                            'location': '',\n",
    "                                            'protocol': 's3',\n",
    "                                            'secret_key': SECRET_KEY}}\n",
    "dj.config['custom'] = {\n",
    "        'database.prefix': 'group_shared_',\n",
    "        'mlims.database': 'prod_mlims_data',\n",
    "        'flask.database': 'group_shared_flask',\n",
    "        'drive_config': {\n",
    "          'local': 'C:/',\n",
    "          'network': 'N:/'}}\n",
    "\n",
    "analysis = dj.create_virtual_module('analysis', 'group_shared_analysis')\n",
    "analysis_param = dj.create_virtual_module('analysis_param', 'group_shared_analysis_param')\n",
    "\n",
    "key = {'animal_id': 'f5e35afe2bf2a9f8',\n",
    " 'datasource_id': 0,\n",
    " 'session_time': datetime.datetime(2019, 11, 11, 12, 45, 59),\n",
    " 'unit': 167,\n",
    " 'task_type': 'OpenField',\n",
    " 'task_start': '21.00',\n",
    " 'task_spike_tracking_hash': '083ff30e220fbe26815e820e54451af6',\n",
    " 'occu_params_name': 'default',\n",
    " 'smoothing_params_name': 'default',\n",
    " 'analysis_package': 'python',\n",
    " 'cell_selection_params_name': 'default',\n",
    " 'field_detect_params_name': 'default',\n",
    " 'score_params_name': 'default'}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1: Path maps\n",
    "\n",
    "In the first exercise, you will plot a path and spike map\n",
    "\n",
    "Goal: given a `key` to the Datajoint table `analysis.TaskSpikesTracking`, create and display a path Map for that cell in that task. \n",
    "\n",
    "This one is fairly simple: due to the convenience of Datajoint, you need only fetch four arrays from the database, and plot them directly using `matplotlib`. Don't forget to set the correct aspect ratio. Further, because this is about using Python correctly, not Datajoint in particular, the below code will do the database hevay lifting for you"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pos_t, pos_x, pos_y = (analysis.TaskTracking & key).fetch1(\"task_timestamps\", \"x_pos\", \"y_pos\")\n",
    "spk_t, spk_x, spk_y = (analysis.TaskSpikesTracking & key).fetch1(\"spike_times\", \"x_pos\", \"y_pos\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2: Coverage\n",
    "\n",
    "It's useful to know how much of the arena the animal has visited. This requires some manipulation of the data you have acquired so far. IWhat assumptions are you making?\n",
    "\n",
    "**Hint**: You will find the [NumPy Documentation](https://docs.scipy.org/doc/numpy) very useful, and the [2D histogram](https://docs.scipy.org/doc/numpy/reference/generated/numpy.histogram2d.html) function in particular, and it's hard to get anything done with arrays without [meshgrid](https://docs.scipy.org/doc/numpy/reference/generated/numpy.meshgrid.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3: Rate maps\n",
    "\n",
    "A ratemap takes in a list of animal positions, and a list of spike positions, and computes the rate of firing of that unit at each location in an arena. \n",
    "\n",
    "Based on these data, create:\n",
    "* An occupancy map: a binned map of the time the animal spent in each location\n",
    "* A spike map: a binned map of the number of unit spikes in each location\n",
    "* A rate map: the rate at which the unit fires in each location\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise 4: Smoothing\n",
    "\n",
    "Coarse discretisation like this is highly prone to distortionary boundary effects, which can be ameliorated by smoothing. Conventionally, by smoothing, we mean that each pixel is made more like its neighbours. The fascinating detail (and a significant part of the engineering-physics-maths sub-field of signal processing) is how neighbours are weighted in this calculation\n",
    "\n",
    "This is where analysis starts to become extremely complicated by the distressing existence of boundaries. \n",
    "\n",
    "How we we handle unexpected values?\n",
    "\n",
    "The simplest solution for convolution in Python is to use [`scipy.ndimage.filters.convolve`](https://docs.scipy.org/doc/scipy-0.15.1/reference/generated/scipy.ndimage.filters.convolve.html)\n",
    "\n",
    "This requires a `weights` keyword that determines how we weight the value of each neighbour."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}