Lecture-6.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Python Performance Tuning\n",
    "### Setup \n",
    "\n",
    "- Install `line_profiler`, using either `pip install line_profiler` or `conda install line_profiler`\n",
    "\n",
    "### What is profile?\n",
    "\n",
    "#### Profiling\n",
    "\n",
    "#### Deterministic profiling\n",
    "- Monitor all function calls/returns and other events\n",
    "- Precise timing\n",
    "- By instrumenting the program - inserting intructions into the program that collect this timing information\n",
    "\n",
    "#### Statistical profiling \n",
    "- Randomly samples the effective instruction pointer\n",
    "- Less overhead, but less precise\n",
    "\n",
    "### Why call counts & profiling statistics are useful?\n",
    "- identify bugs\n",
    "- identify possible tuning points\n",
    "- identify \"hot loops\" that should be carefully optimized\n",
    "- identify high level system design or algorithm adoption\n",
    "\n",
    "\n",
    "### Python **standard** profiling modules\n",
    "\n",
    "#### `cProfile`\n",
    "- lowest overhead, written in C, may not be as widely available\n",
    "\n",
    "#### `profile`\n",
    "- Written in Python, so much higher overhead, but easier to extend\n",
    "\n",
    "#### Non-standard `line_profiler`, but popular for ipython & juputer-notebook\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import cProfile\n",
    "import re"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "         185 function calls (180 primitive calls) in 0.001 seconds\n",
      "\n",
      "   Ordered by: standard name\n",
      "\n",
      "   ncalls  tottime  percall  cumtime  percall filename:lineno(function)\n",
      "        1    0.000    0.000    0.001    0.001 <string>:1(<module>)\n",
      "        1    0.000    0.000    0.001    0.001 re.py:222(compile)\n",
      "        1    0.000    0.000    0.001    0.001 re.py:278(_compile)\n",
      "        1    0.000    0.000    0.000    0.000 sre_compile.py:221(_compile_charset)\n",
      "        1    0.000    0.000    0.000    0.000 sre_compile.py:248(_optimize_charset)\n",
      "        1    0.000    0.000    0.000    0.000 sre_compile.py:412(_compile_info)\n",
      "        2    0.000    0.000    0.000    0.000 sre_compile.py:513(isstring)\n",
      "        1    0.000    0.000    0.000    0.000 sre_compile.py:516(_code)\n",
      "        1    0.000    0.000    0.001    0.001 sre_compile.py:531(compile)\n",
      "      3/1    0.000    0.000    0.000    0.000 sre_compile.py:64(_compile)\n",
      "        3    0.000    0.000    0.000    0.000 sre_parse.py:105(__init__)\n",
      "        5    0.000    0.000    0.000    0.000 sre_parse.py:153(__len__)\n",
      "       12    0.000    0.000    0.000    0.000 sre_parse.py:157(__getitem__)\n",
      "        7    0.000    0.000    0.000    0.000 sre_parse.py:165(append)\n",
      "      3/1    0.000    0.000    0.000    0.000 sre_parse.py:167(getwidth)\n",
      "        1    0.000    0.000    0.000    0.000 sre_parse.py:217(__init__)\n",
      "        8    0.000    0.000    0.000    0.000 sre_parse.py:226(__next)\n",
      "        2    0.000    0.000    0.000    0.000 sre_parse.py:242(match)\n",
      "        6    0.000    0.000    0.000    0.000 sre_parse.py:247(get)\n",
      "        1    0.000    0.000    0.000    0.000 sre_parse.py:276(tell)\n",
      "        1    0.000    0.000    0.000    0.000 sre_parse.py:429(_parse_sub)\n",
      "        2    0.000    0.000    0.000    0.000 sre_parse.py:491(_parse)\n",
      "        1    0.000    0.000    0.000    0.000 sre_parse.py:70(__init__)\n",
      "        2    0.000    0.000    0.000    0.000 sre_parse.py:75(groups)\n",
      "        1    0.000    0.000    0.000    0.000 sre_parse.py:797(fix_flags)\n",
      "        1    0.000    0.000    0.000    0.000 sre_parse.py:819(parse)\n",
      "        1    0.000    0.000    0.000    0.000 {built-in method _sre.compile}\n",
      "        1    0.000    0.000    0.001    0.001 {built-in method builtins.exec}\n",
      "       17    0.000    0.000    0.000    0.000 {built-in method builtins.isinstance}\n",
      "    25/24    0.000    0.000    0.000    0.000 {built-in method builtins.len}\n",
      "        2    0.000    0.000    0.000    0.000 {built-in method builtins.max}\n",
      "        9    0.000    0.000    0.000    0.000 {built-in method builtins.min}\n",
      "        6    0.000    0.000    0.000    0.000 {built-in method builtins.ord}\n",
      "       48    0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}\n",
      "        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}\n",
      "        5    0.000    0.000    0.000    0.000 {method 'find' of 'bytearray' objects}\n",
      "        1    0.000    0.000    0.000    0.000 {method 'items' of 'dict' objects}\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "cProfile.run('re.compile(\"foo|bar\")')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What does it mean\n",
    "\n",
    "- The first line: the number of calls that were monitored. Primitive call is call not induced via recursion. \n",
    "- The next line, “Ordered by: standard name”: sorting method\n",
    "  - ncalls: the number of calls\n",
    "  - tottime: the total time spent in the given function (and excluding time made in calls to sub-functions)\n",
    "  - percall: is the quotient of tottime divided by ncalls\n",
    "  - cumtime: is the cumulative time spent in this and all subfunctions (from invocation till exit). This figure is accurate even for recursive functions\n",
    "  - percall: is the quotient of cumtime divided by primitive calls\n",
    "  - filename:lineno(function) : provides the respective data of each functio\n",
    "  \n",
    "- two numbers in the first column (for example 3/1), it means that the function recursed. The second value is the number of primitive calls and the former is the total number of calls. Note that when the function does not recurse, these two values are the same, and only the single figure is printed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### From IPython\n",
    "Use the `%prun` magic command or `%%prun` cell magic command."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### From Command Line\n",
    "\n",
    "`python -m cProfile [-o output_file] [-s sort_order] filename.py`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Practise:  Basel problem, approximate $\\pi$\n",
    "$$\\pi^2=\\sum_{k=1}^\\infty \\frac{1}{k^2} = 6 \\lim_{k -> \\infty} (\\frac{1}{1^2} + \\frac{1}{2^2} + ... + \\frac{1}{k^2})$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "### Example 1\n",
    "def recip_square(i):\n",
    "    return 1./i**2\n",
    "\n",
    "def approx_pi(n=10000000):\n",
    "    val = 0.\n",
    "    for k in range(1,n+1):\n",
    "        val += recip_square(k)\n",
    "    return (6 * val)**.5\n",
    "\n",
    "### Example 2\n",
    "def approx_pi2(n=10000000):\n",
    "    val = 0.\n",
    "    for k in range(1,n+1):\n",
    "        val += 1./k**2\n",
    "    return (6 * val)**.5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 loop, best of 3: 6.51 s per loop\n"
     ]
    }
   ],
   "source": [
    "%timeit approx_pi()\n",
    "%prun approx_pi()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Using `%timeit approx_pi()`\n",
    "1 loop, best of 3: 6.51 s per loop\n",
    "\n",
    "#### You will get a pop-up window in the Notebook showing:\n",
    " 10000004 function calls in 22.087 seconds\n",
    "\n",
    "   Ordered by: internal time\n",
    "<pre>\n",
    "   ncalls  tottime  percall  cumtime  percall filename:lineno(function)\n",
    " 10000000   15.834    0.000   15.834    0.000 <ipython-input-6-8fdaa89ffea3>:1(recip_square)\n",
    "        1    6.253    6.253   22.087   22.087 <ipython-input-6-8fdaa89ffea3>:4(approx_pi)\n",
    "        1    0.000    0.000   22.087   22.087 {built-in method builtins.exec}\n",
    "        1    0.000    0.000   22.087   22.087 <string>:1(<module>)\n",
    "        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}\n",
    "</pre>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " "
     ]
    }
   ],
   "source": [
    "%prun approx_pi2()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<pre>\n",
    "4 function calls in 16.854 seconds\n",
    "\n",
    "   Ordered by: internal time\n",
    "\n",
    "   ncalls  tottime  percall  cumtime  percall filename:lineno(function)\n",
    "        1   16.854   16.854   16.854   16.854 <ipython-input-9-28f45ff46d91>:12(approx_pi2)\n",
    "        1    0.000    0.000   16.854   16.854 {built-in method builtins.exec}\n",
    "        1    0.000    0.000   16.854   16.854 <string>:1(<module>)\n",
    "        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}\n",
    "</pre>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import itertools\n",
    "import math\n",
    "\n",
    "def approx_pi3(n=10000000): \n",
    "    \n",
    "    \n",
    "approx_pi3()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Part-3 line_profiler\n",
    "\n",
    "#### The downside of cProfile: \n",
    "- cProfile only times explicit function calls, \n",
    "   - e.g. `a[large_index_array] = some_other_large_array` never gets explicitly shown by cProfile\n",
    "\n",
    "#### line_profiler\n",
    "- load line_profile extension \n",
    "  - From ipython/ jupyter-notebook: `%load_ext line_profiler`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%load_ext line_profiler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def primes(n=1000): \n",
    "    A = [True] * (n+1)  # declare fix size array, and initizlize\n",
    "    A[0] = False\n",
    "    A[1] = False\n",
    "    for i in range(2, int(n**0.5)+1):\n",
    "        if A[i]:\n",
    "            for j in range(i**2, n+1, i):\n",
    "                A[j] = False\n",
    "\n",
    "    return [x for x in range(2, n) if A[x]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%lprun -f primes primes(10000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Showing in pop-up message window\n",
    "<pre>\n",
    "Timer unit: 3.95112e-07 s\n",
    "\n",
    "Total time: 0.189868 s\n",
    "File: <ipython-input-2-0ccce2a0061a>\n",
    "Function: primes at line 1\n",
    "\n",
    "Line #      Hits         Time  Per Hit   % Time  Line Contents\n",
    "==============================================================\n",
    "     1                                           def primes(n=1000): \n",
    "     2         1          420    420.0      0.1      A = [True] * (n+1)\n",
    "     3         1           16     16.0      0.0      A[0] = False\n",
    "     4         1           10     10.0      0.0      A[1] = False\n",
    "     5        99          719      7.3      0.1      for i in range(2, int(n**0.5)):\n",
    "     6        98         1966     20.1      0.4          if A[i]:\n",
    "     7     17006       247514     14.6     51.5              for j in range(i**2, n+1, i):\n",
    "     8     16981       215724     12.7     44.9                  A[j] = False\n",
    "     9                                           \n",
    "    10         1        14172  14172.0      2.9      return [x for x in range(2, n) if A[x]]\n",
    "</pre>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Line # - The line number in the code\n",
    "- Hits - The number of times that line was executed\n",
    "- Time - The total amount of time spent executing the line in the timer’s units\n",
    "- Per Hit - The average amount of time spent executing the line once in the timer’s unit\n",
    "- % Time - The percentage of time spent on that line relative to the total amount of recorded time spent in the function\n",
    "- Line Contents - The actual source code of the line\n",
    "\n",
    "#### Insighte on the profiling result & Possible improment\n",
    "- use built-in function\n",
    "- use a fast library such as NumPy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Practise: improve the prime function\n",
    "Use the following test code to check the results:\n",
    "<pre>\n",
    "from nose.tools import assert_equal, assert_less\n",
    "import timeit\n",
    "assert_equal(primes(1000), faster_primes(1000))\n",
    "assert_less(timeit.timeit(faster_primes), timeit.timeit(primes))\n",
    "</pre>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "def faster_primes(n=1000):\n",
    "    s = np.arange(3, n, 2)\n",
    "    for m in range(3, int(n ** 0.5)+1, 2): \n",
    "        if s[(m-3)/2]: \n",
    "            s[(m*m-3)/2::m]=0\n",
    "    return np.r_[2, s[s>0]].tolist()\n",
    "\n",
    "#     \"\"\" Input n>=6, Returns a array of primes, 2 <= p < n \"\"\"\n",
    "#     sieve = np.ones(n/3 + (n%6==2), dtype=np.bool)\n",
    "#     sieve[0] = False\n",
    "#     for i in range(int(n**0.5)/3+1):\n",
    "#         if sieve[i]:\n",
    "#             k=3*i+1|1\n",
    "#             sieve[      ((k*k)/3)      ::2*k] = False\n",
    "#             sieve[(k*k+4*k-2*k*(i&1))/3::2*k] = False\n",
    "#     return np.r_[2,3,((3*np.nonzero(sieve)[0]+1)|1)].tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\DevHome\\Python\\Anaconda3\\lib\\site-packages\\ipykernel\\__main__.py:5: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future\n",
      "C:\\DevHome\\Python\\Anaconda3\\lib\\site-packages\\ipykernel\\__main__.py:6: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future\n"
     ]
    },
    {
     "ename": "AssertionError",
     "evalue": "712.8869785655612 not less than 342.5819903142208",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mAssertionError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-27-2fff9ce88b30>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[1;32mimport\u001b[0m \u001b[0mtimeit\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0massert_equal\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mprimes\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m1000\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mfaster_primes\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m1000\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0massert_less\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mtimeit\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtimeit\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mfaster_primes\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtimeit\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtimeit\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mprimes\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32mC:\\DevHome\\Python\\Anaconda3\\lib\\unittest\\case.py\u001b[0m in \u001b[0;36massertLess\u001b[0;34m(self, a, b, msg)\u001b[0m\n\u001b[1;32m   1197\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[0ma\u001b[0m \u001b[1;33m<\u001b[0m \u001b[0mb\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m   1198\u001b[0m             \u001b[0mstandardMsg\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;34m'%s not less than %s'\u001b[0m \u001b[1;33m%\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0msafe_repr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0ma\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0msafe_repr\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mb\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1199\u001b[0;31m             \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfail\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_formatMessage\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmsg\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstandardMsg\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1200\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m   1201\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0massertLessEqual\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0ma\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mb\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmsg\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mNone\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[0;32mC:\\DevHome\\Python\\Anaconda3\\lib\\unittest\\case.py\u001b[0m in \u001b[0;36mfail\u001b[0;34m(self, msg)\u001b[0m\n\u001b[1;32m    663\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0mfail\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmsg\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mNone\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m    664\u001b[0m         \u001b[1;34m\"\"\"Fail immediately, with the given message.\"\"\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m--> 665\u001b[0;31m         \u001b[1;32mraise\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mfailureException\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mmsg\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    666\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m    667\u001b[0m     \u001b[1;32mdef\u001b[0m \u001b[0massertFalse\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mexpr\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmsg\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mNone\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mAssertionError\u001b[0m: 712.8869785655612 not less than 342.5819903142208"
     ]
    }
   ],
   "source": [
    "from nose.tools import assert_equal, assert_less\n",
    "import timeit\n",
    "assert_equal(primes(1000), faster_primes(1000))\n",
    "assert_less(timeit.timeit(faster_primes), timeit.timeit(primes))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}