4-Structuring-Datasets.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# Structuring Datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "> Objectives:\n",
    "> * How to use a hierarchy to structure datasets inside the same file\n",
    "> * Learn how to store tabular data in normalized and denormalized forms\n",
    "> * Use compression to get rid of duplicates (specially in denormalized tables)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Using the Hierarchy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "In HDF5, all nodes stem from a root (\"/\").  The nodes can be either `Groups` or `Datasets` (also know as `Leaves` in PyTables).  `Groups` are the equivalent of directories on a filesystem and can container `Datasets` or other `Groups`.  A `Dataset` is a container for data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "In PyTables, you may access nodes as attributes on a Python object, namely `f.root.a_group.some_data`.  This is known as natural naming.  Creating new nodes must be done on the file handle:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import tables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import shutil\n",
    "data_dir = \"structuring\"\n",
    "if os.path.exists(data_dir):\n",
    "    shutil.rmtree(data_dir)\n",
    "os.mkdir(data_dir)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "/a_group (Group) ''\n",
       "  children := []"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f = tables.open_file(\"%s/layout.h5\" % data_dir, \"w\")\n",
    "group = f.create_group('/', 'a_group')\n",
    "group"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Inside this group we can create many datasets:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "f.create_array(group, \"my_array1\", np.arange(10))\n",
    "f.create_array(group, \"my_array2\", np.ones(100).reshape(10, 10));"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "structuring/layout.h5 (File) ''\n",
      "Last modif.: 'Sat May 20 13:10:19 2017'\n",
      "Object Tree: \n",
      "/ (RootGroup) ''\n",
      "/a_group (Group) ''\n",
      "/a_group/my_array1 (Array(10,)) ''\n",
      "/a_group/my_array2 (Array(10, 10)) ''\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(f)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "With that, you can endow your datasets with any hierachy that would fit better to your needs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Normalizing and denormalizing tables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Many data sources are expressed in terms of related tables.  For example, part of the [MovieLens dataset](https://grouplens.org/datasets/movielens/) is structured in tables having the next columns:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "ratings = ['user_id', 'movie_id', 'rating', 'unix_timestamp']\n",
    "movies = ['movie_id', 'title', 'genres']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The relation that links the two tables above is the `movie_id` field.  This way, one can query parts of the dataset that involve the two tables, like for example, which users ('user_id') gave a rating of 5 to some movie ('title').  This is called the `normalized` version and we have already dealt with that in a previous section."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "On the other hand, one can fuse the above 2 tables into a single one:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "ratings_movies = ['title', 'genres', 'user_id', 'rating', 'unix_timestamp']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "As you see, we still keep all the data fields, except for the 'movie_id' that is not needed anymore.  This is called the `denormalized` version."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The advantage of this one is that we have all the fields readily available in one single table, so querying it and getting info about all the fileds is straighforward.  The disadvantage is that this table will have many duplicated information, i.e. the 'title' and 'genres' fields will appear for all the ratings, which can be seen as a waste of space."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "However, many times compression can get rid of many of the duplicated info in denormalized tables.  Let's see how to produce a denormalized table and how it fares compared with the normalized version."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Denormalizing tables using pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import tables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# Import CSV files via pandas\n",
    "dset = 'movielens-1m'\n",
    "fdata = os.path.join(dset, 'ratings.dat.gz')\n",
    "fitem = os.path.join(dset, 'movies.dat.gz')\n",
    "\n",
    "# pass in column names for each CSV\n",
    "r_cols = ['user_id', 'movie_id', 'rating', 'unix_timestamp']\n",
    "ratings = pd.read_csv(fdata, sep=';', names=r_cols)\n",
    "\n",
    "m_cols = ['movie_id', 'title', 'genres']\n",
    "movies = pd.read_csv(fitem, sep=';', names=m_cols,\n",
    "                     dtype={'title': object, 'genres': object})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "# create one merged DataFrame\n",
    "lens = pd.merge(movies, ratings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "movie_id           int64:dense\n",
       "title             object:dense\n",
       "genres            object:dense\n",
       "user_id            int64:dense\n",
       "rating             int64:dense\n",
       "unix_timestamp     int64:dense\n",
       "dtype: object"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lens.ftypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "def to_hdf5_denorm(lens, filters):\n",
    "\n",
    "    class Lens(tables.IsDescription):\n",
    "        user_id = tables.Int32Col(pos=0)\n",
    "        rating = tables.Int8Col(pos=1)\n",
    "        unix_timestamp = tables.Int64Col(pos=2)\n",
    "        title = tables.StringCol(100, pos=3)\n",
    "        genres = tables.StringCol(50, pos=4)\n",
    "        \n",
    "    def get_filename(filters):\n",
    "        if filters.complevel != 0:\n",
    "            complib = filters.complib if \":\" not in filters.complib else filters.complib.replace(\":\", \"-\")\n",
    "            shuffle = \"shuffle\" if filters.shuffle else \"noshuffle\"\n",
    "            filename = \"%s/%s-%d-%s.h5\" % (data_dir, complib, filters.complevel, shuffle)\n",
    "        else:\n",
    "            filename = \"%s/no-compressed.h5\" % (data_dir,)\n",
    "        return filename\n",
    "\n",
    "    filename = get_filename(filters)\n",
    "    print(\"Creating file:\", filename)\n",
    "    with tables.open_file(filename, \"w\", filters=filters) as f:\n",
    "        table_lens = f.create_table(f.root, \"lens\", Lens)\n",
    "        table_lens.append([lens[col].values for col in table_lens.dtype.names])\n",
    "    return filename"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating file: structuring/no-compressed.h5\n",
      "CPU times: user 218 ms, sys: 159 ms, total: 377 ms\n",
      "Wall time: 429 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "filters = tables.Filters(complevel=0)\n",
    "h5file = to_hdf5_denorm(lens, filters)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/ (RootGroup) ''\r\n",
      "/lens (Table(1000209,)) ''\r\n",
      "  description := {\r\n",
      "  \"user_id\": Int32Col(shape=(), dflt=0, pos=0),\r\n",
      "  \"rating\": Int8Col(shape=(), dflt=0, pos=1),\r\n",
      "  \"unix_timestamp\": Int64Col(shape=(), dflt=0, pos=2),\r\n",
      "  \"title\": StringCol(itemsize=100, shape=(), dflt=b'', pos=3),\r\n",
      "  \"genres\": StringCol(itemsize=50, shape=(), dflt=b'', pos=4)}\r\n",
      "  byteorder := 'little'\r\n",
      "  chunkshape := (402,)\r\n",
      "  Data dump:\r\n",
      "[0] (1, 5, 978824268, b'Toy Story (1995)', b\"Animation|Children's|Comedy\")\r\n",
      "[1] (6, 4, 978237008, b'Toy Story (1995)', b\"Animation|Children's|Comedy\")\r\n",
      "[2] (8, 4, 978233496, b'Toy Story (1995)', b\"Animation|Children's|Comedy\")\r\n",
      "[3] (9, 5, 978225952, b'Toy Story (1995)', b\"Animation|Children's|Comedy\")\r\n",
      "[4] (10, 5, 978226474, b'Toy Story (1995)', b\"Animation|Children's|Comedy\")\r\n",
      "[5] (18, 4, 978154768, b'Toy Story (1995)', b\"Animation|Children's|Comedy\")\r\n",
      "[6] (19, 5, 978555994, b'Toy Story (1995)', b\"Animation|Children's|Comedy\")\r\n",
      "[7] (21, 3, 978139347, b'Toy Story (1995)', b\"Animation|Children's|Comedy\")\r\n",
      "[8] (23, 4, 978463614, b'Toy Story (1995)', b\"Animation|Children's|Comedy\")\r\n",
      "[9] (26, 3, 978130703, b'Toy Story (1995)', b\"Animation|Children's|Comedy\")\r\n"
     ]
    }
   ],
   "source": [
    "!ptdump -v -R0,10 {h5file}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "compression:\r\n",
      "total 121712\r\n",
      "-rw-r--r--  1 faltet  staff   5.0M May 20 13:07 blosc-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   5.0M May 20 13:07 blosc-blosclz-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   5.5M May 20 13:07 blosc-lz4-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   4.8M May 20 13:07 blosc-lz4hc-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   5.5M May 20 13:07 blosc-snappy-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   4.3M May 20 13:07 blosc-zlib-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   4.3M May 20 13:07 blosc-zstd-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   4.1M May 20 13:07 bzip2-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff    17M May 20 13:07 no-compressed.h5\r\n",
      "-rw-r--r--  1 faltet  staff   4.2M May 20 13:07 zlib-5-shuffle.h5\r\n",
      "\r\n",
      "structuring:\r\n",
      "total 318752\r\n",
      "-rw-r--r--  1 faltet  staff   5.2K May 20 13:10 layout.h5\r\n",
      "-rw-r--r--  1 faltet  staff   156M May 20 13:10 no-compressed.h5\r\n"
     ]
    }
   ],
   "source": [
    "%ls -lh structuring compression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "As can be seen, the size of the denormalized table is much larger than the normalized one (156 MB vs 17 MB).  But that is without using compression."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Exercise 1\n",
    "\n",
    "Create a compressed version of the denormalized table and compare it with the same table in the normalized state.\n",
    "What's the difference in size now?  Why do you think the compression process works much better in this case?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating file: structuring/blosc-blosclz-5-shuffle.h5\n",
      "CPU times: user 332 ms, sys: 90.8 ms, total: 423 ms\n",
      "Wall time: 445 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'structuring/blosc-blosclz-5-shuffle.h5'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filters = tables.Filters(complevel=5, complib=\"blosc:blosclz\")\n",
    "%time to_hdf5_denorm(lens, filters)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "compression:\r\n",
      "total 121712\r\n",
      "-rw-r--r--  1 faltet  staff   5.0M May 20 13:07 blosc-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   5.0M May 20 13:07 blosc-blosclz-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   5.5M May 20 13:07 blosc-lz4-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   4.8M May 20 13:07 blosc-lz4hc-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   5.5M May 20 13:07 blosc-snappy-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   4.3M May 20 13:07 blosc-zlib-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   4.3M May 20 13:07 blosc-zstd-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   4.1M May 20 13:07 bzip2-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff    17M May 20 13:07 no-compressed.h5\r\n",
      "-rw-r--r--  1 faltet  staff   4.2M May 20 13:07 zlib-5-shuffle.h5\r\n",
      "\r\n",
      "structuring:\r\n",
      "total 333600\r\n",
      "-rw-r--r--  1 faltet  staff   7.2M May 20 13:10 blosc-blosclz-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   5.2K May 20 13:10 layout.h5\r\n",
      "-rw-r--r--  1 faltet  staff   156M May 20 13:10 no-compressed.h5\r\n"
     ]
    }
   ],
   "source": [
    "%ls -lh structuring compression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "### Exercise 2\n",
    "\n",
    "Create different files containing the denormalized table using different codecs.  Which one reduces the size better?  How does it compare with the files for the normalized version?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating file: structuring/zlib-5-shuffle.h5\n",
      "CPU times: user 1.26 s, sys: 94 ms, total: 1.36 s\n",
      "Wall time: 1.39 s\n",
      "Creating file: structuring/bzip2-5-shuffle.h5\n",
      "CPU times: user 9.02 s, sys: 133 ms, total: 9.15 s\n",
      "Wall time: 9.17 s\n",
      "Creating file: structuring/blosc-blosclz-5-shuffle.h5\n",
      "CPU times: user 278 ms, sys: 69.3 ms, total: 348 ms\n",
      "Wall time: 351 ms\n",
      "Creating file: structuring/blosc-lz4-5-shuffle.h5\n",
      "CPU times: user 251 ms, sys: 67.1 ms, total: 318 ms\n",
      "Wall time: 320 ms\n",
      "Creating file: structuring/blosc-lz4hc-5-shuffle.h5\n",
      "CPU times: user 1.9 s, sys: 68.5 ms, total: 1.97 s\n",
      "Wall time: 1.97 s\n",
      "Creating file: structuring/blosc-snappy-5-shuffle.h5\n",
      "CPU times: user 275 ms, sys: 70.1 ms, total: 345 ms\n",
      "Wall time: 347 ms\n",
      "Creating file: structuring/blosc-zlib-5-shuffle.h5\n",
      "CPU times: user 1.18 s, sys: 72.7 ms, total: 1.25 s\n",
      "Wall time: 1.25 s\n",
      "Creating file: structuring/blosc-zstd-5-shuffle.h5\n",
      "CPU times: user 638 ms, sys: 72.6 ms, total: 711 ms\n",
      "Wall time: 712 ms\n"
     ]
    }
   ],
   "source": [
    "for complib in (\"zlib\", \"bzip2\", \"blosc:blosclz\", \"blosc:lz4\", \"blosc:lz4hc\", \"blosc:snappy\", \"blosc:zlib\", \"blosc:zstd\"):\n",
    "    filters = tables.Filters(complevel=5, complib=complib)\n",
    "    %time to_hdf5_denorm(lens, filters)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "compression:\r\n",
      "total 121712\r\n",
      "-rw-r--r--  1 faltet  staff   5.0M May 20 13:07 blosc-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   5.0M May 20 13:07 blosc-blosclz-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   5.5M May 20 13:07 blosc-lz4-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   4.8M May 20 13:07 blosc-lz4hc-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   5.5M May 20 13:07 blosc-snappy-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   4.3M May 20 13:07 blosc-zlib-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   4.3M May 20 13:07 blosc-zstd-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   4.1M May 20 13:07 bzip2-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff    17M May 20 13:07 no-compressed.h5\r\n",
      "-rw-r--r--  1 faltet  staff   4.2M May 20 13:07 zlib-5-shuffle.h5\r\n",
      "\r\n",
      "structuring:\r\n",
      "total 438952\r\n",
      "-rw-r--r--  1 faltet  staff   7.2M May 20 13:10 blosc-blosclz-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   7.8M May 20 13:10 blosc-lz4-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   6.6M May 20 13:10 blosc-lz4hc-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff    13M May 20 13:10 blosc-snappy-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   6.0M May 20 13:10 blosc-zlib-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   5.4M May 20 13:10 blosc-zstd-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   6.4M May 20 13:10 bzip2-5-shuffle.h5\r\n",
      "-rw-r--r--  1 faltet  staff   5.2K May 20 13:10 layout.h5\r\n",
      "-rw-r--r--  1 faltet  staff   156M May 20 13:10 no-compressed.h5\r\n",
      "-rw-r--r--  1 faltet  staff   6.0M May 20 13:10 zlib-5-shuffle.h5\r\n"
     ]
    }
   ],
   "source": [
    "%ls -lh structuring compression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "In the next section we will see the effect of querying normalized and denormalized tables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}