01_constructors.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to DataFrames\n",
    "**[Bogumił Kamiński](http://bogumilkaminski.pl/about/), May 6, 2020**\n",
    "\n",
    "Let's get started by loading the `DataFrames` package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "using DataFrames, Random"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Constructors and conversion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Constructors\n",
    "\n",
    "In this section, you'll see many ways to create a `DataFrame` using the `DataFrame()` constructor.\n",
    "\n",
    "First, we could create an empty DataFrame,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th></tr><tr><th></th></tr></thead><tbody><p>0 rows × 0 columns</p></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|}\n",
       "\t& \\\\\n",
       "\t\\hline\n",
       "\t& \\\\\n",
       "\t\\hline\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "0×0 DataFrame\n"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or we could call the constructor using keyword arguments to add columns to the `DataFrame`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Float64</th><th>String</th></tr></thead><tbody><p>3 rows × 3 columns</p><tr><th>1</th><td>1</td><td>0.050034</td><td>e5U</td></tr><tr><th>2</th><td>2</td><td>0.746714</td><td>OuX</td></tr><tr><th>3</th><td>3</td><td>0.754752</td><td>HXE</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& A & B & C\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Float64 & String\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 0.050034 & e5U \\\\\n",
       "\t2 & 2 & 0.746714 & OuX \\\\\n",
       "\t3 & 3 & 0.754752 & HXE \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "3×3 DataFrame\n",
       "│ Row │ A     │ B        │ C      │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mFloat64\u001b[39m  │ \u001b[90mString\u001b[39m │\n",
       "├─────┼───────┼──────────┼────────┤\n",
       "│ 1   │ 1     │ 0.050034 │ e5U    │\n",
       "│ 2   │ 2     │ 0.746714 │ OuX    │\n",
       "│ 3   │ 3     │ 0.754752 │ HXE    │"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(A=1:3, B=rand(3), C=randstring.([3,3,3]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can create a `DataFrame` from a dictionary, in which case keys from the dictionary will be sorted to create the `DataFrame` columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Bool</th><th>Char</th></tr></thead><tbody><p>2 rows × 3 columns</p><tr><th>1</th><td>1</td><td>1</td><td>'a'</td></tr><tr><th>2</th><td>2</td><td>0</td><td>'b'</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& A & B & C\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Bool & Char\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 'a' \\\\\n",
       "\t2 & 2 & 0 & 'b' \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×3 DataFrame\n",
       "│ Row │ A     │ B    │ C    │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mBool\u001b[39m │ \u001b[90mChar\u001b[39m │\n",
       "├─────┼───────┼──────┼──────┤\n",
       "│ 1   │ 1     │ 1    │ 'a'  │\n",
       "│ 2   │ 2     │ 0    │ 'b'  │"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = Dict(\"A\" => [1,2], \"B\" => [true, false], \"C\" => ['a', 'b'])\n",
    "DataFrame(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rather than explicitly creating a dictionary first, as above, we could pass `DataFrame` arguments with the syntax of dictionary key-value pairs. \n",
    "\n",
    "Note that in this case, we use `Symbol`s to denote the column names and arguments are not sorted. For example, `:A`, the symbol, produces `A`, the name of the first column here:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Bool</th><th>Char</th></tr></thead><tbody><p>2 rows × 3 columns</p><tr><th>1</th><td>1</td><td>1</td><td>'a'</td></tr><tr><th>2</th><td>2</td><td>0</td><td>'b'</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& A & B & C\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Bool & Char\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 'a' \\\\\n",
       "\t2 & 2 & 0 & 'b' \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×3 DataFrame\n",
       "│ Row │ A     │ B    │ C    │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mBool\u001b[39m │ \u001b[90mChar\u001b[39m │\n",
       "├─────┼───────┼──────┼──────┤\n",
       "│ 1   │ 1     │ 1    │ 'a'  │\n",
       "│ 2   │ 2     │ 0    │ 'b'  │"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(:A => [1,2], :B => [true, false], :C => ['a', 'b'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Although, in general, using `Symbol`s rather than strings to denote column names is preferred (as it is faster) DataFrames.jl accepts passing strings as column names, so this also works:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Bool</th><th>Char</th></tr></thead><tbody><p>2 rows × 3 columns</p><tr><th>1</th><td>1</td><td>1</td><td>'a'</td></tr><tr><th>2</th><td>2</td><td>0</td><td>'b'</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& A & B & C\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Bool & Char\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 'a' \\\\\n",
       "\t2 & 2 & 0 & 'b' \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×3 DataFrame\n",
       "│ Row │ A     │ B    │ C    │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mBool\u001b[39m │ \u001b[90mChar\u001b[39m │\n",
       "├─────┼───────┼──────┼──────┤\n",
       "│ 1   │ 1     │ 1    │ 'a'  │\n",
       "│ 2   │ 2     │ 0    │ 'b'  │"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(\"A\" => [1,2], \"B\" => [true, false], \"C\" => ['a', 'b'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we create a `DataFrame` from a vector of vectors, and each vector becomes a column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th></tr><tr><th></th><th>Float64</th><th>Float64</th><th>Float64</th></tr></thead><tbody><p>3 rows × 3 columns</p><tr><th>1</th><td>0.150228</td><td>0.596849</td><td>0.0807212</td></tr><tr><th>2</th><td>0.752575</td><td>0.803518</td><td>0.420067</td></tr><tr><th>3</th><td>0.0497649</td><td>0.749983</td><td>0.696673</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x1 & x2 & x3\\\\\n",
       "\t\\hline\n",
       "\t& Float64 & Float64 & Float64\\\\\n",
       "\t\\hline\n",
       "\t1 & 0.150228 & 0.596849 & 0.0807212 \\\\\n",
       "\t2 & 0.752575 & 0.803518 & 0.420067 \\\\\n",
       "\t3 & 0.0497649 & 0.749983 & 0.696673 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "3×3 DataFrame\n",
       "│ Row │ x1        │ x2       │ x3        │\n",
       "│     │ \u001b[90mFloat64\u001b[39m   │ \u001b[90mFloat64\u001b[39m  │ \u001b[90mFloat64\u001b[39m   │\n",
       "├─────┼───────────┼──────────┼───────────┤\n",
       "│ 1   │ 0.150228  │ 0.596849 │ 0.0807212 │\n",
       "│ 2   │ 0.752575  │ 0.803518 │ 0.420067  │\n",
       "│ 3   │ 0.0497649 │ 0.749983 │ 0.696673  │"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame([rand(3) for i in 1:3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is not allowed to pass a vector of scalars to `DataFrame` constructor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "ename": "ArgumentError",
     "evalue": "ArgumentError: 'Array{Float64,1}' iterates 'Float64' values, which doesn't satisfy the Tables.jl `AbstractRow` interface",
     "output_type": "error",
     "traceback": [
      "ArgumentError: 'Array{Float64,1}' iterates 'Float64' values, which doesn't satisfy the Tables.jl `AbstractRow` interface",
      "",
      "Stacktrace:",
      " [1] invalidtable(::Array{Float64,1}, ::Float64) at /home/bkamins/.julia/packages/Tables/okt7x/src/tofromdatavalues.jl:42",
      " [2] iterate at /home/bkamins/.julia/packages/Tables/okt7x/src/tofromdatavalues.jl:48 [inlined]",
      " [3] buildcolumns at /home/bkamins/.julia/packages/Tables/okt7x/src/fallbacks.jl:185 [inlined]",
      " [4] columns at /home/bkamins/.julia/packages/Tables/okt7x/src/fallbacks.jl:237 [inlined]",
      " [5] DataFrame(::Array{Float64,1}; copycols::Bool) at /home/bkamins/.julia/packages/DataFrames/3ZmR2/src/other/tables.jl:43",
      " [6] DataFrame(::Array{Float64,1}) at /home/bkamins/.julia/packages/DataFrames/3ZmR2/src/other/tables.jl:34",
      " [7] top-level scope at In[8]:1"
     ]
    }
   ],
   "source": [
    "DataFrame(rand(3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Instead use a transposed vector if you have a vector of single values (in this way you effectively pass a two dimensional array to the constructor which is supported)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>1 rows × 3 columns</p><tr><th>1</th><td>1</td><td>2</td><td>3</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x1 & x2 & x3\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 2 & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "1×3 DataFrame\n",
       "│ Row │ x1    │ x2    │ x3    │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n",
       "├─────┼───────┼───────┼───────┤\n",
       "│ 1   │ 1     │ 2     │ 3     │"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(permutedims([1, 2, 3]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note, however, that in other cases you can pass a single value to a `DataFrame` constrtuctor and it will be broadcasted appropriately"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th></tr><tr><th></th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>3 rows × 2 columns</p><tr><th>1</th><td>1</td><td>1</td></tr><tr><th>2</th><td>1</td><td>2</td></tr><tr><th>3</th><td>1</td><td>3</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& A & B\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 \\\\\n",
       "\t2 & 1 & 2 \\\\\n",
       "\t3 & 1 & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "3×2 DataFrame\n",
       "│ Row │ A     │ B     │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n",
       "├─────┼───────┼───────┤\n",
       "│ 1   │ 1     │ 1     │\n",
       "│ 2   │ 1     │ 2     │\n",
       "│ 3   │ 1     │ 3     │"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(\"A\" => 1, \"B\" => [1,2,3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Array…</th></tr></thead><tbody><p>3 rows × 3 columns</p><tr><th>1</th><td>1</td><td>1</td><td>[\"a\", \"b\"]</td></tr><tr><th>2</th><td>1</td><td>2</td><td>[\"a\", \"b\"]</td></tr><tr><th>3</th><td>1</td><td>3</td><td>[\"a\", \"b\"]</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& A & B & C\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Array…\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & [\"a\", \"b\"] \\\\\n",
       "\t2 & 1 & 2 & [\"a\", \"b\"] \\\\\n",
       "\t3 & 1 & 3 & [\"a\", \"b\"] \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "3×3 DataFrame\n",
       "│ Row │ A     │ B     │ C          │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │ \u001b[90mArray…\u001b[39m     │\n",
       "├─────┼───────┼───────┼────────────┤\n",
       "│ 1   │ 1     │ 1     │ [\"a\", \"b\"] │\n",
       "│ 2   │ 1     │ 2     │ [\"a\", \"b\"] │\n",
       "│ 3   │ 1     │ 3     │ [\"a\", \"b\"] │"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(A=1, B=[1,2,3], C=Ref([\"a\", \"b\"]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, as a particular rule, like in standard broadcasting - values wrapped in `Ref` get unwrapped as can be seen in the example above. Now column `:C` contains three times the same vector `[\"a\", \"b\"]` (the values are aliases, not copies)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also pass a vector of `NamedTuple`s to construct a `DataFrame`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th></tr><tr><th></th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td>2</td></tr><tr><th>2</th><td>3</td><td>4</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& a & b\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 2 \\\\\n",
       "\t2 & 3 & 4 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ a     │ b     │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n",
       "├─────┼───────┼───────┤\n",
       "│ 1   │ 1     │ 2     │\n",
       "│ 2   │ 3     │ 4     │"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "v = [(a=1, b=2), (a=3, b=4)]\n",
    "DataFrame(v)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you pass a vector of vectors then pass a second argument to give the columns names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>3 rows × 3 columns</p><tr><th>1</th><td>1</td><td>4</td><td>7</td></tr><tr><th>2</th><td>2</td><td>5</td><td>8</td></tr><tr><th>3</th><td>3</td><td>6</td><td>9</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& A & B & C\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 4 & 7 \\\\\n",
       "\t2 & 2 & 5 & 8 \\\\\n",
       "\t3 & 3 & 6 & 9 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "3×3 DataFrame\n",
       "│ Row │ A     │ B     │ C     │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n",
       "├─────┼───────┼───────┼───────┤\n",
       "│ 1   │ 1     │ 4     │ 7     │\n",
       "│ 2   │ 2     │ 5     │ 8     │\n",
       "│ 3   │ 3     │ 6     │ 9     │"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame([1:3, 4:6, 7:9], [:A, :B, :C])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>3 rows × 3 columns</p><tr><th>1</th><td>1</td><td>4</td><td>7</td></tr><tr><th>2</th><td>2</td><td>5</td><td>8</td></tr><tr><th>3</th><td>3</td><td>6</td><td>9</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& A & B & C\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 4 & 7 \\\\\n",
       "\t2 & 2 & 5 & 8 \\\\\n",
       "\t3 & 3 & 6 & 9 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "3×3 DataFrame\n",
       "│ Row │ A     │ B     │ C     │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n",
       "├─────┼───────┼───────┼───────┤\n",
       "│ 1   │ 1     │ 4     │ 7     │\n",
       "│ 2   │ 2     │ 5     │ 8     │\n",
       "│ 3   │ 3     │ 6     │ 9     │"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame([1:3, 4:6, 7:9], [\"A\", \"B\", \"C\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively you can pass a `NamedTuple` of vectors:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th></tr><tr><th></th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>3 rows × 2 columns</p><tr><th>1</th><td>1</td><td>11</td></tr><tr><th>2</th><td>2</td><td>12</td></tr><tr><th>3</th><td>3</td><td>13</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& a & b\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 11 \\\\\n",
       "\t2 & 2 & 12 \\\\\n",
       "\t3 & 3 & 13 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "3×2 DataFrame\n",
       "│ Row │ a     │ b     │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n",
       "├─────┼───────┼───────┤\n",
       "│ 1   │ 1     │ 11    │\n",
       "│ 2   │ 2     │ 12    │\n",
       "│ 3   │ 3     │ 13    │"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n = (a=1:3, b=11:13)\n",
    "DataFrame(n)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we create a `DataFrame` from a matrix,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th>Float64</th><th>Float64</th><th>Float64</th><th>Float64</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>0.214943</td><td>0.627109</td><td>0.106318</td><td>0.545103</td></tr><tr><th>2</th><td>0.889415</td><td>0.704955</td><td>0.0711917</td><td>0.123521</td></tr><tr><th>3</th><td>0.910963</td><td>0.519658</td><td>0.345197</td><td>0.628368</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Float64 & Float64 & Float64 & Float64\\\\\n",
       "\t\\hline\n",
       "\t1 & 0.214943 & 0.627109 & 0.106318 & 0.545103 \\\\\n",
       "\t2 & 0.889415 & 0.704955 & 0.0711917 & 0.123521 \\\\\n",
       "\t3 & 0.910963 & 0.519658 & 0.345197 & 0.628368 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "3×4 DataFrame\n",
       "│ Row │ x1       │ x2       │ x3        │ x4       │\n",
       "│     │ \u001b[90mFloat64\u001b[39m  │ \u001b[90mFloat64\u001b[39m  │ \u001b[90mFloat64\u001b[39m   │ \u001b[90mFloat64\u001b[39m  │\n",
       "├─────┼──────────┼──────────┼───────────┼──────────┤\n",
       "│ 1   │ 0.214943 │ 0.627109 │ 0.106318  │ 0.545103 │\n",
       "│ 2   │ 0.889415 │ 0.704955 │ 0.0711917 │ 0.123521 │\n",
       "│ 3   │ 0.910963 │ 0.519658 │ 0.345197  │ 0.628368 │"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(rand(3,4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and here we do the same but also pass column names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th><th>c</th><th>d</th></tr><tr><th></th><th>Float64</th><th>Float64</th><th>Float64</th><th>Float64</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>0.805002</td><td>0.321617</td><td>0.973277</td><td>0.607407</td></tr><tr><th>2</th><td>0.657194</td><td>0.591434</td><td>0.88867</td><td>0.811593</td></tr><tr><th>3</th><td>0.023967</td><td>0.123394</td><td>0.450165</td><td>0.237314</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& a & b & c & d\\\\\n",
       "\t\\hline\n",
       "\t& Float64 & Float64 & Float64 & Float64\\\\\n",
       "\t\\hline\n",
       "\t1 & 0.805002 & 0.321617 & 0.973277 & 0.607407 \\\\\n",
       "\t2 & 0.657194 & 0.591434 & 0.88867 & 0.811593 \\\\\n",
       "\t3 & 0.023967 & 0.123394 & 0.450165 & 0.237314 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "3×4 DataFrame\n",
       "│ Row │ a        │ b        │ c        │ d        │\n",
       "│     │ \u001b[90mFloat64\u001b[39m  │ \u001b[90mFloat64\u001b[39m  │ \u001b[90mFloat64\u001b[39m  │ \u001b[90mFloat64\u001b[39m  │\n",
       "├─────┼──────────┼──────────┼──────────┼──────────┤\n",
       "│ 1   │ 0.805002 │ 0.321617 │ 0.973277 │ 0.607407 │\n",
       "│ 2   │ 0.657194 │ 0.591434 │ 0.88867  │ 0.811593 │\n",
       "│ 3   │ 0.023967 │ 0.123394 │ 0.450165 │ 0.237314 │"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(rand(3,4), Symbol.('a':'d'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th><th>c</th><th>d</th></tr><tr><th></th><th>Float64</th><th>Float64</th><th>Float64</th><th>Float64</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>0.425596</td><td>0.669391</td><td>0.015079</td><td>0.353956</td></tr><tr><th>2</th><td>0.507025</td><td>0.751827</td><td>0.88226</td><td>0.120275</td></tr><tr><th>3</th><td>0.0190897</td><td>0.524012</td><td>0.217928</td><td>0.517043</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& a & b & c & d\\\\\n",
       "\t\\hline\n",
       "\t& Float64 & Float64 & Float64 & Float64\\\\\n",
       "\t\\hline\n",
       "\t1 & 0.425596 & 0.669391 & 0.015079 & 0.353956 \\\\\n",
       "\t2 & 0.507025 & 0.751827 & 0.88226 & 0.120275 \\\\\n",
       "\t3 & 0.0190897 & 0.524012 & 0.217928 & 0.517043 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "3×4 DataFrame\n",
       "│ Row │ a         │ b        │ c        │ d        │\n",
       "│     │ \u001b[90mFloat64\u001b[39m   │ \u001b[90mFloat64\u001b[39m  │ \u001b[90mFloat64\u001b[39m  │ \u001b[90mFloat64\u001b[39m  │\n",
       "├─────┼───────────┼──────────┼──────────┼──────────┤\n",
       "│ 1   │ 0.425596  │ 0.669391 │ 0.015079 │ 0.353956 │\n",
       "│ 2   │ 0.507025  │ 0.751827 │ 0.88226  │ 0.120275 │\n",
       "│ 3   │ 0.0190897 │ 0.524012 │ 0.217928 │ 0.517043 │"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(rand(3,4), string.('a':'d'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also construct an uninitialized DataFrame.\n",
    "\n",
    "Here we pass column types, names and number of rows to get an unitialized `DataFrame`; we get `missing` in column :D because `Any >: Missing`.\n",
    "\n",
    "(in all examples below we omit the version with column names passed as strings, but it is allowed everywhere where `Symbol`s are accepted)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th><th>D</th></tr><tr><th></th><th>Int64</th><th>Float64</th><th>String</th><th>Any</th></tr></thead><tbody><p>1 rows × 4 columns</p><tr><th>1</th><td>35</td><td>3.9e-322</td><td><em>#undef</em></td><td><em>missing</em></td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& A & B & C & D\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Float64 & String & Any\\\\\n",
       "\t\\hline\n",
       "\t1 & 35 & 3.9e-322 & \\emph{\\#undef} & \\emph{missing} \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "1×4 DataFrame\n",
       "│ Row │ A     │ B        │ C      │ D       │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mFloat64\u001b[39m  │ \u001b[90mString\u001b[39m │ \u001b[90mAny\u001b[39m     │\n",
       "├─────┼───────┼──────────┼────────┼─────────┤\n",
       "│ 1   │ 35    │ 3.9e-322 │ \u001b[90m#undef\u001b[39m │ \u001b[90mmissing\u001b[39m │"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame([Int, Float64, String, Any], [:A, :B, :C, :D], 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To initialize a `DataFrame` with column names, but no rows use"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Float64</th><th>String</th></tr></thead><tbody><p>0 rows × 3 columns</p></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& A & B & C\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Float64 & String\\\\\n",
       "\t\\hline\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "0×3 DataFrame\n"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame([Int, Float64, String], [:A, :B, :C])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>B</th><th>C</th></tr><tr><th></th><th>Int64</th><th>Float64</th><th>String</th></tr></thead><tbody><p>0 rows × 3 columns</p></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& A & B & C\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Float64 & String\\\\\n",
       "\t\\hline\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "0×3 DataFrame\n"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(A=Int[], B=Float64[], C=String[])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can create a `DataFrame` by copying an existing `DataFrame`.\n",
    "\n",
    "Note that `copy` also copies the vectors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(false, true, true, false)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(a=1:2, b='a':'b')\n",
    "y = copy(x)\n",
    "(x === y), isequal(x, y), (x.a == y.a), (x.a === y.a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calling `DataFrame` on a `DataFrame` object works like `copy`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(false, true, true, false)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(a=1:2, b='a':'b')\n",
    "y = DataFrame(x)\n",
    "(x === y), isequal(x, y), (x.a == y.a), (x.a === y.a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can avoid copying of columns of a data frame (if it is possible) by passing `copycols=false` keyword argument or using `DataFrame!` constructor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(false, true, true, true)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(a=1:2, b='a':'b')\n",
    "y = DataFrame(x, copycols=false)\n",
    "(x === y), isequal(x, y), (x.a == y.a), (x.a === y.a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(false, true, true, true)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(a=1:2, b='a':'b')\n",
    "y = DataFrame!(x)\n",
    "(x === y), isequal(x, y), (x.a == y.a), (x.a === y.a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The same rule applies to other constructors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(false, true)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = [1, 2, 3]\n",
    "df1 = DataFrame(a=a)\n",
    "df2 = DataFrame(a=a, copycols=false)\n",
    "df1.a === a, df2.a === a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can create a similar uninitialized `DataFrame` based on an original one:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th></tr><tr><th></th><th>Int64</th><th>Char</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>139778165174576</td><td>'\\xa3\\xe1\\xf2\\xc0'</td></tr><tr><th>2</th><td>139777388798976</td><td>'\\x00\\x00\\x7f\\x20'</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& a & b\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Char\\\\\n",
       "\t\\hline\n",
       "\t1 & 139778165174576 & '\\textbackslash{}xa3\\textbackslash{}xe1\\textbackslash{}xf2\\textbackslash{}xc0' \\\\\n",
       "\t2 & 139777388798976 & '\\textbackslash{}x00\\textbackslash{}x00\\textbackslash{}x7f\\textbackslash{}x20' \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ a               │ b                  │\n",
       "│     │ \u001b[90mInt64\u001b[39m           │ \u001b[90mChar\u001b[39m               │\n",
       "├─────┼─────────────────┼────────────────────┤\n",
       "│ 1   │ 139778165174576 │ '\\xa3\\xe1\\xf2\\xc0' │\n",
       "│ 2   │ 139777388798976 │ '\\x00\\x00\\x7f\\x20' │"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similar(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "number of rows in a new DataFrame can be passed as a second argument"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th></tr><tr><th></th><th>Int64</th><th>Char</th></tr></thead><tbody><p>0 rows × 2 columns</p></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& a & b\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Char\\\\\n",
       "\t\\hline\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "0×2 DataFrame\n"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "similar(x, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also create a new `DataFrame` from `SubDataFrame` or `DataFrameRow` (discussed in detail later in the tutorial; in particular although `DataFrameRow` is considered a 1-dimensional object similar to a `NamedTuple` it gets converted to a 1-row `DataFrame` for convinience)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th></tr><tr><th></th><th>Int64</th><th>Char</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td>'a'</td></tr><tr><th>2</th><td>1</td><td>'a'</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& a & b\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Char\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 'a' \\\\\n",
       "\t2 & 1 & 'a' \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 SubDataFrame\n",
       "│ Row │ a     │ b    │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mChar\u001b[39m │\n",
       "├─────┼───────┼──────┤\n",
       "│ 1   │ 1     │ 'a'  │\n",
       "│ 2   │ 1     │ 'a'  │"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sdf = view(x, [1,1], :)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SubDataFrame{DataFrame,DataFrames.Index,Array{Int64,1}}"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "typeof(sdf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th></tr><tr><th></th><th>Int64</th><th>Char</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td>'a'</td></tr><tr><th>2</th><td>1</td><td>'a'</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& a & b\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Char\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 'a' \\\\\n",
       "\t2 & 1 & 'a' \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ a     │ b    │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mChar\u001b[39m │\n",
       "├─────┼───────┼──────┤\n",
       "│ 1   │ 1     │ 'a'  │\n",
       "│ 2   │ 1     │ 'a'  │"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(sdf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<p>DataFrameRow (2 columns)</p><table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th></tr><tr><th></th><th>Int64</th><th>Char</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>'a'</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& a & b\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Char\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 'a' \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "DataFrameRow\n",
       "│ Row │ a     │ b    │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mChar\u001b[39m │\n",
       "├─────┼───────┼──────┤\n",
       "│ 1   │ 1     │ 'a'  │"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfr = x[1, :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th></tr><tr><th></th><th>Int64</th><th>Char</th></tr></thead><tbody><p>1 rows × 2 columns</p><tr><th>1</th><td>1</td><td>'a'</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& a & b\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Char\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 'a' \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "1×2 DataFrame\n",
       "│ Row │ a     │ b    │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mChar\u001b[39m │\n",
       "├─────┼───────┼──────┤\n",
       "│ 1   │ 1     │ 'a'  │"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(dfr)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conversion to a matrix\n",
    "\n",
    "Let's start by creating a `DataFrame` with two rows and two columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr><tr><th></th><th>Int64</th><th>String</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td>A</td></tr><tr><th>2</th><td>2</td><td>B</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x & y\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & String\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & A \\\\\n",
       "\t2 & 2 & B \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ x     │ y      │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mString\u001b[39m │\n",
       "├─────┼───────┼────────┤\n",
       "│ 1   │ 1     │ A      │\n",
       "│ 2   │ 2     │ B      │"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(x=1:2, y=[\"A\", \"B\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can create a matrix by passing this `DataFrame` to `Matrix` or `Array`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2×2 Array{Any,2}:\n",
       " 1  \"A\"\n",
       " 2  \"B\""
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Matrix(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2×2 Array{Any,2}:\n",
       " 1  \"A\"\n",
       " 2  \"B\""
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Array(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This would work even if the `DataFrame` had some `missing`s:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr><tr><th></th><th>Int64</th><th>String?</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td><em>missing</em></td></tr><tr><th>2</th><td>2</td><td>B</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x & y\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & String?\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & \\emph{missing} \\\\\n",
       "\t2 & 2 & B \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ x     │ y       │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mString?\u001b[39m │\n",
       "├─────┼───────┼─────────┤\n",
       "│ 1   │ 1     │ \u001b[90mmissing\u001b[39m │\n",
       "│ 2   │ 2     │ B       │"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(x=1:2, y=[missing,\"B\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2×2 Array{Any,2}:\n",
       " 1  missing\n",
       " 2  \"B\""
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Matrix(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the two previous matrix examples, Julia created matrices with elements of type `Any`. We can see more clearly that the type of matrix is inferred when we pass, for example, a `DataFrame` of integers to `Matrix`, creating a 2D `Array` of `Int64`s:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr><tr><th></th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td>3</td></tr><tr><th>2</th><td>2</td><td>4</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x & y\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 3 \\\\\n",
       "\t2 & 2 & 4 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ x     │ y     │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n",
       "├─────┼───────┼───────┤\n",
       "│ 1   │ 1     │ 3     │\n",
       "│ 2   │ 2     │ 4     │"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(x=1:2, y=3:4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2×2 Array{Int64,2}:\n",
       " 1  3\n",
       " 2  4"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Matrix(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this next example, Julia correctly identifies that `Union` is needed to express the type of the resulting `Matrix` (which contains `missing`s)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr><tr><th></th><th>Int64</th><th>Int64?</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td><em>missing</em></td></tr><tr><th>2</th><td>2</td><td>4</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x & y\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64?\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & \\emph{missing} \\\\\n",
       "\t2 & 2 & 4 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ x     │ y       │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64?\u001b[39m  │\n",
       "├─────┼───────┼─────────┤\n",
       "│ 1   │ 1     │ \u001b[90mmissing\u001b[39m │\n",
       "│ 2   │ 2     │ 4       │"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(x=1:2, y=[missing,4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2×2 Array{Union{Missing, Int64},2}:\n",
       " 1   missing\n",
       " 2  4"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Matrix(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that we can't force a conversion of `missing` values to `Int`s!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "ename": "ArgumentError",
     "evalue": "ArgumentError: cannot convert a DataFrame containing missing values to Matrix{Int64} (found for column y)",
     "output_type": "error",
     "traceback": [
      "ArgumentError: cannot convert a DataFrame containing missing values to Matrix{Int64} (found for column y)",
      "",
      "Stacktrace:",
      " [1] convert(::Type{Array{Int64,2}}, ::DataFrame) at /home/bkamins/.julia/packages/DataFrames/3ZmR2/src/abstractdataframe/abstractdataframe.jl:1104",
      " [2] Array{Int64,2}(::DataFrame) at /home/bkamins/.julia/packages/DataFrames/3ZmR2/src/abstractdataframe/abstractdataframe.jl:1116",
      " [3] top-level scope at In[43]:1"
     ]
    }
   ],
   "source": [
    "Matrix{Int}(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conversion to `NamedTuple` related tabular structures"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First define some data frame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr><tr><th></th><th>Int64</th><th>String</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td>A</td></tr><tr><th>2</th><td>2</td><td>B</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x & y\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & String\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & A \\\\\n",
       "\t2 & 2 & B \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ x     │ y      │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mString\u001b[39m │\n",
       "├─────┼───────┼────────┤\n",
       "│ 1   │ 1     │ A      │\n",
       "│ 2   │ 2     │ B      │"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(x=1:2, y=[\"A\", \"B\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we convert a `DataFrame` into a `NamedTuple` of vectors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(x = [1, 2], y = [\"A\", \"B\"])"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ct = Tables.columntable(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we convert it into a vector of `NamedTuples`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2-element Array{NamedTuple{(:x, :y),Tuple{Int64,String}},1}:\n",
       " (x = 1, y = \"A\")\n",
       " (x = 2, y = \"B\")"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rt = Tables.rowtable(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can perform the conversions back to a `DataFrame` using a standard constructor call:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr><tr><th></th><th>Int64</th><th>String</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td>A</td></tr><tr><th>2</th><td>2</td><td>B</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x & y\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & String\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & A \\\\\n",
       "\t2 & 2 & B \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ x     │ y      │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mString\u001b[39m │\n",
       "├─────┼───────┼────────┤\n",
       "│ 1   │ 1     │ A      │\n",
       "│ 2   │ 2     │ B      │"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(ct)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr><tr><th></th><th>Int64</th><th>String</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td>A</td></tr><tr><th>2</th><td>2</td><td>B</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x & y\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & String\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & A \\\\\n",
       "\t2 & 2 & B \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ x     │ y      │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mString\u001b[39m │\n",
       "├─────┼───────┼────────┤\n",
       "│ 1   │ 1     │ A      │\n",
       "│ 2   │ 2     │ B      │"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(rt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Iterating data frame by rows or columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes it is useful to create a wrapper around a `DataFrame` that produces its rows or columns."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For iterating columns you can use the `eachcol` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<p>2×2 DataFrameColumns</p><table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr><tr><th></th><th>Int64</th><th>String</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>A</td></tr><tr><th>2</th><td>2</td><td>B</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x & y\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & String\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & A \\\\\n",
       "\t2 & 2 & B \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrameColumns\n",
       "│ Row │ x     │ y      │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mString\u001b[39m │\n",
       "├─────┼───────┼────────┤\n",
       "│ 1   │ 1     │ A      │\n",
       "│ 2   │ 2     │ B      │"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ec = eachcol(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`DataFrameColumns` object behaves as a vector (note though that is is a view - it does not create)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "true"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ec isa AbstractVector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2-element Array{Int64,1}:\n",
       " 1\n",
       " 2"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ec[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "similarly `eachrow` creates a `DataFrameRows` object that is a vector of its rows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<p>2×2 DataFrameRows</p><table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr><tr><th></th><th>Int64</th><th>String</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>A</td></tr><tr><th>2</th><td>2</td><td>B</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x & y\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & String\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & A \\\\\n",
       "\t2 & 2 & B \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrameRows\n",
       "│ Row │ x     │ y      │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mString\u001b[39m │\n",
       "├─────┼───────┼────────┤\n",
       "│ 1   │ 1     │ A      │\n",
       "│ 2   │ 2     │ B      │"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "er = eachrow(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "true"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "er isa AbstractVector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<p>DataFrameRow (2 columns)</p><table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr><tr><th></th><th>Int64</th><th>String</th></tr></thead><tbody><tr><th>2</th><td>2</td><td>B</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x & y\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & String\\\\\n",
       "\t\\hline\n",
       "\t2 & 2 & B \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "DataFrameRow\n",
       "│ Row │ x     │ y      │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mString\u001b[39m │\n",
       "├─────┼───────┼────────┤\n",
       "│ 2   │ 2     │ B      │"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "er[end]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that both data frame and also `DataFrameColumns` and `DataFrameRows` objects are not type stable (they do not know the types of their columns). This is useful to avoid compilation cost if you have very wide data frames with heterogenous column types."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, often (especially if a data frame is narrows) it is useful to create a lazy iterator that produces `NamedTuple`s for each row of the `DataFrame`. Its key benefit is that it is type stable (so it is useful when you want to perform some operations in a fast way on a small subset of columns of a `DataFrame` - this strategy is often used internally by DataFrames.jl package):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Tables.NamedTupleIterator{Tables.Schema{(:x, :y),Tuple{Int64,String}},Tables.RowIterator{NamedTuple{(:x, :y),Tuple{Array{Int64,1},Array{String,1}}}}}(Tables.RowIterator{NamedTuple{(:x, :y),Tuple{Array{Int64,1},Array{String,1}}}}((x = [1, 2], y = [\"A\", \"B\"]), 2))"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nti = Tables.namedtupleiterator(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "row = (1, (x = 1, y = \"A\"))\n",
      "row = (2, (x = 2, y = \"B\"))\n"
     ]
    }
   ],
   "source": [
    "for row in enumerate(nti)\n",
    "    @show row\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "similarly to the previous options you can easily convert `NamedTupleIterator` back to a `DataFrame`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th></tr><tr><th></th><th>Int64</th><th>String</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>1</td><td>A</td></tr><tr><th>2</th><td>2</td><td>B</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x & y\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & String\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & A \\\\\n",
       "\t2 & 2 & B \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×2 DataFrame\n",
       "│ Row │ x     │ y      │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mString\u001b[39m │\n",
       "├─────┼───────┼────────┤\n",
       "│ 1   │ 1     │ A      │\n",
       "│ 2   │ 2     │ B      │"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(nti)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Handling of duplicate column names\n",
    "\n",
    "We can pass the `makeunique` keyword argument to allow passing duplicate names (they get deduplicated)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>a_2</th><th>a_1</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>1 rows × 3 columns</p><tr><th>1</th><td>1</td><td>2</td><td>3</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& a & a\\_2 & a\\_1\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 2 & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "1×3 DataFrame\n",
       "│ Row │ a     │ a_2   │ a_1   │\n",
       "│     │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │ \u001b[90mInt64\u001b[39m │\n",
       "├─────┼───────┼───────┼───────┤\n",
       "│ 1   │ 1     │ 2     │ 3     │"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame(:a=>1, :a=>2, :a_1=>3; makeunique=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Otherwise, duplicates are not allowed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "ename": "ArgumentError",
     "evalue": "ArgumentError: Duplicate variable names: :a. Pass makeunique=true to make them unique using a suffix automatically.",
     "output_type": "error",
     "traceback": [
      "ArgumentError: Duplicate variable names: :a. Pass makeunique=true to make them unique using a suffix automatically.",
      "",
      "Stacktrace:",
      " [1] make_unique!(::Array{Symbol,1}, ::Array{Symbol,1}; makeunique::Bool) at /home/bkamins/.julia/packages/DataFrames/3ZmR2/src/other/utils.jl:35",
      " [2] #make_unique#2 at /home/bkamins/.julia/packages/DataFrames/3ZmR2/src/other/utils.jl:57 [inlined]",
      " [3] #Index#3 at /home/bkamins/.julia/packages/DataFrames/3ZmR2/src/other/index.jl:27 [inlined]",
      " [4] DataFrame(::Pair{Symbol,Int64}, ::Vararg{Pair{Symbol,Int64},N} where N; makeunique::Bool, copycols::Bool) at /home/bkamins/.julia/packages/DataFrames/3ZmR2/src/dataframe/dataframe.jl:171",
      " [5] DataFrame(::Pair{Symbol,Int64}, ::Vararg{Pair{Symbol,Int64},N} where N) at /home/bkamins/.julia/packages/DataFrames/3ZmR2/src/dataframe/dataframe.jl:169",
      " [6] top-level scope at In[59]:1"
     ]
    }
   ],
   "source": [
    "df = DataFrame(:a=>1, :a=>2, :a_1=>3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finallly observe that `nothing` is not printed when displaying a `DataFrame`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x</th><th>y</th><th>z</th></tr><tr><th></th><th>Union…</th><th>Union…</th><th>String?</th></tr></thead><tbody><p>2 rows × 3 columns</p><tr><th>1</th><td>1</td><td></td><td><em>missing</em></td></tr><tr><th>2</th><td></td><td>a</td><td>c</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x & y & z\\\\\n",
       "\t\\hline\n",
       "\t& Union… & Union… & String?\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 &  & \\emph{missing} \\\\\n",
       "\t2 &  & a & c \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "2×3 DataFrame\n",
       "│ Row │ x      │ y      │ z       │\n",
       "│     │ \u001b[90mUnion…\u001b[39m │ \u001b[90mUnion…\u001b[39m │ \u001b[90mString?\u001b[39m │\n",
       "├─────┼────────┼────────┼─────────┤\n",
       "│ 1   │ 1      │        │ \u001b[90mmissing\u001b[39m │\n",
       "│ 2   │        │ a      │ c       │"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DataFrame(x=[1, nothing], y=[nothing, \"a\"], z=[missing, \"c\"])"
   ]
  }
 ],
 "metadata": {
  "@webio": {
   "lastCommId": null,
   "lastKernelId": null
  },
  "kernelspec": {
   "display_name": "Julia 1.4.1",
   "language": "julia",
   "name": "julia-1.4"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.4.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}