lecture_3_calculus_1.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create anaconda environment\n",
    "<br>\n",
    "```bash\n",
    "conda create -n ml python=3.7.4 jupyter\n",
    "```\n",
    "Install fastai library\n",
    "<br>\n",
    "```bash\n",
    "conda install -c pytorch -c fastai fastai\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install https://github.com/ipython-contrib/jupyter_contrib_nbextensions/tarball/master\n",
    "!pip install jupyter_nbextensions_configurator\n",
    "!jupyter contrib nbextension install --user\n",
    "!jupyter nbextensions_configurator enable --user"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Metric spaces"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let $X$ be a set and $d:X \\times X \\to [0, +\\infty)$ (for every $x, y \\in X$: $d(u, v) \\geq 0$) be a function such that:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- for every $x \\in X$: $d(x, x) = 0$\n",
    "- for every $x, y \\in X$: $d(x, y) = d(y, x)$ (symmetry)\n",
    "- for every $x, y, z \\in X$: $d(x, z) \\leq d(x, y) + d(y, z)$ (triangle inequality)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$X$ set with such function $d:X \\times X \\to [0, +\\infty)$ is called metric space and function $d$ - metrics on $X$, we denote it with $(X, d)$ or simple with $X$ if metrics is well defined"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The $r \\in \\mathbb{R}$ radius open ball with center $x \\in X$ in metric space $(X, d)$ is the set $B(x, r) = \\{y|d(x, y) \\lt r \\}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Proposition:  \n",
    "- For each $x \\in X$ and each $r \\in \\mathbb{R}$ and every point $y \\in B(x, r)$ there exists $r' \\in \\mathbb{R}$ such that $B(y, r') \\subset B(x, r)$\n",
    "#### Proof:\n",
    "- Let $x \\in R$, $r \\in \\mathbb{R}$ and $y \\in B(x, r)$, let $l = d(x, y)$ then: $l = d(x, y) \\lt r$ by the definition and thus $r' = r - l \\gt 0$ and consider $z \\in B(y, r')$ we have that $d(x, z) \\lt r$ $d(x, z) \\leq d(x, y) + d(y, z)$ (traiangle inequality), but $d(x, y) = l$ and $d(y, z) \\lt r' = r - l$, therefore, $d(x, z) \\lt l + r - l = r$ which implies that $z \\in B(x, r)$. So we've got that, each element of $B(y, r')$ belongs to $B(x, r)$ which means that $B(y, r') \\subset B(x, r)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The set $U \\subset X$ is open in metric space (X, d) if for each point $x \\in U$ there exists $r \\in \\mathbb{R}$ such that $B(x, r) \\subset U$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Prove that:\n",
    "For every metric space (X, d):\n",
    "- For any $\\mathcal{U} = \\{U | U$ is open in $(X, d)\\}$ holds $\\bigcup_{U \\in \\mathcal{U}} U$ is open in $(X, d)$\n",
    "- For any two $U, V \\subset X$ open in $(X, d)$ holds: $U \\cap V$ is open in $(X, d)$\n",
    "- $X$ is open in $(X, d)$\n",
    "- $\\emptyset$ is open in $(X, d)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Prove that:\n",
    "- For any finity set $(U)_{i=1}^{n}$ of open sets in $(X, d)$, $\\bigcap_{i=1}^{n}U$ is open in $(X, d)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Open neighbourhood of point $x \\in X$ is an open set $U \\subset X$ such that $x \\in U$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We proved that an open ball is open subset in $(X, d)$ for any metric space"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Subset $U \\subset A$ in metric space $(X, d)$ is called open subset if it is open in $(X, d)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interior of the set $A$ is $Int(A) = \\bigcup \\{U \\subset A | U $ is open in $(X, d)\\}$ (the biggest open subset of $A$)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Prove that for set $U \\subset X$ is open if and only if for each point $x \\in U$ there exists the open neighbourhood $V$ of $x$ such that $V \\subset U$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The set $F \\subset X$ is closed if $X - F$ is open\n",
    "<br>\n",
    "Closed ball with center $x \\in X$ with radius $r \\in \\mathbb{R}$ is $\\bar{B}(x, r) = \\{y | d(x, y) \\le r \\}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Prove that the closed ball is closed subset in $(X, d)$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Prove that\n",
    "For every metric space (X, d):\n",
    "- For any $\\mathcal{F} = \\{F | F$ is closed in $(X, d)\\}$ holds $\\bigcap_{F \\in \\mathcal{F}} F$ is closed in $(X, d)$\n",
    "- For any two $F_1, F_2 \\subset X$ closed in $(X, d)$ holds: $F_1 \\cup F_2$ is closed in $(X, d)$\n",
    "- $X$ is closed in $(X, d)$\n",
    "- $\\emptyset$ is closed in $(X, d)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Prove that:\n",
    "- For any finite set $(F)_{i=1}^{n}$ of closed sets in $(X, d)$, $\\bigcup_{i=1}^{n}F$ is closed in $(X, d)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Closure of the set $A \\subset X$ is $cl(A) = \\bigcap\\{F \\subset X | F \\text{ is closed in } (X, d)$ (the smallest set which contains $A$)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The frontier of the set $A \\subset X$ is $Cl(A) - Int(A)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Prove that, if $F \\subset X$ is closed then $X - F$ is open in $(X, d)$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The set $A \\subset X$ in metric space $(X, d)$ is called bounded if there exists $r \\in \\mathbb{R}$ such that $d(x, y) < r$ for each $x, y \\in A$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Examples of metric spaces"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For any set $X$ define $d:X \\times X \\to [0, \\infty)$ as \n",
    "$$d(x, y) = \\left\\{\\begin{matrix}\n",
    "0 \\text{ iff } x = y \\\\\n",
    "1 \\text{ otherwise} \\\\\n",
    "\\end{matrix}\\right.$$\n",
    "<br>\n",
    "All three metric properties hold.\n",
    "<br>\n",
    "This metric is called discrete metric and space $(X, d)$ discrete space\n",
    "<br>\n",
    "In discret space each set $U \\subset X$ is open and close"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Metrics in Euclidean spaces"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For $v, u \\in \\mathbb{R}^{n}$ let define distance function $d:\\mathbb{R}^{n} \\times \\mathbb{R}^{n} \\to \\mathbb{R}$ by $d(u, v) = ||u-v||_2$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prove that:\n",
    "- for every $u, v \\in \\mathbb{R}^{n}$: $d(u, v) \\geq 0$\n",
    "- for every $v \\in \\mathbb{R}^{n}$: $d(v, v) = 0$\n",
    "- for every $u, v \\in \\mathbb{R}^{n}$: $d(u, v) = d(v, u)$ (symmetry)\n",
    "- for every $u, v, w \\in \\mathbb{R}^{n}$: $d(u, w) \\leq d(v, u) + d(v, w)$ (triangle inequality)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prove the same properties hold for $d(u, v) = ||u-v||_1$ ($||u-v||_1 = \\sum_{i = 1}^{n}|u_i - v_i|$)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sequences and limits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Every bijection from $\\mathbb{N}$ or $N_0 \\subset \\mathbb{N}$ to some set $A$ is called sequence"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can write sequence as $(x_i)_{i=1}^{\\infty} = (x_1, x_2, \\dots x_n)$ if $N_0 \\subset \\mathbb{N}$ is finite of $(x_i)_{i=1}^{\\infty} = (x_1, x_2, \\dots, x_n)$ if it's infinite\n",
    "<br>\n",
    "We say that point $x \\in X$ is a limit of the sequence $(x_i)_{i=1}^{\\infty}$ and write $\\lim_{n \\to \\infty}{x_n} = x$ if for every open set $U \\in X$ such that $x \\in U$ there exists $n_0 \\in \\mathbb{R}$ such that $x_i \\in U$ for every $i \\gt n_0$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Prove that $x = \\lim_{n\\to\\infty}{x_n}$ in $(X, d)$ if and only if (iff) for every $r \\in \\mathbb{R}$ there exists $n_0 \\in \\mathbb{N}$ such that: $x_i \\in B(x, r)$ for every $i \\gt n_0$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A subsequence of the sequence $(x_i)_{i=1}^{\\infty} = (x_1, x_2, \\dots, x_n)$ is an equivalent subset of this sequence"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convergent (Cauchy) sequence is a sequence $(x_i)_{i=1}^{\\infty} = (x_1, x_2, \\dots, x_n)$ such that, for every $r \\in \\mathbb{R}$ there exists $n_0 \\in \\mathbb{N}$ such that $d(x_i, x_j) < r$ for every $i, j > n_0$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Prove that if $x = \\lim_{n\\to\\infty}{x_n}$ and $x \\notin \\{-\\infty, \\infty\\}$ then $(x_i)_{i=1}^{\\infty} = (x_1, x_2, \\dots, x_n)$ is a Cauchy sequence\n",
    "<br>\n",
    "For closed set $F \\subset \\mathbb{R}^n$ and convergent sequence $(x_i)_{i=1}^{\\infty} = (x_1, x_2, \\dots, x_n)$ such that there exists $n_0 \\in \\mathbb{N}$ such that $x_i \\in F$ for each $i \\gt n_0$ then: $\\lim_{n\\to\\infty}{x_n} \\in F$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Prove that if $F$ is open from the previous example, the statement does not hold. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Properties of limits\n",
    "- The limit of a sequence is unique.\n",
    "- $\\lim _{n\\to \\infty }(a_{n}\\pm b_{n})=\\lim _{n\\to \\infty }a_{n}\\pm \\lim _{n\\to \\infty }b_{n}$\n",
    "<br>\n",
    "- $\\lim _{n\\to \\infty }ca_{n}=c\\cdot \\lim _{n\\to \\infty }a_{n}$\n",
    "<br>\n",
    "- $\\lim _{n\\to \\infty }(a_{n}\\cdot b_{n})=(\\lim _{n\\to \\infty }a_{n})\\cdot (\\lim _{n\\to \\infty }b_{n})$\n",
    "<br>\n",
    "- $\\lim _{n\\to \\infty }\\left({\\frac {a_{n}}{b_{n}}}\\right)={\\frac {\\lim \\limits _{n\\to \\infty }a_{n}}{\\lim \\limits _{n\\to \\infty }b_{n}}}$ provided $\\lim _{n\\to \\infty }b_{n}\\neq 0$\n",
    "<br>\n",
    "- $\\lim _{n\\to \\infty }a_{n}^{p}=\\left[\\lim _{n\\to \\infty }a_{n}\\right]^{p}$\n",
    "<br>\n",
    "- if $a_{i}\\leq b_{i}$ for some $n_0 \\in \\mathbb{R}^n$ and all $i > n_0$ then ${\\displaystyle \\lim _{n\\to \\infty }a_{n}\\leq \\lim _{n\\to \\infty }b_{n}}$\n",
    "<br>\n",
    "- If $a_{i}\\leq c_{i}\\leq b_{i}$ for some $n_0 \\in \\mathbb{R}^n$ and all $i > n_0$ and $\\lim _{n\\to \\infty }a_{n}=\\lim _{n\\to \\infty }b_{n}=L$ then $\\lim _{n\\to \\infty }{c_n} = L$\n",
    "- If a sequence is bounded and monotonic then it is convergent.\n",
    "<br>\n",
    "- A sequence is convergent if and only if every subsequence is convergent.\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If each convergent sequence has a limit in some metric space it is called complete"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets define subspace as a subset of metric space $(X, d)$ as a subset with the same metric, which is called inherited metric of subspace"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Prove that inherited metric is a metric"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example of sequence which is bounded but does not converge"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$x_n = \\left\\{\\begin{matrix}\n",
    "-\\frac{1}{2} \\text{ iff } n \\text{ is odd} \\\\\n",
    "\\frac{1}{2} \\text{ otherwise} \\\\\n",
    "\\end{matrix}\\right.$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Every subset in discrete space is open and close simultaneously"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The limit point of the set $A \\subset X$ in a metric space $(X, d)$ is a point $x \\in X$ such that for each open neighborhood of $x \\in U \\subset X$ there exists at least one $y \\neq x$ such that $y \\in A \\cap U$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If $x \\in X$ is a limit point of $A$ if there exists a sequence $(x_i)_{i=1}{\\infty} \\subset A$ such that $\\lim _{n\\to \\infty }{x_n} = x$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we take sequence $(\\frac{1}{n})_{n=1}^{\\infty}$ and sequence $(x_n)_{n=1}^{\\infty} = \\{x_n| x_n \\in B(x, \\frac{1}{n}) \\cap A\\}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let for every function $f:X \\to Y$ and $A \\subset X$ define $f(A) \\subset Y$ $f(A) = \\{f(x) | x \\in A\\}$ or $\\bigcup_{x \\in A}{f(x)}$\n",
    "<br>\n",
    "For $f:X \\to Y$ function let define $f^{-1}:Y \\to 2^{X}$ by $\\{x \\in f^{-1}(y), |f(x) = y\\}$\n",
    "<br>\n",
    "in a similar way, we can define $f^{-1}(B)$ for each $B \\subset Y = \\bigcup\\{f^{-1}(y) | y\\in B\\}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Limits of functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let $f:S \\to Y$ is a function between subset $S \\subset X$ of a metric space $(X, d_x)$ and metric space $(Y, d_Y)$\n",
    "- We say that the limit of function $f:(S, d_x) \\to (Y, d_Y)$ between metric spaces in some limit point $c \\in X$ of the subset $S$ is $y \\in Y $if for each open neighborhood of $y \\in V \\subset Y$ there exists the open neighbourhood of $c \\in U \\subset X$ such that $f(U \\cap S) \\subset V$\n",
    "<br>\n",
    "This definition is equivalent of definition:\n",
    "<br>\n",
    "- The limit of function $f:(S, d_X) \\to (Y, d_Y)$ between metric spaces in limit point $c \\in X$ of the subset $S$ is $y \\in Y $ if for each open ball $B(x, r) \\subset Y$ there exists the ball $B(c, l) \\subset X$ such that $f(B(c, l) \\cap S) \\subset B(y, r)$\n",
    "<br>\n",
    "or\n",
    "<br>\n",
    "- The limit of function $f:(S, d_X) \\to (Y, d_Y)$ between metric spaces in limit point $c \\in X$ of subset $S$ is $y \\in Y $ if for any $r \\in \\mathbb{R}$ there exists $l \\in \\mathbb{R}$ such that for every $x \\in S$ with $d_X(x, c) < l$ implies that $d_Y(f(x), y) < r$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We write: $$\\lim_{x \\to c}f(x) = y$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Prove that these three definitions are equivalent for any function between any two metric spaces"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For one dimensional euclidean space with natural metrics the definition coincides with our definition"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Continuous functions "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- The function $f:(X, d_X) \\to (Y, d_Y)$ is called continuous in $c \\in X$ if $$\\lim_{x \\to c}f(x) = f(c)$$\n",
    "-  The function $f:(X, d_X) \\to (Y, d_Y)$ is called continuous if for every open neighbourhood of $f(c) \\in U \\subset Y$ there exists the open neighborhood $c \\in V \\subset X$ such that $f(V) \\subset U$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Prove that function is continuous in $c$ if for any sequence $(x_n)_{n=1}^{\\infty} \\subset X$ such that $\\lim_{n \\to \\infty}x_n = c$ we have $\\lim_{n \\to \\infty}f(x_n) = f(c)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Function $f:(X, d_X) \\to (Y, d_Y)$ is called continuous if it is continuous in each point $x \\in X$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Prove that function is continuous if for every open set $V \\subset Y$ the $f^{-1}(V)$ is open in $X$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Prove that function is continuous if for every closed set $F \\subset Y$ the $f^{-1}(F)$ is open in $X$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Prove that any composition of continuous functions is continuous"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Continuous functions in Euclidean spaces"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Every linear function is continuous"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.2"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}