notebooks/PROB_low_of_large_numbers.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 大数の法則について\n",
    "\n",
    "参考：\n",
    "* [The Laws of Large Numbers Compared](http://www.dklevine.com/archive/strong-law.pdf)\n",
    "\n",
    "準備：\n",
    "* $X_1, X_2, \\dots$を独立な確率変数とします．それぞれの期待値を$\\mu_i$とします．\n",
    "* $\\bar{X}_n=\\frac{1}{n} \\sum_{i=1}^n X_i$\n",
    "* $\\bar{\\mu}_n=\\frac{1}{n} \\sum_{i=1}^n \\mu_i$\n",
    "* $\\mathbb{E}[X_n^2] < \\infty$のとき，$X_n \\in L^2$ と書きます．ちなみに$X_n \\in L^2$ならば$\\mathbb{E}[|X_n|] < \\infty$です．（[これなど](https://homepage.univie.ac.at/erhard.reschenhofer/pdf/advanced/L2.pdf)）\n",
    "\n",
    "大数の法則には大きく弱法則と強法則の２種類があります．ちゃんと見てみましょう．"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 大数の弱法則\n",
    "\n",
    "次の定理を大数の弱法則といいます．\n",
    "\n",
    "---\n",
    "\n",
    "独立な$(X_n)_{n=1}^\\infty$が$X_n \\in L^2$かつ\n",
    "$$\n",
    "\\mathbb{V}[X_n] = \\mathbb{E}[(X_n - \\mu_n)^2] = \\sigma_n^2\n",
    "$$\n",
    "が\n",
    "$$\n",
    "\\sup_n \\frac{\\sum^n_{k=1} \\sigma_k^2}{n} < \\infty\n",
    "$$\n",
    "をみたすとする（例えば分散が上から抑えられていればこれは成り立つ）．このとき，\n",
    "\n",
    "$$\n",
    "\\forall_{\\varepsilon>0} \\lim _{n \\rightarrow \\infty} \\operatorname{Pr}\\left(\\left|\\bar{X}_n-\\bar{\\mu}_n\\right| \\leq \\varepsilon\\right)=1\n",
    "$$\n",
    "\n",
    "---\n",
    "\n",
    "このとき，**$\\bar{X}_n$は$\\bar{\\mu}_n$に確率収束する**といいます．\n",
    "丁寧に書くと，$\\forall \\varepsilon$について\n",
    "$$\n",
    "\\lim_{n \\to \\infty}\n",
    "\\operatorname{Pr}\\left(\\{\\omega \\in \\Omega \\mid |X_n(\\omega) - X(\\omega)| > \\varepsilon\\}\\right) = 0\n",
    "$$\n",
    "のとき，$X_n$は$X$に確率収束する，といいます．\n",
    "\n",
    "また，ε-δ論法を使えば\n",
    "\n",
    "$$\n",
    "\\forall_{\\varepsilon>0} \\forall_{\\delta>0} \\exists_{N>0} \\forall_{n \\geq N} \\operatorname{Pr}\\left(\\left|\\bar{X}_n-\\bar{\\mu}_n\\right| \\leq \\varepsilon\\right) \\geq 1-\\delta\n",
    "$$\n",
    "と書くこともできます．\n",
    "\n",
    "**証明**\n",
    "\n",
    "独立性より，$i \\neq j$のとき，\n",
    "$$\n",
    "\\mathbb{E}[(X_i - \\mu_i)(X_j - \\mu_j)] = \\mathbb{E}[X_i]\\mathbb{E}[X_j] - \\mu_i \\mu_j = 0\n",
    "$$\n",
    "よって，\n",
    "$$\n",
    "\\begin{aligned}\n",
    "\\mathbb{E}[(\\bar{X}_n - \\bar{\\mu}_n)^2]\n",
    "&= \\frac{1}{n^2} \\mathbb{E}\\left[\\left(\\sum^n_{i=1} X_i - \\mu_i\\right)^2\\right]\\\\\n",
    "&= \\frac{1}{n^2} \\sum^n_{i=1}\\mathbb{E}\\left[\\left(X_i - \\mu_i\\right)^2\\right]\\\\\n",
    "&= \\frac{1}{n} \\frac{\\sum^n_{i=1}\\sigma^2_i}{n}\n",
    "\\end{aligned}\n",
    "$$\n",
    "仮定より二項目は$< \\infty$なので，$n\\to \\infty$でこれは$0$に収束します．\n",
    "\n",
    "これは$p$次平均収束そのものです．\n",
    "\n",
    "---\n",
    "\n",
    "**$p$次平均収束**\n",
    "\n",
    "$p > 0$とする．$L^p \\triangleq \\{確率変数 X \\mid |X|^pが可積分\\}$という表記を導入する．\n",
    "\n",
    "このとき，\n",
    "$X_n, X\\in L^p$ かつ$\\lim_{n\\to \\infty} \\mathbb{E}[|X_n - X|^p]=0$のとき，$X_n$は$X$に$p$次平均収束するという．\n",
    "\n",
    "実は$p$次平均収束するならば確率収束します．\n",
    "証明はチェビシェフの不等式を使えば簡単です．実際，$\\forall \\varepsilon > 0$に対して，\n",
    "\n",
    "$$\n",
    "\\mathbb{P}(|X_n - X| \\geq \\varepsilon)\n",
    "= \\mathbb{E}(1\\{|X_n - X| \\geq \\varepsilon\\})\n",
    "\\leq \\frac{\\mathbb{E}|X_n - X|^p}{\\varepsilon^p} \\to 0\n",
    "$$\n",
    "ここで不等式はチェビシェフの不等式を使ってます（[一般版](https://en.wikipedia.org/wiki/Chebyshev%27s_inequality)）\n",
    "\n",
    "---\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 中心極限定理\n",
    "\n",
    "弱法則では，$\\sigma^2_n$が有界な場合は成立し，$\\bar{X}_n$が$\\mu$に確率収束することが言えます．\n",
    "一方で，有界ならば中心極限定理が使え，もっと強いことが言えます．\n",
    "\n",
    "---\n",
    "\n",
    "**中心極限定理**\n",
    "\n",
    "$(X_n)_{n=1}^\\infty$が平均$\\mu$，分散$\\sigma^2$のi.i.d.の確率変数とする．\n",
    "このとき，\n",
    "$$\n",
    "\\frac{\\sqrt{n}}{\\sigma}\\left(\\bar{X}_n - \\mu\\right) \\rightsquigarrow \\mathcal{N}(0, 1)\n",
    "$$\n",
    "が成立する．\n",
    "\n",
    "TODO: 分布収束について書く．特性関数について書く．\n",
    "\n",
    "---\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 大数の強法則\n",
    "\n",
    "$(X_n)_{n=1}^\\infty$はi.i.d.であり，$X_1 \\in L^4$とします．このとき，\n",
    "\n",
    "$$\n",
    "\\operatorname{Pr}\\left(\\lim _{n \\rightarrow \\infty} \\bar{X}_n=\\mu\\right)=1\n",
    "$$\n",
    "が成立します．\n",
    "\n",
    "このとき，**$\\bar{X}_n$が$X$に概収束する**といいます．\n",
    "\n",
    "\n",
    "丁寧に書くと，\n",
    "$$\n",
    "\\operatorname{Pr}\\left(\\{w \\in \\Omega \\mid \\lim_{n \\to \\infty} X_n(\\omega) = X(\\omega)\\}\\right) = 1\n",
    "$$\n",
    "のとき，$X_n$は$X$に概収束するといいます．\n",
    "\n",
    "証明にはBorel-Cantelliの補題と特性関数などを使います．TODO: 証明\n",
    "\n",
    "TODO: これ書く：概収束と確率収束の違いは[これ](https://wiis.info/math/probability/asymptotic-theory/probability-limit-and-almost-sure-convergence/)を参考にしてください．"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}