Lab09/Lab09-1_YounchanChoi.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lab09-1 XOR - Logistic Regression - Eager Excuetion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- XOR 문제를 Logistic Regression을 활용해 풀어보도록 하겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.6.0\n"
     ]
    }
   ],
   "source": [
    "# 기본 Library 선언 및 Tensorflow 버전 확인\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "import tensorflow as tf\n",
    "\n",
    "tf.random.set_seed(777)  # for reproducibility\n",
    "\n",
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 강의에 설명할 Data입니다.\n",
    "- x_data가 2차원 배열이기에 2차원 공간에 표현하여 x1과 x2를 기준으로 y_data 0과 1로 구분하는 예제입니다.\n",
    "- 붉은색과 푸른색으로 0과 1을 표시해 보도록 하겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAQRUlEQVR4nO3db4xcV33G8e9Tmwgof0Lxgqgdajcyf9wqQWETohbaAGqx0xcWFaoSUKJGqawUQnmZCAlQ674oqlohRMCyIieCtlgVRMSggFW1glRKU7yWEicmDd06IdnaUTZAKQqttrZ/fTHjMl3P2mtn7gzj8/1Iq517z9mZ39ldnWfOnZl7U1VIktr1c5MuQJI0WQaBJDXOIJCkxhkEktQ4g0CSGrd20gWcq3Xr1tXGjRsnXYYkTZWDBw8+V1Uzw9qmLgg2btzI3NzcpMuQpKmS5HsrtXloSJIaZxBIUuMMAklqnEEgSY0zCCSpcZ0FQZI9SZ5N8ugK7Uny6STzSQ4luaKrWgCOHYNLL4VnnunyUSSpIx1OYl2uCO4Gtp6hfRuwuf+1A/hch7Wwcyc8+WTvuyRNnQ4nsc6CoKruB35whi7bgc9Xz4PAxUle10Utx47BXXfByZO9764KJE2VjiexSb5GsB54emB7ob/vNEl2JJlLMre4uHjOD7RzZ+/3B3DihKsCSVOm40lskkGQIfuGXiWnqnZX1WxVzc7MDP2E9IpOBenSUm97aclVgaQpMoZJbJJBsABcMrC9ATg66gcZDNJTXBVImhpjmMQmGQT7gBv77x66GvhRVR0b+YPs+2mQnrK0BPfeO+pHkqQOjGES6+ykc0m+CFwDrEuyAHwCeBFAVe0C7gOuBeaBnwA3dVHHwkIX9ypJYzKGSayzIKiq68/SXsCHunp8SdLq+MliSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIa12kQJNma5PEk80luH9L+yiRfTfJwksNJbuqyHknS6ToLgiRrgDuAbcAW4PokW5Z1+xDwnaq6HLgG+IskF3VVkyTpdF2uCK4C5qvqSFUtAXuB7cv6FPDyJAFeBvwAON5hTZKkZboMgvXA0wPbC/19gz4DvBk4CjwCfKSqTi6/oyQ7kswlmVtcXOyqXklqUpdBkCH7atn2e4CHgF8E3gJ8JskrTvuhqt1VNVtVszMzM6OuU5Ka1mUQLACXDGxvoPfMf9BNwD3VMw88Abypw5okSct0GQQHgM1JNvVfAL4O2Lesz1PAuwGSvBZ4I3Ckw5okScus7eqOq+p4kluB/cAaYE9VHU5yS799F7ATuDvJI/QOJd1WVc91VZMk6XSdBQFAVd0H3Lds366B20eB3+6yBknSmfnJYklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktS4ToMgydYkjyeZT3L7Cn2uSfJQksNJvtVlPZKk063t6o6TrAHuAH4LWAAOJNlXVd8Z6HMx8Flga1U9leQ1XdUjSRquyxXBVcB8VR2pqiVgL7B9WZ/3A/dU1VMAVfVsh/VIkoboMgjWA08PbC/09w16A/CqJN9McjDJjcPuKMmOJHNJ5hYXFzsqV5La1GUQZMi+Wra9Fngr8DvAe4CPJXnDaT9UtbuqZqtqdmZmZvSVSlLDOnuNgN4K4JKB7Q3A0SF9nquq54Hnk9wPXA58t8O6JEkDulwRHAA2J9mU5CLgOmDfsj73Au9IsjbJS4G3AY91WJMkaZnOVgRVdTzJrcB+YA2wp6oOJ7ml376rqh5L8g3gEHASuLOqHu2qJknS6VK1/LD9z7bZ2dmam5ubdBmSNFWSHKyq2WFtfrJYkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklq3BmDIMkrklw6ZP9l3ZUkSRqnFYMgye8B/wJ8uX9h+SsHmu/uujBJ0nicaUXwUeCtVfUW4CbgC0l+t9827OpjkqQpdKbrEaypqmMAVfXtJO8EvpZkA6dfclKSNKXOtCL48eDrA/1QuAbYDvxKx3VJksbkTEHwh8DPJdlyakdV/RjYCvxB14VJksZjxSCoqoer6l+Bv01yW3peAvwl8MGxVShJ6tRqPkfwNuAS4AF6F6Q/Cvx6l0VJksZnNUHwP8B/AS8BXgw8UVUnO61KkjQ2qwmCA/SC4Erg7cD1Sb7UaVWSpLE509tHT7m5qub6t58Btie5ocOaJEljdNYVwUAIDO77QjflSJLGzZPOSVLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhrXaRAk2Zrk8STzSW4/Q78rk5xI8r4u65Ekna6zIEiyBrgD2AZsoXeOoi0r9PsksL+rWiRJK+tyRXAVMF9VR6pqCdhL7+pmy30Y+DLwbIe1SJJW0GUQrAeeHthe6O/7P0nWA+8Fdp3pjpLsSDKXZG5xcXHkhUpSy7oMggzZt/yi958CbquqE2e6o6raXVWzVTU7MzMzqvokSazuNNTna4Helc1O2UDv6maDZoG9SQDWAdcmOV5VX+mwLknSgC6D4ACwOckm4N+B64D3D3aoqk2nbie5G/iaISBJ49VZEFTV8SS30ns30BpgT1UdTnJLv/2MrwtIksajyxUBVXUfcN+yfUMDoKp+v8taJEnD+cliSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1LhOgyDJ1iSPJ5lPcvuQ9g8kOdT/eiDJ5V3WI0k6XWdBkGQNcAewDdgCXJ9ky7JuTwC/WVWXATuB3V3VI0karssVwVXAfFUdqaolYC+wfbBDVT1QVT/sbz4IbOiwHknSEF0GwXrg6YHthf6+ldwMfH1YQ5IdSeaSzC0uLo6wRElSl0GQIftqaMfknfSC4LZh7VW1u6pmq2p2ZmZmhCVKktZ2eN8LwCUD2xuAo8s7JbkMuBPYVlXf77AeSdIQXa4IDgCbk2xKchFwHbBvsEOS1wP3ADdU1Xc7rEWStILOVgRVdTzJrcB+YA2wp6oOJ7ml374L+DjwauCzSQCOV9VsVzVJkk6XqqGH7X9mzc7O1tzc3KTLkKSpkuTgSk+0/WSxJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmN6zQIkmxN8niS+SS3D2lPkk/32w8luaKzYo4dg0svhWee6ewhJKkrXU5hnQVBkjXAHcA2YAtwfZIty7ptAzb3v3YAn+uqHnbuhCef7H2XpCnT5RTW5YrgKmC+qo5U1RKwF9i+rM924PPV8yBwcZLXjbySY8fgrrvg5Mned1cFkqZI11NYl0GwHnh6YHuhv+9c+5BkR5K5JHOLi4vnXsnOnb3fIMCJE64KJE2VrqewLoMgQ/bVefShqnZX1WxVzc7MzJxbFaeidGmpt7205KpA0tQYxxTWZRAsAJcMbG8Ajp5HnxdmMEpPcVUgaUqMYwrrMggOAJuTbEpyEXAdsG9Zn33Ajf13D10N/Kiqjo20in37fhqlpywtwb33jvRhJKkL45jC1o7urv6/qjqe5FZgP7AG2FNVh5Pc0m/fBdwHXAvMAz8Bbhp5IQsLI79LSRqXcUxhnQUBQFXdR2+yH9y3a+B2AR/qsgZJ0pn5yWJJapxBIEmNMwgkqXEGgSQ1Lr3Xa6dHkkXge+f54+uA50ZYzjRwzG1wzG14IWP+paoa+oncqQuCFyLJXFXNTrqOcXLMbXDMbehqzB4akqTGGQSS1LjWgmD3pAuYAMfcBsfchk7G3NRrBJKk07W2IpAkLWMQSFLjLsggSLI1yeNJ5pPcPqQ9ST7dbz+U5IpJ1DlKqxjzB/pjPZTkgSSXT6LOUTrbmAf6XZnkRJL3jbO+LqxmzEmuSfJQksNJvjXuGkdtFf/br0zy1SQP98c8+rMYj1GSPUmeTfLoCu2jn7+q6oL6onfK638Dfhm4CHgY2LKsz7XA1+ldIe1q4J8nXfcYxvxrwKv6t7e1MOaBfv9A7yy475t03WP4O18MfAd4fX/7NZOuewxj/ijwyf7tGeAHwEWTrv0FjPk3gCuAR1doH/n8dSGuCK4C5qvqSFUtAXuB7cv6bAc+Xz0PAhcned24Cx2hs465qh6oqh/2Nx+kdzW4abaavzPAh4EvA8+Os7iOrGbM7wfuqaqnAKpq2se9mjEX8PIkAV5GLwiOj7fM0amq++mNYSUjn78uxCBYDzw9sL3Q33eufabJuY7nZnrPKKbZWcecZD3wXmAXF4bV/J3fALwqyTeTHExy49iq68ZqxvwZ4M30LnP7CPCRqlp2cccLysjnr04vTDMhGbJv+XtkV9Nnmqx6PEneSS8I3t5pRd1bzZg/BdxWVSd6Txan3mrGvBZ4K/Bu4CXAPyV5sKq+23VxHVnNmN8DPAS8C7gU+Lsk/1hV/9lxbZMy8vnrQgyCBeCSge0N9J4pnGufabKq8SS5DLgT2FZV3x9TbV1ZzZhngb39EFgHXJvkeFV9ZSwVjt5q/7efq6rngeeT3A9cDkxrEKxmzDcBf1a9A+jzSZ4A3gR8ezwljt3I568L8dDQAWBzkk1JLgKuA/Yt67MPuLH/6vvVwI+q6ti4Cx2hs445yeuBe4AbpvjZ4aCzjrmqNlXVxqraCHwJ+OAUhwCs7n/7XuAdSdYmeSnwNuCxMdc5SqsZ81P0VkAkeS3wRuDIWKscr5HPXxfciqCqjie5FdhP7x0He6rqcJJb+u276L2D5FpgHvgJvWcUU2uVY/448Grgs/1nyMdris/cuMoxX1BWM+aqeizJN4BDwEngzqoa+jbEabDKv/NO4O4kj9A7bHJbVU3t6amTfBG4BliXZAH4BPAi6G7+8hQTktS4C/HQkCTpHBgEktQ4g0CSGmcQSFLjDAJJapxBII1Qkm8k+Y8kX5t0LdJqGQTSaP05cMOki5DOhUEgnYf+NQ4OJXlxkp/vnwf/V6vq74EfT7o+6VxccJ8slsahqg4k2Qf8Kb2Tu/3VNH+CV20zCKTz9yf0zoXz38AfTbgW6bx5aEg6f79A70IoLwdePOFapPNmEEjnbzfwMeCvgU9OuBbpvHloSDoP/St/Ha+qv0myBnggybuAP6Z3LvyX9c8ceXNV7Z9krdLZePZRSWqch4YkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWrc/wJBV9aPpKHk7wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 데이터 생성\n",
    "x_data = [[0, 0],\n",
    "          [0, 1],\n",
    "          [1, 0],\n",
    "          [1, 1]]\n",
    "y_data = [[0],\n",
    "          [1],\n",
    "          [1],\n",
    "          [0]]\n",
    "\n",
    "# 그래프에 각각의 x_data에 따른 y_data값 나타내기\n",
    "plt.scatter(x_data[0][0],x_data[0][1], c='red' , marker='^')\n",
    "plt.scatter(x_data[3][0],x_data[3][1], c='red' , marker='^')\n",
    "plt.scatter(x_data[1][0],x_data[1][1], c='blue' , marker='^')\n",
    "plt.scatter(x_data[2][0],x_data[2][1], c='blue' , marker='^')\n",
    "\n",
    "plt.xlabel(\"x1\")\n",
    "plt.ylabel(\"x2\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tensorflow Eager\n",
    "#### 위 Data를 기준으로 XOR처리를 위한 모델을 만들도록 하겠습니다.\n",
    "- Tensorflow data API를 통해 학습시킬 값들을 담는다 (Batch Size는 한번에 학습시킬 Size로 정한다)\n",
    "- preprocess function으로 features,labels는 실재 학습에 쓰일 Data 연산을 위해 Type를 맞춰준다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# x_data, y_data 값을 tensorflow로 Dataset화 하기\n",
    "dataset = tf.data.Dataset.from_tensor_slices((x_data, y_data)).batch(len(x_data))\n",
    "\n",
    "# 위의 integer값을 연산을 위한 값으로 변환\n",
    "def preprocess_data(features, labels):\n",
    "    features = tf.cast(features, tf.float32)\n",
    "    labels = tf.cast(labels, tf.float32)\n",
    "    return features, labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1) Logistic Regression으로 XOR모델을 만들어 보겠습니다.\n",
    "##### W와 b은 학습을 통해 생성되는 모델에 쓰이는 Weight와 Bias (초기값을 variable : 0이나 Random값으로 가능 tf.random_normal([2, 1]) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "W = [[0.]\n",
      " [0.]], B = [0.]\n"
     ]
    }
   ],
   "source": [
    "# Weight와 Bias의 초기값을 0으로 지정\n",
    "W = tf.Variable(tf.zeros((2,1)), name='weight')\n",
    "b = tf.Variable(tf.zeros((1,)), name='bias')\n",
    "print(\"W = {}, B = {}\".format(W.numpy(), b.numpy()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sigmoid 함수를 가설(예측값)로 선언합니다.\n",
    "- Sigmoid는 아래 그래프와 같이 0과 1의 값만을 리턴합니다 tf.sigmoid(tf.matmul(X, W) + b)와 같습니다.\n",
    "$$\n",
    "\\begin{align}\n",
    "sigmoid(x)  = \\frac{1}{1+e^{-x}}  \\\\\\\\\\\n",
    "\\end{align}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def logistic_regression(features):\n",
    "    hypothesis  = tf.divide(1., 1. + tf.exp(tf.matmul(features, W) + b))\n",
    "    return hypothesis"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 가설을 검증할 Cost 함수를 정의합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def loss_fn(hypothesis, features, labels):\n",
    "    cost = -tf.reduce_mean(labels * tf.math.log(logistic_regression(features)) + (1 - labels) * tf.math.log(1 - hypothesis))\n",
    "    return cost\n",
    "\n",
    "optimizer = tf.keras.optimizers.SGD(learning_rate=0.01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 추론한 값은 0.5를 기준(Sigmoid 그래프 참조)로 0과 1의 값을 리턴합니다.\n",
    "- Sigmoid 함수를 통해 예측값이 0.5보다 크면 1을 반환하고 0.5보다 작으면 0으로 반환합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def accuracy_fn(hypothesis, labels):\n",
    "    predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)\n",
    "    accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, labels), dtype=tf.float32))\n",
    "    return accuracy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### GradientTape를 통해 경사값을 계산합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def grad(hypothesis, features, labels):\n",
    "    with tf.GradientTape() as tape:\n",
    "        loss_value = loss_fn(logistic_regression(features),features,labels)\n",
    "    return tape.gradient(loss_value, [W,b])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tensorflow를 통한 실행을 위해 Session를 선언합니다.\n",
    "- 위의 Data를 Cost함수를 통해 학습시킨 후 모델을 생성합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iter: 0, Loss: 0.6931\n",
      "Iter: 100, Loss: 0.6931\n",
      "Iter: 200, Loss: 0.6931\n",
      "Iter: 300, Loss: 0.6931\n",
      "Iter: 400, Loss: 0.6931\n",
      "Iter: 500, Loss: 0.6931\n",
      "Iter: 600, Loss: 0.6931\n",
      "Iter: 700, Loss: 0.6931\n",
      "Iter: 800, Loss: 0.6931\n",
      "Iter: 900, Loss: 0.6931\n",
      "Iter: 1000, Loss: 0.6931\n",
      "W = [[0.]\n",
      " [0.]], B = [0.]\n",
      "Testset Accuracy: 0.5000\n"
     ]
    }
   ],
   "source": [
    "EPOCHS = 1001\n",
    "\n",
    "for step in range(EPOCHS):\n",
    "    for features, labels  in dataset:\n",
    "        features, labels = preprocess_data(features, labels)\n",
    "        grads = grad(logistic_regression(features), features, labels)\n",
    "        optimizer.apply_gradients(grads_and_vars=zip(grads,[W,b]))\n",
    "        if step % 100 == 0:\n",
    "            print(\"Iter: {}, Loss: {:.4f}\".format(step, loss_fn(logistic_regression(features),features,labels)))\n",
    "print(\"W = {}, B = {}\".format(W.numpy(), b.numpy()))\n",
    "x_data, y_data = preprocess_data(x_data, y_data)\n",
    "test_acc = accuracy_fn(logistic_regression(x_data),y_data)\n",
    "print(\"Testset Accuracy: {:.4f}\".format(test_acc))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-> Iteration에 따라 Loss 감소하는 경향 보이지 않고 Accuracy 또한 50% 수준에 못 미친다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}