Lab09/Lab09-4_JunYoungSeo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lab 09 XOR - tensorboard - eager\n",
    "* XOR 문제를 Deep Neural Network 활용해 풀어보고 Tensorboard에 출력해 보도록 하겠습니다.\n",
    "\n",
    "### 기본 Library 선언 및 Tensorflow 버전 확인"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.12.0\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "import tensorflow as tf\n",
    "import tensorflow.contrib.eager as tfe\n",
    "\n",
    "tf.enable_eager_execution()\n",
    "tf.set_random_seed(777)  # for reproducibility\n",
    "\n",
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 강의에 설명할 Data입니다\n",
    "* x_data가 2차원 배열이기에 2차원 공간에 표현하여 x1과 x2를 기준으로 y_data 0과 1로 구분하는 예제입니다\n",
    "* 붉은색과 푸른색으로 0과 1을 표시해 보도록 하겠습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAEFNJREFUeJzt3W2MXGd5h/HrH5sQCgGqeJFQbOM0dVTcgEq6hFSoJTRp5eSDrQqEEolXpViChlYFoaalBWr3C0WlEpJbcAUJhEIwVCIrMHUlCIoEOPVGKRFOGro1Ads4igkhQkph83L3w4wfJht7d7zZM+NdXz/J8syZRzP38dpz+czZnUlVIUkSwFnjHkCSdPowCpKkxihIkhqjIElqjIIkqTEKkqTGKEiSGqMgSWqMgiSpWT3uAU7VmjVrasOGDeMeQ5KWlTvvvPPHVTWx0LplF4UNGzYwPT097jEkaVlJ8oNh1vnykSSpMQqSpMYoSJIaoyBJaoyCJKkxCpKkxihIkhqjIElqOotCkk8meTDJd09ye5J8NMlMkruTXNLVLHMdPQoXXggPPDCqR5SkRRjDk1WXRwo3AZvnuf0qYGP/1zbgnzuc5Sl27ID77+/9LkmnrTE8WXUWhaq6HfjJPEu2Ap+unn3AC5O8uKt5jjt6FG68EZ58sve7RwuSTktjerIa5zmF84FDA9cP97c9TZJtSaaTTB87duwZPeiOHb0/Y4AnnvBoQdJpakxPVsviRHNV7aqqyaqanJhY8E3+Tup4eGdne9dnZz1akHQaGuOT1TijcARYN3B9bX9bZwbDe5xHC5JOO2N8shpnFKaAN/e/C+ky4JGqOtrpA079MrzHzc7Crbd2+aiSdIrG+GTV2ecpJPkccDmwJslh4APAswCq6mPAHuBqYAZ4FHhbV7Mcd/hw148gSUtgjE9WnUWhqq5d4PYC/qSrx5cknbplcaJZkjQaRkGS1BgFSVJjFCRJjVGQJDVGQZLUGAVJUmMUJEmNUZAkNUZBktQYBUlSYxQkSY1RkCQ1RkGS1BgFSVJjFCRJjVGQJDVGQZLUGAVJUmMUJEmNUZAkNUZBktQYBUlSYxQkSY1RkCQ1RkGS1BgFSVJjFCRJjVGQJDWdRiHJ5iT3JZlJcsMJbl+f5LYkdyW5O8nVXc4jSZpfZ1FIsgrYCVwFbAKuTbJpzrK/BnZX1SuAa4B/6moeSdLCujxSuBSYqaqDVTUL3AJsnbOmgOf3L78A+FGH80iSFrC6w/s+Hzg0cP0w8Ko5az4I/EeSdwHPBa7scB5J0gLGfaL5WuCmqloLXA3cnORpMyXZlmQ6yfSxY8dGPqQknSm6jMIRYN3A9bX9bYOuA3YDVNW3gXOANXPvqKp2VdVkVU1OTEx0NK4kqcso7Ac2Jrkgydn0TiRPzVnzQ+AKgCQvpRcFDwUkaUw6i0JVPQ5cD+wF7qX3XUYHkmxPsqW/7D3A25N8B/gc8Naqqq5mkiTNr8sTzVTVHmDPnG3vH7h8D/DqLmeQJA1v3CeaJUmnEaMgSWqMgiSpMQqSpMYoSJIaoyBJaoyCJKkxCpKkxihIkhqjIElqjIIkqTEKkqTGKEiSGqMgSWqMgiSpMQqSpMYoSJIaoyBJaoyCJKkxCpKkxihIkhqjIElqjIIkqTEKkqTGKEiSGqMgSWqMgiSpMQqSpMYoSJKaTqOQZHOS+5LMJLnhJGvekOSeJAeSfLbLeSRJ81vd1R0nWQXsBP4AOAzsTzJVVfcMrNkI/CXw6qp6OMmLuppHkrSwLo8ULgVmqupgVc0CtwBb56x5O7Czqh4GqKoHO5xHkrSALqNwPnBo4Prh/rZBFwEXJflmkn1JNnc4jyRpAZ29fHQKj78RuBxYC9ye5GVV9dPBRUm2AdsA1q9fP+oZJemM0eWRwhFg3cD1tf1tgw4DU1X1WFV9H/gevUg8RVXtqqrJqpqcmJjobGBJOtN1GYX9wMYkFyQ5G7gGmJqz5kv0jhJIsobey0kHO5xJkjSPzqJQVY8D1wN7gXuB3VV1IMn2JFv6y/YCDyW5B7gNeG9VPdTVTJKk+aWqxj3DKZmcnKzp6elxjyFJy0qSO6tqcqF1/kSzJKkxCpKkxihIkhqjIElqjIIkqTEKkqTGKEiSGqMgSWqMgiSpMQqSpMYoSJKaeaOQ5PlJLjzB9pd3N5IkaVxOGoUkbwD+G/i3JAeSvHLg5pu6HkySNHrzHSn8FfDbVfVbwNuAm5P8Uf+2dD6ZJGnk5vs4zlVVdRSgqv4zyWuBLydZByyv99uWJA1lviOFnw2eT+gH4nJgK/CbHc8lSRqD+aLwDuCsJJuOb6iqnwGbgT/uejBJ0uidNApV9Z2q+h9gd5K/SM9zgI8A7xzZhJKkkRnm5xReBawDvgXsB34EvLrLoSRJ4zFMFB4D/g94DnAO8P2qerLTqSRJYzFMFPbTi8Irgd8Frk3yhU6nkiSNxXzfknrcdVU13b98FNia5E0dziRJGpMFjxQGgjC47eZuxpEkjZNviCdJaoyCJKkxCpKkxihIkhqjIElqjIIkqTEKkqSm0ygk2ZzkviQzSW6YZ93rklSSyS7nkSTNr7MoJFkF7ASuAjbRe3uMTSdYdy7wZ8AdXc0iSRpOl0cKlwIzVXWwqmaBW+h9QM9cO4APAT/vcBZJ0hC6jML5wKGB64f725oklwDrquor891Rkm1JppNMHzt2bOknlSQBYzzRnOQseh/Y856F1lbVrqqarKrJiYmJ7oeTpDNUl1E4Qu/DeY5b29923LnAxcA3ktwPXAZMebJZksanyyjsBzYmuSDJ2cA1wNTxG6vqkapaU1UbqmoDsA/YcqJ3ZZUkjUZnUaiqx4Hrgb3AvcDuqjqQZHuSLV09riRp8Yb5kJ1Fq6o9wJ45295/krWXdzmLJGlh/kSzJKkxCpKkxihIkhqjIElqjIIkqTEKkqTGKEiSGqMgSWqMgiSpMQqSpMYoSJIaoyBJaoyCJKkxCpKkxihIkhqjIElqjIIkqTEKkqTGKEiSGqMgSWqMgiSpMQqSpMYoSJIaoyBJaoyCJKkxCpKkxihIkhqjIElqjIIkqek0Ckk2J7kvyUySG05w+7uT3JPk7iRfS/KSLueRJM2vsygkWQXsBK4CNgHXJtk0Z9ldwGRVvRz4IvD3Xc0jSVpYl0cKlwIzVXWwqmaBW4Ctgwuq6raqerR/dR+wtsN5JEkL6DIK5wOHBq4f7m87meuAr3Y4jyRpAavHPQBAkjcCk8BrTnL7NmAbwPr160c4mSSdWbo8UjgCrBu4vra/7SmSXAm8D9hSVb840R1V1a6qmqyqyYmJiU6GlSR1G4X9wMYkFyQ5G7gGmBpckOQVwMfpBeHBDmeRJA2hsyhU1ePA9cBe4F5gd1UdSLI9yZb+sg8DzwO+kOS/kkyd5O4kSSPQ6TmFqtoD7Jmz7f0Dl6/s8vElSafGn2iWJDVGQZLUGAVJUmMUJEmNUZAkNUZBktQYBUlSYxQkSY1RkCQ1RkGS1BgFSVJjFCRJjVGQJDVGQZLUGAVJUmMUJEmNUZAkNUZBktQYBUlSYxQkSY1RkCQ1RkGS1BgFSVJjFCRJjVGQJDVGQZLUGAVJUmMUJEmNUZAkNZ1GIcnmJPclmUlywwluf3aSz/dvvyPJhi7nkSTNr7MoJFkF7ASuAjYB1ybZNGfZdcDDVfXrwD8CH+pqnqc4ehQuvBAeeGAkDydJizGOp6oujxQuBWaq6mBVzQK3AFvnrNkKfKp/+YvAFUnS4Uw9O3bA/ff3fpek09Q4nqq6jML5wKGB64f72064pqoeBx4Bzutwpl56b7wRnnyy97tHC5JOQ+N6qloWJ5qTbEsynWT62LFjz+zOduzo/SkDPPGERwuSTkvjeqrqMgpHgHUD19f2t51wTZLVwAuAh+beUVXtqqrJqpqcmJhY/ETH0zs727s+O+vRgqTTzjifqrqMwn5gY5ILkpwNXANMzVkzBbylf/n1wNerqjqbaDC9x3m0IOk0M86nqs6i0D9HcD2wF7gX2F1VB5JsT7Klv+wTwHlJZoB3A0/7ttUlNTX1y/QeNzsLt97a6cNK0qkY51NVuvyPeRcmJydrenp63GNI0rKS5M6qmlxo3bI40SxJGg2jIElqjIIkqTEKkqTGKEiSGqMgSWqMgiSpMQqSpGbZ/fBakmPAD5bgrtYAP16C+1ku3N+V60zaV3B/F+slVbXgm8ctuygslSTTw/x030rh/q5cZ9K+gvvbNV8+kiQ1RkGS1JzJUdg17gFGzP1duc6kfQX3t1Nn7DkFSdLTnclHCpKkOVZ8FJJsTnJfkpkkT/sQnyTPTvL5/u13JNkw+imXzhD7++4k9yS5O8nXkrxkHHMuhYX2dWDd65JUkmX9HSvD7G+SN/S/vgeSfHbUMy6lIf4ur09yW5K7+n+frx7HnEshySeTPJjkuye5PUk+2v+zuDvJJZ0NU1Ur9hewCvhf4NeAs4HvAJvmrHkn8LH+5WuAz4977o7397XAr/Qvv2O57u8w+9pfdy5wO7APmBz33B1/bTcCdwG/2r/+onHP3fH+7gLe0b+8Cbh/3HM/g/39PeAS4Lsnuf1q4KtAgMuAO7qaZaUfKVwKzFTVwaqaBW4Bts5ZsxX4VP/yF4ErkmSEMy6lBfe3qm6rqkf7V/cBa0c841IZ5msLsAP4EPDzUQ7XgWH29+3Azqp6GKCqHhzxjEtpmP0t4Pn9yy8AfjTC+ZZUVd0O/GSeJVuBT1fPPuCFSV7cxSwrPQrnA4cGrh/ubzvhmup9rvQjwHkjmW7pDbO/g66j97+P5WjBfe0fYq+rqq+McrCODPO1vQi4KMk3k+xLsnlk0y29Yfb3g8AbkxwG9gDvGs1oY3Gq/7YXbXUXd6rTX5I3ApPAa8Y9SxeSnAV8BHjrmEcZpdX0XkK6nN4R4O1JXlZVPx3rVN25Fripqv4hye8ANye5uKqeHPdgy9lKP1I4AqwbuL62v+2Ea5KspncY+tBIplt6w+wvSa4E3gdsqapfjGi2pbbQvp4LXAx8I8n99F6HnVrGJ5uH+doeBqaq6rGq+j7wPXqRWI6G2d/rgN0AVfVt4Bx67xO0Eg31b3sprPQo7Ac2Jrkgydn0TiRPzVkzBbylf/n1wNerf2ZnGVpwf5O8Avg4vSAs59ec593XqnqkqtZU1Yaq2kDv/MmWqpoez7jP2DB/l79E7yiBJGvovZx0cJRDLqFh9veHwBUASV5KLwrHRjrl6EwBb+5/F9JlwCNVdbSLB1rRLx9V1eNJrgf20vtuhk9W1YEk24HpqpoCPkHvsHOG3omea8Y38TMz5P5+GHge8IX++fQfVtWWsQ29SEPu64ox5P7uBf4wyT3AE8B7q2pZHvUOub/vAf4lyZ/TO+n81uX6H7okn6MX9DX9cyQfAJ4FUFUfo3fO5GpgBngUeFtnsyzTP0NJUgdW+stHkqRTYBQkSY1RkCQ1RkGS1BgFSVJjFKQllOTfk/w0yZfHPYu0GEZBWlofBt407iGkxTIK0iIkeWX/fe3PSfLc/ucXXFxVXwN+Nu75pMVa0T/RLHWlqvYnmQL+DngO8JmqOuEHpEjLiVGQFm87vffo+Tnwp2OeRVoSvnwkLd559N5H6lx6b8YmLXtGQVq8jwN/A/wrvU93k5Y9Xz6SFiHJm4HHquqzSVYB30ry+8DfAr8BPK//bpfXVdXecc4qnQrfJVWS1PjykSSpMQqSpMYoSJIaoyBJaoyCJKkxCpKkxihIkhqjIElq/h+CL7kPwpocagAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_data = [[0, 0],\n",
    "          [0, 1],\n",
    "          [1, 0],\n",
    "          [1, 1]]\n",
    "y_data = [[0],\n",
    "          [1],\n",
    "          [1],\n",
    "          [0]]\n",
    "\n",
    "plt.scatter(x_data[0][0],x_data[0][1], c='red' , marker='^')\n",
    "plt.scatter(x_data[3][0],x_data[3][1], c='red' , marker='^')\n",
    "plt.scatter(x_data[1][0],x_data[1][1], c='blue' , marker='^')\n",
    "plt.scatter(x_data[2][0],x_data[2][1], c='blue' , marker='^')\n",
    "\n",
    "plt.xlabel(\"x1\")\n",
    "plt.ylabel(\"x2\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  Tensorboard\n",
    "### 위 Data를 기준으로 XOR처리를 위한 모델을 만들도록 하겠습니다\n",
    "* 위의 Data를 4Layer의 Neural Network를 통해 학습시킨 후 모델을 생성합니다.\n",
    "* tensorboard --logdir=./logs/ 실행합니다.\n",
    "* summary 값을 logs폴더에 저장하고 아래 명령어로 실행해서 확인한다 (http://0.0.0.0:6006)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = tf.data.Dataset.from_tensor_slices((x_data, y_data)).batch(len(x_data))\n",
    "\n",
    "def preprocess_data(features, labels):\n",
    "    features = tf.cast(features, tf.float32)\n",
    "    labels = tf.cast(labels, tf.float32)\n",
    "    return features, labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Deep Neural Network를 통해 XOR해결 \n",
    "* 위의 Data를 4Layer의 Neural Network를 통해 학습시킨 후 모델을 생성합니다. \n",
    "* 각각의 값을 histogram으로 tensorboard에 저장한다 (Model)\n",
    "* 각각의 값을 scalar값으로 tensorboard에 저장한다 (cost, accuracy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "W1 = tf.Variable(tf.random_normal([2, 10]), name='weight1')\n",
    "b1 = tf.Variable(tf.random_normal([10]), name='bias1')\n",
    "\n",
    "W2 = tf.Variable(tf.random_normal([10, 10]), name='weight2')\n",
    "b2 = tf.Variable(tf.random_normal([10]), name='bias2')\n",
    "\n",
    "W3 = tf.Variable(tf.random_normal([10, 10]), name='weight3')\n",
    "b3 = tf.Variable(tf.random_normal([10]), name='bias3')\n",
    "\n",
    "W4 = tf.Variable(tf.random_normal([10, 1]), name='weight4')\n",
    "b4 = tf.Variable(tf.random_normal([1]), name='bias4')\n",
    "\n",
    "def neural_net(features):\n",
    "    layer1 = tf.sigmoid(tf.matmul(features, W1) + b1)\n",
    "    layer2 = tf.sigmoid(tf.matmul(layer1, W2) + b2)\n",
    "    layer3 = tf.sigmoid(tf.matmul(layer2, W3) + b3)\n",
    "    hypothesis = tf.sigmoid(tf.matmul(layer3, W4) + b4)\n",
    "\n",
    "    with tf.contrib.summary.record_summaries_every_n_global_steps(1):\n",
    "        tf.contrib.summary.histogram(\"weights1\", W1)\n",
    "        tf.contrib.summary.histogram(\"biases1\", b1)\n",
    "        tf.contrib.summary.histogram(\"layer1\", layer1)\n",
    "\n",
    "        tf.contrib.summary.histogram(\"weights2\", W2)\n",
    "        tf.contrib.summary.histogram(\"biases2\", b2)\n",
    "        tf.contrib.summary.histogram(\"layer2\", layer2)\n",
    "\n",
    "        tf.contrib.summary.histogram(\"weights3\", W3)\n",
    "        tf.contrib.summary.histogram(\"biases3\", b3)\n",
    "        tf.contrib.summary.histogram(\"layer3\", layer3)\n",
    "\n",
    "        tf.contrib.summary.histogram(\"weights4\", W4)\n",
    "        tf.contrib.summary.histogram(\"biases4\", b4)\n",
    "        tf.contrib.summary.histogram(\"hypothesis\", hypothesis)\n",
    "    return hypothesis\n",
    "\n",
    "def loss_fn(hypothesis, labels):\n",
    "    cost = -tf.reduce_mean(labels * tf.log(hypothesis) + (1 - labels) * tf.log(1 - hypothesis))\n",
    "    with tf.contrib.summary.record_summaries_every_n_global_steps(1):\n",
    "        tf.contrib.summary.scalar('loss', cost)\n",
    "    return cost\n",
    "\n",
    "optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.1)\n",
    "\n",
    "def accuracy_fn(hypothesis, labels):\n",
    "    predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)\n",
    "    accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, labels), dtype=tf.float32))\n",
    "    return accuracy\n",
    "\n",
    "def grad(hypothesis, features, labels):\n",
    "    with tf.GradientTape() as tape:\n",
    "        loss_value = loss_fn(neural_net(features),labels)\n",
    "    return tape.gradient(loss_value, [W1, W2, W3, W4, b1, b2, b3, b4])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* summary 값을 logs폴더에 저장하고 아래 명령어로 실행해서 확인한다 (http://0.0.0.0:6006)\n",
    "* tensorboard --logdir=./logs/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iter: 0, Loss: 0.9517\n",
      "Iter: 50, Loss: 0.6936\n",
      "Iter: 100, Loss: 0.6923\n",
      "Iter: 150, Loss: 0.6912\n",
      "Iter: 200, Loss: 0.6901\n",
      "Iter: 250, Loss: 0.6890\n",
      "Iter: 300, Loss: 0.6879\n",
      "Iter: 350, Loss: 0.6867\n",
      "Iter: 400, Loss: 0.6855\n",
      "Iter: 450, Loss: 0.6842\n",
      "Iter: 500, Loss: 0.6827\n",
      "Iter: 550, Loss: 0.6811\n",
      "Iter: 600, Loss: 0.6793\n",
      "Iter: 650, Loss: 0.6772\n",
      "Iter: 700, Loss: 0.6749\n",
      "Iter: 750, Loss: 0.6722\n",
      "Iter: 800, Loss: 0.6690\n",
      "Iter: 850, Loss: 0.6654\n",
      "Iter: 900, Loss: 0.6611\n",
      "Iter: 950, Loss: 0.6561\n",
      "Iter: 1000, Loss: 0.6502\n",
      "Iter: 1050, Loss: 0.6432\n",
      "Iter: 1100, Loss: 0.6349\n",
      "Iter: 1150, Loss: 0.6251\n",
      "Iter: 1200, Loss: 0.6134\n",
      "Iter: 1250, Loss: 0.5992\n",
      "Iter: 1300, Loss: 0.5821\n",
      "Iter: 1350, Loss: 0.5612\n",
      "Iter: 1400, Loss: 0.5357\n",
      "Iter: 1450, Loss: 0.5047\n",
      "Iter: 1500, Loss: 0.4677\n",
      "Iter: 1550, Loss: 0.4248\n",
      "Iter: 1600, Loss: 0.3774\n",
      "Iter: 1650, Loss: 0.3281\n",
      "Iter: 1700, Loss: 0.2798\n",
      "Iter: 1750, Loss: 0.2356\n",
      "Iter: 1800, Loss: 0.1971\n",
      "Iter: 1850, Loss: 0.1649\n",
      "Iter: 1900, Loss: 0.1385\n",
      "Iter: 1950, Loss: 0.1173\n",
      "Iter: 2000, Loss: 0.1003\n",
      "Iter: 2050, Loss: 0.0865\n",
      "Iter: 2100, Loss: 0.0754\n",
      "Iter: 2150, Loss: 0.0663\n",
      "Iter: 2200, Loss: 0.0588\n",
      "Iter: 2250, Loss: 0.0526\n",
      "Iter: 2300, Loss: 0.0474\n",
      "Iter: 2350, Loss: 0.0429\n",
      "Iter: 2400, Loss: 0.0392\n",
      "Iter: 2450, Loss: 0.0359\n",
      "Iter: 2500, Loss: 0.0331\n",
      "Iter: 2550, Loss: 0.0306\n",
      "Iter: 2600, Loss: 0.0285\n",
      "Iter: 2650, Loss: 0.0265\n",
      "Iter: 2700, Loss: 0.0248\n",
      "Iter: 2750, Loss: 0.0233\n",
      "Iter: 2800, Loss: 0.0220\n",
      "Iter: 2850, Loss: 0.0207\n",
      "Iter: 2900, Loss: 0.0196\n",
      "Iter: 2950, Loss: 0.0186\n",
      "Testset Accuracy: 1.0000\n"
     ]
    }
   ],
   "source": [
    "EPOCHS = 3000\n",
    "log_path = \"./logs/xor_eager_logs_r0_01\"\n",
    "writer = tf.contrib.summary.create_file_writer(log_path)\n",
    "global_step=tf.train.get_or_create_global_step()  # global step variable\n",
    "writer.set_as_default()\n",
    "\n",
    "for step in range(EPOCHS):\n",
    "    global_step.assign_add(1)\n",
    "    for features, labels  in tfe.Iterator(dataset):\n",
    "        features, labels = preprocess_data(features, labels)\n",
    "        grads = grad(neural_net(features), features, labels)\n",
    "        optimizer.apply_gradients(grads_and_vars=zip(grads,[W1, W2, W3, W4, b1, b2, b3, b4]))\n",
    "        if step % 50 == 0:\n",
    "            loss_value = loss_fn(neural_net(features),labels)\n",
    "            print(\"Iter: {}, Loss: {:.4f}\".format(step, loss_value))\n",
    "x_data, y_data = preprocess_data(x_data, y_data)\n",
    "test_acc = accuracy_fn(neural_net(x_data),y_data)\n",
    "print(\"Testset Accuracy: {:.4f}\".format(test_acc))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}