Activity_C_Dense_Neural_Networks.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "879ff612",
   "metadata": {},
   "source": [
    "# Machine Learning (and Deep Learning) in just over 200 minutes with R\n",
    "Created by [Ajay Hemanth](https://www.linkedin.com/in/ajayhemanth/?originalSubdomain=au) with [Yoni Nazarathy](https://yoninazarathy.com/). \n",
    "\n",
    "See more material at the [workshop's GitHub page](https://github.com/ajayhemanth/Machine-Learning-Workshop).\n",
    "\n",
    "--- \n",
    "# Activity C: Dense Neural Networks\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "260c6f2d",
   "metadata": {},
   "source": [
    "# Dense Neural Networks (aka as Multi-Layer Perceptrons)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3e19119",
   "metadata": {},
   "source": [
    "## Classification"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "693dbc1d",
   "metadata": {},
   "source": [
    "### Dataset\n",
    "We will be using the [Covertype dataset](https://archive.ics.uci.edu/ml/datasets/covertype) available in the dataset folder to work with MLP.\n",
    "<br>With this dataset we are supposed to predict `Forest Cover type` using the following features.\n",
    "\n",
    "<img src=\"img/Dataset.png\" />\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b73ed5fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Make + as string concat operator as well.. Dont worry about this cell...\n",
    "\"+\" = function(x,y) { if(is.character(x) || is.character(y)) return(paste(x , y, sep=\"\")) else .Primitive(\"+\")(x,y) }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a5eed64e",
   "metadata": {},
   "outputs": [],
   "source": [
    "library(\"keras\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67a1951d",
   "metadata": {},
   "outputs": [],
   "source": [
    "d1 = read.csv(\"dataset/covtype.data\", header=FALSE, stringsAsFactors=FALSE)\n",
    "colnames(d1) = c('Elevation', 'Aspect', 'Slope', 'HorHydro', 'VertHydro', 'HorRoad', 'Shade9', 'Shade12', 'Shade15', 'HorFire', 'WildernessArea'+(1:4), 'SoilType'+(1:40), 'CoverType')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d2bfb2de",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data  =  d1[  , -ncol(d1) ]\n",
    "train_labels = d1[  ,  ncol(d1) ]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be972a0d",
   "metadata": {},
   "source": [
    "#### Input Data Format\n",
    "Keras requires the input data to be a matrix with features as columns and observations as rows.\n",
    "<br>\n",
    "The response variable is required to be a binary matrix with each class in a seraparate column, which is achieved using the `to_categorical` function of Keras. `to_categorical` requires the labels to start from `0`. But since in our dataset the labels start with `1` we, subtract labels values by 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5051422e",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data = as.matrix(train_data)\n",
    "train_labels = to_categorical(train_labels-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19ac313a",
   "metadata": {},
   "source": [
    "<img src=\"img/Network.png\" width=\"500\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca89b7a9",
   "metadata": {},
   "source": [
    "#### Linear Stack of Layers\n",
    "A Keras model is composed of layers of compute units, the most common of which is a Linear Stack of Layers.\n",
    "<br> We define a Linear stack using \"keras_model_sequential\" functions.\n",
    "<br> We then add layer by layer to the stack"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c76294ed",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = keras_model_sequential() "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31d3b724",
   "metadata": {},
   "source": [
    "#### Input Layer\n",
    "The first layer of MLP is the input layer which receives the input data.\n",
    "<br> The total number of nodes in this layer is equal to the total number of input features."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a7e0782",
   "metadata": {},
   "source": [
    "#### Hidden Layer\n",
    "A hidden layer is where the computation takes place. There are many parameters in hidden layers which changes the performance of the model, few of which are `activation function`, `number of hidden layers in a nerwork`, `number of hidden units in a layer`. The code below implements an example with 1 hidden layer having 200 units, with `relu` activation function for each layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9d9f424a",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = model  %>% layer_dense(\n",
    "    input_shape = ncol(train_data), \n",
    "    units = 200, \n",
    "    activation = 'relu',\n",
    "    name=\"layer1\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dca5963c",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = model  %>% layer_dense(\n",
    "    units = 200, \n",
    "    activation = 'relu',\n",
    "    name=\"layer2\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90a21192",
   "metadata": {},
   "source": [
    "#### Output Layer\n",
    "\n",
    "The output layer is the last layer of the network, which gives the prediction.\n",
    "<br> The nature of the `activation function` tells if the prediction is classification or regression. For classification there are a couple of options, with `softmax` being one of them for multi-class classification. \n",
    "<br> Total number of units in output class should be equal to the total number of classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e4c62fa8",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = model  %>% layer_dense(units = ncol(train_labels), activation = 'softmax', name=\"layer3\")\n",
    "model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1678a8b",
   "metadata": {},
   "source": [
    "#### Calculating total number of model parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb9c7486",
   "metadata": {},
   "source": [
    "Printing `model` shows the following. Below is the calculation for total number of parameters for each layer.<br>\n",
    "- Since the 1st layer (input layer) is connected to input, total number of parameter = number of input columns(54) X number of units in 1st layer (200) + 200 bias (1 for each node) = 11,000 parameters <br>\n",
    "- Since the 2nd layer (hidden layer) is connected to input layer, total number of parameter = number of units in 1st layer (200) X number of units in 2nd layer (200) + 200 bias (1 for each node) = 40,200 parameters <br>\n",
    "- Since the 3rd layer (output layer) is connected to input layer, total number of parameter = number of units in 2nd layer (200) X number of units in 3rd layer (7) + 7 bias (1 for each node) = 1,407 parameters <br>\n",
    "- The total number of parameters is the sum of all the parameters in each layer -- provided if a layer weights are set to non-trainable\n",
    "\n",
    "<br>\n",
    "<div>\n",
    "<img src=\"img/Model.PNG\" width=\"500\"/>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a1a849e",
   "metadata": {},
   "source": [
    "#### Configure Loss Function, Optimizer Algorithm, Error Metric using `compile` function\n",
    "\n",
    "- Loss Functions - https://keras.io/api/losses/\n",
    "- Optimizer Algorithms - https://keras.io/api/optimizers/\n",
    "- Error Metrics - https://keras.io/api/metrics/ \n",
    "\n",
    "compile R function - https://keras.rstudio.com/reference/compile.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6bf7b30c",
   "metadata": {},
   "outputs": [],
   "source": [
    "model %>% compile(\n",
    "    loss = 'categorical_crossentropy',\n",
    "    optimizer = optimizer_adam(),\n",
    "    metrics = 'accuracy'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5125428",
   "metadata": {},
   "source": [
    "#### Fit Model to Training Data\n",
    "\n",
    "When fitting the model you get to choose \n",
    "- batch size\n",
    "- epochs\n",
    "- validation split\n",
    "- class weight\n",
    "- ... \n",
    "\n",
    "The arguments of the `fit` function are in\n",
    "https://keras.rstudio.com/reference/fit.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e722ba7c",
   "metadata": {},
   "outputs": [],
   "source": [
    "history <- fit(\n",
    "    object = model,\n",
    "    x = train_data,\n",
    "    y = train_labels,\n",
    "    epochs = 25,\n",
    "    validation_split = 0.2\n",
    ")\n",
    "\n",
    "plot(history)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f966f860",
   "metadata": {},
   "source": [
    "#### Output of De-Normalized Data\n",
    "\n",
    "Below is the output of the denormalized data. Ideally MLP requires the data to be normalized. If the data is not normalized then it will take many epochs for the model to converge to reasonably optimum weights, which will require more compute capacity. Also if the scale between the features are vastly different, chances are that it might get held up in a local minima far from the global minima.\n",
    "\n",
    "<img src=\"img/denorm.png\" width=\"400\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03c5adb6",
   "metadata": {},
   "source": [
    "#### Predicting New Data using the Built Model\n",
    "The built model can be used to predict for new data points using the `predict` function. The test data should have the same structure of the train data. For demo purposes, the test data is taken to be the first two rows of the train data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eb23d087",
   "metadata": {},
   "outputs": [],
   "source": [
    "test_data = train_data[1:2,]\n",
    "predict(model, test_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "636bb0b4",
   "metadata": {},
   "source": [
    "The prediction output is the probability of the new data points belonging to each class, and we classify the data into the class with maximum probability. <br>\n",
    "<img src=\"img/PredictKeras.PNG\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "525f6628",
   "metadata": {},
   "source": [
    "#### Performance of MLP with Normalized Input Data\n",
    "\n",
    "On check the summary of the training data, it can be observed that the 1st ten columns' scale is very large."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2b5f5660",
   "metadata": {},
   "outputs": [],
   "source": [
    "summary(train_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "431f640d",
   "metadata": {},
   "source": [
    "One way of normalizing is by z-score -> ($x-\\mu) / \\sigma$ <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a8da3804",
   "metadata": {},
   "outputs": [],
   "source": [
    "d2 = d1 \n",
    "\n",
    "means1 = sapply(d1[,1:10], mean)\n",
    "sd1 = sapply(d1[,1:10], sd)\n",
    "\n",
    "d2[ , 1:10 ] = t((t(d1[ , 1:10 ]) - means1)  / sd1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5b6e475",
   "metadata": {},
   "source": [
    "Rest of the steps follow as before"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "15883405",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_data  =  d2[  , -ncol(d2) ]\n",
    "train_labels = d2[  ,  ncol(d2) ]\n",
    "\n",
    "train_data = as.matrix(train_data)\n",
    "train_labels = to_categorical(train_labels-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f3ad7a32",
   "metadata": {},
   "outputs": [],
   "source": [
    "model <- keras_model_sequential()  %>% \n",
    "\n",
    "layer_dense(\n",
    "    input_shape = ncol(train_data), \n",
    "    units = 200, \n",
    "    activation = 'relu'\n",
    ") %>% \n",
    "\n",
    "layer_dense(\n",
    "    units = 200, \n",
    "    activation = 'relu'\n",
    ") %>% \n",
    "\n",
    "layer_dense(units = ncol(train_labels), activation = 'softmax')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e5119c5a",
   "metadata": {},
   "outputs": [],
   "source": [
    "model %>% compile(\n",
    "    loss = 'categorical_crossentropy',\n",
    "    optimizer = optimizer_adam(),\n",
    "    metrics = 'accuracy'\n",
    ")\n",
    "\n",
    "history <- fit(\n",
    "    object = model,\n",
    "    x = train_data,\n",
    "    y = train_labels,\n",
    "    epochs = 25,\n",
    "    validation_split = 0.2\n",
    ")\n",
    "\n",
    "plot(history)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01fd4229",
   "metadata": {},
   "source": [
    "#### Output of Normalized Data\n",
    "\n",
    "It can be observed that the model converges faster to optimum than de-normalized data.\n",
    "There are many methods of normalization. [This article](https://developers.google.com/machine-learning/data-prep/transform/normalization) provides a good explanation of the available methods.\n",
    "\n",
    "<div>\n",
    "<img src=\"img/norm.png\" width=\"400\" />\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "876045e9",
   "metadata": {},
   "source": [
    "#### More Hidden Layers\n",
    "\n",
    "Deeper neural networks can help predict complex patterns. The code below works with the same models parameters of the previous except for an additional hidden layer of 200 units."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49f641e6",
   "metadata": {},
   "outputs": [],
   "source": [
    "model <- keras_model_sequential()  %>% \n",
    "\n",
    "layer_dense(\n",
    "    input_shape = ncol(train_data), \n",
    "    units = 200, \n",
    "    activation = 'relu'\n",
    ") %>% \n",
    "\n",
    "############ added layer ############\n",
    "\n",
    "layer_dense(\n",
    "    units = 200, \n",
    "    activation = 'relu'\n",
    ") %>% \n",
    "\n",
    "############ added layer ############\n",
    "\n",
    "layer_dense(\n",
    "    units = 200, \n",
    "    activation = 'relu'\n",
    ") %>% \n",
    "\n",
    "layer_dense(units = ncol(train_labels), activation = 'softmax')\n",
    "\n",
    "model %>% compile(\n",
    "    loss = 'categorical_crossentropy',\n",
    "    optimizer = optimizer_adam(),\n",
    "    metrics = 'accuracy'\n",
    ")\n",
    "\n",
    "\n",
    "\n",
    "history <- fit(\n",
    "    object = model,\n",
    "    x = train_data,\n",
    "    y = train_labels,\n",
    "    epochs = 25,\n",
    "    validation_split = 0.2\n",
    ")\n",
    "\n",
    "plot(history)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6caf49d",
   "metadata": {},
   "source": [
    "The accuracy of test and validation set has improved compared to model with 1 hidden layer.\n",
    "Note that the performance of the model is not proportional to the number of hidden layers / units in each hidden unit. \n",
    "<img src=\"img/TwoHidden.png\" width=\"400\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf7bb1c7",
   "metadata": {},
   "source": [
    "#### Batch Size\n",
    "Batch size defines the number of instances after which the weights are updated.\n",
    "<br>If the batch size of 1 then after each instance, the weight is updated. If the batch size is equal to size of dataset then the weights are updated once for each epoch.\n",
    "<br>The default batch size is 32.\n",
    "<br> Higher the batch size, faster is the execution with GPUs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6909e166",
   "metadata": {},
   "outputs": [],
   "source": [
    "############ same as before ############\n",
    "model <- keras_model_sequential()  %>% \n",
    "\n",
    "layer_dense(\n",
    "    input_shape = ncol(train_data), \n",
    "    units = 200, \n",
    "    activation = 'relu'\n",
    ") %>% \n",
    "\n",
    "layer_dense(\n",
    "    units = 200, \n",
    "    activation = 'relu'\n",
    ") %>% \n",
    "\n",
    "layer_dense(units = ncol(train_labels), activation = 'softmax')\n",
    "\n",
    "############ same as before ############\n",
    "\n",
    "model %>% compile(\n",
    "    loss = 'categorical_crossentropy',\n",
    "    optimizer = optimizer_adam(),\n",
    "    metrics = 'accuracy'\n",
    ")\n",
    "\n",
    "\n",
    "############ same as before ############\n",
    "\n",
    "history <- fit(\n",
    "    object = model,\n",
    "    x = train_data,\n",
    "    y = train_labels,\n",
    "    epochs = 25,\n",
    "    validation_split = 0.2,\n",
    "    batch_size=nrow(train_data) # set the batch size to size of dataset\n",
    ")\n",
    "\n",
    "plot(history)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f15123d5",
   "metadata": {},
   "source": [
    "Result shows that large batch size takes more epoch to converge. Ideally with smaller batch size there would be greater variation in error across each epoch, however it would reach higher accuracy/performance with fewer epochs, on the other hand if the batch size is very large then the variation of model performance across epochs is small after convergence, but it takes longer to converge.\n",
    "\n",
    "<div>\n",
    "<img src=\"img/BatchSizeMax.png\" width=\"400\"/>\n",
    "</div>\n",
    "\n",
    "[This link](https://machinelearningmastery.com/how-to-control-the-speed-and-stability-of-training-neural-networks-with-gradient-descent-batch-size/) has good information on the pros and cons of varying batch size, from where the below plot has been taken from showing the performance of model on mnist data for varying batch size.\n",
    "\n",
    "<div>\n",
    "<img src=\"img/BatchSizeWebSite.png\" width=\"400\"/>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85ec9614",
   "metadata": {},
   "source": [
    "#### Learning Rate\n",
    "Batch Size and learning rate are usually tuned together. Smaller learning rate takes more epochs to converge, whereas bigger learning rate can cause model to jump around the optimal point. Since large batch size takes more number of epochs to reach close to convergence, it is a usual practice that learning rate is increased with batch size. However, this can depend on specific requirement. <br>\n",
    "Below is shown a model with learning rate 100 times less than the default."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fdf78ac1",
   "metadata": {},
   "outputs": [],
   "source": [
    "############ same as before ############\n",
    "model <- keras_model_sequential()  %>% \n",
    "\n",
    "layer_dense(\n",
    "    input_shape = ncol(train_data), \n",
    "    units = 200, \n",
    "    activation = 'relu'\n",
    ") %>% \n",
    "\n",
    "layer_dense(\n",
    "    units = 200, \n",
    "    activation = 'relu'\n",
    ") %>% \n",
    "\n",
    "layer_dense(units = ncol(train_labels), activation = 'softmax')\n",
    "\n",
    "\n",
    "############ same as before ############\n",
    "\n",
    "\n",
    "model %>% compile(\n",
    "    loss = 'categorical_crossentropy',\n",
    "    optimizer = optimizer_adam(lr=0.00001),\n",
    "    metrics = 'accuracy'\n",
    ")\n",
    "\n",
    "\n",
    "\n",
    "history <- fit(\n",
    "    object = model,\n",
    "    x = train_data,\n",
    "    y = train_labels,\n",
    "    epochs = 25,\n",
    "    validation_split = 0.2,\n",
    "    batch_size=nrow(train_data)\n",
    ")\n",
    "\n",
    "plot(history)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60b5599e",
   "metadata": {},
   "source": [
    "With learning rate 100 times less than defauly it can be observed that even after 25 epochs, the model doesn't seems to be anywhere close to the optimum because the steps taken by weights across each epoch is very small.\n",
    "\n",
    "<img src=\"img/LearningRate.png\" width=\"500\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82cff366",
   "metadata": {},
   "source": [
    "## Regression\n",
    "\n",
    "For regression, the main changes in parameters include\n",
    "- activation function of the output layer changes (Eg. linear)\n",
    "- loss function (Eg. mean square error)\n",
    "- error metric (Eg. mean absolute error)\n",
    "\n",
    "For demo, lets predict Elevation using all the other features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d4ddadb0",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "d2 = d1 \n",
    "\n",
    "means1 = sapply(d1[,1:10], mean)\n",
    "sd1 = sapply(d1[,1:10], sd)\n",
    "\n",
    "d2[ , 1:10 ] = t((t(d1[ , 1:10 ]) - means1)  / sd1)\n",
    "\n",
    "# Converting Cover Type into Binary Matrix\n",
    "d2 = cbind(\n",
    "                    d2[  , -ncol(d2) ],  \n",
    "    to_categorical( d2[  ,  ncol(d2) ] - 1 )\n",
    ")\n",
    "\n",
    "\n",
    "train_data  =  d2[  , -1 ]\n",
    "train_labels = d2[  ,  1 ]\n",
    "\n",
    "train_data = as.matrix(train_data)\n",
    "train_labels = as.matrix(train_labels)\n",
    "\n",
    "\n",
    "\n",
    "# Changing Output Activation to Linear\n",
    "\n",
    "model <- keras_model_sequential()  %>% \n",
    "\n",
    "layer_dense(\n",
    "    input_shape = ncol(train_data), \n",
    "    units = 200, \n",
    "    activation = 'relu',\n",
    "    name=\"layer1\"\n",
    ") %>% \n",
    "\n",
    "layer_dense(\n",
    "    units = 200, \n",
    "    activation = 'relu',\n",
    "    name=\"layer2\"\n",
    ") %>% \n",
    "\n",
    "layer_dense(units = ncol(train_labels), activation = 'linear', name=\"layer3\")\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "# Changing Loss Function to Mean Square Error, and error metric to Mean Absolute Error\n",
    "model %>% compile(\n",
    "    loss = 'mse',\n",
    "    optimizer = optimizer_adam(),\n",
    "    metrics = 'mean_absolute_error'\n",
    ")\n",
    "\n",
    "# Setting Batch Size very high for faster computation\n",
    "history <- fit(\n",
    "    object = model,\n",
    "    x = train_data,\n",
    "    y = train_labels,\n",
    "    epochs = 100,\n",
    "    validation_split = 0.2,\n",
    "    batch_size=nrow(train_data)\n",
    ")\n",
    "\n",
    "plot(history)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d91a422",
   "metadata": {},
   "source": [
    "<img src=\"img/Regression.png\" width=\"500\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41ff0f59",
   "metadata": {},
   "source": [
    "More Example of MLP and other Deep Neural Netowork are available [here](https://keras.rstudio.com/articles/examples/index.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8fe3b24",
   "metadata": {},
   "source": [
    "## Task\n",
    "The MLP network of MNIST dataset available [here](https://tensorflow.rstudio.com/guide/keras/examples/mnist_mlp/) has a test set accuracy of 98.40%. Modify the model to tune its hyper-parameters to improve the model performance against test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3dcc57eb",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "4.0.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}