Ex10.SampleSolution-ShapleyValues.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://www.th-koeln.de/img/logo.svg\" style=\"float:right;\" width=\"200\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Musterlösung / Sample solution \n",
    "## 10th exercise: <font color=\"#C70039\">Interpretable Machine Learning with Shapley Values for image classification</font>\n",
    "* Course: AML\n",
    "* Lecturer: <a href=\"https://www.gernotheisenberg.de/\">Gernot Heisenberg</a>\n",
    "* Author of notebook: <a href=\"https://www.gernotheisenberg.de/\">Gernot Heisenberg</a>\n",
    "* Date:   28.10.2024\n",
    "\n",
    "---------------------------------\n",
    "\n",
    "### <font color=\"ce33ff\">DESCRIPTION</font>:\n",
    "This is one implementation example to demo XAI for image classification using the inbuild cifar-10 data set, that you have come across with in exercise 8 already.\n",
    "\n",
    "### <font color=\"FF0000\">IMPORTANT NOTE</font>:\n",
    "\n",
    "This code needs the shap library version 0.44.0 !\n",
    "Hence, numpy and tensorflow then also need earlier versions to work with this version of shap. These are: numpy==1.26.4 and tensorflow==2.15.0."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "L1MnrZSd06ED"
   },
   "source": [
    "## Imports\n",
    "Import all necessary utilities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import shap # v0.44.0\n",
    "import numpy as np # v1.26.4\n",
    "\n",
    "import tensorflow as tf # v2.15.0\n",
    "from   tensorflow import keras\n",
    "import matplotlib.pyplot as plt\n",
    "from   tensorflow.keras.models import Sequential\n",
    "import ssl"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## load build-in dataset\n",
    "take the cifar-10 data set from exercise 8"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ssl._create_default_https_context = ssl._create_unverified_context\n",
    "(x_train, y_train), (x_test, y_test) = tf.keras.datasets.cifar10.load_data()\n",
    "# reshape and normalize data\n",
    "x_train = x_train.reshape(50000, 32, 32, 3).astype(\"float32\") / 255\n",
    "x_test = x_test.reshape(10000, 32, 32, 3).astype(\"float32\") / 255\n",
    "y_train = y_train.reshape(50000,)\n",
    "y_test = y_test.reshape(10000,)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build a simple CNN, compile and fit the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "inputs = tf.keras.Input(shape=(32, 32, 3))\n",
    "x = tf.keras.layers.Conv2D(32, (3, 3), activation='relu')(inputs)\n",
    "x = tf.keras.layers.MaxPooling2D((2, 2))(x)\n",
    "x = tf.keras.layers.Conv2D(128, (3, 3), activation='relu')(inputs)\n",
    "x = tf.keras.layers.MaxPooling2D((2, 2))(x)\n",
    "x = tf.keras.layers.Conv2D(64, (3, 3), activation='relu')(inputs)\n",
    "x = tf.keras.layers.MaxPooling2D((2, 2))(x)\n",
    "x = tf.keras.layers.Conv2D(32, (3, 3), activation='relu')(inputs)\n",
    "x = tf.keras.layers.MaxPooling2D((2, 2))(x)\n",
    "x = tf.keras.layers.Flatten()(x)\n",
    "x = tf.keras.layers.Dense(256, activation='relu')(x)\n",
    "x = tf.keras.layers.Dense(64, activation='relu')(x)\n",
    "outputs = tf.keras.layers.Dense(10, activation='softmax')(x)\n",
    "\n",
    "# inputs and outputs\n",
    "model = tf.keras.Model(inputs=inputs, outputs=outputs, name=\"test_for_shap\")\n",
    "# compile the model\n",
    "model.compile(\n",
    "      loss=tf.keras.losses.SparseCategoricalCrossentropy(),\n",
    "      optimizer=tf.keras.optimizers.Adam(),\n",
    "      metrics=[tf.keras.metrics.SparseCategoricalAccuracy()]\n",
    "  )\n",
    "model.fit(x_train, y_train, validation_data=(x_test, y_test), epochs = 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### predict on the test set (one image for each class)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# class label list\n",
    "class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer',\n",
    "               'dog', 'frog', 'horse', 'ship', 'truck']\n",
    "\n",
    "# example image for each class\n",
    "images_dict = dict()\n",
    "\n",
    "for i, l in enumerate(y_train):\n",
    "    if len(images_dict)==10:\n",
    "        break\n",
    "    if l not in images_dict.keys():\n",
    "        images_dict[l] = x_train[i].reshape((32, 32,3))\n",
    "images_dict = dict(sorted(images_dict.items()))\n",
    "    \n",
    "# example image for each class for test set\n",
    "x_test_dict = dict()\n",
    "for i, l in enumerate(y_test):\n",
    "    if len(x_test_dict)==10:\n",
    "        break\n",
    "    if l not in x_test_dict.keys():\n",
    "        x_test_dict[l] = x_test[i]\n",
    "        \n",
    "# order by class\n",
    "x_test_each_class = [x_test_dict[i] for i in sorted(x_test_dict)]\n",
    "x_test_each_class = np.asarray(x_test_each_class)\n",
    "\n",
    "# Compute predictions\n",
    "predictions = model.predict(x_test_each_class)\n",
    "predicted_class = np.argmax(predictions, axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualization\n",
    "#### plot function\n",
    "define a plot function for actual and predicted class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# plot actual and predicted class\n",
    "def plot_actual_predicted(images, pred_classes):\n",
    "    fig, axes = plt.subplots(1, 11, figsize=(16, 15))\n",
    "    axes = axes.flatten()\n",
    "  \n",
    "    # plot\n",
    "    ax = axes[0]\n",
    "    dummy_array = np.array([[[0, 0, 0, 0]]], dtype='uint8')\n",
    "    ax.set_title(\"Base reference\")\n",
    "    ax.set_axis_off()\n",
    "    ax.imshow(dummy_array, interpolation='nearest')\n",
    "    \n",
    "    # plot image\n",
    "    for k,v in images.items():\n",
    "        ax = axes[k+1]\n",
    "        ax.imshow(v, cmap=plt.cm.binary)\n",
    "        ax.set_title(f\"True: %s \\nPredict: %s\" % (class_names[k], class_names[pred_classes[k]]))\n",
    "        ax.set_axis_off()\n",
    "    \n",
    "    plt.tight_layout()\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### XAI using SHAP\n",
    "Now use the SHAP library to generate the Shapley values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# select background for shap to take an expectation over\n",
    "background = x_train[np.random.choice(x_train.shape[0], 1000, replace=False)]\n",
    "\n",
    "# use the Explainer to explain predictions of the model\n",
    "explainer = shap.DeepExplainer(model, background)\n",
    "shap_values = explainer.shap_values(x_test[1:5])\n",
    "\n",
    "# compute the shapley values\n",
    "#shap_values = explainer.shap_values(x_test_each_class)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### plot the Shapley values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#plot_actual_predicted(images_dict, predicted_class)\n",
    "#print()\n",
    "#shap.image_plot(shap_values, x_test_each_class * 255)\n",
    "\n",
    "x_train[np.random.choice(x_train.shape[0], 1000, replace=False)].shape # select a random 1000 datapoints.\n",
    "# This is the data that is going to form our \"overall\" that we will compare each prediction to.\n",
    "#shap.image_plot(shap_values, x_test_each_class, true_labels=class_names)\n",
    "\n",
    "# plot the feature attributions\n",
    "shap.image_plot(shap_values, -x_test[1:5], true_labels=class_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "plot_actual_predicted(images_dict, predicted_class)\n",
    "print()\n",
    "shap.image_plot(shap_values, x_test_each_class * 255)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "lime_image.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.17"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}