PNG_preparation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Preprocessing .png data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contents of this notebook:\n",
    "<ol>\n",
    "<li>Set-up</li>\n",
    "<li>Experimenting with scikit-image (skimage)</li>\n",
    "<li>Experimenting with Pillow (PIL)</li>\n",
    "<li>Plotting the histograms of pixel values</li>\n",
    "<li>Mini assignment: feature scaling images</li> \n",
    "<li>Contrast enhancement</li>\n",
    "<li>A brief note on segmentation</li>\n",
    "</ol>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Set-up"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For more information, check out [10 Python image manipulation tools](https://opensource.com/article/19/3/python-image-manipulation-tools)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll be using a Kaggle dataset: [Pulmonary Chest X-Ray Abnormalities](https://www.kaggle.com/kmader/pulmonary-chest-xray-abnormalities)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Locate your images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!ls data_png # you can use some Bash commands in Jupyter Notebook, just prefix them with \"!\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls data_png/chest_xrays_pngs/*png"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Experimenting with scikit-image (skimage)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from skimage import io\n",
    "filename = 'data_png/chest_xrays_pngs/CHNCXR_0001_0.png'\n",
    "img=io.imread(filename) # loads it into a numpy array!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f'Img data type: {img.dtype}')\n",
    "print(f'Img shape: {img.shape}')\n",
    "print(f'Img min value: {img.min()}')\n",
    "print(f'Img max value: {img.max()}')\n",
    "print(f'Img array: {img}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt  \n",
    "plt.imshow(img,cmap='gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For loop to load & display all of your images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import glob # Python module\n",
    "\n",
    "images=[]\n",
    "for file in glob.iglob('data_png/chest_xrays_pngs/*png'):\n",
    "    img=io.imread(file) \n",
    "    images.append(img)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Images is now a list of 20 numpy arrays. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "images_array = np.array(images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Problem:** all of these images have different dimensions! Let's take a closer look: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize=(20, 15))\n",
    "rows = 5\n",
    "columns = 4\n",
    "for i in range(20):\n",
    "    fig.add_subplot(rows, columns, i+1)\n",
    "    img=images[i]\n",
    "    plt.imshow(img)\n",
    "    plt.axis('off')\n",
    "    plt.title(img.shape) # height x width [x channel]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- It seems as though all of the images with a default **grayscale colour map (colour map often abbreviated to _cmap_)** are 3-tensors, meaning that each pixel has a **R** (red), **G** (green) and **B** (blue) value associated with it. This means that for 1 _N x M_ image, you have 3 _N x M_ matrices and each one is often referred to as a **channel**.\n",
    "- Normally, an image with 3 channels would be a colour image. A _N x M_ black & white image usually only has 1 channel. Each entry of the _N x M_ matrix corresponds to 1 grayscale value per pixel.\n",
    "- Note that here, the default colour map of the 1-tensor images is actually the viridis _cmap_ (yellow to dark purple), which is unexpected. This is a great example of why you should always check the dimensions of the image array, and never trust your naked eye alone!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from skimage.color import rgb2gray\n",
    "\n",
    "images_adjusted = []\n",
    "for i in range(20):\n",
    "    img = images[i]\n",
    "    if len(img.shape) != 2: grayscale=rgb2gray(img)\n",
    "    images_adjusted.append(grayscale)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize=(20, 15))\n",
    "rows = 5\n",
    "columns = 4\n",
    "for i in range(20):\n",
    "    fig.add_subplot(rows, columns, i+1)\n",
    "    img=images_adjusted[i]\n",
    "    plt.imshow(img)\n",
    "    plt.axis('off')\n",
    "    plt.title(img.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nice! Our images are now all 2-tensors instead of 3-tensors. Note that you can display the images with [other colour maps](https://matplotlib.org/stable/tutorials/colors/colormaps.html):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize=(12, 12))\n",
    "rows = 1\n",
    "columns = 5\n",
    "\n",
    "fig.add_subplot(rows, columns, 1)\n",
    "img=images_adjusted[0]\n",
    "plt.imshow(img,cmap=\"gray\")\n",
    "plt.axis('off')\n",
    "plt.title(\"Gray scale\")\n",
    "\n",
    "fig.add_subplot(rows, columns, 2)\n",
    "img=images_adjusted[0]\n",
    "plt.imshow(img,cmap=\"binary\")\n",
    "plt.axis('off')\n",
    "plt.title(\"Binary\")\n",
    "\n",
    "fig.add_subplot(rows, columns, 3)\n",
    "img=images_adjusted[0]\n",
    "plt.imshow(img,cmap=\"summer\")\n",
    "plt.axis('off')\n",
    "plt.title(\"Summer\")\n",
    "\n",
    "fig.add_subplot(rows, columns, 4)\n",
    "img=images_adjusted[0]\n",
    "plt.imshow(img,cmap=\"viridis\")\n",
    "plt.axis('off')\n",
    "plt.title(\"Viridis\")\n",
    "\n",
    "fig.add_subplot(rows, columns, 5)\n",
    "img=images_adjusted[0]\n",
    "plt.imshow(img,cmap=\"cool\")\n",
    "plt.axis('off')\n",
    "plt.title(\"Cool\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's resize our images!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "resized_images=[]\n",
    "for i in range(20):\n",
    "    img=np.resize(images_adjusted[i],(2300,2300))\n",
    "    resized_images.append(img)\n",
    "resized_images_array = np.array(resized_images)\n",
    "resized_images_array.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Mission accomplished! The shape seems right (20 images that are each 2300 x 2300). Now one last sanity check to make sure our resized images still make sense:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# resized_images=[]\n",
    "# fig = plt.figure(figsize=(20, 15))\n",
    "# rows = 5\n",
    "# columns = 4\n",
    "# for i in range(20):\n",
    "#     fig.add_subplot(rows, columns, i+1)\n",
    "#     img=resized_images_array[i]\n",
    "#     plt.imshow(img)\n",
    "#     plt.axis('off')\n",
    "#     plt.title(img.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Whoops! That did NOT work. It is important to always have **check points** where you **manually** look at some of your data points to make that your image modifications are not doing anything unexpected.\\\n",
    "What if we tried **cropping** our images?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "height=[]\n",
    "width=[]\n",
    "for i in range(20):\n",
    "    img=images_adjusted[i]\n",
    "    height.append(img.shape[0])\n",
    "    width.append(img.shape[1])\n",
    "print(f'Smallest image height: {min(height)}')\n",
    "print(f'Smallest image width: {min(width)}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "trimmed_images=[]\n",
    "for i in range(20):\n",
    "    img=images_adjusted[i]\n",
    "    img_trim=img[0:min(height),0:min(width)]\n",
    "    trimmed_images.append(img_trim)\n",
    "trimmed_images_array = np.array(trimmed_images)\n",
    "trimmed_images_array.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize=(20, 15))\n",
    "rows = 5\n",
    "columns = 4\n",
    "for i in range(20):\n",
    "    fig.add_subplot(rows, columns, i+1)\n",
    "    img=trimmed_images_array[i]\n",
    "    plt.imshow(img)\n",
    "    plt.axis('off')\n",
    "    plt.title(img.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It worked! However, now you cropped out a huge portion of the lungs in some patients. Thus, the cropping solution was definitely not perfect either. What if we tried another one of the many available 2D image processing packages?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Experimenting with Pillow (PIL)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!cd data_png\n",
    "!mkdir adjusted_images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Save our adjusted (ie 1-tensor) images into a new directory, since PIL seems to work better when it reads in images directly:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib\n",
    "\n",
    "for i in range(20):\n",
    "    img = images_adjusted[i]\n",
    "    matplotlib.image.imsave(f'adjusted_images/img_{i}.png', img) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from PIL import Image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's read in and display our adjusted .png files, as well as their dimensions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "resize_images_PIL=[]\n",
    "\n",
    "fig = plt.figure(figsize=(20, 15))\n",
    "rows = 5\n",
    "columns = 4\n",
    "\n",
    "for i in range(20):\n",
    "    image = Image.open(f'adjusted_images/img_{i}.png')\n",
    "    new_image = image.resize((2000, 2000))\n",
    "    new_images_array=np.array(new_image)\n",
    "    resize_images_PIL.append(new_images_array)\n",
    "    \n",
    "    fig.add_subplot(rows, columns, i+1)\n",
    "    plt.imshow(new_image)\n",
    "    plt.axis('off')\n",
    "    plt.title(new_images_array.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Why 4 channels? Because we saved .png files! Let's try the same thing with .jpgs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(20):\n",
    "    img=images_adjusted[i]\n",
    "    matplotlib.image.imsave(f'adjusted_images/img_{i}.jpg', img)\n",
    "    \n",
    "resize_images_PIL_jpg=[]\n",
    "\n",
    "fig = plt.figure(figsize=(20, 15))\n",
    "rows = 5\n",
    "columns = 4\n",
    "\n",
    "for i in range(20):\n",
    "    image = Image.open(f'adjusted_images/img_{i}.jpg')\n",
    "    new_image = image.resize((2000, 2000))\n",
    "    new_images_array=np.array(new_image)\n",
    "    resize_images_PIL_jpg.append(new_images_array)\n",
    "    \n",
    "    fig.add_subplot(rows, columns, i+1)\n",
    "    plt.imshow(new_image)\n",
    "    plt.axis('off')\n",
    "    plt.title(new_images_array.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we're back to our tensor format!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Plotting the histograms of pixel values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plotting the histograms of the pixels values of the 1-tensor images we obtained with skimage:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from skimage.exposure import histogram\n",
    "fig = plt.figure(figsize=(20, 15))\n",
    "rows = 5\n",
    "columns = 4\n",
    "for i in range(20):\n",
    "    fig.add_subplot(rows, columns, i+1)\n",
    "    hist,edges=histogram(images_adjusted[i])\n",
    "    plt.plot(edges,hist)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plotting the histograms of the pixels values of the 3-tensor images we obtained with PIL:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from skimage.exposure import histogram\n",
    "fig = plt.figure(figsize=(20, 15))\n",
    "rows = 5\n",
    "columns = 4\n",
    "for i in range(20):\n",
    "    fig.add_subplot(rows, columns, i+1)\n",
    "    img=resize_images_PIL_jpg[i]\n",
    "    hist_0,edges_0=histogram(img[:,:,0])\n",
    "    hist_1,edges_1=histogram(img[:,:,1])\n",
    "    hist_2,edges_2=histogram(img[:,:,2])\n",
    "    plt.plot(edges_0,hist_0)\n",
    "    plt.plot(edges_1,hist_1)\n",
    "    plt.plot(edges_2,hist_2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Mini assignment: feature scaling images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are [many](https://machinelearningmastery.com/how-to-manually-scale-image-pixel-data-for-deep-learning/) [different](https://machinelearningmastery.com/how-to-normalize-center-and-standardize-images-with-the-imagedatagenerator-in-keras/) ways to scale the pixel values in an image. You can do (a) within-channel, within-image (local) feature scaling; (b) cross-channel within-image (local) feature scaling and (c) cross-channel and cross-sample (global) feature scaling. **For the following exercise, implement a cross-channel within-image standard scaling of the 3-tensor images (recall: _resize_images_PIL_jpg_ numpy array). Then, display the new histograms and images.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Contrast enhancement"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "img = plt.imread('adjusted_images/img_0.jpg')\n",
    "plt.imshow(img)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "markers = np.zeros_like(img) # tensor of 0s that has the same shape as img\n",
    "\n",
    "markers[img < 10] = 100 \n",
    "# for pixel value in the img that has a value below 10, set the corresponding value in the tensor of 0s to 100\n",
    "\n",
    "markers[img > 120] = 200\n",
    "# for pixel value in the img that has a value above 120, set the corresponding value in the tensor of 0s to 200\n",
    "plt.imshow(markers)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. A brief note on segmentation\n",
    "\n",
    "Image segmentation is one of the perennial challenges of computer vision. It is the process of assigning all of the pixels in an image that are part of some label of interest with \"1\", and assigning all of the other pixels a value of \"0\". It is a huge and fascinating field that you will learn more about when you get to convolutional neural networks (CNNs). However, segmentation can also be part of your image preprocessing, depending on the machine learning problem at hand. For instance, if the purpose of your model is to be able to output a clinical diagnosis based on an image, segmentation could potentially be of use to you."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}