Topics/Computer Vision/Introduction_to_Computer_Vision_using_OpenCV_and_Python.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Introduction to Computer Vision using OpenCV and Python.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "metadata": {
        "id": "OmqAkANMsWql",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Introduction to Computer Vision using OpenCV and Python\n",
        "\n",
        "---\n",
        "\n",
        "![Introduction](https://i.ibb.co/L03Rvcq/introduction.jpg)\n",
        " \n",
        "In this guide, we will introduce a brief overview of Deep Learning. Then, we will discuss the purpose of Computer Vision in Python. After that, we' ll be taught the basics of dealing with data using OpenCV libraries by creating and displaying images. The fundamental tasks of Computer Vision such as object recognition and semantic segmentation will be explained. We will also cover the process of feature extraction, edge and face detection and object classification..\n",
        "\n",
        "## Prerequisites\n",
        "Before starting this guide, it is essential to be familiar with the basics of Python programming and Image Processing concepts.\n",
        "\n",
        "## Guide map\n",
        "We will provide a structured content according to the following map:\n",
        "\n",
        "1.\tIntroduction;\n",
        "2. A brief introduction to Deep Learning;\n",
        "3.\tComputer vision tasks;\n",
        "4.\tComputer Vision Systems;\n",
        "5.\tPython libraries for Computer Vision;\n",
        "6. OpenCV library on Windows and Ubuntu;\n",
        "7. Processing images with OpenCV;\n",
        "8. Use cases for Computer Vision;\n",
        "9. Conclusion.\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "## 1.\tIntroduction: \n",
        "\n",
        "Computer Vision is a branch of Computer Science, which aims to build up intelligent systems that can understand the content in images as they are perceived by humans. The data may be presented in different modalities such as sequential (video) images from multiple sensors (cameras) or multidimensional data from a biomedical camera, and so on. It is the discipline that integrates the methods of acquiring, processing, analyzing and understanding large-scale images from the real world. It is also about depicting and reconstructing the world that we perceive in images, such as edge, lighting, color and pattern. The recognition of images, by decoding them into meaningful information from image-based data using models created by engineering, physics, statistics and learning theories. It is intended to simulate human vision, including the ability to learn, make decisions and react to actions based on visual information. Computer Vision is one aspect of Artificial Intelligence and Image Processing, which generally aims to simulate intelligent human capabilities. In computer Vision concept, object recognition is one of the fundamental tasks, which depends on how these objects are defined, whether in the form of images or video sequences, and human beings are able to recognize many entities, even if these objects, which are images, vary greatly in size and lighting.\n",
        "\n",
        "![Computer Vision](https://i.ibb.co/X5d74Ft/CVV.jpg)\n",
        "\n",
        "Some examples of Computer Vision applications:\n",
        "* Any application that can recognize objects or humans in an image;\n",
        "*  Automatic control applications (industrial robots, vehicles);\n",
        "*  Object construction models (industrial inspection, medical image analysis);\n",
        "*  Applications make it possible to track a moving object.\n",
        "\n",
        "\n",
        "\n",
        "Useful books for learning various aspects of Computer Vision: [Multiple View Geometry in Computer Vision](http://www.robots.ox.ac.uk/~vgg/hzbook/),  [Computer Vision: Algorithms and Applications](http://szeliski.org/Book/)"
      ]
    },
    {
      "metadata": {
        "id": "VyNV5akJzJEP",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## 2.\tA brief introduction to Deep Learning: \n",
        "\n",
        "###2.1. What is Deep Learning?\n",
        "Deep Learning is an Machine Learning strategy that has greatly enhanced performance in many fields such as Computer Vision, Speech Recognition, Machine Tanslation, and so on.  The use of deep learning techniques, through raw data, allows many challenges to be solved in many economic sectors such as health, transport, finance, etc.\n",
        "\n",
        "The favourable conditions that allowed the rise of Deep Learning:\n",
        "\n",
        "*   Availability of very large spatio-temporal datasets (Big Data);\n",
        "*   Availability of high-performance computing (GPU);\n",
        "*   Flexibility of new training models (Deep Neural Networks).\n",
        "\n",
        "###2.2. Deep Learning Frameworks\n",
        "\n",
        "In this section, we present the most popular frameworks for Deep Learning.\n",
        "\n",
        "| Framework | Features   | Supports languages |  Download     |\n",
        "|------|------|------|------|\n",
        "|   Tensorflow  |Highly flexible system architecture|  Python, C++ and R  | [here](https://www.tensorflow.org/)\n",
        "|   Caffe  |Speed, transposability and applicability in modelling Convolution Neural Networks (CNN)|  C, C++, Python, MATLAB  | [here](http://caffe.berkeleyvision.org/)\n",
        "|   CNTK  |Easy training and combination of popular model types across servers|  Python, C++ and the Command Line Interface  | [here](https://www.microsoft.com/en-us/cognitive-toolkit/)\n",
        "|   Torch/PyTorch  | The entire deep modeling process is far more simpler as well as transparent | Lua, Python    | [here](http://torch.ch/) or [here](http://pytorch.org/)\n",
        "|   Keras  | Provide a simplistic interface for the purpose of quick prototyping by constructing effective neural networks that can work with TensorFlow |Python    | [here](https://keras.io/) "
      ]
    },
    {
      "metadata": {
        "id": "5PEB8FNsuEFX",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##3.\tComputer vision tasks:\n",
        " \n",
        "In this section, we will successively examine some tasks of Computer Vision, in particular Image Recognition, Semantic Segmentation, Image Retrieval, Image Restoration, Object Recognition, Video Tracking, and so on.\n",
        "\n",
        "![CV tasks](https://i.ibb.co/NxVsrHf/cvt.jpg)\n",
        "\n",
        "###3.1.\t Image Recognition\n",
        "\n",
        "Traditionally, Computer Vision is about deciding whether or not the image contains an object. This task can be solved simply with little effort by human beings, but a certain activity is still not solved effectively and finely by computer in its general state. The only way to solve this issue is to find the best solutions to match certain features (edges, shapes, etc), and in some cases only, often with specific lighting conditions, a background and a certain position for the camera.\n",
        "\n",
        "####Types of recognition:\n",
        "\n",
        "**A - Identification:**\n",
        "Predefined objects are often identified from different viewpoints of the camera in their different locations.\n",
        "\n",
        "**B - Selection:**\n",
        "Define a unique identifier in the shape. For example: identify a person's face or identify the specific type of a person or car.\n",
        "\n",
        "**C - Examination:**\n",
        "Image data is treated for a specific object. For example: check for the presence of diseased cells in medical form, check if a car is present on a highway.\n",
        "\n",
        "You will find here some projects and scripts based on Image Recognition task using a Deep Learning paradigm:\n",
        "\n",
        "- Fast MPN-COV: [here](https://github.com/jiangtaoxie/fast-MPN-COV);\n",
        "- Fine-Grained Representation Learning and Recognition by Exploiting Hierarchical Semantic Embedding: [here](https://github.com/HCPLab-SYSU/HSE); \n",
        "- Fine grained classification: [here](https://github.com/xcnkx/fine_grained_classification)."
      ]
    },
    {
      "metadata": {
        "id": "zrHJj09uvaWC",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "###3.2.\tImage Retrieval:\n",
        "Images stored in a visual dataset are retrieved based on the content as well as similar concepts of the database query where an image is inserted, and the output is a similar set of images. Content-based visual information retrieval is the implementation of the computer vision system in order to target images, i.e. the problem of retrieving images from large datasets. Image retrieval systems seek to find images similar to a query image among a dataset. The following figure represents the general process of retrieving images from content.\n",
        "\n",
        "![Image retrieval](https://i.ibb.co/VTLpZgD/SFS.jpg)\n",
        "\n",
        "You will find here some projects and scripts based on Image Retrieval tasks using a Deep Learning paradigm:\n",
        "\n",
        "- Deep Local Feature (DeLF): [here](https://github.com/nashory/DeLF-pytorch);\n",
        "- MILDNet: [here](https://github.com/gofynd/mildnet); \n",
        "- MultiGrain: [here](https://github.com/facebookresearch/multigrain)."
      ]
    },
    {
      "metadata": {
        "id": "Dq13xQrBwOvf",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "###3.3.\tImage Restoration:\n",
        "This is the process of restoring degraded images that cannot be recovered. Original images can be restored by prior-knowledge of damage or distortions that cause deterioration of images such as scratches, dust and stains. Restoration also includes images taken by sophisticated cameras that have been distorted due to the weather conditions in which they were taken, such as scanned images.\n",
        "\n",
        "You will find here some projects and scripts based on Image Restoration task using a Deep Learning paradigm:\n",
        "\n",
        "- Image Super Resolution using in Keras 2+: [here](https://github.com/titu1994/Image-Super-Resolution);\n",
        "- RED-net: [here](https://github.com/ved27/RED-net);\n",
        "- Noise2Noise: [here](https://github.com/NVlabs/noise2noise)."
      ]
    },
    {
      "metadata": {
        "id": "s6pzqNlKwaMf",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "###3.4.\tObject Recognition:\n",
        "It is a branch of Computer Vision dedicated to the detection of a particular object in an image or video. Humans can recognize many objects in images with little effort, although the image may differ slightly from different aspects, such as variations, or even when they are moved or rotated. Although humans can recognize objects when they are partially hidden, this task remains a challenge for computer vision systems. The  object recognition process is given by the following figure:\n",
        "\n",
        "![Object recognition](https://www.mathworks.com/content/mathworks/www/en/solutions/deep-learning/object-recognition/jcr:content/mainParsys/band_copy_1227855798_2052140687/mainParsys/columns_1606542234/2/image_copy_copy_copy.adapt.full.low.svg/1541451136966.svg)\n",
        "\n",
        "Unlike Machine Learning, the Deep Learning paradigm consists of an end-to-end feature representation learning from raw data without any prior data processing steps.\n",
        "\n",
        "You will find here some projects and scripts based on Object Recognition task using a Deep Learning paradigm:\n",
        "\n",
        "- TF-slim: [here](https://github.com/tensorflow/models/tree/master/research/slim);\n",
        "- DenseNet: [here](https://github.com/liuzhuang13/DenseNet);\n",
        "- DeepBeliefSDK: [here](https://github.com/jetpacapp/DeepBeliefSDK).\n"
      ]
    },
    {
      "metadata": {
        "id": "3utQKUl1wotP",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "###3.5. Semantic Segmentation\n",
        "Semantic segmentation is a Deep Learning algorithm that assigns a label or category for each pixel in an image. It makes it possible to recognize a set of pixels that are in distinct classes. For example, an autonomous vehicle must be able to recognize vehicles, pedestrians, traffic signs, sidewalks and other environmental components of the road network. Semantic segmentation is involved in a wide range of solutions such as computer-controlled driving, autonomous vehicles, diagnostic imaging, industrial controls, and so on. The splitting of images into two classes is a simple example of semantic segmentation. In fact, it has no restriction in terms of categories. The number of classes can be changed in order to classify image content. For example, the image could be segmented into 4 classes: person, sky, sea and background. The example in the following figure is based on ICCV 2015 paper [Conditional Random Fields as Recurrent Neural Networks](http://www.robots.ox.ac.uk/~szheng/papers/CRFasRNN.pdf), which utilizes deep learning techniques and probabilistic graphical models for semantic image segmentation. \n",
        "\n",
        "\n",
        "![Semantic Segmentation](https://i.ibb.co/gdWgGd4/ss.jpg)\n",
        "\n",
        "You will find here some projects and scripts based on Semantic Segmentation task using a Deep Learning paradigm:\n",
        "\n",
        "- PSPNet: [here](https://github.com/hszhao/PSPNet);\n",
        "- TorchSeg: [here](https://github.com/ycszen/TorchSeg);\n",
        "- Deeplab: [here](https://github.com/tensorflow/models/tree/master/research/deeplab).\n"
      ]
    },
    {
      "metadata": {
        "id": "7G6p8zL4xIKC",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "###3.6.\tVideo Tracking\n",
        "It is the process of locating or tracking a moving object (or several moving objects) using static or mobile cameras, while having many uses, such as human-computer interaction, security, three-dimensional reality, medical images and video editing. Tracking can be time-consuming due to video content and the need to use complex algorithms to identify and track objects.\n",
        "Tracking aims to follow the desired object to be tracked in a sequence of successive images. Tracking is a difficult task when this object moves faster than the capture-rate of these successive images. It is even more difficult when this entity changes direction as it shifts. For this reason, tracking systems apply a motion model that explains how this object's image will change as it moves in different directions. The following figure illustrates the overall scheme of the object tracking process:\n",
        "\n",
        "![Object Tracking](https://i.ibb.co/hY5GhdB/track.jpg)\n",
        "\n",
        "You will find here some projects and scripts based on Video Tracking task using a Deep Learning paradigm:\n",
        "\n",
        "- GOT-10k Python Toolkit: [here](https://github.com/got-10k/toolkit);\n",
        "- SiamMask: [here](https://github.com/foolwood/SiamMask);\n",
        "- Deep SORT: [here](https://github.com/guanfuchen/deep_sort);\n",
        "- Object tracking (tutorial): [here](https://www.pyimagesearch.com/2018/07/30/opencv-object-tracking/)."
      ]
    },
    {
      "metadata": {
        "id": "9TF1Sp6PxcTf",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##4.\tComputer Vision Systems\n",
        "Computer vision systems are very diverse and are divided into large and sophisticated systems that perform general and complete tasks as well as small systems that perform specific and simple ones. Most computer vision systems mainly include the following:\n",
        "\n",
        "### 4.1.\tCollecting images\n",
        "\n",
        "The image is generated by using one or more image sensors. These include many digital camera sensors, distance sensors, radars, and ultrasonic cameras.  \n",
        "\n",
        "###4.2.\tPre-processing operations\n",
        "\n",
        "Before applying the computer vision algorithm in order to extract valuable information, it is necessary to perform prior data operations to ensure that the data are consistent with the algorithm's specific hypotheses. Some examples of these processes include:  \n",
        "\n",
        "1. Select the image resolution to confirm that its coordinate system is correct.\n",
        "2. Reduce the interference to ensure that the sensor does not provide inaccurate information.\n",
        "3. Increase the variance in order to ensure that the required information will be available.\n",
        "\n",
        "###4.3.\tFeatures extraction\n",
        "\n",
        "Visual data features are extracted at different levels of abstraction from data raw. These benchmarks are categorized into:\n",
        "1. Global features such as color and shape.\n",
        "2. Local features such as edges and points.\n",
        "More complex features related to colors and patterns can be obtained.\n",
        "\n",
        "###4.4.\tSegmentation\n",
        "All zones of the image can be recognized as important locations for subsequent operations. For example: select a set of key points, divide one or more images that contain the region of interest.\n",
        "3.5.\tHigh-level processing operations\n",
        "At this stage, the input data consists of a small set of data, such as a set of points or a portion of the image that is suspected to contain the interest object. The other operations are:\n",
        "1. Ensure that the collected data are consistent with the hypotheses of the intended application.\n",
        "2. Evaluate the transaction values assigned to the request, such as steering or shape size.\n",
        "3. Classify the recognized objects into several classes\n"
      ]
    },
    {
      "metadata": {
        "id": "cktRfUeFx_Xy",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##5.\tPython libraries for Computer Vision\n",
        "The main toolkits for image processing in python are OpenCV, scikit-image and Pillow. The most general Python libraries (Numpy and Scipy) also provide some image processing tools. All these libraries can easily dialog with each other due to the common use of Numpy arrays to store images. A grayscale image is usually stored in a 2-dimensional integer or real value Numpy array with H rows and W columns (W=width,H=height). A color image is stored in a 3-dimensional Numpy array (H, W, 3).\n",
        "\n",
        "* OpenCV is a library that is written in C++, which is rich and widely used in computer vision.\n",
        "* [Pillow](http://pillow.readthedocs.org/en/latest/) is a PIL Fork (Python IMage Library). It is a library that is specific to Python, but is mainly written in C. It allows basic operations to be performed on images including read/write, transformations, histograms, filtering.\n",
        "* [Scikit-Imag](http://scikit-image.org/)e is a fairly recent and actively developed library. The advantage of this library is that it is written in Python and Cython (Python typed and compiled for acceleration) which makes it easy to read its code.\n",
        "*\t[Scipy.ndimage](http://docs.scipy.org/doc/scipy/reference/ndimage.\n",
        "htm): Scipy's ndimage module provides a number of functions for shaping, interpolation, mathematical morphology and statistics.\n"
      ]
    },
    {
      "metadata": {
        "id": "zIk09vPayrOn",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##6.\t\tOpenCV library on Windows and Ubuntu\n",
        "Gary Bradsky started OpenCV at Intel in 1999. Compatible with a variety of languages such as C++, Python, etc., OpenCV-Python is an API that allows OpenCV to simultaneously release the power of Python and C++ API. In the case of Python, it is a library of binaries intended to address computer vision challenges. This library is based on NumPy and its array structures. That means we can also integrate it easily into other libraries such as SciPy and Matplotlib.\n",
        "\n",
        "As we have explained previously, all operations on images are purely mathematical operations. But we can't say that programmers will do all these operations every time they use images, hence the development of OpenCV library, which includes functions that perform the most necessary operations in the images.\n",
        "\n",
        "###Windows:\n",
        "In order to download the Python program (x,y), click [here](https://python-xy.github.io/downloads.html, it’s possible to download each file individually).  First, download the following [file](https://drive.google.com/file/d/0B0kFf-FN5r9tS1EzSlppUGItNUU/view?usp=sharing) which contains the collection of the OpenCV library. Then, install the python program (x,y) as shown in the figures:\n",
        "![Windows](https://i.ibb.co/WgLMMHs/122.png)\n",
        "\n",
        "###Ubuntu:\n",
        "There are two ways to install OpenCV on Linux systems. The first one consists in installing pre-compiled files from repositories. For instance, in the case of the Ubuntu platform, it is sufficient to execute the following command:\n"
      ]
    },
    {
      "metadata": {
        "id": "WYwqBYgvzzvg",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "sudo apt-get install libopencv-dev python-opencv"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "koAtONVGz2yX",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "The second method consists in compiling the source files immediately beforehand (this method allows you to obtain the latest version of the library).\n",
        "\n",
        "Open the terminal line and proceed as follows:\n"
      ]
    },
    {
      "metadata": {
        "id": "rr05G_iDz6RP",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "sudo apt-get update \n",
        "sudo apt-get upgrade\n",
        "sudo apt-get install build-essential cmake git pkg-config\n",
        "sudo apt-get install libjpeg8-dev libtiff4-dev libjasper-dev libpng12-dev \n",
        "sudo apt-get install libatlas-base-dev gfortran\n",
        "# install Pip package \n",
        "wget https://bootstrap.pypa.io/get-pip.py \n",
        "sudo python get-pip.py\n",
        "sudo pip install virtualenv virtualenvwrapper \n",
        "sudo rm -rf ~/.cache/pip\n",
        "# virtualenv and virtualenvwrapper \n",
        "export WORKON_HOME=$HOME/.virtualenvs source /usr/local/bin/virtualenvwrapper.sh\n",
        "source ~/.bashrc\n",
        "mkvirtualenv cv\n",
        "# Install Python2.7 \n",
        "sudo apt-get install python2.7-dev\n",
        "# Install Numby libraries\n",
        "pip install numpy\n",
        "# Download OpenCV library\n",
        "\n",
        "cd ~ \n",
        "git clone https://github.com/Itseez/opencv.git \n",
        "cd opencv \n",
        "git checkout 3.0.0\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "S__70Mvu0JYp",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "After that, try downloading the opencv_contrib package. It will be used to use some features such as SIFT, SURF, which were in the OpenCV 2.4.2 library, and then deleted in OpenCV 3.0."
      ]
    },
    {
      "metadata": {
        "id": "cu2qCiI90KYK",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "cd ~ \n",
        "git clone https://github.com/Itseez/opencv_contrib.git \n",
        "cd opencv_contrib\n",
        "git checkout 3.0.0\n",
        "cd ~/opencv \n",
        "mkdir build \n",
        "cd build\n",
        "cmake -D CMAKE_BUILD_TYPE=RELEASE \\ -D CMAKE_INSTALL_PREFIX=/usr/local \\ -D INSTALL_C_EXAMPLES=ON \\ -D INSTALL_PYTHON_EXAMPLES=ON \\ -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \\ -D BUILD_EXAMPLES=ON ..\n",
        "make\n",
        "sudo make install\n",
        "sudo ldconfig\n",
        "cd ~/.virtualenvs/cv/lib/python2.7/site-packages/ \n",
        "ln -s /usr/local/lib/python2.7/site-packages/cv2.so cv2.so\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "0SAAY5fQd0iI",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##7. Processing images with OpenCV\n",
        "\n",
        "Now we have successfully installed OpenCV, let's start by doing it.\n",
        "\n",
        "###7.1. Reading images in Python\n",
        "\n",
        "To read an image, we have the ` imread ()` function. It should be mentioned that previously, we have moved to the directory that contains the image.\n"
      ]
    },
    {
      "metadata": {
        "id": "Vu3cyESDeIn_",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "img = cv2.imread ('img.jpg')\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "2Z9OfGtkeSzI",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "As an alternative, it is also possible to pass a value for a flag, which is the second argument\n",
        "\n",
        "cv2.IMREAD_COLOR: For loading a color image by overlooking existing transparency;\n",
        "cv2.IMREAD_GRAYSCALE: For loading a grayscale image;\n",
        "cv2.IMREAD_UNCHANGED: For loading an image that includes an alpha channel\n",
        "It is possible to use integers 1, 0 or -1:"
      ]
    },
    {
      "metadata": {
        "id": "zB8tW_3ceU7o",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "img = cv2. imread ('img.jpg', 0)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "oUBkmlUUeYCF",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Note that sending an invalid image path does not result in any errors.\n",
        "\n",
        "###7.2. Displaying images in Python\n",
        "\n",
        "The `cv2.imshow ()` function enables to display an image in a frame that can be adjusted to its size. The first argument is the name of the frame and the second one is the image.\n"
      ]
    },
    {
      "metadata": {
        "id": "ClkgZCvhehgG",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "\n",
        "img = cv2. imread ('img.jpg')\n",
        "cv2.imshow('Images', img)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "9jG40wsVejLm",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Note that we have two frames at once as we have not attempted to title them in the same way. `cv2.destroyAllWindows ()` function is another function that destroys all the frames that we have already created. `cv2.destroyWindow ()` also destroys a specific frame.\n",
        "\n",
        "###7.3. Creating images in Python\n",
        "\n",
        "To do this, there is the ` cv2.imwrite ()`function. The first argument is the file name and the second one is the image to be saved."
      ]
    },
    {
      "metadata": {
        "id": "srTWfL38e6aO",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "cv2.imwrite('img_gray.png', img)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "kzEjMU7ke681",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "This will store the grayscale image named \"img_gray.png\" in the current location.\n",
        "\n",
        "###7.4. Displaying images using Matplotlib\n",
        "\n",
        "By using [Matplotlib](https://matplotlib.org/index.html) library, we can display that image."
      ]
    },
    {
      "metadata": {
        "id": "s7XZT_McfGJH",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import matplotlib.pyplot as plt\n",
        "plt.imshow(img, cmap = \"gray\", interpolation = \"bilinear\")\n",
        "plt.xticks([]), pl.ticks ([])\n",
        "(([], ), ([], ))\n",
        "plt.display ()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "EVqyYGe5fHwf",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "\n",
        "###7.5. Core operations on images\n",
        "\n",
        "Let's now look at the basic operations applicable on the image."
      ]
    },
    {
      "metadata": {
        "id": "ia5rzt_cfMJi",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import cv2\n",
        "img = cv2.imread ('img.jpg')\n",
        "y, x = 100,50"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "FmrfgC9AfN9_",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Reading of color values at positions y, x:\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "arC6GdqpfP68",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "(b, g, r) = img[y,x]\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "mwa-7_0ufRRF",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Region of interest at (x, y) whose dimensions are 100x100:\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "B1YJOQ9YfT7c",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "roi = img[y:y+100,x:x+100] \n",
        "cv2.imshow ('image', img)\n",
        "cv2.imshow('ROI', roi)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "yHVyOsVefVyp",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Pixelization of the new color :\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "rxmt6fXnfXcO",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "\n",
        "roi[:,:]= (55,44,87) \n",
        "cv2.imshow('New image', img)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "IOsx9s8Pf0ZE",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##8. Use cases for Computer Vision\n",
        "In this section, we will look at some tasks related to computer vision such as *edge detection, face detection, *feature detection and description, object classification performed by OpenCV and Python.\n",
        "\n",
        "\n",
        "###8.1.Edge detection \n",
        "In OpenCV we can choose only to display the edges of objects with the `Canny ()` function:\n"
      ]
    },
    {
      "metadata": {
        "id": "sI-NcW48f56o",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "img = cv2.imread('img.jpg')\n",
        "cv2.imwrite ('edge_img.jpg', cv2.Canny (img, 512, 415))\n",
        "cv2.imshow ('edges', cv2.imread('edge_img.jpg'))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "3MwM6bgwf7an",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "###8.2. Face detection\n",
        "\n",
        "OpenCV will also enable to detect faces in images. Let's now use Haar's cascading classifier.\n",
        "\n",
        "Now, there is one last point that we would really like to address, and that is the face detection. The Haar classifier is used. It is a matter of locating the position of faces in an image in order to standardize the size of the face area."
      ]
    },
    {
      "metadata": {
        "id": "TbvZE86RgGl3",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import sys, os\n",
        "import cv2\n",
        "  \n",
        "def face_detection(image, image_out, show = False):\n",
        "    # Load the image in memory\n",
        "    img = cv2.imread(image)\n",
        "    # Load the face detection model\n",
        "    face_model = cv2.CascadeClassifier(\"haarcascade_frontalface_alt2.xml\")\n",
        "     \n",
        "     \n",
        "    # detection of the face(s)\n",
        "    faces = face_model.detectMultiScale(img)\n",
        "     \n",
        "    # we place a bounding boxe around the faces\n",
        "    print (\"number of faces\", len(faces), \"image size\", img.shape, \"image\", image)\n",
        "    for face in faces:\n",
        "        cv2.rectangle(img, (face[0], face[1]), (face[0] + face[2], face[0] + face[3]), (255, 0, 0), 3)\n",
        "         \n",
        "    # we store the final result\n",
        "    cv2.imwrite(image_out, img)\n",
        "     \n",
        "    # to see the image, press ESC to exit\n",
        "    if show :\n",
        "        cv2.imshow(\"face\",img)\n",
        "        if cv2.waitKey(5000) == 27: cv2.destroyWindow(\"face\")\n",
        "   \n",
        "if __name__ == \"__main__\":\n",
        "    # wall lamp \n",
        "    for file in os.listdir(\".\"\") :\n",
        "        if file.startswith(\"face\") : continues # already processed\n",
        "        if os.path.splitext(file)[-1].lower() in [\".jpg\",\".jpeg\",\".png\" ] :\n",
        "            face_detect (file, \"face_\" + file)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "OZtycY15gLSv",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "As you can see, it drew a blue square (bounding boxe) around the face in the image."
      ]
    },
    {
      "metadata": {
        "id": "wrce4QQ80YBa",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "###8.3.\tFeature Detection and Description\n",
        "In this section, we will present a brief description of the SIFT (Scale-Invariant Feature Transform) algorithm. \n",
        " The main idea of this approach is to transform an image into feature vectors (feature maps), which should ideally be invariant to geometric transformations (rotation and scaling). This involves the detection of interest points, which will make it possible to detect an object. The detection of these points leads to the implementation of feature vectors whose components are specific to the point under consideration.\n",
        " \n",
        "\n",
        "***SIFT in OpenCV and Python:***"
      ]
    },
    {
      "metadata": {
        "id": "rNTXBl4B0q6Z",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import cv2\n",
        "import numpy as np\n",
        "\n",
        "img = cv2.imread('my_img.jpg')\n",
        "gray= cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)\n",
        "\n",
        "sift = cv2.SIFT()\n",
        "kp = sift.detect(gray,None)\n",
        "\n",
        "img=cv2.drawKeypoints(gray,kp)\n",
        "\n",
        "cv2.imwrite('sift_img.jpg',img)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "TYWymgUI0tyQ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "###8.4.\tObject Classification\n",
        "To correctly identify an object in an image, it may be interesting to simply detect its edges and shapes when extracting features.\n",
        "How will we proceed to recognize objects?\n",
        "These are the 3 steps that we will perform: (1) extracting features in the image, (2) estimating each feature and (3) classifying of edges.\n",
        "* Let's start by importing and loading an image.\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "P4nlnByg02do",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import numpy as np \n",
        "import cv2 \n",
        "image = cv2.imread('my_image.bmp')\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "i18X_cDl07Aq",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "* Step 1: Edge detection\n",
        "In order to improve edge detection, we will convert the color image to grayscale before performing a thresholding.\n"
      ]
    },
    {
      "metadata": {
        "id": "W-KDt17J1FYg",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)\n",
        "ret,thresh = cv2.threshold(gray,250,255,cv2.THRESH_BINARY_INV)\n",
        "\n",
        "img,edges,h=cv2.findEdges(thresh,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "iIROM1FR1HvM",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "* Step 2: Edge estimation"
      ]
    },
    {
      "metadata": {
        "id": "OIYl__da1OmC",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "for cnt in edges:\n",
        "perimeter =cv2.arcLength(cnt,True)\n",
        "approx = cv2.approxPolyDP(cnt,0.01* perimeter,True)\n",
        "\n",
        "M = cv2.moments(cnt)\n",
        "cX = int(M[\"m10\"] / M[\"m00\"])\n",
        "cY = int(M[\"m01\"] / M[\"m00\"])\n",
        "cv2.drawEdges(image,[cnt],-1,(0,255,0),2)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "8snGzFoo1Sec",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "* Step 3 : Pattern classification"
      ]
    },
    {
      "metadata": {
        "id": "JxtYXTKq1WkN",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "All you need to do is to recall how many peaks there are in each shape."
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "RVRgjJe11ZLD",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "if len(approx)==3:\n",
        "shape = \"triangle\"\n",
        "elif len(approx)==4:\n",
        "(x, y, w, h) = cv2.boundingRect(approx)\n",
        "ratio = w / float(h)\n",
        "if ratio >= 0.95 and ratio <= 1.05:\n",
        "shape = \" square\"\n",
        "else:\n",
        "shape = \"rectangle\"\n",
        "elif len(approx)==5:\n",
        "shape = \" pentagon\"\n",
        "elif len(approx)==6:\n",
        "shape = \" hexagon \"\n",
        "else:\n",
        "shape= \"circle\"\n",
        "cv2.putText(image, shape, (cX, cY), cv2.FONT_HERSHEY_SIMPLEX,0.5, (255, 255, 255), 2)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "XwA6Ph0u1ct4",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "We just have to display the result to check out our work:"
      ]
    },
    {
      "metadata": {
        "id": "N54tEW351emc",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "cv2.imshow('Final_image',image)\n",
        "cv2.waitKey(0)\n",
        "cv2.destroyAllWindows()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "J1DxqSPp4yF1",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "##9. Conclusion\n",
        "\n",
        "In this guide, we discussed the topic of Computer Vision using OpenCV and Python. We presented some fundamental tasks of Computer Vision such as Object Recognition and Semantic Segmentation. We also examined some case studies about the process of edge and face detection, feature extraction and object classification."
      ]
    }
  ]
}