Internal/coco_python_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2z9E1mUmfiKT"
   },
   "source": [
    "# COCO Python Tutorial\n",
    "COCO is a large-scale object detection, segmentation, and captioning dataset. \n",
    "\n",
    "In this tutorial we will show how to:\n",
    "\n",
    "1.   Download and use the COCO dataset\n",
    "2.   Load and show instance annotations for object detection, segmentation, key-point detection "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HnsBcPWYUgGr"
   },
   "source": [
    "Here is some information about the COCO API-\n",
    "\n",
    "*Throughout the API \"ann\"=annotation, \"cat\"=category, and \"img\"=image.*\n",
    "\n",
    "Get methods:\n",
    "* `getAnnIds` - Get ann ids that satisfy given filter conditions. \n",
    "* `getCatIds` - Get cat ids that satisfy given filter conditions. \n",
    "* `getImgIds` - Get img ids that satisfy given filter conditions. \n",
    "\n",
    "Load methods:\n",
    "* `loadAnns` - Load anns with the specified ids. \n",
    "* `loadCats` - Load cats with the specified ids. \n",
    "* `loadImgs` - Load imgs with the specified ids. \n",
    "* `loadRes`  - Load algorithm results and create API for accessing them.\n",
    "\n",
    "Show methods:\n",
    "* `showAnns` - Display the specified annotations.\n",
    "\n",
    "See source:\n",
    "https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocotools/coco.py\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NNBPKYY5fl_V"
   },
   "source": [
    "### Install packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "inm5aNBlBlbO"
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "from pycocotools.coco import COCO\n",
    "import numpy as np\n",
    "import skimage\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SCq20l38RpaU"
   },
   "source": [
    "### Download the COCO Dataset\n",
    "Use the github [gist](https://gist.github.com/mkocabas/a6177fc00315403d31572e17700d7fd9) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "j4kx9wO7teh8"
   },
   "outputs": [],
   "source": [
    "!wget http://images.cocodataset.org/annotations/annotations_trainval2017.zip\n",
    "!unzip annotations_trainval2017.zip\n",
    "!rm annotations_trainval2017.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Initialize COCO api for instance annotations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "MF6OOHIcBu-B"
   },
   "outputs": [],
   "source": [
    "dataType = \"val2017\"\n",
    "annFile = f\"annotations/instances_{dataType}.json\"\n",
    "coco = COCO(annFile)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Display COCO categories and supercategories\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "k6Hf3go-B1F4"
   },
   "outputs": [],
   "source": [
    "cats = coco.loadCats(coco.getCatIds())\n",
    "nms = [cat[\"name\"] for cat in cats]\n",
    "print(\"COCO categories: \\n{}\\n\".format(\" \".join(nms)))\n",
    "\n",
    "nms = set([cat[\"supercategory\"] for cat in cats])\n",
    "print(\"COCO supercategories: \\n{}\".format(\" \".join(nms)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get all images containing given categories and select one at random\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "kftMr0vOB7Qm"
   },
   "outputs": [],
   "source": [
    "catIds = coco.getCatIds(catNms=[\"person\", \"sheep\", \"dog\"])\n",
    "imgIds = coco.getImgIds(catIds=catIds)\n",
    "img = coco.loadImgs(imgIds[np.random.randint(0, len(imgIds))])[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load and display image using the url"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "CTEOrmoZB7Ob"
   },
   "outputs": [],
   "source": [
    "I = skimage.io.imread(img[\"coco_url\"])\n",
    "plt.axis(\"off\")\n",
    "plt.imshow(I)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "myTJZaxcU_Db"
   },
   "source": [
    "### Object Detection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load and display image with bounding box\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "MKPnmQagB7L7"
   },
   "outputs": [],
   "source": [
    "plt.imshow(I)\n",
    "plt.axis(\"off\")\n",
    "annIds = coco.getAnnIds(imgIds=img[\"id\"], catIds=catIds, iscrowd=None)\n",
    "anns = coco.loadAnns(annIds)\n",
    "coco.showAnns(anns, draw_bbox=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load and display image with image classification\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "K73JokkOd2uF"
   },
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(figsize=(10, 8))\n",
    "ax.imshow(I)\n",
    "coco.showAnns(anns, draw_bbox=True)\n",
    "for i, ann in enumerate(anns):\n",
    "    ax.text(\n",
    "        anns[i][\"bbox\"][0],\n",
    "        anns[i][\"bbox\"][1],\n",
    "        coco.loadCats(anns[i][\"category_id\"])[0][\"name\"],\n",
    "        style=\"italic\",\n",
    "        bbox={\"facecolor\": \"white\", \"alpha\": 0.7, \"pad\": 5},\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dIYCU8QWg6US"
   },
   "source": [
    "### Segementation\n",
    "Load and display instance annotations only"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "4pMJj32FHtjd"
   },
   "outputs": [],
   "source": [
    "plt.imshow(I)\n",
    "plt.axis(\"off\")\n",
    "annIds = coco.getAnnIds(imgIds=img[\"id\"], catIds=catIds, iscrowd=None)\n",
    "anns = coco.loadAnns(annIds)\n",
    "coco.showAnns(anns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load and display image with segemtation\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "-IY5LWDaAUEf"
   },
   "outputs": [],
   "source": [
    "mask = coco.annToMask(anns[0])\n",
    "for i in range(len(anns)):\n",
    "    mask += coco.annToMask(anns[i])\n",
    "\n",
    "plt.imshow(mask)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EJ06_60hfUb7"
   },
   "source": [
    "### Person Keypoints\n",
    "Initialize COCO api for person keypoints annotations\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "fiIXWBuYB7JU"
   },
   "outputs": [],
   "source": [
    "annFile = f\"annotations/person_keypoints_{dataType}.json\"\n",
    "coco_kps = COCO(annFile)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load and display keypoints annotations\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "RxYREhTMWa5V"
   },
   "outputs": [],
   "source": [
    "plt.imshow(I)\n",
    "plt.axis(\"off\")\n",
    "ax = plt.gca()\n",
    "annIds = coco_kps.getAnnIds(imgIds=img[\"id\"], catIds=catIds, iscrowd=None)\n",
    "anns = coco_kps.loadAnns(annIds)\n",
    "coco_kps.showAnns(anns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qKVrzmOLSRh8"
   },
   "source": [
    "### Image Captions\n",
    "Initialize COCO api for caption annotations\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "O_3XfpphX164"
   },
   "outputs": [],
   "source": [
    "annFile = f\"annotations/captions_{dataType}.json\"\n",
    "coco_caps = COCO(annFile)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load and display caption annotations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "YdirU_FSa0mF"
   },
   "outputs": [],
   "source": [
    "annIds = coco_caps.getAnnIds(imgIds=img[\"id\"])\n",
    "anns = coco_caps.loadAnns(annIds)\n",
    "coco_caps.showAnns(anns)\n",
    "plt.imshow(I)\n",
    "plt.axis(\"off\")\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "provenance": []
  },
  "gpuClass": "standard",
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}