Pix2PixHD_next_frame.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Pix2PixHD_next_frame.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "machine_shape": "hm",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/duskvirkus/colab-notebooks/blob/main/Pix2PixHD_next_frame.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VXoBaXRDKuMV"
      },
      "source": [
        "#Next Frame Prediction using Pix2PixHD\n",
        "\n",
        "By Derrick Schultz\n",
        "\n",
        "Forked repo and tutorial based on [JC Testud’s excellent repo and Medium](https://medium.com/@jctestud/video-generation-with-pix2pix-aed5b1b69f57) article."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xfOexYWJX3Pt"
      },
      "source": [
        "## Mount Google Drive"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K--AsrIzpH58"
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive', force_remount=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cUvLbCJtLqaV"
      },
      "source": [
        "### **Install libraries and dependencies**\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "y3dczlxhXwe6"
      },
      "source": [
        "### Run this cell if you haven’t previously install Pix2PixHD in your Google Drive"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bQbRsmWdvjUo"
      },
      "source": [
        "%cd /content/drive/My\\ Drive\n",
        "!mkdir nfp-colab\n",
        "%cd nfp-colab\n",
        "!git clone -b video https://github.com/dvschultz/pix2pixHD.git\n",
        "!pip install dominate\n",
        "%cd pix2pixHD/"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-yDOMtTH42xl"
      },
      "source": [
        "### Run this cell if you’ve already installed Pix2PixHD in Google Drive before"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "V94ah70BxPUn"
      },
      "source": [
        "%cd /content/drive/My\\ Drive/nfp-colab/pix2pixHD/\n",
        "# !git pull\n",
        "!pip install dominate\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nzLGf05WXMFV"
      },
      "source": [
        "## Extract frames from video\n",
        "\n",
        "Upload a video to either Colab or Google Drive.\n",
        "\n",
        "* `-video` is the path to the video file\n",
        "* `-name` should be a unique name (think of it like a project name)\n",
        "* `-width` and `-height` **must** to be a multiple of 32\n",
        "* `-p2pdir` leave as `.` unless you know it shouldn’t be that ;)\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lWRL2ty6N9LD"
      },
      "source": [
        "!python extract_frames.py -video /content/1961.mp4 -name 1961 -p2pdir . -width 1280 -height 736"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qL9BZkBA_QRR"
      },
      "source": [
        "## Train your model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4X7qahzMX05u"
      },
      "source": [
        "### Initial training\n",
        "\n",
        "Note: if you have a large dataset, this may timeout initially (`ValueError: __len__() should return >= 0`). Give it a minute or two and run it again.\n",
        "\n",
        "*   `--name` should be a unique name (think of it like a project name). For your sanity I recommend using the same `-name` as above.\n",
        "*   `--dataroot` should point to your dataset. This will usually be in `./datasets/[your project name]`\n",
        "\n",
        " \n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fzHBGVnUKEzE"
      },
      "source": [
        "!python train_video.py --name chiquitettas2 --dataroot ./datasets/chiquitettas2/ --save_epoch_freq 5 #--continue_train"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0GDg3CeW_1TD"
      },
      "source": [
        "### Continue Training\n",
        "To pick up from a previous training session run the same command you ran to start with and append `--continue_train` to the end of the command."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F5q3dE9S_5eg"
      },
      "source": [
        "!python train_video.py --name chiquitettas2 --dataroot ./datasets/chiquitettas2/ --save_epoch_freq 1 --continue_train"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jly_3OyBoGg2"
      },
      "source": [
        "#Generating Videos\n",
        "\n",
        "To generate a video from your completed model, run the `generate_video.py` script. I recommend keeping the `--name` and `--dataroot` arguments the same.\n",
        "\n",
        "There are a number of additional arguments you’ll need to include in this command:\n",
        "\n",
        "\n",
        "*   `--fps` frame rate for your video\n",
        "*   `--how_many` how many frames you want to create (this + fps = video length)\n",
        "*   `--which_epoch` which epoch you want to generate videos from (note: if you choose `133` but there’s no epoch 133 model file, this will throw an error)\n",
        "*   `--start_from` by default your video will start predicting images from the 60th frame of your training set. You can pass in the path to a different frame to have it start from that frame\n",
        "\n",
        "I recommend playing with both the `--which_epoch` and `--start_from` arguments as you can get dramatically different results by changing in the inputs here.\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "INVUtG-pt_F6"
      },
      "source": [
        "!python generate_video.py --name chiquitettas2 --dataroot ./datasets/chiquitettas2/ --fps 24 --how_many 600 --which_epoch latest"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K1ed1wiF1SMq"
      },
      "source": [
        "import os\n",
        "\n",
        "def exportMultiple():\n",
        "  for epoch in range(10, 200, 10):\n",
        "    command = 'python generate_video.py --name chiquitettas2 --dataroot ./datasets/chiquitettas2/ --fps 24 --how_many 600 --which_epoch {e}'.format(e = epoch)\n",
        "    print('running: ' + command)\n",
        "    os.system(command)\n",
        "\n",
        "exportMultiple()"
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}