Download_Model_Convert_to_NeMo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e4c9fc31-b77e-4007-8901-d186024675cd",
   "metadata": {},
   "source": [
    "# Step 2: Download and Convert the Base Model\n",
    "\n",
    "This notebook performs the preparatory tasks needed for obtaining the base model that we will use for fine-tuning.\n",
    "\n",
    "## Setup and Requirements\n",
    "Before proceeding, you need to install one dependency to follow along. Execute the following cell before getting started."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "3db9f6f9-b0c3-4c15-92d4-5c7ff0e5a286",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com\n",
      "Requirement already satisfied: huggingface-hub in /usr/local/lib/python3.10/dist-packages (0.26.1)\n",
      "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub) (3.15.4)\n",
      "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub) (2024.5.0)\n",
      "Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub) (24.0)\n",
      "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub) (6.0.1)\n",
      "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from huggingface-hub) (2.32.3)\n",
      "Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub) (4.66.5)\n",
      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub) (4.12.0)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub) (3.4.0)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub) (3.7)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub) (1.26.20)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub) (2024.7.4)\n",
      "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.\u001b[0m\u001b[33m\n",
      "\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.1.2\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.3.1\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpython -m pip install --upgrade pip\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "! pip install huggingface-hub"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fccfc187",
   "metadata": {},
   "source": [
    "Please run the following cell to incorporate patches required for this hackathon material."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "11920d0b",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "--2024-10-28 00:39:22--  https://raw.githubusercontent.com/NVIDIA/NeMo/721f9101f92c9dc3976689b2bf45bad1d5075d07/nemo/collections/nlp/models/language_modeling/megatron/gemma2/gemma2_modules.py\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.109.133, 185.199.108.133, ...\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 10842 (11K) [text/plain]\n",
      "Saving to: ‘/opt/NeMo/nemo/collections/nlp/models/language_modeling/megatron/gemma2/gemma2_modules.py’\n",
      "\n",
      "     0K ..........                                            100%  100M=0s\n",
      "\n",
      "2024-10-28 00:39:22 (100 MB/s) - ‘/opt/NeMo/nemo/collections/nlp/models/language_modeling/megatron/gemma2/gemma2_modules.py’ saved [10842/10842]\n",
      "\n",
      "--2024-10-28 00:39:22--  https://raw.githubusercontent.com/NVIDIA/NeMo/721f9101f92c9dc3976689b2bf45bad1d5075d07/scripts/checkpoint_converters/convert_gemma2_hf_to_nemo.py\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.110.133, 185.199.109.133, ...\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 13502 (13K) [text/plain]\n",
      "Saving to: ‘/opt/NeMo/scripts/checkpoint_converters/convert_gemma2_hf_to_nemo.py’\n",
      "\n",
      "     0K .......... ...                                        100% 15.9M=0.001s\n",
      "\n",
      "2024-10-28 00:39:22 (15.9 MB/s) - ‘/opt/NeMo/scripts/checkpoint_converters/convert_gemma2_hf_to_nemo.py’ saved [13502/13502]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "\n",
    "wget -O /opt/NeMo/nemo/collections/nlp/models/language_modeling/megatron/gemma2/gemma2_modules.py \\\n",
    "        https://raw.githubusercontent.com/NVIDIA/NeMo/721f9101f92c9dc3976689b2bf45bad1d5075d07/nemo/collections/nlp/models/language_modeling/megatron/gemma2/gemma2_modules.py\n",
    "\n",
    "\n",
    "wget -O /opt/NeMo/scripts/checkpoint_converters/convert_gemma2_hf_to_nemo.py \\\n",
    "        https://raw.githubusercontent.com/NVIDIA/NeMo/721f9101f92c9dc3976689b2bf45bad1d5075d07/scripts/checkpoint_converters/convert_gemma2_hf_to_nemo.py"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74332120-9e1a-4a08-96bc-b56308d3eeb7",
   "metadata": {},
   "source": [
    "\n",
    "### HuggingFace API Key\n",
    "\n",
    "Next, please specify your HuggingFace API token. This token will be used for downloading the base model.\n",
    "If you do not have an access token, follow [this tutorial](https://huggingface.co/docs/hub/en/security-tokens#how-to-manage-user-access-tokens) to generate one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "68abb5ee",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Configure your HuggingFace token here.\n",
    "HF_TOKEN=\"HF TOKEN\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aeb293b5",
   "metadata": {},
   "source": [
    "## Base Model Specificaions\n",
    "\n",
    "For this work, we will use the Gemma-2-2B model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b93dc17a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "model_to_use = \"google/gemma-2-2b\"\n",
    "model_name = model_to_use.split('/')[-1].lower()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2efcaaeb-59d6-4a71-a4fe-eb577a6b439c",
   "metadata": {},
   "source": [
    "---\n",
    "## Download the Base Model from Hugging Face"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "621ad3a5-3050-407e-bcd4-3644d638450a",
   "metadata": {},
   "source": [
    "Gemma 2 is a gated model on Hugging Face. To download it, you must first request access to the model by following the link [here](https://huggingface.co/google/gemma-2-2b-it)\n",
    "\n",
    "After access has been granted, run the following cell to download the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "d9e353d3-9232-4538-968d-882ea44ccd15",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "04d1802d04e74814bee8256e5a19714e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Fetching 12 files:   0%|          | 0/12 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ebaf57954a5b4ad199e3bbe3b4c399aa",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       ".gitattributes:   0%|          | 0.00/1.57k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9ebe4c3402214105aef7976b3a53e150",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "model.safetensors.index.json:   0%|          | 0.00/24.2k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3790e7008a834eb388b1b17744907297",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "generation_config.json:   0%|          | 0.00/168 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d65a1e7f61f94548bc1c7dbeb0975e1b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "model-00002-of-00003.safetensors:   0%|          | 0.00/4.98G [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d81153a757d14608b8c83628ccbc2888",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "model-00001-of-00003.safetensors:   0%|          | 0.00/4.99G [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6c8bf605ddbb40539eaf5868c5f70c25",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "README.md:   0%|          | 0.00/25.8k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "27af14a25b674c9087e7a0568aa5d19d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "special_tokens_map.json:   0%|          | 0.00/636 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b69d1c5b96754b9fa6e58ba153dc7eea",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "model-00003-of-00003.safetensors:   0%|          | 0.00/481M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3b2118abeaa94d87984421dfb20d6ec1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "config.json:   0%|          | 0.00/818 [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a6f0ea667b2e4a3b9b098e6a7d4b0cc2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer.model:   0%|          | 0.00/4.24M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "db54cebd31ba4e95b4975b589c752a6f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer.json:   0%|          | 0.00/17.5M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f611351a6ed14477ad7ab3121cd2811f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "tokenizer_config.json:   0%|          | 0.00/46.4k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "'/root/ODSC-Hackathon-Repository/models/gemma-2-2b'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from huggingface_hub import login, snapshot_download\n",
    "\n",
    "# Download the model\n",
    "login(token=HF_TOKEN)\n",
    "snapshot_download(repo_id=model_to_use, local_dir=f\"models/{model_name}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be275449",
   "metadata": {},
   "source": [
    "---\n",
    "## Convert the Model to `.nemo` Format\n",
    "\n",
    "To use the downloaded model with the NeMo Framework, we need to convert the checkpoint to the NeMo format. This is done thorugh a helper script that is already provided in the NeMo Framework container.\n",
    "\n",
    "> NOTE: The conversion is a one-time process and may take some time to finish.\n",
    "\n",
    "To begin the conversion process, execute the following cell.\n",
    "\n",
    "> NOTE: If you encounter any errors during the conversion process, a log file named `model_conversion.log` will be produced in the current working directory. Please include this file when filing support requests."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "3bd666e8",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model conversion started. This will take some time...\n",
      "Model conversion completed successfully!\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import subprocess\n",
    "\n",
    "command = f\"\"\"\n",
    "python3 /opt/NeMo/scripts/checkpoint_converters/convert_gemma2_hf_to_nemo.py \\\\\n",
    "    --input_name_or_path ./models/{model_name} \\\\\n",
    "    --tokenizer_path ./models/{model_name}/tokenizer.model \\\\\n",
    "    --output_path ./models/{model_name}.nemo \\\\\n",
    "    --run_verification\n",
    "\"\"\"\n",
    "\n",
    "# The log file to capture any messages or errors from the conversion process\n",
    "log_filename = \"model_conversion.log\"\n",
    "\n",
    "if os.path.exists(log_filename):\n",
    "    os.remove(log_filename)\n",
    "\n",
    "print(\"Model conversion started. This will take some time...\")\n",
    "process = subprocess.run(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)\n",
    "\n",
    "# Did the conversion process succeed?\n",
    "if process.returncode == 0:\n",
    "    print(\"Model conversion completed successfully!\")\n",
    "else:\n",
    "    # Write the logs to a file\n",
    "    with open(log_filename, \"w\") as f:\n",
    "        f.write(process.stdout)\n",
    "\n",
    "    print(f\"Model conversion failed!\")\n",
    "    print(f\"{'#'*80}\\nLogs:\\n{'#'*80}\")\n",
    "    print(process.stdout)\n",
    "    print(f\"{'#'*80}\\n\")\n",
    "    print(f\"Logs also saved to '{os.path.abspath(log_filename)}'.\\nPlease share this file with us if you need help debugging the issue.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17f8c3f4",
   "metadata": {},
   "source": [
    "---\n",
    "# Freeing Memory and Other Resources\n",
    "\n",
    "Before moving to the next notebook, please execute the following cell to free up all the allocated resources to avoid running into out-of-memory or other issues.\n",
    "\n",
    "Alternatively, please restart the kernel by navigating to `Kernel > Restart Kernel` (if using Jypyter notebook), or clicking the `Restart` button in VS Code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "0c590d6c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "exit(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5cd9c119-ae84-4db6-a87b-81175c2789c4",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}