update

Sandeep42 · Sep 17, 2024 · 70256bd · 70256bd
1 parent 1c30718
commit 70256bd
Show file tree

Hide file tree

Showing 2 changed files with 12 additions and 6 deletions.
diff --git a/machine_learning_with_python/05.llms.ipynb b/machine_learning_with_python/05.llms.ipynb
@@ -1479,12 +1479,15 @@
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "1334a233-14af-4c60-be1e-59a8f28163bc",
+   "cell_type": "markdown",
+   "id": "ff97b77a-b92e-49e0-b64a-8afc9f803935",
    "metadata": {},
-   "outputs": [],
-   "source": []
+   "source": [
+    "* LLama models change positional embeddings to RoPE: RoPE incorporates position information directly into the attention mechanism by rotating the input embeddings in a way that allows the model to capture relative positional relationships, rather than absolute ones. This makes the model more efficient at handling long-range dependencies.\n",
+    "* LLama uses prenormalization unlike GPT.\n",
+    "* Custom tokenizer with support for tokens in other langugages.\n",
+    "* Large context length with the recent Llama3: 128k"
+   ]
   }
  ],
  "metadata": {

diff --git a/requirements.txt b/requirements.txt
@@ -3,4 +3,7 @@ matplotlib
 numpy
 ghp-import
 scipy
-scikit-learn
+scikit-learn
+torch
+langchain_core
+langchain_groq