add sentence-transformers and trl docs (#52)

Ascend · Oct 16, 2024 · 641ac35 · 641ac35
1 parent 79b3369
commit 641ac35
Show file tree

Hide file tree

Showing 9 changed files with 217 additions and 0 deletions.
diff --git a/_static/images/trl.png b/_static/images/trl.png
diff --git a/index.rst b/index.rst
@@ -33,6 +33,8 @@
    sources/wenet/index.rst
    sources/whisper_cpp/index.rst
    sources/llama_cpp/index.rst
+   sources/sentence_transformers/index.rst
+   sources/trl/index.rst
 
 .. warning::
 
@@ -282,5 +284,41 @@
                     <a href="sources/llama_cpp/quick_start.html">快速上手</a>
                 </div>
             </div>
+            <!-- Card 14 -->
+            <div class="box rounded-lg p-4 flex flex-col items-center">
+                <div class="flex items-center mb-4">
+                    <div class="img w-16 h-16 rounded-md mr-4" style="background-image: url('_static/images/huggingface.png')"></div>
+                    <div>
+                        <h2 class="text-lg font-semibold">Sentence Transformers</h2>
+                        <p class="text-gray-600 desc">适用于文本和图像的高性能Embedding库</p>
+                    </div>
+                </div>
+                <div class="flex-grow"></div>
+                <div class="flex space-x-4 text-blue-600">
+                    <a href="https://github.com/UKPLab/sentence-transformers">官方链接</a>
+                    <span class="split">|</span>
+                    <a href="sources/sentence_transformers/install.html">安装指南</a>
+                    <span class="split">|</span>
+                    <a href="sources/sentence_transformers/quick_start.html">快速上手</a>
+                </div>
+            </div>
+            <!-- Card 15 -->
+            <div class="box rounded-lg p-4 flex flex-col items-center">
+                <div class="flex items-center mb-4">
+                    <div class="img w-16 h-16 rounded-md mr-4" style="background-image: url('_static/images/trl.png')"></div>
+                    <div>
+                        <h2 class="text-lg font-semibold">Transformer Reinforcement Learning</h2>
+                        <p class="text-gray-600 desc">适用于SFT、PPO、DPO等方法的模型后训练库</p>
+                    </div>
+                </div>
+                <div class="flex-grow"></div>
+                <div class="flex space-x-4 text-blue-600">
+                    <a href="https://github.com/huggingface/trl">官方链接</a>
+                    <span class="split">|</span>
+                    <a href="sources/trl/install.html">安装指南</a>
+                    <span class="split">|</span>
+                    <a href="sources/trl/quick_start.html">快速上手</a>
+                </div>
+            </div>
         </div>
     </div>
diff --git a/sources/sentence_transformers/index.rst b/sources/sentence_transformers/index.rst
@@ -0,0 +1,8 @@
+Sentence Transformers
+=============================
+
+.. toctree::
+   :maxdepth: 2
+
+   install.rst
+   quick_start.rst
diff --git a/sources/sentence_transformers/install.rst b/sources/sentence_transformers/install.rst
@@ -0,0 +1,30 @@
+安装指南
+===============
+
+本教程面向使用 sentence-transformers & 昇腾的开发者，帮助完成昇腾环境下 sentence-transformers 的安装。
+
+昇腾环境安装
+---------------
+
+请根据已有昇腾产品型号及 CPU 架构等按照 :doc:`快速安装昇腾环境指引 <../ascend/quick_install>` 进行昇腾环境安装。
+
+.. warning::
+  CANN 最低版本为 8.0.rc1，安装 CANN 时，请同时安装 Kernel 算子包。
+
+sentence-transformers 下载安装
+---------------------------------------------------
+
+1. 安装项目所需依赖
+
+.. code-block:: shell
+
+    pip install sentence-transformers -i https://pypi.tuna.tsinghua.edu.cn/simple
+
+2. 安装 torch_npu
+
+.. code-block:: shell
+
+    pip install torch==2.1.0 torch_npu==2.1.0.post6 -i https://pypi.tuna.tsinghua.edu.cn/simple
+
+.. hint::
+  torch_npu 的版本需要匹配 torch 的版本，详细信息请参考：`Ascend Extension for PyTorch <https://github.com/Ascend/pytorch>`_。
diff --git a/sources/sentence_transformers/quick_start.rst b/sources/sentence_transformers/quick_start.rst
@@ -0,0 +1,46 @@
+快速开始
+===============
+
+.. note::
+
+    阅读本篇前，请确保已按照 :doc:`安装教程 <./install>` 准备好昇腾环境及 sentence-transformers ！
+
+本教程以 `all-MiniLM-L6-v2` 模型为例，讲述如何使用 sentence-transformers 在昇腾 NPU 上实现文本数据的 Embedding。
+
+前置准备
+---------------
+
+本篇样例代码为 sentence-transformers 的官方样例，需提前进行下载：
+
+.. code-block::
+
+    git clone https://github.com/UKPLab/sentence-transformers.git
+
+使用模型
+---------------
+
+进入 sentence-transformers 项目目录，依次执行如下命令：
+
+.. code-block::
+
+    cd examples/applications/computing-embeddings
+    python computing_embeddings.py
+
+出现如下日志则代表执行成功：
+
+::
+    
+    2024-10-15 08:11:36 - Use pytorch device_name: npu
+    2024-10-15 08:11:36 - Load pretrained SentenceTransformer: all-MiniLM-L6-v2
+    [W compiler_depend.ts:623] Warning: expandable_segments currently defaults to false. You can enable this feature by `export PYTORCH_NPU_ALLOC_CONF = expandable_segments:True`. (function operator())
+    Batches: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.61it/s]
+    Sentence: This framework generates embeddings for each input sentence
+    Embedding: [-0.01375547 -0.04301599 -0.01562478 ...  0.10029524  0.12379668 -0.04230832]
+
+    Sentence: Sentences are passed as a list of string.
+    Embedding: [ 0.05640831  0.05488579  0.03137118 ...  0.06652435  0.08493122 -0.03337045]
+
+    Sentence: The quick brown fox jumps over the lazy dog.
+    Embedding: [0.04393559 0.05903088 0.04824848 ... 0.05215353 0.05615513 0.10205095]
+
+可以看到该模型成功生成了这些句子对应的 Embedding 向量。
diff --git a/sources/trl/images/image.png b/sources/trl/images/image.png
diff --git a/sources/trl/index.rst b/sources/trl/index.rst
@@ -0,0 +1,8 @@
+Transformer Reinforcement Learning
+===================================================
+
+.. toctree::
+   :maxdepth: 2
+
+   install.rst
+   quick_start.rst
diff --git a/sources/trl/install.rst b/sources/trl/install.rst
@@ -0,0 +1,38 @@
+安装指南
+===============
+
+本教程面向使用 TRL (Transformer Reinforcement Learning) & 昇腾的开发者，帮助完成昇腾环境下 TRL 的安装。
+
+昇腾环境安装
+---------------
+
+请根据已有昇腾产品型号及 CPU 架构等按照 :doc:`快速安装昇腾环境指引 <../ascend/quick_install>` 进行昇腾环境安装。
+
+.. warning::
+  CANN 最低版本为 8.0.rc1，安装 CANN 时，请同时安装 Kernel 算子包。
+
+TRL 下载安装
+---------------
+
+1. 安装项目所需依赖
+
+.. code-block:: shell
+
+    pip install trl -i https://pypi.tuna.tsinghua.edu.cn/simple
+
+另外，本项目需要手动安装 transformers 仓库的最新 main 分支，否则可能会出现如下错误：
+
+.. image:: ./images/image.png
+
+.. code-block:: shell
+
+    pip install git+https://github.com/huggingface/transformers.git
+
+2. 安装 torch_npu
+
+.. code-block:: shell
+
+    pip install torch==2.1.0 torch_npu==2.1.0.post6 -i https://pypi.tuna.tsinghua.edu.cn/simple
+
+.. hint::
+  torch_npu 的版本需要匹配 torch 的版本，详细信息请参考：`Ascend Extension for PyTorch <https://github.com/Ascend/pytorch>`_。
diff --git a/sources/trl/quick_start.rst b/sources/trl/quick_start.rst
@@ -0,0 +1,49 @@
+快速开始
+===============
+
+.. note::
+
+    阅读本篇前，请确保已按照 :doc:`安装教程 <./install>` 准备好昇腾环境及 TRL (Transformer Reinforcement Learning) ！
+
+本教程以 DPO 方法为例，讲述如何使用 TRL 在昇腾 NPU 上进行模型的后训练。
+
+前置准备
+---------------
+
+本篇样例代码为 TRL 官方样例，需提前进行下载：
+
+.. code-block::
+
+    git clone https://github.com/huggingface/trl.git
+
+模型训练
+---------------
+
+进入 TRL 项目目录，依次执行如下命令：
+
+.. code-block::
+
+    cd examples/scripts
+    python dpo.py
+
+出现如下日志则代表训练成功：
+
+::
+    
+    Tokenizing train dataset: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 62135/62135 [07:11<00:00, 143.85 examples/s]
+    Tokenizing eval dataset: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1000/1000 [00:06<00:00, 144.73 examples/s]
+    Detected kernel version 4.19.90, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher.
+      0%| ... | 0/3883 [00:00<?, ?it/s]/home/sss/github/trl/trl/trainer/dpo_trainer.py:1355: UserWarning: AutoNonVariableTypeMode is deprecated and will be removed in 1.10 release. For kernel implementations please use AutoDispatchBelowADInplaceOrView instead, If you are looking for a user facing API to enable running your inference-only workload, please use c10::InferenceMode. Using AutoDispatchBelowADInplaceOrView in user code is under risk of producing silent wrong result in some edge cases. See Note [AutoDispatchBelowAutograd] for more details. (Triggered internally at build/CMakeFiles/torch_npu.dir/compiler_depend.ts:74.)
+      labels[labels == label_pad_token_id] = 0
+    Could not estimate the number of tokens of the input, floating-point operations will not be computed
+    {'loss': 0.6598, 'grad_norm': 84.0019760131836, 'learning_rate': 4.967808395570435e-07, 'rewards/chosen': -0.15474730730056763, 'rewards/rejected': -0.24127893149852753, 'rewards/accuracies': 0.5799999833106995, 'rewards/margins': 0.0865316167473793, 'logps/rejected': -283.3350830078125, 'logps/chosen': -322.8130187988281, 'logits/rejected': -2.3705289363861084, 'logits/chosen': -2.455843925476074, 'epoch': 0.01}
+    {'loss': 0.635, 'grad_norm': 88.99441528320312, 'learning_rate': 4.93561679114087e-07, 'rewards/chosen': -0.4325330853462219, 'rewards/rejected': -0.6220334768295288, 'rewards/accuracies': 0.625, 'rewards/margins': 0.1895003467798233, 'logps/rejected': -298.9117431640625, 'logps/chosen': -323.9031982421875, 'logits/rejected': -2.384589433670044, 'logits/chosen': -2.4548392295837402, 'epoch': 0.01}
+    {'eval_loss': 0.6398493647575378, 'eval_runtime': 66.0493, 'eval_samples_per_second': 15.14, 'eval_steps_per_second': 1.893, 'eval_rewards/chosen': -0.5118070840835571, 'eval_rewards/rejected': -0.6984029412269592, 'eval_rewards/accuracies': 0.6269999742507935, 'eval_rewards/margins': 0.18659590184688568, 'eval_logps/rejected': -314.8978271484375, 'eval_logps/chosen': -344.0274658203125, 'eval_logits/rejected': -2.242685556411743, 'eval_logits/chosen': -2.3058021068573, 'epoch': 0.01}
+    {'loss': 0.6284, 'grad_norm': 76.736572265625, 'learning_rate': 4.903425186711305e-07, 'rewards/chosen': -0.5593773722648621, 'rewards/rejected': -0.7793089151382446, 'rewards/accuracies': 0.6575000286102295, 'rewards/margins': 0.21993154287338257, 'logps/rejected': -305.3935241699219, 'logps/chosen': -327.55230712890625, 'logits/rejected': -2.3826913833618164, 'logits/chosen': -2.4632484912872314, 'epoch': 0.02}
+    {'loss': 0.6126, 'grad_norm': 84.96641540527344, 'learning_rate': 4.87123358228174e-07, 'rewards/chosen': -0.5430492162704468, 'rewards/rejected': -0.8355176448822021, 'rewards/accuracies': 0.6700000166893005, 'rewards/margins': 0.29246845841407776, 'logps/rejected': -327.23089599609375, 'logps/chosen': -353.1753845214844, 'logits/rejected': -2.42580509185791, 'logits/chosen': -2.513734817504883, 'epoch': 0.03}
+    {'eval_loss': 0.6248273253440857, 'eval_runtime': 66.049, 'eval_samples_per_second': 15.14, 'eval_steps_per_second': 1.893, 'eval_rewards/chosen': -0.4807929992675781, 'eval_rewards/rejected': -0.7240013480186462, 'eval_rewards/accuracies': 0.6439999938011169, 'eval_rewards/margins': 0.2432083934545517, 'eval_logps/rejected': -315.15380859375, 'eval_logps/chosen': -343.7173767089844, 'eval_logits/rejected': -2.2496635913848877, 'eval_logits/chosen': -2.3131723403930664, 'epoch': 0.03}
+    {'loss': 0.6089, 'grad_norm': 78.2168960571289, 'learning_rate': 4.839041977852176e-07, 'rewards/chosen': -0.4145514667034149, 'rewards/rejected': -0.7176669239997864, 'rewards/accuracies': 0.6600000262260437, 'rewards/margins': 0.3031154274940491, 'logps/rejected': -332.7577209472656, 'logps/chosen': -364.76898193359375, 'logits/rejected': -2.436467409133911, 'logits/chosen': -2.530369997024536, 'epoch': 0.03}
+    {'loss': 0.5769, 'grad_norm': 75.62930297851562, 'learning_rate': 4.806850373422611e-07, 'rewards/chosen': -0.6155031323432922, 'rewards/rejected': -1.012223720550537, 'rewards/accuracies': 0.7300000190734863, 'rewards/margins': 0.3967204988002777, 'logps/rejected': -291.11419677734375, 'logps/chosen': -331.4106750488281, 'logits/rejected': -2.378931999206543, 'logits/chosen': -2.4724509716033936, 'epoch': 0.04}
+    {'eval_loss': 0.6191915273666382, 'eval_runtime': 66.0551, 'eval_samples_per_second': 15.139, 'eval_steps_per_second': 1.892, 'eval_rewards/chosen': -0.7790046334266663, 'eval_rewards/rejected': -1.1102681159973145, 'eval_rewards/accuracies': 0.6460000276565552, 'eval_rewards/margins': 0.33126339316368103, 'eval_logps/rejected': -319.0165100097656, 'eval_logps/chosen': -346.699462890625, 'eval_logits/rejected': -2.2563016414642334, 'eval_logits/chosen': -2.3217742443084717, 'epoch': 0.04}
+    {'loss': 0.6159, 'grad_norm': 80.91998291015625, 'learning_rate': 4.774658768993046e-07, 'rewards/chosen': -0.7987264394760132, 'rewards/rejected': -1.1548289060592651, 'rewards/accuracies': 0.6225000023841858, 'rewards/margins': 0.3561025857925415, 'logps/rejected': -307.766357421875, 'logps/chosen': -319.15777587890625, 'logits/rejected': -2.369903326034546, 'logits/chosen': -2.430453300476074, 'epoch': 0.05}
+    {'loss': 0.5967, 'grad_norm': 87.26203155517578, 'learning_rate': 4.7424671645634816e-07, 'rewards/chosen': -0.6392844319343567, 'rewards/rejected': -1.015390396118164, 'rewards/accuracies': 0.699999988079071, 'rewards/margins': 0.3761059045791626, 'logps/rejected': -313.5637512207031, 'logps/chosen': -319.4141845703125, 'logits/rejected': -2.3648269176483154, 'logits/chosen': -2.4581611156463623, 'epoch': 0.05}