Skip to content

Commit

Permalink
add title number for pytorch
Browse files Browse the repository at this point in the history
  • Loading branch information
@hipudding
hipudding committed Jul 18, 2024
1 parent b1057e4 commit d26cc93
Show file tree
Hide file tree
Showing 5 changed files with 23 additions and 28 deletions.
Binary file modified _static/images/pytorch_wechat.jpg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 2 additions & 2 deletions _static/pytorch_actions.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -80,9 +80,9 @@ docker run \\
$('#install-pytorch-docker-section').hide();
$('#install-pytorch-pip-section').show();
} else {
$("#codecell4").html("# install requirements\nconda install cmake ninja git\n\n# get torch source\ngit clone -b v"+options['pytorch']+" --recursive https://github.com/pytorch/pytorch\ncd pytorch\ngit submodule sync\ngit submodule update --init --recursive\n\n# install torch\npip install -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt\nexport CMAKE_PREFIX_PATH=${CONDA_PREFIX:-\"$(dirname $(which conda))/../\"}\nUSE_CUDA=0 python setup.py develop");
$("#codecell3").html("# install requirements\nconda install cmake ninja git\n\n# get torch source\ngit clone -b v"+options['pytorch']+" --recursive https://github.com/pytorch/pytorch\ncd pytorch\ngit submodule sync\ngit submodule update --init --recursive\n\n# install torch\npip install -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt\nexport CMAKE_PREFIX_PATH=${CONDA_PREFIX:-\"$(dirname $(which conda))/../\"}\nUSE_CUDA=0 python setup.py develop");

$('#codecell4').append("\n\n# get torch-npu source\ngit clone https://github.com/ascend/pytorch.git -b "+match_versions['npu_branch']+" --depth 1 pytorch_npu\ncd pytorch_npu\n\n# install torch-npu\npip install -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt\nbash ci/build.sh --python=$(python --version 2>&1 | awk '{print $2}' | cut -d '.' -f 1,2)\npip install dist/torch_npu*.whl");
$('#codecell3').append("\n\n# get torch-npu source\ngit clone https://github.com/ascend/pytorch.git -b "+match_versions['npu_branch']+" --depth 1 pytorch_npu\ncd pytorch_npu\n\n# install torch-npu\npip install -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt\nbash ci/build.sh --python=$(python --version 2>&1 | awk '{print $2}' | cut -d '.' -f 1,2)\npip install dist/torch_npu*.whl");

$('#install-pytorch-pip-section').hide();
$('#install-pytorch-docker-section').hide();
Expand Down
10 changes: 5 additions & 5 deletions sources/pytorch/examples.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,13 @@

在运行下述示例之前，需要您已经安装了PyTorch-NPU环境，有关环境安装，请参考 :doc:`./install`

数据并行
1. 数据并行
-----------------------

PyTorch的数据并行主要分为以下几种：DP、DDP以及FSDP（HSDP变种），接下来将简单描述在Ascend NPU场景下如何实现上述数据并行。

DDP
^^^^^^
1.1 DDP
^^^^^^^^^^

.. code-block:: python
:linenos:
Expand Down Expand Up @@ -83,8 +83,8 @@ DDP
if __name__ == "__main__":
main()
FSDP
^^^^^^
1.2 FSDP
^^^^^^^^^^

.. code-block:: python
:linenos:
Expand Down
29 changes: 12 additions & 17 deletions sources/pytorch/install.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
跟随指导，安装在NPU上运行的PyTorch版本。


选择需要安装的 PyTorch 版本
1. 选择需要安装的 PyTorch 版本
------------------------------
准备安装 PyTorch：

Expand Down Expand Up @@ -50,8 +50,12 @@
</div>


安装 PyTorch
2. 安装 PyTorch
----------------

.. warning::

如果使用了非CANN安装时的Python环境（如Conda），请确保CANN-toolkit依赖的Python包在该环境中已经 `安装 <../ascend/quick_install.html>`_ 。
.. raw:: html

<section id="install-pytorch-docker-section">
Expand All @@ -78,37 +82,28 @@
<p class="admonition-title">备注</p>
<p>请确保已经根据上述表格建议<a class="reference internal" href="../ascend/quick_install.html"><span class="doc">安装</span></a>了对应的CANN-toolkit版本以及相应的固件和驱动，并应用了CANN-toolkit环境变量。</p>
</div>
<h3>环境依赖</h3>
<ol>
<h3>2.1 环境依赖</h3>
<ul>
<li>Python 3.8 ~ 3.10</li>
<li>支持C++17的编译器，例如clang 或者 gcc (9.4.0及以上)</li>
<li><a class="reference internal" href="https://docs.anaconda.com/free/miniconda/#quick-command-line-install"><span class="doc">Conda (可选)</span></a></li>
</ol>
<div class="admonition note">
<p class="admonition-title">备注</p>
<p>如果使用Conda，请确保CANN-toolkit依赖的Python包在Conda环境中已经<a class="reference internal" href="../ascend/quick_install.html"><span class="doc">安装</span></a></p>
</div>
<div class="admonition warning">
<p class="admonition-title">警告</p>
<p>不建议您使用Conda提供的gcc，可能需要解决一些编译依赖问题，如果一定要使用，可参考以下命令：</p>
</div>
<div class="highlight"><pre>conda install -c conda-forge libstdcxx-ng libgcc-ng gcc=9.4 gxx=9.4</pre></div>
<li><a class="reference internal" href="https://docs.anaconda.com/free/miniconda/#quick-command-line-install"><span class="doc">Conda</span></a></li>
</ul>
</section>
<section>
<div class="admonition note">
<p class="admonition-title">备注</p>
<p>请确认CXX11_ABI是关闭的，如果无法确定，建议显式关闭：</p>
</div>
<div class="highlight"><pre>export _GLIBCXX_USE_CXX11_ABI=0</pre></div>
<h3>构建</h3>
<h3>2.2 构建</h3>
<div class="highlight">
<pre></pre>
</div>
</section>
</div>


验证安装结果
3. 验证安装结果
------------------

.. code-block:: python
Expand Down
8 changes: 4 additions & 4 deletions sources/pytorch/quick_start.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
下面的实例演示了如何使用NPU进行训练和推理任务：

单卡训练
1. 单卡训练
-----------------------
以下代码使用了cifar10数据集在NPU上训练模型（截取自 `PyTorch tutorials <https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html>`_），请关注高亮的内容。

Expand Down Expand Up @@ -211,7 +211,7 @@
accuracy = 100 * float(correct_count) / total_pred[classname]
print(f'Accuracy for class: {classname:5s} is {accuracy:.1f} %')
使用DeepSpeed多卡并行训练
2. 使用DeepSpeed多卡并行训练
-------------------------------
以下代码使用了cifar10数据集，使用DeepSpeed训练模型在多张NPU卡上进行模型训练（来自 `DeepSpeed Examples <https://github.com/microsoft/DeepSpeedExamples/blob/master/training/cifar/cifar10_deepspeed.py>`_），自DeepSpeed v0.12.6之后，代码无需任何修改，即可自动检测NPU并进行训练。

Expand All @@ -220,7 +220,7 @@
:linenos:


使用Transforms进行模型微调
3. 使用Transforms进行模型微调
---------------------------------
以下代码使用了Transforms对LLM进行微调（来自 `transforms examples <https://github.com/huggingface/transformers/blob/main/examples/pytorch/language-modeling/run_clm.py>`_），自transforms xxx版本以及accelerator 0.21.0版本以后，代码无需任何修改，即可自动检测NPU并进行。

Expand All @@ -242,7 +242,7 @@
--do_eval \
--output_dir /tmp/test-clm
使用Diffusers进行模型微调
4. 使用Diffusers进行模型微调
---------------------------------
以下代码使用了Diffusers对文生图模型进行微调（来自 `diffusers examples <https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image.py>`_），自diffusers v0.27.0版本以后，代码无需任何修改，即可自动检测NPU并进行。

Expand Down

0 comments on commit d26cc93

Please sign in to comment.