diff --git a/index.rst b/index.rst index f8e9f75..9420f39 100644 --- a/index.rst +++ b/index.rst @@ -22,6 +22,7 @@ sources/llamafactory/index.rst sources/accelerate/index.rst sources/transformers/index.rst + sources/deepspeed/index.rst sources/onnxruntime/index.rst sources/open_clip/index.rst sources/timm/index.rst diff --git a/sources/deepspeed/index.rst b/sources/deepspeed/index.rst new file mode 100644 index 0000000..29bff2a --- /dev/null +++ b/sources/deepspeed/index.rst @@ -0,0 +1,8 @@ +DeepSpeed +=========== + +.. toctree:: + :maxdepth: 2 + + install.rst + quick_start.rst diff --git a/sources/deepspeed/install.rst b/sources/deepspeed/install.rst new file mode 100644 index 0000000..58437a2 --- /dev/null +++ b/sources/deepspeed/install.rst @@ -0,0 +1,73 @@ +安装DeepSpeed +============== + +.. note:: + 在本示例之前,请确保已经安装了 `昇腾环境 <../ascend/quick_install.html>`_ 和 `PyTorch <../pytorch/install.html>`_ 环境。 + +1. 安装DeepSpeed +----------------- +安装DeepSpeed最简单的方式是通过 ``pip`` 。 + +.. code-block:: shell + :linenos: + + pip install deepspeed + + +2. 通过源码安装 +------------------ +从 `GitHub `_ 克隆DeepSpeed项目后,可以通过 ``pip`` 来通过源码编译。 + +.. code-block:: shell + :linenos: + + pip install . + + +3. 预编译DeepSpeed算子(可选) +---------------------------------- +如果不想使用JIT编译模式,而想要预编译DeepSpeed算子,可以通过设置环境变量的方式完成算子的预编译。 + +.. code-block:: shell + :linenos: + + DS_BUILD_OPS=1 pip install deepspeed + +4. 安装验证 +----------- + +安装完成后,可以通过 ``ds_report`` 命令查看安装结果 + +.. code-block:: shell + :linenos: + + -------------------------------------------------- + DeepSpeed C++/CUDA extension op report + -------------------------------------------------- + NOTE: Ops not installed will be just-in-time (JIT) compiled at + runtime if needed. Op compatibility means that your system + meet the required dependencies to JIT install the op. + -------------------------------------------------- + JIT compiled ops requires ninja + ninja .................. [OKAY] + -------------------------------------------------- + op name ................ installed .. compatible + -------------------------------------------------- + deepspeed_not_implemented [NO] ....... [OKAY] + async_io ............... [NO] ....... [OKAY] + cpu_adagrad ............ [NO] ....... [OKAY] + cpu_adam ............... [NO] ....... [OKAY] + cpu_lion ............... [NO] ....... [OKAY] + fused_adam ............. [NO] ....... [OKAY] + transformer_inference .. [NO] ....... [OKAY] + -------------------------------------------------- + DeepSpeed general environment info: + torch install path ............... ['/root/miniconda3/envs/ds/lib/python3.10/site-packages/torch'] + torch version .................... 2.2.0 + deepspeed install path ........... ['/root/miniconda3/envs/ds/lib/python3.10/site-packages/deepspeed'] + deepspeed info ................... 0.14.4, unknown, unknown + deepspeed wheel compiled w. ...... torch 2.2 + torch_npu install path ........... ['/root/miniconda3/envs/ds/lib/python3.10/site-packages/torch_npu'] + torch_npu version ................ 2.2.0 + ascend_cann version .............. 8.0.RC2.alpha002 + shared memory (/dev/shm) size .... 20.00 GB \ No newline at end of file diff --git a/sources/deepspeed/quick_start.rst b/sources/deepspeed/quick_start.rst new file mode 100644 index 0000000..00baae3 --- /dev/null +++ b/sources/deepspeed/quick_start.rst @@ -0,0 +1,34 @@ +快速开始 +========== + +.. note:: + 在本示例之前,请确保已经安装了 `DeepSpeed <./install.html>`_ 环境。 如果还未安装,可以执行 ``pip install deepspeed`` 完成安装。 + + +1. 使用DeepSpeed多卡并行训练 +------------------------------- +以下代码使用了cifar10数据集,使用DeepSpeed训练模型在多张NPU卡上进行模型训练(来自 `DeepSpeed Examples `_),自DeepSpeed v0.12.6之后,代码无需任何修改,即可自动检测NPU并进行训练。 + +.. rli:: https://raw.githubusercontent.com/microsoft/DeepSpeedExamples/master/training/cifar/cifar10_deepspeed.py + :language: python + :linenos: + +2. 训练结果查看 +---------------- +训练完成后,会打印模型对图像识别的结果。 + +.. code-block:: shell + :linenos: + + Finished Training + Accuracy of the network on the 10000 test images: 57 % + Accuracy of plane : 65 % + Accuracy of car : 67 % + Accuracy of bird : 52 % + Accuracy of cat : 34 % + Accuracy of deer : 52 % + Accuracy of dog : 49 % + Accuracy of frog : 59 % + Accuracy of horse : 66 % + Accuracy of ship : 66 % + Accuracy of truck : 56 %