[Wait for 2759][GPU/OpenCL] Initial version of LM Head layer with OpenCl ops and updated Addition Layer with latest pipeline changes. @open sesame 12/02 16:21 #2752

yashSingh0723 · 2024-10-09T09:56:24Z

Added initial version of LM Head Layer for GPU. Also Updated the addition layer for GPU with latest pipeline changes.
Changes added with this PR:

lm_head_layer_cl.cpp added containing the new CustomLMHeadLayerCl class for OpenCL implementation.
Updated with new GPU pipeline changes
Added unittest_layers_lm_head_cl.cpp to test LM Head Layer on GPU.
Removed cl_context dependencies for addition layer

Signed-off-by: Yash Singh [email protected]

taos-ci · 2024-10-09T09:56:27Z

📝 TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #2752. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/.

taos-ci · 2024-10-09T09:56:30Z

cibot: @yashSingh0723, nntrainer/layers/cl_layers/lm_head_layer_cl.h does not include Doxygen tags such as @file @brief @author @bug. You must include the Doxygen tags in the source code. Please refer to a Doxygen manual at http://github.com/nnstreamer/TAOS-CI/blob/main/ci/doc/doxygen-documentation.md

taos-ci · 2024-10-09T10:11:50Z

cibot: @yashSingh0723, nntrainer/layers/cl_layers/lm_head_layer_cl.h does not include Doxygen tags such as @file @brief @author @bug. You must include the Doxygen tags in the source code. Please refer to a Doxygen manual at http://github.com/nnstreamer/TAOS-CI/blob/main/ci/doc/doxygen-documentation.md

taos-ci · 2024-10-09T10:32:30Z

cibot: @yashSingh0723, nntrainer/layers/cl_layers/lm_head_layer_cl.h does not include Doxygen tags such as @file @brief @author @bug. You must include the Doxygen tags in the source code. Please refer to a Doxygen manual at http://github.com/nnstreamer/TAOS-CI/blob/main/ci/doc/doxygen-documentation.md

taos-ci

@yashSingh0723, 💯 All CI checkers are successfully verified. Thanks.

EunjuYang

I'm glad to see the implementation of the key blocks for LLM in opencl version.
Here are some opinions in my side:

nntrainer/layers/cl_layers/custom_vocab_selection.cpp

nntrainer/layers/cl_layers/lm_head_layer_cl.cpp

test/input_gen/gen_layer_tests.py

nntrainer/utils/custom_properties.h

taos-ci

@yashSingh0723, 💯 All CI checkers are successfully verified. Thanks.

taos-ci · 2024-11-22T10:11:22Z

cibot: @yashSingh0723, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://ci.nnstreamer.ai/nntrainer/ci/repo-workers/pr-checker/2752-202411221838150.57073402404785-0b9ff2aa74fcfcdb01e1dff03a229df8f4457b55/.

taos-ci · 2024-11-22T12:18:30Z

cibot: @yashSingh0723, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://ci.nnstreamer.ai/nntrainer/ci/repo-workers/pr-checker/2752-202411222045320.65086197853088-57ae5ed2a75efcba64c20bad9b312233da9d6a95/.

taos-ci · 2024-11-25T07:10:21Z

cibot: @yashSingh0723, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://ci.nnstreamer.ai/nntrainer/ci/repo-workers/pr-checker/2752-202411251537140.93050694465637-5c0e60a0a35fdc17a75ad5e5420b9086781d56df/.

taos-ci · 2024-12-02T11:27:10Z

cibot: @yashSingh0723, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://ci.nnstreamer.ai/nntrainer/ci/repo-workers/pr-checker/2752-202412021954220.7212700843811-d0b5e1c22ba9c2426ecf23fb18344942573209b0/.

taos-ci

@yashSingh0723, 💯 All CI checkers are successfully verified. Thanks.

EunjuYang

LGTM

djeong20 · 2024-12-04T02:11:55Z

api/ccapi/include/layer.h

@@ -108,7 +108,8 @@ enum LayerType {
  LAYER_UPSAMPLE2D,               /**< Upsample 2D Layer type */
  LAYER_RMSNORM = ML_TRAIN_LAYER_TYPE_RMSNORM,     /**<RMS NORM Layer */
  LAYER_TRANSPOSE = ML_TRAIN_LAYER_TYPE_TRANSPOSE, /**< Transpose Layer type */
-  LAYER_UNKNOWN = ML_TRAIN_LAYER_TYPE_UNKNOWN      /**< Unknown */
+  LAYER_UNKNOWN = ML_TRAIN_LAYER_TYPE_UNKNOWN,     /**< Unknown */
+  LAYER_LM_HEAD = ML_TRAIN_LAYER_TYPE_LM_HEAD,     /**< LM Head Layer */


This is not a big deal, but I think setting the LAYER_UNKNOWN at the end would be better.

djeong20 · 2024-12-04T02:13:57Z

nntrainer/layers/cl_layers/lm_head_layer_cl.h

+  /**
+   * @copydoc bool supportBackwarding() const
+   */
+  bool supportBackwarding() const override { return true; };


Suggested change

bool supportBackwarding() const override { return true; };

bool supportBackwarding() const override { return false; };

Updated in the latest commit, Thank you for pointing it out.

s-debadri

LGTM

… and Update Addition Layer on GPU with latest Pipeline changes Added initial version of LM head layer fpr GPU and removed dependencies of cl_context for addition_layer. Signed-off-by: Yash Singh <[email protected]>

Clang format fixed and comments removed. Rebased PR with latest changes. Signed-off-by: Yash Singh <[email protected]>

Addressed multiple review comments Removed unnecessary code Added nnlayer golden files Signed-off-by: Yash Singh <[email protected]>

Reviewed PR comments SupportBackwarding() set to false Signed-off-by: Yash Singh <[email protected]>

yashSingh0723 requested review from myungjoo, jijoongmoon, again4you, jaeyun-jung, leemgs, wooksong, helloahn, kparichay, gichan-jang, anyj0527, zhoonit, lhs8928, songgot, jihochu, DonghakPark, SeoHyungjun, baek2sm, skykongkong8, djeong20, EunjuYang and a team as code owners October 9, 2024 09:56

github-actions bot added the Need Review label Oct 9, 2024

yashSingh0723 force-pushed the opencl_lmhead_layer branch from 781e16e to 8bd85e8 Compare October 9, 2024 10:11

yashSingh0723 force-pushed the opencl_lmhead_layer branch from 8bd85e8 to 210d217 Compare October 9, 2024 10:32

taos-ci approved these changes Nov 18, 2024

View reviewed changes

EunjuYang reviewed Nov 19, 2024

View reviewed changes

taos-ci approved these changes Nov 20, 2024

View reviewed changes

yashSingh0723 force-pushed the opencl_lmhead_layer branch 3 times, most recently from 27cf3d5 to 0b9ff2a Compare November 22, 2024 09:38

yashSingh0723 force-pushed the opencl_lmhead_layer branch from 0b9ff2a to 57ae5ed Compare November 22, 2024 11:45

yashSingh0723 force-pushed the opencl_lmhead_layer branch from 57ae5ed to 5c0e60a Compare November 25, 2024 06:37

yashSingh0723 force-pushed the opencl_lmhead_layer branch from 5c0e60a to d0b5e1c Compare December 2, 2024 10:54

yashSingh0723 force-pushed the opencl_lmhead_layer branch from d0b5e1c to 2d8c2e8 Compare December 3, 2024 06:45

taos-ci approved these changes Dec 3, 2024

View reviewed changes

EunjuYang approved these changes Dec 4, 2024

View reviewed changes

djeong20 reviewed Dec 4, 2024

View reviewed changes

s-debadri approved these changes Dec 4, 2024

View reviewed changes

github-actions bot added PR/READY2MERGE and removed Need Review labels Dec 4, 2024

yashSingh0723 added 3 commits December 6, 2024 11:10

[Trivial] Clang format fixed and Rebased the PR

5a55098

Clang format fixed and comments removed. Rebased PR with latest changes. Signed-off-by: Yash Singh <[email protected]>

[Update] Removed SmartReply property and added nnlayergolden files

0c9e7aa

Addressed multiple review comments Removed unnecessary code Added nnlayer golden files Signed-off-by: Yash Singh <[email protected]>

yashSingh0723 force-pushed the opencl_lmhead_layer branch from 2d8c2e8 to 0c9e7aa Compare December 6, 2024 05:42

[Update] Backwarding support set to false

d08a0ec

Reviewed PR comments SupportBackwarding() set to false Signed-off-by: Yash Singh <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Wait for 2759][GPU/OpenCL] Initial version of LM Head layer with OpenCl ops and updated Addition Layer with latest pipeline changes. @open sesame 12/02 16:21 #2752

[Wait for 2759][GPU/OpenCL] Initial version of LM Head layer with OpenCl ops and updated Addition Layer with latest pipeline changes. @open sesame 12/02 16:21 #2752

yashSingh0723 commented Oct 9, 2024 •

edited

Loading

taos-ci commented Oct 9, 2024

taos-ci commented Oct 9, 2024

taos-ci commented Oct 9, 2024

taos-ci commented Oct 9, 2024

taos-ci left a comment

EunjuYang left a comment

taos-ci left a comment

taos-ci commented Nov 22, 2024

taos-ci commented Nov 22, 2024

taos-ci commented Nov 25, 2024

taos-ci commented Dec 2, 2024

taos-ci left a comment

EunjuYang left a comment

djeong20 Dec 4, 2024

djeong20 Dec 4, 2024

yashSingh0723 Dec 6, 2024

s-debadri left a comment

	bool supportBackwarding() const override { return true; };
	bool supportBackwarding() const override { return false; };

[Wait for 2759][GPU/OpenCL] Initial version of LM Head layer with OpenCl ops and updated Addition Layer with latest pipeline changes. @open sesame 12/02 16:21 #2752

Are you sure you want to change the base?

[Wait for 2759][GPU/OpenCL] Initial version of LM Head layer with OpenCl ops and updated Addition Layer with latest pipeline changes. @open sesame 12/02 16:21 #2752

Conversation

yashSingh0723 commented Oct 9, 2024 • edited Loading

taos-ci commented Oct 9, 2024

taos-ci commented Oct 9, 2024

taos-ci commented Oct 9, 2024

taos-ci commented Oct 9, 2024

taos-ci left a comment

Choose a reason for hiding this comment

EunjuYang left a comment

Choose a reason for hiding this comment

taos-ci left a comment

Choose a reason for hiding this comment

taos-ci commented Nov 22, 2024

taos-ci commented Nov 22, 2024

taos-ci commented Nov 25, 2024

taos-ci commented Dec 2, 2024

taos-ci left a comment

Choose a reason for hiding this comment

EunjuYang left a comment

Choose a reason for hiding this comment

djeong20 Dec 4, 2024

Choose a reason for hiding this comment

djeong20 Dec 4, 2024

Choose a reason for hiding this comment

yashSingh0723 Dec 6, 2024

Choose a reason for hiding this comment

s-debadri left a comment

Choose a reason for hiding this comment

yashSingh0723 commented Oct 9, 2024 •

edited

Loading