[Wait for #2566] [ Layers ] Update Layers to support FP16 @open sesame 05/07 15:58 #2567
Conversation
We will add Var32 Tensor if the Variable Weight is not Full precision (FP32). This eables the Weight Update with full precision and only Apply Gradient Process ueses this Tensor. Therefore, the lifespan of this tensor should be "ApplyGradient". . Modify TensorPool to generate Weigth considering Mixed Precsion. **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: jijoong.moon <[email protected]>
This pr create the variable fp32 tensor when we create the Weight and Optimizer Weight. . update the manager to create Weight with var32 tensor which requested to weight pool. . update the weight requests with Weight Spec and var, grad and var32 tensors which created already. . add clone Tensor with specific type in tensor.h Resolves: **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: jijoong.moon <[email protected]>
This PR enables the FP16 support for the layers below: . input layer . mse loss layer Resolves: **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: jijoong.moon <[email protected]>
jijoongmoon
requested review from
myungjoo,
again4you,
jaeyun-jung,
leemgs,
wooksong,
helloahn,
kparichay,
gichan-jang,
anyj0527,
zhoonit,
lhs8928,
songgot,
jihochu,
DonghakPark and
SeoHyungjun
as code owners
|
📝 TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #2567. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/. |
jijoongmoon
requested review from
baek2sm,
skykongkong8,
djeong20,
EunjuYang and
a team
as code owners
|
|
cibot:
jijoongmoon
changed the title
There was a problem hiding this comment.
@jijoongmoon, 💯 All CI checkers are successfully verified. Thanks.
| std::string var32_suffix = ":fp32"; | ||
| std::string var32_name = name + var32_suffix; |
There was a problem hiding this comment.
| std::string var32_suffix = ":fp32"; | |
| std::string var32_name = name + var32_suffix; |
duplicate with line 78, 79
|
closed by #2663 |
In this PR
This PR enables the FP16 support for the layers below:
. input layer
. mse loss layer
Resolves:
Self evaluation:
Signed-off-by: jijoong.moon [email protected]