[SYCLomatic] Enable migration of CUBLASLT_EPILOGUE_BIAS & CUBLASLT_EPILOGUE_GELU_AUX_BIAS #2460
Conversation
| } else if (compute_desc->_epilogue == epilogue_t::bias) { | ||
| matmul_ops.append_binary(::dnnl::algorithm::binary_add, bias_md); | ||
| } else if (compute_desc-> epilogue == epilogue::gelu_aux_bias) { | ||
| matmul_ops.append_eltwise(::dnnl::algorithm::eltwise_gelu_erf, 0.f, 0.f); |
There was a problem hiding this comment.
CUBLASLT_EPILOGUE_GELU_AUX_BIASneeds to output thebiasresult toCUBLASLT_MATMUL_DESC_EPILOGUE_AUX_POINTER. But here, your implementation does not.- Please write an E2E test to validate your implementation. I tried something similar before but encountered a runtime error or an incorrect result.
There was a problem hiding this comment.
Ok. Please help to review.
abhilash1910
force-pushed
the
bias_au_gelu_epilogues
branch
from
0a657ab
to
c5b19af
Compare
| } else if (compute_desc-> epilogue == epilogue::gelu_aux_bias) { | ||
| matmul_ops.append_eltwise(::dnnl::algorithm::eltwise_gelu_erf, 0.f, 0.f); | ||
| matmul_ops.append_binary(::dnnl::algorithm::binary_add, bias_md); | ||
| compute_desc->set_attribute(matmul_desc_t::attribute::epilogue_aux_pointer, bias_mem); |
There was a problem hiding this comment.
We need to copy the data in bias_mem to _epilogue_aux_pointer. Meanwhile, please also consider the data layout (row_major/col_major) and the leading dimension (CUBLASLT_MATMUL_DESC_EPILOGUE_AUX_LD).
And according to the cublaslt doc, CUBLASLT_MATMUL_DESC_BIAS_POINTER is also related to this epilogue. Please check the usage.
There was a problem hiding this comment.
I directly added matrix memcpy to copy epilogue_aux_pointer contents from bias_mem pointer, using the epilogue_aux_ld is and new_ldc (bias ld). This usage is for matrix memcpy, and will try to see if tests are ok .
Please suggest in meantime.
|
Also could you please prepare an E2E test first? like https://github.com/oneapi-src/SYCLomatic-test/blob/SYCLomatic/features/feature_case/cublasLt/matmul.cu . |
abhilash1910
force-pushed
the
bias_au_gelu_epilogues
branch
from
6b14fa2
to
a2dc3f5
Compare
@zhiweij1 @zhimingwang36