Parallelize unary tanh on cpu, generalize ADD to allow more shapes #580
+107
−48
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ggerganov
approved these changes
Oct 20, 2023
Comment on lines
+9383
to
9402
| } else { | ||
| // all are not contiguous | ||
| for (int ie = ie0; ie < ie1; ++ie) { | ||
| // src1 is broadcastable across src0 and dst in i1, i2, i3 | ||
| const int64_t i03 = ie/(ne02*ne01*ne00); | ||
| const int64_t i02 = (ie - i03*ne02*ne01*ne00)/(ne01*ne00); | ||
| const int64_t i01 = (ie - i03*ne02*ne01*ne00 - i02*ne01*ne00); | ||
| const int64_t i00 = (ie - i03*ne02*ne01*ne00 - i02*ne01*ne00 - i01*ne00); | ||
|
|
||
| const int64_t i13 = i03 % ne13; | ||
| const int64_t i12 = i02 % ne12; | ||
| const int64_t i11 = i01 % ne11; | ||
| const int64_t i10 = i00 % ne10; | ||
|
|
||
| float * dst_ptr = (float *) ((char *) dst->data + i03*nb3 + i02*nb2 + i01*nb1 + i00*nb0 ); | ||
| float * src0_ptr = (float *) ((char *) src0->data + i03*nb03 + i02*nb02 + i01*nb01 + i00*nb00); | ||
| float * src1_ptr = (float *) ((char *) src1->data + i13*nb13 + i12*nb12 + i11*nb11 + i10*nb10); | ||
|
|
||
| *dst_ptr = *src0_ptr + *src1_ptr; | ||
| } |
There was a problem hiding this comment.
This branch will probably be very slow - most of the computation will probably go into computing the indices
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I'm working on a project which needed those operations.
tanh was parallelized in the same manner as other unary ops
ADD is generalized to allow for ggml_can_repeat constraint, instead of the ggml_can_repeat_rows
This was done adding two extra branches in the function, one of them is likely very slow and handles the most general case. The second is particularly optimized for my project's need (adding MxN and 1xP tensors) and uses ggml_vec_add1_f32.