Implementations for Q4_0_8_8 quantization based functions in AVX2 SIMD architecture#8713

Merged

ggerganov merged 4 commits intoggerganov:masterfrom Srihari-mcw:block_interleaving_q4_0_8_8_avx2_implementation

Sep 4, 2024

+612

Commits on Sep 4, 2024

Add AVX2 based implementations for quantize_q8_0_4x8, ggml_gemv_q4_0_8x8_q8_0 and ggml_gemm_q4_0_8x8_q8_0 functions

Srihari-mcw
authored and
Srihari-mcw
committed
Update code to fix issues occuring due to non alignment of elements to be processed as multiple of 16 in MSVC

Srihari-mcw
authored and
Srihari-mcw
committed
Update comments and indentation

Srihari-mcw
authored and
Srihari-mcw
committed
Make updates to reduce number of load instructions

Srihari-mcw
authored and
Srihari-mcw
committed