You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The fp16 support using Faiss scalar quantizer(SQfp16) provides 50% memory reduction besides providing at par performance and mostly similar recall compared to fp32 vectors. But, it has a range limitation where the input vectors needs to be within the range of [-65504, 65504] which is a bottleneck to use fp16 as a default datatype instead of fp32.
To overcome this problem, we can add support for BFloat16 with Faiss Scalar Quantizer(SQbf16). BFloat16 comes with an extended range(same as fp32) by trading off precision(supports upto 2 or 3 decimal values or 7 mantissa bits) and still uses 16 bits per dimension (provides 50% memory reduction).
Intel AVX512 also has BF16 instruction set which can be used to further improve the performance on newer-generation processors.
The text was updated successfully, but these errors were encountered:
Description
The fp16 support using Faiss scalar quantizer(SQfp16) provides 50% memory reduction besides providing at par performance and mostly similar recall compared to fp32 vectors. But, it has a range limitation where the input vectors needs to be within the range of [-65504, 65504] which is a bottleneck to use fp16 as a default datatype instead of fp32.
To overcome this problem, we can add support for BFloat16 with Faiss Scalar Quantizer(SQbf16). BFloat16 comes with an extended range(same as fp32) by trading off precision(supports upto 2 or 3 decimal values or 7 mantissa bits) and still uses 16 bits per dimension (provides 50% memory reduction).
Intel AVX512 also has BF16 instruction set which can be used to further improve the performance on newer-generation processors.
The text was updated successfully, but these errors were encountered: