Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for BFloat16 with Faiss Scalar Quantizer for extended range #2510

Open
naveentatikonda opened this issue Feb 8, 2025 · 0 comments
Assignees
Labels
Features Introduces a new unit of functionality that satisfies a requirement v3.0.0

Comments

@naveentatikonda
Copy link
Member

naveentatikonda commented Feb 8, 2025

Description

The fp16 support using Faiss scalar quantizer(SQfp16) provides 50% memory reduction besides providing at par performance and mostly similar recall compared to fp32 vectors. But, it has a range limitation where the input vectors needs to be within the range of [-65504, 65504] which is a bottleneck to use fp16 as a default datatype instead of fp32.

To overcome this problem, we can add support for BFloat16 with Faiss Scalar Quantizer(SQbf16). BFloat16 comes with an extended range(same as fp32) by trading off precision(supports upto 2 or 3 decimal values or 7 mantissa bits) and still uses 16 bits per dimension (provides 50% memory reduction).

Intel AVX512 also has BF16 instruction set which can be used to further improve the performance on newer-generation processors.

@naveentatikonda naveentatikonda added Features Introduces a new unit of functionality that satisfies a requirement v3.0.0 labels Feb 8, 2025
@naveentatikonda naveentatikonda moved this from Backlog to Backlog (Hot) in Vector Search RoadMap Feb 8, 2025
@naveentatikonda naveentatikonda self-assigned this Feb 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Features Introduces a new unit of functionality that satisfies a requirement v3.0.0
Projects
Status: Backlog (Hot)
Development

No branches or pull requests

1 participant