Enable SVE Support for L2 Metric Computation in FP32

[adarshs1310]

Description:
This PR introduces SVE (Scalable Vector Extension) enablement for L2 metric computation in FP32. The changes enhance performance for most indexing methods compared to NEON, with observed speed-ups across multiple algorithms.

Changes in This PR:

• Added SVE optimizations for L2 metric computation in FP32.
• Updated CMakeLists.txt to include support for -march=armv8-a+sve.
• Refactored compute kernels to leverage SVE intrinsics.

Benchmark Results:

Performance benchmarks(32 vcpus) were conducted using both NEON and SVE on ARM architecture. Below are the results showcasing execution times (in seconds):

Key observations:

• All of the algorithms exhibit performance gains with SVE when compared with NEON.

Note: SVE support has been made optional, as not all functions have been fully enabled yet. To utilize SVE, please compile with -march=armv8-a+sve.

/kind feature
Fixes #782

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: adarshs1310
To complete the pull request process, please assign zhengbuqian after the PR has been reviewed.
You can assign the PR to them by writing /assign @zhengbuqian in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[sre-ci-robot]

Welcome @adarshs1310! It looks like this is your first PR to zilliztech/knowhere 🎉

[mergify]

@adarshs1310 🔍 Important: PR Classification Needed!

For efficient project management and a seamless review process, it's essential to classify your PR correctly. Here's how:

1. If you're fixing a bug, label it as kind/bug.
2. For small tweaks (less than 20 lines without altering any functionality), please use kind/improvement.
3. Significant changes that don't modify existing functionalities should be tagged as kind/enhancement.
4. Adjusting APIs or changing functionality? Go with kind/feature.

For any PR outside the kind/improvement category, ensure you link to the associated issue using the format: “issue: #”.

Thanks for your efforts and contribution to the community!.

[alexanderguzhva]

I believe that this if condition is not needed, just pg = svwhilelt_b32(i, d); should be sufficient

[adarshs1310]

Thank you so much, @alexanderguzhva , for the valuable suggestions. During development, we considered this approach as well, and our reason for going with the current approach is as follows:

Using the if condition to update pg only in the last iteration avoids unnecessary updates and reduces the dependency chain introduced by the svwhilelt instruction. This optimization minimizes stalls caused by these dependencies, allowing the processor pipeline to operate more efficiently.

[alexanderguzhva]

I believe that this is not sufficient.
Knowhere is designed as a library that picks function pointers according to CPU capabilities, detected upon the start.
For example, SSE / AVX2 / AVX512 code files have different corresponding compile options

knowhere/cmake/libs/libfaiss.cmake

Lines 37 to 40 in 1cb9937

	target_compile_options(utils_sse PRIVATE -msse4.2 -mpopcnt)
	target_compile_options(utils_avx PRIVATE -mfma -mf16c -mavx2 -mpopcnt)
	target_compile_options(utils_avx512 PRIVATE -mfma -mf16c -mavx512f -mavx512dq
	-mavx512bw -mpopcnt -mavx512vl)

So, it seems to be logical that the new SVE code should also contain some form of different flags, such as -march=armv8-a+sve for distances_sve.cc. And I don't believe that I see this.

[adarshs1310]

Thank you for your valuable feedback @alexanderguzhva

We considered two approaches: first, to keep Neon as the default until all FP32 functions for SVE have been fully implemented. Based on this, we decided to proceed with this approach.

Your approach is absolutely correct, and we agree that the Neon fallback for non-SVE functions can be effectively handled using hook.cc.

We have incorporated these changes into our solution.

[adarshs1310]

@alexanderguzhva @foxspy @hhy3

Requesting updates to the CI pipeline to address GCC header file conflicts.

Since Ubuntu 22.04 defaults to GCC-11, there is a risk of incorrect header usage when GCC-12 is installed. To resolve this, the GCC-11 header files should be removed after installing GCC-12.

The necessary changes have been incorporated into the ARM-based Ubuntu 22.04 Docker configuration as part of this PR, ensuring proper SVE compatibility. Kindly review and consider these adjustments for consistent and accurate builds. Thank you!

[adarshs1310]

/kind feature

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.01%. Comparing base (3c46f4c) to head (da98fcf).
Report is 297 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##           main     #969       +/-   ##
=========================================
+ Coverage      0   73.01%   +73.01%     
=========================================
  Files         0       82       +82     
  Lines         0     7507     +7507     
=========================================
+ Hits          0     5481     +5481     
- Misses        0     2026     +2026     

see 82 files with indirect coverage changes

[alexanderguzhva]

@adarshs1310 would you please rebase on top of master? it contains a fix for UT. Thanks.

[adarshs1310]

Sure @alexanderguzhva! The rebase has been done now!

[adarshs1310]

Hi @alexanderguzhva , @foxspy , @hhy3 ,

Could you please look into the SSE fix when you have a chance? The ARM CI issue has already been handled in our code.

Thank you so much for your support!

[foxspy]

Hi @alexanderguzhva , @foxspy , @hhy3 ,

Could you please look into the SSE fix when you have a chance? The ARM CI issue has already been handled in our code.

Thank you so much for your support!

SSE's CI will not block; but ARM's CI fails

[alexanderguzhva]

@adarshs1310 I was able to compile and run knowhere on AWS Graviton 3 using GCC-12. I see the following error during unit tests, which seems to be a minor precision issue. Still, would you be able to find the root of this problem?

-------------------------------------------------------------------------------
Test Brute Force with input ids
-------------------------------------------------------------------------------
/home/ubuntu/zilliz/knowhere_sve/knowhere/tests/ut/test_bruteforce.cc:201
...............................................................................

/home/ubuntu/zilliz/knowhere_sve/knowhere/tests/ut/test_bruteforce.cc:243: FAILED:
  REQUIRE( gt_dis[i] == dis[i] )
with expansion:
  156769.3125f == 156769.32812f

Other unit tests pass.

The compilation is the following (given that you have created a profile for GCC 12 for conan):

mkdir build
cd build
conan install .. --build=missing -o with_diskann=True -o with_ut=True -o with_benchmark=True -s compiler.libcxx=libstdc++11 -c tools.build:cxxflags+=[\"-mcpu=neoverse-512tvb\",\"-march=native\"] -s build_type=Release --profile=gcc12
conan build ..

[adarshs1310]

Rebase has been done

@foxspy @alexanderguzhva @hhy3

[alexanderguzhva]

@adarshs1310 Could you please just remove this hacky pragma?
Thanks.

[adarshs1310]

Sure @alexanderguzhva I have removed it. Thanks!

[alexanderguzhva]

lgtm

[adarshs1310]

Hi @foxspy and @hhy3! Can we Please get an update on this?

Thank you so much!

[Presburger]

At this stage, I am not inclined to accept this PR, as it would require us to maintain two sets of ARM binaries when releasing binaries or containers. At this stage, I am not inclined to accept this PR, as it would require us to maintain two sets of ARM binaries when releasing binaries or containers.

[adarshs1310]

At this stage, I am not inclined to accept this PR, as it would require us to maintain two sets of ARM binaries when releasing binaries or containers. At this stage, I am not inclined to accept this PR, as it would require us to maintain two sets of ARM binaries when releasing binaries or containers.

Thank you, @Presburger , for sharing this! We have now implemented a supports_sve() function that dynamically checks if the machine supports SVE at runtime. This approach eliminates the need to maintain separate binaries for ARM architectures.

[adarshs1310]

Hi @Presburger @foxspy @alexanderguzhva @hhy3!

I am following up regarding the recent updates to this PR. As noted, we have implemented the supports_sve() function to dynamically verify SVE support at runtime hereby removing the need to maintain two binaries for arm.

In addition, we have been making further progress and remain committed to improving the Knowhere ecosystem, particularly by optimizing performance. We will continue to contribute actively to this project.

Could you kindly review the latest changes and let us know if this is ready to proceed?

Your feedback and guidance are highly valued.

[Presburger]

@adarshs1310 Thank you very much for your contribution. Could you please test on a CPU that does not support SVE to see if there are any illegal instruction issues? Additionally, when dynamic selection is supported, is the performance improvement still significant? Thank you very much for your work.

[adarshs1310]

Hi @Presburger!

Thank you for your feedback and for pointing this out. We’ve tested the latest commit on a CPU that does not support SVE, and there are no illegal instruction issues on the NEON machine (m6g.16xlarge). Additionally, there is still a significant performance boost even when dynamic selection is supported. All test cases are passing successfully on the m7g.16xlarge instance.

However, on the main branch, in the test_simd function with the disabled BF16 patch, the threshold for validation is very strict, occasionally leading to test failures with a value difference of less than 0.1. It might be worth revisiting the tolerance in this case.

Your feedback is highly valued, and we look forward to continuing our contributions together.

[adarshs1310]

Hi @Presburger @alexanderguzhva @foxspy @hhy3 ,

Just following up to check if our PR looks good to you or if there’s anything that needs to be addressed.

[Presburger]

By the way, please also implement the batch4 for the IP sve.

[Presburger]

The handling of the tail block here could be more elegant, as SVE's predicate registers are inherently friendly to out-of-bounds memory access.

[Presburger]

You can directly place the SVE source file into the UTILS_SRC variable.