Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix iterator: set omp to 1 #989

Closed

Conversation

alwayslove2013
Copy link
Collaborator

@alwayslove2013 alwayslove2013 commented Dec 16, 2024

Use search_pool to handle iterator's next_batch instead of init

Copy link

mergify bot commented Dec 16, 2024

@alwayslove2013 🔍 Important: PR Classification Needed!

For efficient project management and a seamless review process, it's essential to classify your PR correctly. Here's how:

  1. If you're fixing a bug, label it as kind/bug.
  2. For small tweaks (less than 20 lines without altering any functionality), please use kind/improvement.
  3. Significant changes that don't modify existing functionalities should be tagged as kind/enhancement.
  4. Adjusting APIs or changing functionality? Go with kind/feature.

For any PR outside the kind/improvement category, ensure you link to the associated issue using the format: “issue: #”.

Thanks for your efforts and contribution to the community!.

@alwayslove2013
Copy link
Collaborator Author

/kind improvement

Copy link

codecov bot commented Dec 16, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.84%. Comparing base (3c46f4c) to head (64010b8).
Report is 279 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff            @@
##           main     #989       +/-   ##
=========================================
+ Coverage      0   73.84%   +73.84%     
=========================================
  Files         0       82       +82     
  Lines         0     6916     +6916     
=========================================
+ Hits          0     5107     +5107     
- Misses        0     1809     +1809     

see 82 files with indirect coverage changes

@cqy123456
Copy link
Collaborator

/lgtm

@cydrain
Copy link
Collaborator

cydrain commented Dec 17, 2024

Generally e2e should finish in 30 minutes.
Comparing a success e2e run with yours

success e2e run:
[2024-12-17 04:35:58 - INFO - test_func.py:48]: Start building index, index_type = HNSW, metric_type = L2
[2024-12-17 04:36:00 - INFO - test_func.py:39]: ### check_build_index runtime: 2.301s
[2024-12-17 04:36:00 - INFO - test_func.py:39]: ### iterator_search runtime: 0.023s
[2024-12-17 04:36:00 - INFO - test_func.py:39]: ### iterator_search runtime: 0.055s
[2024-12-17 04:36:01 - INFO - test_func.py:39]: ### iterator_search runtime: 0.219s

yours:
[2024-12-17 06:58:15 - INFO - test_func.py:48]: Start building index, index_type = HNSW, metric_type = L2
[2024-12-17 06:58:17 - INFO - test_func.py:39]: ### check_build_index runtime: 2.460s

I cannot see "iterator_search runtime" in your e2e log, there must be something wrong

@cqy123456
Copy link
Collaborator

cqy123456 commented Dec 17, 2024

Deadlock caused by nested calls to search thread pool.
We can let iterator workspace hold a shared_ptr of search thread pool, and generate thread tasks itself.
@alwayslove2013

@sre-ci-robot
Copy link
Collaborator

New changes are detected. LGTM label has been removed.

@sre-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: alwayslove2013
To complete the pull request process, please ask for approval from cqy123456 after the PR has been reviewed.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@alwayslove2013
Copy link
Collaborator Author

/hold

@alexanderguzhva
Copy link
Collaborator

@alwayslove2013 is it good to review this PR? thanks

@alwayslove2013
Copy link
Collaborator Author

@alexanderguzhva Sorry, there are still some compatibility issues with bruteforce-iterator. I will fix it asap. please hold~

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants