[Cherry-pick]Async generate diskann cache after deserialize#215

Closed

cqy123456 wants to merge 197 commits intozilliztech:mainfrom cqy123456:async-generate-diskann

+29,588-56,924

Commits on Jul 19, 2022

Use release instead of debug (zilliztech#267 )
Jenny Li
authored
Update cpu-e2e
Jenny Li
authored
Update Dockfile for ubuntu20.04 (zilliztech#269 )
cydrain
authored
Update publish-builder.yaml
Jenny Li
authored
Update cpu-e2e to 8c32g (zilliztech#274 )
Jenny Li
authored

Commits on Jul 20, 2022

[skip e2e]Add skip e2e (zilliztech#277 )
Jenny Li
authored
Separate build & e2e test (zilliztech#276 )
Jenny Li
authored
Change distance_compute_blas_threshold default value to 16384 (zilliztech#279 )
cydrain
authored
Replace docker image for build & e2e (zilliztech#280 )
Jenny Li
authored
[skip e2e]Add action to build docker images (zilliztech#281 )
Jenny Li
authored
Update publish-docker.yaml
Jenny Li
authored
Move archive logs to post (zilliztech#285 )
Jenny Li
authored

Commits on Jul 21, 2022

Commits on Jul 22, 2022

Commits on Jul 25, 2022

Add id validation check for GetVectorById() (zilliztech#308 )
cydrain
authored

Commits on Jul 27, 2022

Commits on Jul 28, 2022

Commits on Aug 2, 2022

[skip e2e]Add deps for diskAnn (zilliztech#346 )
Jenny Li
authored

Commits on Aug 3, 2022

Commits on Aug 5, 2022

fix dataset memory leak when use python api (zilliztech#360 )
Presburger
authored

Commits on Aug 8, 2022

Commits on Aug 15, 2022

Commits on Aug 16, 2022

[skip e2e] Add parallel test in benchmark (zilliztech#399 )
cydrain
authored

Commits on Aug 18, 2022

Commits on Aug 22, 2022

Add api to control omp threads for all index (zilliztech#413 )
smellthemoon
authored

Commits on Aug 23, 2022

[skip e2e]Add deps for new e2e docker image (zilliztech#421 )
Jenny Li
authored
[skip e2e]Upgrade hdf for debug image (zilliztech#423 )
Jenny Li
authored
Support DiskANN in Knowhere

committed

Commits on Aug 24, 2022

Commits on Sep 7, 2022

add default values to diskann (zilliztech#452 )
cqy123456
authored

Commits on Sep 8, 2022

Update OWNERS
liliu-z
authored

Commits on Sep 9, 2022

Commits on Sep 13, 2022

Commits on Sep 15, 2022

Commits on Sep 16, 2022

Remove unnecessary denpendancies for DiskANN (zilliztech#470 )
liliu-z
authored
Update ci env

Jenny Li
authored and
Presburger
committed
Update ci env (zilliztech#475 )
Jenny Li
authored

Commits on Sep 20, 2022

Support infinite dim for DiskANN (zilliztech#476 )
liliu-z
authored

Commits on Sep 22, 2022

Correct range search radius scope (zilliztech#479 )
cydrain
authored

Commits on Sep 23, 2022

Add api for multithreading test (zilliztech#459 )

smellthemoon
and
lixinguo
authored

Commits on Sep 24, 2022

Use brute force search to get all element link lists (zilliztech#480 )
cydrain
authored

Commits on Sep 26, 2022

Commits on Sep 27, 2022

Add some logs for diskann (zilliztech#484 )
cqy123456
authored
Fix bugs use async index (zilliztech#487 )

smellthemoon
and
lixinguo
authored

Commits on Sep 29, 2022

Fix large degree error (zilliztech#489 )
liliu-z
authored

Commits on Sep 30, 2022

Print error message when the function abnormal returns (zilliztech#491 )
cqy123456
authored

Commits on Oct 3, 2022

Add ut for AsyncIndex (zilliztech#492 )

smellthemoon
and
lixinguo
authored

Commits on Oct 9, 2022

[skip e2e] Add deps for code checker into build image (zilliztech#495 )
Jenny Li
authored
[skip e2e]Add python2 for cpp lint (zilliztech#497 )
Jenny Li
authored
[skip e2e]Add python2 for cpp lint (zilliztech#498 )
Jenny Li
authored

Commits on Oct 10, 2022

[skip e2e]Update ut to 20.04 (zilliztech#488 )
Jenny Li
authored

Commits on Oct 11, 2022

Commits on Oct 12, 2022

Commits on Oct 14, 2022

Remove assemble and disassemble util func (zilliztech#502 )
xige-16
authored

Commits on Oct 18, 2022

Update benchmark SSNPP (zilliztech#503 )
cydrain
authored

Commits on Oct 19, 2022

Commits on Oct 20, 2022

Commits on Oct 21, 2022

Add benchmark qps for search and range search (zilliztech#508 )
cydrain
authored

Commits on Oct 25, 2022

Remove HNSW range search parameter range_k, use ef instead (zilliztech#509 )
cydrain
authored

Commits on Oct 28, 2022

[skip e2e] Update benchmark readme (zilliztech#515 )
cydrain
authored

Commits on Oct 31, 2022

Commits on Nov 1, 2022

Release free mem in diskann aligned_free function (zilliztech#520 )
cqy123456
authored

Commits on Nov 2, 2022

Make the Query thread pool global (zilliztech#517 )
liliu-z
authored

Commits on Nov 7, 2022

Fix HNSW/DiskANN feder memory leak (zilliztech#529 )
cydrain
authored

Commits on Nov 9, 2022

Commits on Nov 16, 2022

Use elkan's algo to accelerate KMeans (zilliztech#537 )
hhy3
authored

Commits on Nov 21, 2022

Commits on Nov 22, 2022

Optimize hnsw search algo (zilliztech#553 )
hhy3
authored

Commits on Nov 23, 2022

[skip e2e]Delete Unit Test on windows (zilliztech#561 )
Bennu-Li
authored

Commits on Nov 24, 2022

Add visited list pool in hnsw (zilliztech#559 )
hhy3
authored

Commits on Nov 25, 2022

Replace diskann compute function with faiss's compute function (zilliztech#568 )
cqy123456
authored

Commits on Nov 29, 2022

Commits on Dec 1, 2022

Fix thread safety in hnsw (zilliztech#575 )
hhy3
authored

Commits on Dec 2, 2022

Let range search accept distance square as radius (zilliztech#576 )
cydrain
authored

Commits on Dec 9, 2022

Fix aio io_getevents (zilliztech#593 )
hhy3
authored

Commits on Dec 13, 2022

Commits on Dec 14, 2022

Commits on Dec 19, 2022

Support show knowhere version (zilliztech#607 )
cydrain
authored

Commits on Dec 20, 2022

Enable debug log (zilliztech#609 )
cydrain
authored

Commits on Dec 28, 2022

Fix BruteForce::Search memory leak when metric type invalid (zilliztech#611 )
cydrain
authored

Commits on Jan 12, 2023

Fix range search benchmark wrong with IP metric type (zilliztech#632 )
cydrain
authored

Commits on Jan 13, 2023

replace diskann distance function by faiss distance function (zilliztech#634 )
cqy123456
authored

Commits on Jan 17, 2023

Commits on Feb 6, 2023

Fix diskann filter (zilliztech#658 )
hhy3
authored

Commits on Feb 7, 2023

Update omp dynamic (zilliztech#662 )
hhy3
authored

Commits on Feb 8, 2023

use threaddata scratch as aio and vector buffer (zilliztech#667 )
cqy123456
authored

Commits on Feb 23, 2023

Split queries in IVF and BF

hhy3
authored and
liliu-z
committed

Commits on Mar 7, 2023

Fix TANIMOTO BruteForce distance calculation wrong (zilliztech#718 )
cydrain
authored

Commits on Mar 14, 2023

Fix sort distance crash when distance is nan / inf

cqy123456
authored and
liliu-z
committed

Commits on Mar 16, 2023

free the memory of mediods (zilliztech#738 )
cqy123456
authored

Commits on Mar 21, 2023

Commits on Mar 22, 2023

Commits on Apr 11, 2023

Fix DiskANN Access Invalid Address when Query with Bitset (zilliztech#798 ) (zilliztech#806 )
PwzXxm
authored

Commits on Apr 13, 2023

Change EF Limit to be larger if topk is tuned (zilliztech#809 )
xiaofan-luan
authored

Commits on Apr 18, 2023

Commits on Apr 26, 2023

[Cherry-Pick]Add cache for query (zilliztech#843 )
hhy3
authored

Commits on May 5, 2023

Fix data conversion method in generate_pq_data_from_pivots() function

cqy123456
authored and
liliu-z
committed

Commits on May 8, 2023

Use BF for HNSW and PQ+Refine for DiskANN when Filter Ratio is High (zilliztech#861 )
PwzXxm
authored

Commits on May 22, 2023

Set DiskAnn Filter Threshold to Default Value when Missing (zilliztech#897 )
PwzXxm
authored

Commits on May 24, 2023

fix diskann ip (zilliztech#904 )
foxspy
authored

Commits on Jun 9, 2023

set search_list(ef) default size and throw exception if search list(ef) < topk (zilliztech#923 )
cqy123456
authored

Commits on Jun 10, 2023

Sync perform of HNSW/DiskANN (zilliztech#924 )
liliu-z
authored

Commits on Jun 21, 2023

Support HasRawData interface (zilliztech#947 )
PwzXxm
authored

Commits on Jun 26, 2023

Fix GetVectorById Dim, Return Dataset and Set All HasRawData to False Except HNSW (zilliztech#957 )
PwzXxm
authored

Commits on Aug 21, 2023

Lower kw thread pool thread scheduling priority
zhengbuqian
committed

Commits on Aug 30, 2023

delete release build on centos7 (zilliztech#28 )
Bennu-Li
authored

Commits on Sep 19, 2023

Fix DiskANN LRU Set Invalid Medoid (zilliztech#97 )
PwzXxm
authored

Commits on Nov 1, 2023

revert kw thread prio modify in v1.3.18 (zilliztech#168 )
zhengbuqian
authored

Commits on Nov 22, 2023

async generate diskann cache (zilliztech#191 )
cqy123456
authored

Commits on Nov 28, 2023

Add async thread pool for generating diskann cache
cqy123456
committed