Milbus Speed Up wway

[platic96]

I am currently using Milvus as a vector database. However, when utilizing vector search with HNSW, IVF_FLAT, and GPU_CAGRA, the average search time was around 0.5 seconds, with almost no noticeable difference among them. Of course, I only tested with a single data point, but is it normal for the search times to be so consistent?

Additionally, the parameters for each index type are as follows:
HNSW

m: 16
efConstruction: 200
IVF_FLAT

nlist: 128
GPU_CAGRA

intermediate_graph_degree: 64
graph_degree: 32
Common Parameter

metric_type: L2
I conducted chunking based on sentences.

[yhmo]

I guess your collection is in "Strong" consistency level.

Consistency level is to control the data visibility for search/query in the scenario that data is inserted/updated continually during search/query.
https://milvus.io/docs/consistency.md#Consistency-Level

Let's say there is a client continually sends insert requests to add new data. Another client calls search(). Since insert action is asynchronously, the inserted data might be invisible to search requests and user might get unexpected results. Consistency level allows users to control the data visibility for search requests.

Let's say a search request is received at time point "t1".

Strong level ------- Ensure the data inserted before "t1" is visible for this search request. Since the data is consumed asynchronously, the search request needs to wait a short time, typically 200ms ~ 400ms.
Bounded level ------ Ensure the data inserted before "t1 - 5 seconds" is visible for this search request. Normally, the search request doesn't wait since 5 seconds is quite a large time window. But sometimes due du low disk performance, the data inserted 5 seconds ago is not consumed, then the search request will wait until the data is consumed.
Eventually level ------ Search request is always executed immediately without waiting any data to be consumed. Some data might be invisible to this search request.
Session level ------ No need to use/understand this level. The above 3 levels are enough to use.

Try the "Eventually" level:

res = client.search(
    collection_name="my_collection",
    data=[query_vector],
    limit=3,
    search_params={}，
    consistency_level="Eventually",
)

Normally, for small collection contains no more than one million rows with index. a search latency should be around 10 milliseconds.