Analysis: CLI startup time

[wirthual]

Version: Current master

I did some evaluation on the startup time of the cli.

The majority of the time is required for the loading of sentence_transformers.quantization and within there its torch._C._cuda_getDeviceCount in the cpp_extensions of the torch library

see:

I attached the raw import times which you can visualize using tuna:

tuna import.log

import.log

[michaelfeil]

Cool! Would that change if you would import torch before starting up?

import torch

def profile():
    import infinity_emb 

[wirthual]

Tried that for these runs, difference of 40ms overall.

Seems the variance between runs is high. This run its 6.78s with vs 6.82s without the additional import statement.

import_w_torch.log
import_wo_torch.log

This is how I run it:

poetry run python3 -X importtime infinity_emb/cli.py 2>import_wo_torch.log