bug: Fix Download location of Nitro TensorRT on Windows

[Van-QA]

Describe the bug
The installation is not working, there is a toast msg, but users are blocked from getting to the installed status on Jan UI

Screenshots

Additionally, there is an issue with system monitor in linux, that was resolved via 0.5.3

[louis-menlo]

System monitor issue is addressed. #3492

[imtuyethan]

What installation is this? How can I reproduce?

[louis-menlo]

@imtuyethan TensorrtLLM requires the CUDA Toolkit to be installed, but some users may not have it set up.

Try to uninstall Cuda Toolkit, try to load a model -> Failed -> there will be a guidance message to lead you to the installation page.

[louis-menlo]

Root cause:

The Tensorrt-llm on Jan is broken, as the engine repository name was changed from nitro-tensorrt-llm to cortex.tensorrt-llm.

Jan Tensort-LLM extension is downloading artifact from the repository releases. Now it's pointing to an incorrect repository.

[louis-menlo]

The Linux release artifact has also been removed from the engine repository. It's likely will be a part of #3690.

[dan-menlo]

@louis-jan Is there a quick fix for this that will allow us to keep legacy TensorRT-LLM extension working in Sprint 21?

• Cortex team will be working on TensorRT-LLM in Sprint 23 or 24 (i.e. early November)
• I would just try to keep things working, but if not possible we can also just let it fail for now

[louis-menlo]

Hi @dan-homebrew We can do a quick fix for Windows; it's simply a matter of replacing the URL.

But I'm not sure how Linux is enabled, since we haven't supported it in the extension yet. I'll look into it.

[dan-menlo]

@louis-jan Linux is out of scope - let's just do a quick fix for Windows, and ship it as part of 0.5.5

[louis-menlo]

@louis-jan Linux is out of scope - let's just do a quick fix for Windows, and ship it as part of 0.5.5

Roger that!