Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: #3491 - Unable to use tensorrt-llm #3741

Merged
merged 2 commits into from
Sep 30, 2024

Conversation

louis-menlo
Copy link
Contributor

@louis-menlo louis-menlo commented Sep 29, 2024

Describe Your Changes

This PR fixes the Tensor-LLM engine download URL, making it accessible to Windows users with supported Nvidia GPUs, who can now install and use Tensort-LLM models.

This also fixes the incorrect supported platforms from the extension, which currently only support Windows.

There's an edge case with model downloading on Windows that's been fixed, and it also resolved an issue where the renderer process would send a wrong aborting ID.

Screenshots

Install Tensorrt-llm engine on Windows
Screenshot 2024-09-29 153511
Screenshot 2024-09-29 153727

Fixes Issues

  1. In package.json:

    • Added Jest testing setup and dependencies
    • Updated the scripts section to include a test command
  2. In rollup.config.ts:

    • Updated the download URLs for the runner, changing from janhq/nitro-tensorrt-llm to janhq/cortex.tensorrt-llm
  3. In src/index.ts:

    • Removed the supportedPlatform array
    • Updated the compatibility check to use this.compatibility().platform instead of this.supportedPlatform
  4. In tsconfig.json:

    • Added an exclude field to ignore test files
  5. In web/hooks/useDownloadModel.ts:

    • Updated the abortModelDownload function to handle ModelFile instead of Model
    • Modified the path construction logic in abortModelDownload to use file_path when available

These changes mainly focus on setting up Jest for testing, updating download URLs, refactoring some compatibility checks, and adjusting the model download abort functionality.

@github-actions github-actions bot added the type: bug Something isn't working label Sep 29, 2024
@louis-menlo
Copy link
Contributor Author

Hmmm..checking

Copy link
Contributor

Barecheck - Code coverage report

Total: 64.8%

Your code coverage diff: 0.00% ▴

Uncovered files and lines
FileLines
web/hooks/useDownloadModel.ts41-42, 60, 77-79, 97, 99, 102, 118

@louis-menlo louis-menlo merged commit 8334076 into dev Sep 30, 2024
9 checks passed
@louis-menlo louis-menlo deleted the fix/3491-unable-to-use-nitro.tensorrt-llm branch September 30, 2024 04:58
@github-actions github-actions bot added this to the v0.5.5 milestone Sep 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants