Skip to content

Commit

Permalink
Apply suggestion from code review
Browse files Browse the repository at this point in the history
Co-Authored-By: fria <[email protected]>
Co-Authored-By: Triple T <[email protected]>

Signed-off-by: redoomed1 <[email protected]>
  • Loading branch information
redoomed1 committed Nov 17, 2024
1 parent f3db643 commit 496ffcf
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/ai-chat.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Alternatively, you can run AI models locally so that your data never leaves your

## Hardware for Local AI Models

Local models are also fairly accessible. It's possible to run smaller models at lower speeds on as little as 8GB of RAM. Using more powerful hardware such as a dedicated GPU with sufficient VRAM or a modern system with fast LPDDR5X memory will offer the best experience.
Local models are also fairly accessible. It's possible to run smaller models at lower speeds on as little as 8GB of RAM. Using more powerful hardware such as a dedicated GPU with sufficient VRAM or a modern system with fast LPDDR5X memory offers the best experience.

LLMs can usually be differentiated by the number of parameters, which can vary between 1.3B to 405B for open-source models available for end users. For example, models below 6.7B parameters are only good for basic tasks like text summaries, while models between 7B and 13B are a great compromise between quality and speed. Models with advanced reasoning capabilities are generally around 70B.

Expand Down

0 comments on commit 496ffcf

Please sign in to comment.