Code

More vllama optimizations#992

Code

Merged

EricLBuehler merged 8 commits intomasterEricLBuehler/mistral.rs:masterfrom more_vllama_optEricLBuehler/mistral.rs:more_vllama_optCopy head branch name to clipboardDec 14, 2024

+164-58

Commits

Commits on Dec 13, 2024

• More vllama optimizations

  EricLBuehler

  committed
  4da6b74

• Oops

  EricLBuehler

  committed
  5b0edc5

Commits on Dec 14, 2024

• Use addmm metal

  EricLBuehler

  committed
  a08b0b3

• Make some progress

  EricLBuehler

  committed
  3a9427f

• Conditional

  EricLBuehler

  committed
  91b4318

• Conditional

  EricLBuehler

  committed
  0e4b8c5

• No crossattn quant

  EricLBuehler

  committed
  678990d

• Fix loading from uqff

  EricLBuehler

  committed
  bace225