Code

ExllamaV2 optimizations#88

Code

Merged

noamgat merged 5 commits intonoamgat:mainnoamgat/lm-format-enforcer:mainfrom theroyallab:maintheroyallab/lm-format-enforcer:mainCopy head branch name to clipboardApr 19, 2024

+8-9

Commits

Commits on Feb 18, 2024

• Extract vocab from ExLlamaV2Tokenizer id_to_piece list

  turboderp

  committed
  d34296a

• Optimize initialization of JsonFreetextTokenCache

  turboderp

  committed
  980b632

Commits on Feb 19, 2024

• Precompute token lengths and simplify set construction (current bottleneck), further 2x speedup

  turboderp

  committed
  4695657

Commits on Mar 14, 2024

• Merge remote-tracking branch 'origin/main'

  Show description for 6257b74
  turboderp

  committed
  6257b74

Commits on Apr 14, 2024

• Merge pull request #1 from noamgat/main

  Show description for 13b0ef9
  turboderp

  authored
  13b0ef9