Code

llama : refactor sampling v2#9294

Code

Merged

ggerganov merged 47 commits intomasterggml-org/llama.cpp:masterfrom gg/llama-refactor-sampling-v2ggml-org/llama.cpp:gg/llama-refactor-sampling-v2Copy head branch name to clipboardSep 7, 2024

+3,401-2,818

Commits

Commits on Sep 7, 2024

• llama : add llama_sampling API + move grammar in libllama

  Show description for ab545c8
  ggerganov

  committed
  ab545c8

  Copy full SHA for ab545c8

• llama : sketching new sampling API

  ggerganov

  committed
  86b07cc

  Copy full SHA for 86b07cc

• cont : add llama_constraint_i [no ci]

  ggerganov

  committed
  5116b36

  Copy full SHA for 5116b36

• cont : initial implementation sketch [no ci]

  ggerganov

  committed
  cf4dd10

  Copy full SHA for cf4dd10

• cont : fixes, naming [no ci]

  ggerganov

  committed
  1b07dc5

  Copy full SHA for 1b07dc5

• cont : add rest of the existing samplers [no ci]

  ggerganov

  committed
  71293a6

  Copy full SHA for 71293a6

• cont : fix [no ci]

  ggerganov

  committed
  0daebc6

  Copy full SHA for 0daebc6

• cont : add penalties and logit-bias constraints [no ci]

  ggerganov

  committed
  a2ce91c

  Copy full SHA for a2ce91c

• cont : add comments [no ci]

  ggerganov

  committed
  09ceb68

  Copy full SHA for 09ceb68

• cont : leaner constraint initialization [no ci]

  ggerganov

  committed
  1e8e26c

  Copy full SHA for 1e8e26c

• cont : common/sampling use the new API [no ci]

  ggerganov

  committed
  91cbb40

  Copy full SHA for 91cbb40

• cont : add n_prev to llama_sampler_params

  ggerganov

  committed
  437376e

  Copy full SHA for 437376e

• cont : use new API in examples

  Show description for a0b9121
  ggerganov

  committed
  a0b9121

  Copy full SHA for a0b9121

• examples : fix build

  Show description for ad436e9
  ggerganov

  committed
  ad436e9

  Copy full SHA for ad436e9

• common : fix gpt_sampler_cp

  Show description for fdb52aa
  ggerganov

  committed
  fdb52aa

  Copy full SHA for fdb52aa

• grammar : fix reset call

  Show description for ca5d21c
  ggerganov

  committed
  ca5d21c

  Copy full SHA for ca5d21c

• constraint : clean-up and simplify

  ggerganov

  committed
  c024fe4

  Copy full SHA for c024fe4

• constraint : add name API

  Show description for 1a0de0b
  ggerganov

  committed
  1a0de0b

  Copy full SHA for 1a0de0b

• sampling : convert mirostat samplers to constraints

  Show description for 0e1378c
  ggerganov

  committed
  0e1378c

  Copy full SHA for 0e1378c

• sampler : API to iterate constraints

  Show description for 784a644
  ggerganov

  committed
  784a644

  Copy full SHA for 784a644

• sampling : simplify new llama_sampler calls

  ggerganov

  committed
  e7a11ca

  Copy full SHA for e7a11ca

• sampling : simplify sample API

  Show description for 8e80a1c
  ggerganov

  committed
  8e80a1c

  Copy full SHA for 8e80a1c

• sampling : fix grammar apply

  ggerganov

  committed
  9b95067

  Copy full SHA for 9b95067

• example : fix build + fix speculative

  Show description for b2b36e9
  ggerganov

  committed
  b2b36e9

  Copy full SHA for b2b36e9

• sampling : remove top-k min_keep, fix mirostat init and state

  ggerganov

  committed
  69551ff

  Copy full SHA for 69551ff

• sampling : change _cp/copy to clone

  ggerganov

  committed
  ebeb651

  Copy full SHA for ebeb651

• sampling : add name API + option to disable timings

  ggerganov

  committed
  5957114

  Copy full SHA for 5957114

• llama : restore comments in llama.h

  Show description for a2d8b27
  ggerganov

  committed
  a2d8b27

  Copy full SHA for a2d8b27

• llama : remove llama_constraint

  Show description for 0b6dfce
  ggerganov

  committed
  0b6dfce

  Copy full SHA for 0b6dfce

• sampling : fix cloning of samplers with null ctx

  Show description for 34f4bd0
  ggerganov

  committed
  34f4bd0

  Copy full SHA for 34f4bd0

• ios : try to fix build

  ggerganov

  committed
  bd88352

  Copy full SHA for bd88352

• sampling : improve mirostat implementation

  Show description for 82a89df
  ggerganov

  committed
  82a89df

  Copy full SHA for 82a89df

• swift : fix example

  ggerganov

  committed
  5b01cc8

  Copy full SHA for 5b01cc8

• grammar : restore llama_grammar_accept signature

  Show description for 8c972b6
  ggerganov

  committed
  8c972b6

  Copy full SHA for 8c972b6

• sampling : allow passing m to mirostat sampler

  ggerganov

  committed
  809bdcf

  Copy full SHA for 809bdcf

• sampling : remove redundant indirection calls

  Show description for b448c75
  ggerganov

  committed
  b448c75

  Copy full SHA for b448c75

• sampling : remove _context suffix [no ci]

  ggerganov

  committed
  5ab52c1

  Copy full SHA for 5ab52c1

• llama : add new llama_perf API

  Show description for 757a9bf
  ggerganov

  committed
  757a9bf

  Copy full SHA for 757a9bf

• common : simplify gpt_sampler

  Show description for befcfe7
  ggerganov

  committed
  befcfe7

  Copy full SHA for befcfe7

• batched.swift : fix build [no ci]

  ggerganov

  committed
  9ce9210

  Copy full SHA for 9ce9210

• examples : add missing samplers

  ggerganov

  committed
  4a4530b

  Copy full SHA for 4a4530b

• style : rearrange code + add comments and TODOs

  Show description for 4b27235
  ggerganov

  committed
  4b27235

  Copy full SHA for 4b27235

• batched.swift : fix build

  ggerganov

  committed
  19c3696

  Copy full SHA for 19c3696

• sampling : avoid llama_model in few samplers

  Show description for 0e6d170
  ggerganov

  committed
  0e6d170

  Copy full SHA for 0e6d170

• sampling : fix state cloning

  Show description for 8a82f38
  ggerganov

  committed
  8a82f38

  Copy full SHA for 8a82f38

• sampling : fix repeat penalty out-of-bounds access

  Show description for 2387dbe
  ggerganov

  committed
  2387dbe

  Copy full SHA for 2387dbe

• llama : update doc [no ci]

  ggerganov

  committed
  4ac186a

  Copy full SHA for 4ac186a