Code

b2532#102

Code

Merged

Nexesenex merged 64 commits intoNexesenex:Nexesenex-IQ1_XS-IQ1_S-quant-strategiesNexesenex/croco.cpp:Nexesenex-IQ1_XS-IQ1_S-quant-strategiesfrom ggerganov:masterggerganov/llama.cpp:masterCopy head branch name to clipboardMar 25, 2024

+24,042-16,583

Commits

Commits on Mar 20, 2024

• Server: Handle n_keep parameter in the request (#6174)

  jkarthic

  authored
  47cc7a7

  Copy full SHA for 47cc7a7

• llava : add a MobileVLM_V2-1.7B backup (#6152)

  Show description for f8c4e74
  

  ZiangWu-77

  and

  ggerganov

  authored
  f8c4e74

  Copy full SHA for f8c4e74

• Revert "llava : add a MobileVLM_V2-1.7B backup (#6152)"

  Show description for d795988
  ggerganov

  committed
  d795988

  Copy full SHA for d795988

• server : allow to override -ngl in tests (#6170)

  ggerganov

  committed
  bc0baab

  Copy full SHA for bc0baab

• gitignore : ignore curl-related files

  ggerganov

  committed
  6b7e76d

  Copy full SHA for 6b7e76d

• Server: version bump for httplib and json (#6169)

  Show description for 91f8ad1
  ngxson

  authored
  91f8ad1

  Copy full SHA for 91f8ad1

• cuda : refactor to remove global resources (#6170)

  Show description for ccf58aa
  slaren

  authored
  ccf58aa

  Copy full SHA for ccf58aa

• llava : add MobileVLM_V2 backup (#6175)

  Show description for 272935b
  

  ZiangWu-77

  and

  ggerganov

  authored
  272935b

  Copy full SHA for 272935b

• llava : update MobileVLM-README.md (#6180)

  ZiangWu-77

  authored
  f9c7ba3

  Copy full SHA for f9c7ba3

• cuda : print the returned error when CUDA initialization fails (#6185)

  slaren

  authored
  1c51f98

  Copy full SHA for 1c51f98

Commits on Mar 21, 2024

• cuda : fix conflict with std::swap (#6186)

  slaren

  authored
  42e21c6

  Copy full SHA for 42e21c6

• Add nvidia and amd backends (#6157)

  AidanBeltonS

  authored
  c5b8595

  Copy full SHA for c5b8595

• Add ability to use Q5_0, Q5_1, and IQ4_NL for quantized K cache (#6183)

  Show description for 76aa30a
  

  ikawrakow

  and

  Kawrakow

  authored
  76aa30a

  Copy full SHA for 76aa30a

• build : add mac pre-build binaries (#6182)

  Show description for 5e43ba8
  

  3 peopleauthored
  5e43ba8

  Copy full SHA for 5e43ba8

• ci : fix indentation error (#6195)

  Vaibhavs10

  authored
  1943c01

  Copy full SHA for 1943c01

• json-schema-to-grammar improvements (+ added to server) (#5978)

  Show description for 5b7b0ac
  ochafik

  authored
  5b7b0ac

  Copy full SHA for 5b7b0ac

• ggml : same IQ4_NL quantization for CPU/CUDA/Metal (#6196)

  Show description for cfd3be7
  

  ikawrakow

  and

  Kawrakow

  authored
  cfd3be7

  Copy full SHA for cfd3be7

• cuda : fix LLAMA_CUDA_F16 build (#6197)

  slaren

  authored
  03a8f8f

  Copy full SHA for 03a8f8f

• tests : disable system() calls (#6198)

  Show description for 924ce1d
  ggerganov

  authored
  924ce1d

  Copy full SHA for 924ce1d

• Corrected typo to wrong file (#6199)

  Show description for f372c49
  semidark

  authored
  f372c49

  Copy full SHA for f372c49

• cuda : disable host register by default (#6206)

  slaren

  authored
  d0a7123

  Copy full SHA for d0a7123

• server : update readme doc from slot_id to id_slot (#6213)

  kaetemi

  authored
  be07a03

  Copy full SHA for be07a03

Commits on Mar 22, 2024

• Fix params underscore convert to dash. (#6203)

  Show description for fa046ea
  

  dranger003

  and

  slaren

  authored
  fa046ea

  Copy full SHA for fa046ea

• add blog link (#6222)

  NeoZhangJianyu

  authored
  59c17f0

  Copy full SHA for 59c17f0

• metal : pad n_ctx by 32 (#6177)

  Show description for 95d576b
  ggerganov

  authored
  95d576b

  Copy full SHA for 95d576b

• ci : add CURL flag for the mac builds (#6214)

  Vaibhavs10

  authored
  b2075fd

  Copy full SHA for b2075fd

• metal : proper assert for mat-mat memory alignment (#6225)

  Show description for b3e94f2
  ggerganov

  authored
  b3e94f2

  Copy full SHA for b3e94f2

• server : enable continuous batching by default (#6231)

  ggerganov

  authored
  68e210b

  Copy full SHA for 68e210b

• server : fix n_keep always showing as 0 in response (#6211)

  kaetemi

  authored
  6b8bb3a

  Copy full SHA for 6b8bb3a

• readme : add RecurseChat to the list of UIs (#6219)

  xyc

  authored
  29ab270

  Copy full SHA for 29ab270

• cuda : add LLAMA_CUDA_NO_PEER_COPY to workaround broken ROCm p2p copy (#6208)

  Show description for 2f0e81e
  slaren

  authored
  2f0e81e

  Copy full SHA for 2f0e81e

• json-schema-to-grammar : fix order of props + non-str const/enum (#6232)

  Show description for 72114ed
  ochafik

  authored
  72114ed

  Copy full SHA for 72114ed

• tests : conditional python & node json schema tests (#6207)

  Show description for f77a8ff
  ochafik

  authored
  f77a8ff

  Copy full SHA for f77a8ff

• llama : correction of the attn.v.weight quantization for IQ3_XS (#6209)

  Show description for e80f06d
  Nexesenex

  authored
  e80f06d

  Copy full SHA for e80f06d

• common : add HF arg helpers (#6234)

  Show description for 80bd33b
  ggerganov

  authored
  80bd33b

  Copy full SHA for 80bd33b

• ci: apply concurrency limit for github workflows (#6243)

  mscheong01

  authored
  ee804f6

  Copy full SHA for ee804f6

• llama_model_loader: support multiple split/shard GGUFs (#6187)

  Show description for dba1af6
  

  4 peopleauthored
  dba1af6

  Copy full SHA for dba1af6

• quantize: options for output and token embedding tensors qtype (#6239)

  Show description for 1d0331c
  

  ikawrakow

  and

  Kawrakow

  authored
  1d0331c

  Copy full SHA for 1d0331c

• convert-llama2c-to-ggml : enable conversion of GQA models (#6237)

  Show description for 92397d8
  fraxy-v

  authored
  92397d8

  Copy full SHA for 92397d8

• common : default --hf-file to --model (#6234)

  ggerganov

  committed
  56a00f0

  Copy full SHA for 56a00f0

Commits on Mar 23, 2024

• lookup: complement data from context with general text statistics (#5479)

  Show description for 50ccaf5
  JohannesGaessler

  authored
  50ccaf5

  Copy full SHA for 50ccaf5

• server: flush stdout after logging in both text and json layout (#6253)

  phymbert

  authored
  1b26aeb

  Copy full SHA for 1b26aeb

• split: add gguf-split in the make build target (#6262)

  phymbert

  authored
  21cad01

  Copy full SHA for 21cad01

• llama : add grok-1 support (#6204)

  Show description for 476b025
  

  arki05

  and

  ggerganov

  authored
  476b025

  Copy full SHA for 476b025

• server: docs: --threads and --threads, --ubatch-size, --log-disable (#6254)

  phymbert

  authored
  1997577

  Copy full SHA for 1997577

• common: llama_load_model_from_url split support (#6192)

  Show description for f482bb2
  

  phymbert

  and

  ggerganov

  authored
  f482bb2

  Copy full SHA for f482bb2

• gitignore : gguf-split

  ggerganov

  committed
  9556217

  Copy full SHA for 9556217

• use _wfopen instead of fopen on Windows (#6248)

  Show description for 94d1b3b
  cebtenzzre

  authored
  94d1b3b

  Copy full SHA for 94d1b3b

Commits on Mar 24, 2024

• Support build win release for SYCL (#6241)

  Show description for d03224a
  NeoZhangJianyu

  authored
  d03224a

  Copy full SHA for d03224a

• [SYCL] offload op (#6217)

  Show description for ddf6568
  airMeng

  authored
  ddf6568

  Copy full SHA for ddf6568

• sampling : deduplicated code for probability distribution access (#6240)

  Show description for 586e7bc
  mscheong01

  authored
  586e7bc

  Copy full SHA for 586e7bc

• ci : close inactive issue, increase operations per run (#6270)

  phymbert

  authored
  ea279d5

  Copy full SHA for ea279d5

• Fixed lookup compilation issues on Windows (#6273)

  JohannesGaessler

  authored
  7aed0ff

  Copy full SHA for 7aed0ff

• imatrix : fix wname for mul_mat_id ops (#6271)

  Show description for a0e584d
  

  ggerganov

  and

  slaren

  authored
  a0e584d

  Copy full SHA for a0e584d

• Fix heap corruption from wmode out-of-bound writes on windows (#6272)

  Show description for a32b77c
  TheFlipbook

  authored
  a32b77c

  Copy full SHA for a32b77c

Commits on Mar 25, 2024

• ggml : support AVX512VNNI (#6280)

  Show description for 7733f0c
  jart

  authored
  7733f0c

  Copy full SHA for 7733f0c

• examples : add "retrieval" (#6193)

  Show description for 64e7b47
  

  mscheong01

  and

  ggerganov

  authored
  64e7b47

  Copy full SHA for 64e7b47

• [SYCL] fix SYCL backend build on windows is break by LOG() error (#6290)

  Show description for 95ad616
  NeoZhangJianyu

  authored
  95ad616

  Copy full SHA for 95ad616

• Server: clean up OAI params parsing function (#6284)

  Show description for ad3a050
  ngxson

  authored
  ad3a050

  Copy full SHA for ad3a050

• cuda : refactor into multiple files (#6269)

  slaren

  authored
  ae1f211

  Copy full SHA for ae1f211

• cuda : fix LLAMA_CUDA_F16 build (#6298)

  slaren

  authored
  2f34b86

  Copy full SHA for 2f34b86

• flake.lock: Update (#6266)

  Show description for 43139cc
  

  ggerganov

  and

  github-actions[bot]

  authored
  43139cc

  Copy full SHA for 43139cc

• tests : include IQ2_XXS and IQ2_XS in test-quantize-fns (#6303)

  Show description for 1f2fd4e
  

  ikawrakow

  and

  Kawrakow

  authored
  1f2fd4e

  Copy full SHA for 1f2fd4e

• nix: fix blas support (#6281)

  Show description for b06c16e
  ck3d

  authored
  b06c16e

  Copy full SHA for b06c16e