Code

merge from upstream#48

Code

Merged

l3utterfly merged 83 commits intolayla-buildl3utterfly/llama.cpp:layla-buildfrom masterl3utterfly/llama.cpp:masterCopy head branch name to clipboardDec 11, 2024

+13,888-33,246

Commits

Commits on Nov 29, 2024

• sycl : offload of get_rows set to 0 (ggerganov#10432)

  Alcpz

  authored
  0f77aae

• ggml-cpu: fix typo in gemv/gemm iq4_nl_4_4 (ggerganov#10580)

  FanShupei

  authored
  4b3242b

• ggml : fix I8MM Q4_1 scaling factor conversion (ggerganov#10562)

  Show description for f0678c5
  ggerganov

  authored
  f0678c5

• cleanup UI link list (ggerganov#10577)

  Show description for a3a3048
  slaren

  authored
  a3a3048

• imatrix : support combine-only (ggerganov#10492)

  Show description for 3a8e9af
  robbiemu

  authored
  3a8e9af

• server : add more test cases (ggerganov#10569)

  Show description for b782e5c
  ngxson

  authored
  b782e5c

• ggml : move AMX to the CPU backend (ggerganov#10570)

  Show description for 7cc2d2c
  

  slaren

  and

  ggerganov

  authored
  7cc2d2c

Commits on Nov 30, 2024

• vulkan: Dynamic subgroup size support for Q6_K mat_vec (ggerganov#10536)

  Show description for 0533e7f
  netrunnereve

  authored
  0533e7f

• readme : refresh (ggerganov#10587)

  Show description for abadba0
  ggerganov

  authored
  abadba0

• readme : remove old badge

  ggerganov

  authored
  3e0ba0e

• ggml-cpu: replace AArch64 NEON assembly with intrinsics in ggml_gemv_q4_0_4x4_q8_0() (ggerganov#10567)

  Show description for 0c39f44
  angt

  authored
  0c39f44

Commits on Dec 1, 2024

• build: update Makefile comments for C++ version change (ggerganov#10598)

  wangqin0

  authored
  43957ef

• readme : update the usage section with examples (ggerganov#10596)

  Show description for 6acce39
  ggerganov

  authored
  6acce39

• server : bind to any port when specified (ggerganov#10590)

  alek3y

  authored
  86dc11c

• ggml : automatic selection of best CPU backend (ggerganov#10606)

  Show description for 3420909
  slaren

  authored
  3420909

• ci: add error handling for Python venv creation in run.sh (ggerganov#10608)

  wangqin0

  authored
  5c7a5aa

• grammars : add English-only grammar (ggerganov#10612)

  ggerganov

  authored
  5e1ed95

• Add mistral-v1, mistral-v3, mistral-v3-tekken and mistral-v7 chat template types (ggerganov#10572)

  Show description for 917786f
  jukofyork

  authored
  917786f

Commits on Dec 2, 2024

• contrib : refresh (ggerganov#10593)

  Show description for 4cb003d
  ggerganov

  authored
  4cb003d

• SYCL: Fix and switch to GGML_LOG system instead of fprintf (ggerganov#10579)

  Show description for 991f8aa
  qnixsynapse

  authored
  991f8aa

• server: Add "tokens per second" information in the backend (ggerganov#10548)

  Show description for 64ed209
  

  lhpqaq

  and

  ngxson

  authored
  64ed209

• make : deprecate (ggerganov#10514)

  Show description for 8648c52
  

  ggerganov

  and

  slaren

  authored
  8648c52

• llama : add enum for built-in chat templates (ggerganov#10623)

  Show description for 642330a
  ngxson

  authored
  642330a

Commits on Dec 3, 2024

• server : fix default draft model parameters (ggerganov#10586)

  Show description for 70b98fa
  ggerganov

  authored
  70b98fa

• github : minify link [no ci]

  ggerganov

  committed
  844e2e1

• github : minify link [no ci] (revert)

  Show description for 515d4e5
  ggerganov

  committed
  515d4e5

• metal : small-batch mat-mul kernels (ggerganov#10581)

  Show description for 0115df2
  ggerganov

  authored
  0115df2

• readme : add option, update default value, fix formatting (ggerganov#10271)

  Show description for 82bca22
  pothitos

  authored
  82bca22

• llama : add missing LLAMA_API for llama_chat_builtin_templates (ggerganov#10636)

  ngxson

  authored
  3b4f2e3

• metal : add GGML_OP_CONV_TRANSPOSE_1D kernels (ggml/1026)

  Show description for 667d70d
  

  PABannier

  authored and

  ggerganov

  committed
  667d70d

• feat: add GGML_UNARY_OP_ARGMAX Metal kernel (ggml/1019)

  Show description for efb6ae9
  

  2 peopleauthored and

  ggerganov

  committed
  efb6ae9

• CUDA: remove unnecessary warp reduce in FA (ggml/1032)

  Show description for e9e661b
  

  2 peopleauthored and

  ggerganov

  committed
  e9e661b

• sync : ggml

  ggerganov

  committed
  c505471

• scripts : remove amx sync

  Show description for 1cd3df4
  ggerganov

  committed
  1cd3df4

• server : (web ui) Various improvements, now use vite as bundler (ggerganov#10599)

  Show description for 91c36c2
  ngxson

  authored
  91c36c2

• vulkan: optimize and reenable split_k (ggerganov#10637)

  Show description for cc98896
  jeffbolznv

  authored
  cc98896

Commits on Dec 4, 2024

• clip : add sycl support (ggerganov#10574)

  Show description for 01e6d9b
  

  piDack

  and

  piDack

  authored
  01e6d9b

• Add docs for creating a static build (ggerganov#10268) (ggerganov#10630)

  Show description for da6aac9
  

  mostlygeek

  and

  slaren

  authored
  da6aac9

• Avoid using __fp16 on ARM with old nvcc (ggerganov#10616)

  frankier

  authored
  cd2f37b

• fix typo of README.md (ggerganov#10605)

  WrRan

  authored
  98036d5

• SYCL : Move to compile time oneMKL interface backend selection for NVIDIA backend (ggerganov#10584)

  Show description for 40c6d79
  s-Nick

  authored
  40c6d79

• vulkan: Implement "fast divide" (mul+shift) for unary ops like copy (ggerganov#10642)

  jeffbolznv

  authored
  2759916

• llama: Support MiniCPM-1B (with & w/o longrope) (ggerganov#10559)

  JFLFY2255

  authored
  8d0cfd5

• Fix HF repo commit to clone lora test models (ggerganov#10649)

  ltoniazzi

  authored
  253b7fd

• ggml-cpu : fix HWCAP2_I8MM value (ggerganov#10646)

  slaren

  authored
  2803540

• ggml : add predefined list of CPU backend variants to build (ggerganov#10626)

  Show description for 59f4db1
  slaren

  authored
  59f4db1

• server : fix speculative decoding with context shift (ggerganov#10641)

  Show description for 1da7b76
  ggerganov

  authored
  1da7b76

• Update deprecation-warning.cpp (ggerganov#10619)

  Show description for f112d19
  aryantandon01

  authored
  f112d19

Commits on Dec 5, 2024

• py : update outdated copy-paste instructions [no ci] (ggerganov#10667)

  Show description for d405804
  danbev

  authored
  d405804

• ggml : add GGML_PAD_REFLECT_1D operation (ggml/1034)

  Show description for c2082d9
  

  PABannier

  authored and

  ggerganov

  committed
  c2082d9

• ggml: add GGML_SET Metal kernel + i32 CPU kernel (ggml/1037)

  Show description for a8cbab2
  

  PABannier

  authored and

  ggerganov

  committed
  a8cbab2

• sync : ggml

  ggerganov

  committed
  0cd182e

• llama : add Minerva 7B model support (ggerganov#10673)

  Show description for 6fe6247
  Riccorl

  authored
  6fe6247

• vulkan: Add VK_NV_cooperative_matrix2 support for mul_mat and flash attention (ggerganov#10206)

  jeffbolznv

  authored
  c9c6e01

• fix(server) : not show alert when DONE is received (ggerganov#10674)

  pminev

  authored
  7736837

Commits on Dec 6, 2024

• server : (refactoring) do not rely on JSON internally (ggerganov#10643)

  Show description for 6c5bc06
  ngxson

  authored
  6c5bc06

• common : bring back --no-warmup to server (ggerganov#10686)

  ngxson

  authored
  f162d45

• convert : add custom attention mapping

  ggerganov

  committed
  c5ede38

Commits on Dec 7, 2024

• convert : add support for Roberta embeddings (ggerganov#10695)

  Ssukriti

  authored
  784a14a

• metal : Extend how Llama.cpp locates metal resources (ggerganov#10676)

  Show description for 86a1934
  

  ormandi

  and

  ggerganov

  authored
  86a1934

• Vulkan: VK_KHR_cooperative_matrix support to speed up prompt processing (ggerganov#10597)

  Show description for 3df784b
  0cc4m

  authored
  3df784b

• server : fix free of spec context and batch (ggerganov#10651)

  Show description for c2a16c0
  ggerganov

  authored
  c2a16c0

• ggml : refactor online repacking (ggerganov#10446)

  Show description for 19d8762
  

  Djip007

  and

  ggerganov

  authored
  19d8762

• server : various fixes (ggerganov#10704)

  Show description for ce4a7b8
  

  ggerganov

  and

  ngxson

  authored
  ce4a7b8

• ggml : disable iq4_nl interleave size 8 (ggerganov#10709)

  Show description for d9c3ba2
  ggerganov

  authored
  d9c3ba2

• server : (refactor) no more json in server_task input (ggerganov#10691)

  Show description for 3573fa8
  ngxson

  authored
  3573fa8

• llama : add 128k yarn context for Qwen (ggerganov#10698)

  Show description for 62e84d9
  robbiemu

  authored
  62e84d9

Commits on Dec 8, 2024

• vulkan: compile a test shader in cmake to check for coopmat2 support (ggerganov#10713)

  jeffbolznv

  authored
  ecc93d0

• llama : use cmake for swift build (ggerganov#10525)

  Show description for 43ed389
  

  slaren

  and

  ggerganov

  authored
  43ed389

• Vulkan: fix NaN in tanh.comp with AMD proprietary driver on Windows (ggerganov#10723)

  Show description for 06d7014
  stduhpf

  authored
  06d7014

• server : bring back info of final chunk in stream mode (ggerganov#10722)

  Show description for e52522b
  ngxson

  authored
  e52522b

• server : fix format_infill (ggerganov#10724)

  Show description for ce8784b
  ngxson

  authored
  ce8784b

Commits on Dec 9, 2024

• cmake : simplify msvc charsets (ggerganov#10672)

  iboB

  authored
  1a05004

• vulkan: fix compile warnings (ggerganov#10731)

  jeffbolznv

  authored
  3d98b4c

• Changes to CMakePresets.json to add ninja clang target on windows (ggerganov#10668)

  Show description for c37fb4c
  

  Srihari-mcw

  and

  max-krasnyansky

  authored
  c37fb4c

• CUDA: fix shared memory access condition for mmv (ggerganov#10740)

  JohannesGaessler

  authored
  26a8406

Commits on Dec 10, 2024

• vulkan: disable spirv-opt for coopmat shaders (ggerganov#10763)

  Show description for a05e2af
  jeffbolznv

  authored
  a05e2af

• server : add flag to disable the web-ui (ggerganov#10762) (ggerganov#10751)

  Show description for a86ad84
  

  eugeniosegala

  and

  eugenio.segala

  authored
  a86ad84

• CUDA: rename macros to avoid conflicts with WinAPI (ggerganov#10736)

  Show description for 750cb3e
  aendk

  authored
  750cb3e

• imatrix : Add imatrix to --no-context-shift (ggerganov#10766)

  Show description for ae4b922
  bartowski1182

  authored
  ae4b922

• vulkan: dynamic subgroup size for the remaining k quants (ggerganov#10745)

  Show description for dafae66
  netrunnereve

  authored
  dafae66

• vulkan: request round-to-even for fp16 in im2col/rope_head (ggerganov#10767)

  Show description for b685daf
  jeffbolznv

  authored
  b685daf

Commits on Dec 11, 2024

• ggml: load all backends from a user-provided search path (ggerganov#10699)

  Show description for 43041d2
  

  giladgd

  and

  slaren

  authored
  43041d2