Code

Compilade/bitnet ternary#294

Code

Merged

Nexesenex merged 45 commits intoNexesenex:lcpp_pr_bitnet_v2Nexesenex/croco.cpp:lcpp_pr_bitnet_v2from ggerganov:compilade/bitnet-ternaryggerganov/llama.cpp:compilade/bitnet-ternaryCopy head branch name to clipboardAug 14, 2024

+2,376-175

Commits

Commits on Jun 27, 2024

• ggml-quants : 1.625 bpw ternary packing for BitNet 1.58b

  compilade

  committed
  bd80749

  Copy full SHA for bd80749

• ggml-quants : faster 1.625 bpw AVX2 vec_dot

  Show description for 7ef4254
  compilade

  committed
  7ef4254

  Copy full SHA for 7ef4254

• ggml-quants : substract 1 when back in epi8

  Show description for 48b73b8
  compilade

  committed
  48b73b8

  Copy full SHA for 48b73b8

• ggml-quants : Q2_2 now faster than Q4_K on with AVX2

  compilade

  committed
  ef1e345

  Copy full SHA for ef1e345

• ggml-quants : cleanup Q1_3 code formatting

  compilade

  committed
  638ad52

  Copy full SHA for 638ad52

• ggml-quants : ARM NEON vec_dot for q2_2 and q1_3

  compilade

  committed
  9465ec6

  Copy full SHA for 9465ec6

• ggml-quants : use ceiling division when quantizing q1_3

  compilade

  committed
  89dc3b2

  Copy full SHA for 89dc3b2

• convert-hf : simplify BitNet pre-quantization

  Show description for 961e293
  compilade

  committed
  961e293

  Copy full SHA for 961e293

• convert-hf : allow converting the weird BitNet 1.3B

  Show description for 0996149
  compilade

  committed
  0996149

  Copy full SHA for 0996149

Commits on Jun 29, 2024

• bitnet : replace 1.58b with b1.58, as in the paper

  compilade

  committed
  bfd2f21

  Copy full SHA for bfd2f21

• ggml-quants : fix build failure on Windows

  compilade

  committed
  ec50944

  Copy full SHA for ec50944

• ggml-quants : attempt to fix Arm 32-bit support

  compilade

  committed
  8fbd593

  Copy full SHA for 8fbd593

Commits on Jul 29, 2024

• ggml : add some informative comments in q1_3 vec_dot

  compilade

  committed
  dd3e62a

  Copy full SHA for dd3e62a

• Merge branch 'master' into compilade/bitnet-ternary

  compilade

  committed
  79a278e

  Copy full SHA for 79a278e

Commits on Jul 30, 2024

• ggml : add TQ1_0 and TQ2_0 ternary quantization types

  compilade

  committed
  77b8f84

  Copy full SHA for 77b8f84

Commits on Jul 31, 2024

• ggml : even faster TQ2_0

  compilade

  committed
  560873f

  Copy full SHA for 560873f

• ggml : also faster TQ1_0

  Show description for e971957
  compilade

  committed
  e971957

  Copy full SHA for e971957

Commits on Aug 1, 2024

• ggml : fix build issues in certain environments

  compilade

  committed
  a6dd699

  Copy full SHA for a6dd699

• ggml : add NEON vec_dot implementation for TQ1_0 and TQ2_0

  compilade

  committed
  5417089

  Copy full SHA for 5417089

• ggml : avoid directly using vmlal_high_s8, for 32-bit ARM compat

  Show description for 45719a2
  compilade

  committed
  45719a2

  Copy full SHA for 45719a2

Commits on Aug 3, 2024

• ggml : remove q1_3 and q2_2

  Show description for 04eec58
  compilade

  committed
  04eec58

  Copy full SHA for 04eec58

• ggml-quants : rename fields of TQ1_0 and TQ2_0 structs for consistency

  compilade

  committed
  f034aa1

  Copy full SHA for f034aa1

Commits on Aug 7, 2024

• ggml-quants : allow using vdotq_s32 in TQ2_0 vec_dot

  Show description for 96b3d41
  compilade

  committed
  96b3d41

  Copy full SHA for 96b3d41

Commits on Aug 11, 2024

• Optimize Vulkan backend for better CPU performance and less GPU synchronization overhead. (#8943)

  Show description for 7c5bfd5
  

  mtavenrath

  and

  0cc4m

  authored
  7c5bfd5

  Copy full SHA for 7c5bfd5

• llama : check all graph nodes when searching for result_embd_pooled (#8956)

  Show description for 33309f6
  

  fairydreaming

  and

  sszymczy

  authored
  33309f6

  Copy full SHA for 33309f6

• update guide (#8909)

  Show description for a21c6fd
  arthw

  authored
  a21c6fd

  Copy full SHA for a21c6fd

• flake.lock: Update (#8979)

  ggerganov

  authored
  8cd1bcf

  Copy full SHA for 8cd1bcf

• gguf-py : Numpy dequantization for most types (#8939)

  Show description for 4134999
  compilade

  authored
  4134999

  Copy full SHA for 4134999

• Merge branch 'master' into compilade/bitnet-ternary

  compilade

  committed
  d911cd1

  Copy full SHA for d911cd1

Commits on Aug 12, 2024

• gguf-py : Numpy (de)quantization for TQ1_0 and TQ2_0

  Show description for 3a0bf17
  compilade

  committed
  3a0bf17

  Copy full SHA for 3a0bf17

• server : handle models with missing EOS token (#8997)

  Show description for 5ef07e2
  ggerganov

  authored
  5ef07e2

  Copy full SHA for 5ef07e2

• py : fix requirements check '==' -> '~=' (#8982)

  Show description for d3ae0ee
  ggerganov

  authored
  d3ae0ee

  Copy full SHA for d3ae0ee

• Fix a spelling mistake (#9001)

  Septa2112

  authored
  2589292

  Copy full SHA for 2589292

• ggml: fix div-by-zero (#9003)

  Show description for df5478f
  DavidKorczynski

  authored
  df5478f

  Copy full SHA for df5478f

• grammar-parser : fix possible null-deref (#9004)

  Show description for 1262e7e
  DavidKorczynski

  authored
  1262e7e

  Copy full SHA for 1262e7e

• docs: introduce gpustack and gguf-parser (#8873)

  Show description for 84eb2f4
  thxCode

  authored
  84eb2f4

  Copy full SHA for 84eb2f4

• llama : model-based max number of graph nodes calculation (#8970)

  Show description for 0fd93cd
  

  nicoboss

  and

  slaren

  authored
  0fd93cd

  Copy full SHA for 0fd93cd

• ci : enable RPC in all of the released builds (#9006)

  Show description for 1f67436
  rgerganov

  authored
  1f67436

  Copy full SHA for 1f67436

• ci : fix github workflow vulnerable to script injection (#9008)

  Show description for fc4ca27
  diogoteles08

  authored
  fc4ca27

  Copy full SHA for fc4ca27

Commits on Aug 13, 2024

• export-lora : throw error if lora is quantized (#9002)

  ngxson

  authored
  828d6ff

  Copy full SHA for 828d6ff

• ggml : move rope type enum to ggml.h (#8949)

  Show description for 06943a6
  

  danbev

  and

  slaren

  authored
  06943a6

  Copy full SHA for 06943a6

• convert : allow direct conversion to TQ1_0 and TQ2_0

  Show description for 895004f
  compilade

  committed
  895004f

  Copy full SHA for 895004f

• ggml-quants : allow using ARM dot product instructions for TQ1_0

  compilade

  committed
  69f7726

  Copy full SHA for 69f7726

• Merge branch 'master' into compilade/bitnet-ternary

  compilade

  committed
  82b2404

  Copy full SHA for 82b2404

• ggml-quants : deduplicate TQ1_0 and TQ2_0 __ARM_FEATURE_DOTPROD support

  compilade

  committed
  35cc556

  Copy full SHA for 35cc556