Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history 
Kanzi 2.2
  • Loading branch information
@flanglet
flanglet authored Nov 4, 2023
1 parent 85777ec commit b1039ce
Showing 1 changed file with 64 additions and 82 deletions.
146 changes: 64 additions & 82 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
kanzi
=====
# kanzi


Kanzi is a modern, modular, expandable and efficient lossless data compressor implemented in Go.
Expand Down Expand Up @@ -44,93 +43,76 @@ Use at your own risk. Always keep a copy of your original files.
[![Lines of Code](https://sonarcloud.io/api/project_badges/measure?project=flanglet_kanzi-go&metric=ncloc)](https://sonarcloud.io/summary/new_code?id=flanglet_kanzi-go)


Silesia corpus benchmark
-------------------------

i7-7700K @4.20GHz, 32GB RAM, Ubuntu 22.04

go1.19.beta1
## Benchmarks

Kanzi version 2.1 Go implementation. Block size is 100 MB.
Test machine:

AWS c5a8xlarge: AMD EPYC 7R32 (32 vCPUs), 64 GB RAM

go 1.21.3

Ubuntu 22.04.3 LTS

Kanzi version 2.2

On this machine kanzi can use up to 16 threads depending on compression level
(the default block size at level 9 is 32MB, severly limiting the number of threads
in use, especially with enwik8, but all tests are performed with default values).
bzip3 uses 16 threads. zstd can use 2 for compression, other compressors
are single threaded.


### silesia.tar

| Compressor | Encoding (sec) | Decoding (sec) | Size |
|---------------------------------|-----------------|-----------------|------------------|
|Original | | | 211,938,580 |
|**Kanzi -l 1 -j 1** | **1.9** | **1.0** | **69,399,477** |
|**Kanzi -l 1 -j 6** | **0.7** | **0.4** | **69,399,477** |
|Pigz 2.6 -5 -p6 | 1.0 | 0.7 | 69,170,603 |
|Gzip 1.10 -5 | 4.8 | 1.0 | 69,143,980 |
|Zstd 1.5.3 -2 --long=30 | 0.9 | 0.5 | 68,694,316 |
|Zstd 1.5.3 -2 -T6 --long=30 | 0.4 | 0.3 | 68,694,316 |
|Brotli 1.0.9 -2 --large_window=30| 1.5 | 0.8 | 68,033,377 |
|Pigz 2.6 -9 -p6 | 3.0 | 0.6 | 67,656,836 |
|Gzip 1.10 -9 | 15.5 | 1.0 | 67,631,990 |
|Brotli 1.0.9 -4 --large_window=30| 4.1 | 0.7 | 64,267,169 |
|**Kanzi -l 2 -j 1** | **3.1** | **1.2** | **63,808,747** |
|**Kanzi -l 2 -j 6** | **1.2** | **0.5** | **63,808,747** |
|Zstd 1.5.3 -9 --long=30 | 3.7 | 0.3 | 59,272,590 |
|Zstd 1.5.3 -9 -T6 --long=30 | 2.3 | 0.3 | 59,272,590 |
|**Kanzi -l 3 -j 1** | **5.4** | **2.1** | **59,199,795** |
|**Kanzi -l 3 -j 6** | **1.8** | **0.8** | **59,199,795** |
|Orz 1.5.0 | 7.7 | 2.0 | 57,564,831 |
|Brotli 1.0.9 -9 --large_window=30| 36.7 | 0.7 | 56,232,817 |
|Lzma 5.2.2 -3 | 24.1 | 2.6 | 55,743,540 |
|**Kanzi -l 4 -j 1** | **9.7** | **5.8** | **54,998,198** |
|**Kanzi -l 4 -j 6** | **3.7** | **2.2** | **54,998,198** |
|Bzip2 1.0.6 -9 | 14.9 | 5.2 | 54,506,769 |
|Zstd 1.5.3 -19 --long=30 | 62.0 | 0.3 | 52,828,057 |
|Zstd 1.5.3 -19 -T6 --long=30 | 62.0 | 0.4 | 52,828,057 |
|**Kanzi -l 5 -j 1** | **12.2** | **5.4** | **51,760,244** |
|**Kanzi -l 5 -j 6** | **4.2** | **2.0** | **51,760,244** |
|Brotli 1.0.9 --large_window=30 | 356.2 | 0.9 | 49,383,136 |
|Lzma 5.2.2 -9 | 65.6 | 2.5 | 48,780,457 |
|**Kanzi -l 6 -j 1** | **14.3** | **8.4** | **48,068,000** |
|**Kanzi -l 6 -j 6** | **4.8** | **2.8** | **48,068,000** |
|bsc 3.2.3 -b100 -T -t | 8.8 | 6.0 | 46,932,394 |
|bsc 3.2.3 -b100 | 5.4 | 4.9 | 46,932,394 |
|BCM 1.65 -b100 | 15.5 | 21.1 | 46,506,716 |
|**Kanzi -l 7 -j 1** | **19.4** | **13.3** | **46,447,003** |
|**Kanzi -l 7 -j 6** | **6.8** | **5.1** | **46,447,003** |
|Tangelo 2.4 | 83.2 | 85.9 | 44,862,127 |
|zpaq v7.14 m4 t1 | 107.3 | 112.2 | 42,628,166 |
|zpaq v7.14 m4 t12 | 108.1 | 111.5 | 42,628,166 |
|**Kanzi -l 8 -j 1** | **60.7** | **60.8** | **41,821,127** |
|**Kanzi -l 8 -j 6** | **24.5** | **20.9** | **41,821,127** |
|Tangelo 2.0 | 302.0 | 310.9 | 41,267,068 |
|**Kanzi -l 9 -j 1** | **87.5** | **90.0** | **40,361,391** |
|**Kanzi -l 9 -j 6** | **39.2** | **34.5** | **40,361,391** |
|zpaq v7.14 m5 t1 | 343.1 | 352.0 | 39,112,924 |
|zpaq v7.14 m5 t12 | 344.3 | 350.4 | 39,112,924 |



enwik8
-------

i7-7700K @4.20GHz, 32GB RAM, Ubuntu 22.04

go1.19.beta1

Kanzi version 2.1 Go implementation. Block size is 100 MB. 1 thread


| Compressor | Encoding (sec) | Decoding (sec) | Size |
|-----------------------------|-----------------|-----------------|------------------|
|Original | | | 100,000,000 |
|**Kanzi -l 1 -j 1** | **1.10** | **0.42** | **37,969,539** |
|**Kanzi -l 2 -j 1** | **1.90** | **0.63** | **30,953,719** |
|**Kanzi -l 3 -j 1** | **2.68** | **1.05** | **27,362,969** |
|**Kanzi -l 4 -j 1** | **4.73** | **2.82** | **25,670,924** |
|**Kanzi -l 5 -j 1** | **4.85** | **1.94** | **22,490,875** |
|**Kanzi -l 6 -j 1** | **6.57** | **3.32** | **21,232,300** |
|**Kanzi -l 7 -j 1** | **9.44** | **5.98** | **20,935,519** |
|**Kanzi -l 8 -j 1** | **23.70** | **23.50** | **19,671,786** |
|**Kanzi -l 9 -j 1** | **34.09** | **35.61** | **19,097,946** |


Build
-----
|Original | | | 211,957,760 |
|s2 -cpu 16 | 0.285 | 1.042 | 86,650,932 |
|**Kanzi -l 1** | **0.683** | **0.255** | **80,284,705** |
|Zstd 1.5.5 -2 | 0.761 | 0.286 | 69,590,245 |
|**Kanzi -l 2** | **0.707** | **0.302** | **68,231,498** |
|Brotli 1.1.0 -2 | 1.749 | 2.459 | 68,044,145 |
|Gzip 1.10 -9 | 20.15 | 1.316 | 67,652,229 |
|**Kanzi -l 3** | **1.204** | **0.368** | **64,916,444** |
|**Kanzi -l 4** | **1.272** | **0.681** | **60,770,201** |
|Zstd 1.5.5 -9 | 4.166 | 0.282 | 59,444,065 |
|Brotli 1.1.0 -6 | 14.53 | 4.263 | 58,552,177 |
|Zstd 1.5.5 -15 | 19.15 | 0.276 | 58,061,115 |
|Brotli 1.1.0 -9 | 70.07 | 7.149 | 56,408,353 |
|Bzip2 1.0.8 -9 | 16.94 | 6.734 | 54,572,500 |
|**Kanzi -l 5** | **2.355** | **1.055** | **54,051,139** |
|Zstd 1.5.5 -19 | 92.82 | 0.302 | 52,989,654 |
|**Kanzi -l 6** | **3.414** | **2.235** | **49,517,823** |
|Lzma 5.2.5 -9 | 92.6 | 3.075 | 48,744,632 |
|**Kanzi -l 7** | **4.387** | **3.098** | **47,308,484** |
|bzip3 1.3.2.r4-gb2d61e8 -j 16 | 2.682 | 3.221 | 47,237,088 |
|**Kanzi -l 8** | **19.64** | **21.33** | **43,247,248** |
|**Kanzi -l 9** | **42.41** | **48.37** | **41,807,179** |
|zpaq 7.15 -m5 -t16 | 213.8 | 213.8 | 40,050,429 |



### enwik8

| Compressor | Encoding (sec) | Decoding (sec) | Size |
|------------------------|------------------|------------------|------------------|
|Original | | | 100,000,000 |
|**Kanzi -l 1** | **0.465** | **0.171** | **43,747,730** |
|**Kanzi -l 2** | **0.481** | **0.196** | **37,745,093** |
|**Kanzi -l 3** | **0.761** | **0.301** | **33,839,184** |
|**Kanzi -l 4** | **0.764** | **0.472** | **29,598,635** |
|**Kanzi -l 5** | **0.896** | **0.494** | **26,527,955** |
|**Kanzi -l 6** | **1.433** | **1.104** | **24,076,669** |
|**Kanzi -l 7** | **3.093** | **2.277** | **22,817,376** |
|**Kanzi -l 8** | **13.63** | **13.17** | **21,181,978** |
|**Kanzi -l 9** | **32.44** | **34.11** | **20,035,133** |




# Build

It is recommended to use formal releases (see https://github.com/flanglet/kanzi-go/releases).

Expand Down

0 comments on commit b1039ce

Please sign in to comment.