-
Notifications
You must be signed in to change notification settings - Fork 48
/
Copy pathREADME
124 lines (97 loc) · 4.37 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
cmix version 21
Released by Byron Knoll on September 10, 2024.
http://www.byronknoll.com/cmix.html
https://github.com/byronknoll/cmix
cmix is a lossless data compression program aimed at optimizing compression ratio at the cost of high CPU/memory usage. cmix is free software distributed under the GNU General Public License.
cmix works in Linux, Windows, and Mac OS X. At least 32GB of RAM is recommended to run cmix. Feel free to contact me at [email protected] if you have any questions.
Use "make" to compile cmix. In Windows, cmix can be compiled with MinGW (http://nuwen.net/mingw.html) or Cygwin (https://www.cygwin.com).
cmix can only compress/decompress single files. To compress multiple files or directories, create an archive file using "tar" (or some similar tool).
For some files, preprocessing using "precomp" may improve compression: https://github.com/schnaader/precomp-cpp
Compile with "-Ofast -march=native" for fastest performance. These compiler options might lead to incompatibility between different computers due to floating-point precision differences. Compile with "-O3" to fix compatibility issues (at the cost of slower performance).
Instructions for enwik9 preprocessing:
1) Copy enwik9-preproc and dictionary/new_article_order to a new folder.
2) Run: ./enwik9-preproc c path_to_enwik9
3) .ready4cmix is the preprocessed file. To reproduce enwik9, run: ./enwik9-preproc d .ready4cmix
Changes from version 20 to version 21:
- Improvements from fx2-cmix: https://github.com/kaitz/fx2-cmix
Changes from version 19 to version 20:
- Removed PAQ8HP model
- Added fxcm model: https://github.com/kaitz/fxcm
Changes from version 18 to version 19:
- Improvements from STARLIT and cmix-hp
Changes from version 17 to version 18:
- LSTM improvements
Changes from version 16 to version 17:
- New dictionary from phda 2017 November release
- Rewrote WRT dictionary preprocessor
- Improvements ported from paq8px
- Some other minor improvements, bug fixes, and refactoring
Changes from version 15 to version 16:
- Improvements ported from paq8px
- Context mixer improvements
- Removed some redundant models
- LSTM improvements
Changes from version 14 to version 15:
- Improvements ported from paq8px and paq8pxd
- Enabled pretraining
- LSTM tuning
- Context mixer improvements
- Memory tuning
Changes from version 13 to version 14:
- LSTM performance optimizations and tuning
- Changes from Márcio Pais: improved preprocessing and many PAQ8 model improvements
- Disabled pretraining
- New dictionary (from phda)
Changes from version 12 to version 13:
- Improved LSTM and SSE
Changes from version 11 to version 12:
- Replaced byte mixer with LSTM
Changes from version 10 to version 11:
- Added ppmd model
- Memory tuning and bug fixes
Changes from version 9 to version 10:
- Learning rate decay tuning
- Added "interval" context (adapted from MCM https://github.com/mathieuchartier/mcm)
- Added "only preprocessing" compression option
- Refactoring and memory tuning
Changes from version 8 to version 9:
- Improved Windows compatibility
- Learning rate decay
- New nonstationary state table
- Bracket matching models
- Minor bug fixes and memory tuning
Changes from version 7 to version 8:
- Added a recurrent backpropagation neural network to mix byte-level models
- Added some PPM models
- Memory tuning
- Minor refactoring
Changes from version 6 to version 7:
- Fixed bug that caused cmix to crash in Windows
- Tweaked word model
- Added some image models (JPEG/BMP/TIFF)
- Improved preprocessor (JPEG/BMP/text/EXE)
Changes from version 5 to version 6:
- Memory tuning
- Removed PPM model
Changes from version 4 to version 5:
- Added dictionary pretraining
- Removed dictionary from cmix binary
- Integrated some code from paq8pxd_v12 into paq8l
- Minor refactoring
Changes from version 3 to version 4:
- Added an additional mixer context (longest match length)
- Implemented an additional PPM model
- Integrated some code from paq8pxd_v11 into paq8l
- Integrated mixer contexts from paqar into paq8l
- Minor refactoring and tuning
Changes from version 2 to version 3:
- Merged paq8pxd code into paq8l
- Optimized speed of cmix and paq8 mixers
- Minor refactoring and tuning
Changes from version 1 to version 2:
- Memory tuning
- Increase PPM order to 7
- Replaced dictionary with the one used by decmprs8
- Added paq8pxd_v7 models
- Removed pic models
- Added "facade" models to copy the paq8 model predictions to the cmix mixer