v0.8.0

What's New in libchewing 0.8.0 (May 14, 2024)

BREAKING CHANGE

New dictionary format is not compatible with old tools. Existing user dictionary
will be automatically backed up and migrated however old tools might not be able
to manipulate the dictionary directly.

The chewing-editor program can still be used to edit user dictionaries. In
this release we also introduced a new chewing-cli tool to inspect/dump/build
user dictionaries. See chewing-cli(1) for more details.

This version also closed/fixed many issues. Find the complete list of closed
issues here: https://github.com/chewing/libchewing/milestone/7?closed=1

• Added

  • A new command line tool chewing-cli is included in this release. It can
    create or inspect dictionary files.
  • Reintroduce DictionaryMut as a separate trait.

• Changed

  • CMake default preset switched to build with rust.
  • CMake minimum required version bumped to 3.24.0.
  • System dictionary and user dictionary file now uses platform standard paths.
    See https://docs.rs/chewing/latest/chewing/path/fn.data_dir.html for
    details.
  • Improved TrieBufDictionary::from ergonomics.
  • Rename DictEntries to Entries.
  • Ensure all fallible API returns Error.
  • Improved algorithm so using Tab key to find alternative phrasing is more
    responsive.
  • Easy symbol input now accepts arbitrary long "symbols". (rust)
  • Removed duplicated symbols "●" and "○" from symbols input.

• Buf fixed

  • chewing_cand_CheckDone result was reversed in capi (introduced in v0.7.0)
  • Correctly ignore instead of absorb keys when the pre-edit buffer is empty.
    (introduced in v0.6.0)
  • Reset pagination page number when searching for next candidate range.
    (introduced in v0.7.0)
  • chewing_bopomofo_String_static did not return key sequence for pinyin input
    mode. (introduced in v0.6.0)
  • Different chewing_*_static() functions were using one shared global buffer.
    (introduced in v0.6.0)
  • Special symbol selection missed a couple symbols that was in C version.
    (introduced in v0.7.0)
  • Panics when a new selection replaces more than one existing selections.
    (introduced in v0.7.0)
  • Selection created from alternative bopomofo was not considered in
    conversion. (introduced in v0.6.0)

• Removed

  • HashMap implemented Dictionary was removed.

• Documentation

  • Clarify the accepted value of chewing_handle_Default method.

Full Changelog: v0.7.0...v0.8.0

v0.8.0-rc.4

Download the source tarball from github releases:

https://github.com/chewing/libchewing/releases/tag/v0.8.0-rc.4

libchewing releases can be verified with the following minisign public key

minisign -Vm libchewing-0.8.0-rc.4.tar.zst \
         -P RWRzJFnXiLZleAyCIv1talBjyRewelcy9gzYQq9pd3SKSFBPoy57sf5s

Buf fixed

• Special symbol selection missed a couple symbols that was in C version. (introduced in v0.7.0)
• Panics when a new selection replaces more than one existing selections. (introduced in v0.7.0)
• Selection created from alternative bopomofo was not considered in conversion. (introduced in v0.6.0)

Full Changelog: v0.8.0-rc.3...v0.8.0-rc.4

v0.8.0-rc.3

Download the source tarball from github releases:

https://github.com/chewing/libchewing/releases/tag/v0.8.0-rc.3

libchewing releases can be verified with the following minisign public key

minisign -Vm libchewing-0.8.0-rc.3.tar.zst \
         -P RWRzJFnXiLZleAyCIv1talBjyRewelcy9gzYQq9pd3SKSFBPoy57sf5s

Buf fixed

• Correctly ignore instead of absorb keys when the pre-edit buffer is empty. (introduced in v0.6.0)
• Reset pagination page number when searching for next candidate range. (introduced in v0.7.0)
• chewing_bopomofo_String_static did not return key sequence for pinyin input mode. (introduced in v0.6.0)
• Different chewing_*_static() functions were using one shared global buffer. (introduced in v0.6.0)

Full Changelog: v0.8.0-rc.2...v0.8.0-rc.3

v0.8.0-rc.2

Download the source tarball from github releases:

https://github.com/chewing/libchewing/releases/tag/v0.8.0-rc.2

libchewing releases can be verified with the following minisign public key

minisign -Vm libchewing-0.8.0-rc.2.tar.zst \
         -P RWRzJFnXiLZleAyCIv1talBjyRewelcy9gzYQq9pd3SKSFBPoy57sf5s

Buf fixed

• chewing_cand_CheckDone result was reversed in capi (introduced in v0.7.0)

Full Changelog: v0.8.0-rc.1...v0.8.0-rc.2

v0.8.0-rc.1

Hi,

大家好，從 2022 年 4/29 開始的 Rust Rewrite (89ef2a8) 終於進入一個新的里程碑！
0.8.0 將會是第一個完全用 Rust 重寫的 libchewing 的第一個 release，希望大家幫忙
多多測試。

重大變更：新字典格式與舊有工具不相容

現有使用者字典將會自動備份及轉移，但舊有工具將無法直接操作字典檔案。
請使用新的 chewing-cli 工具來檢視、傾出或建構使用者字典。最終版本將提供更新的文件說明。

Download the source tarball from github releases:

https://github.com/chewing/libchewing/releases/tag/v0.8.0-rc.1

libchewing releases can be verified with the following minisign public key

minisign -Vm libchewing-0.8.0-rc.1.tar.zst \
         -P RWRzJFnXiLZleAyCIv1talBjyRewelcy9gzYQq9pd3SKSFBPoy57sf5s

What's New in libchewing 0.8.0-rc.1 (UNRELEASED)

BREAKING CHANGE: New dictionary format is not compatible with old tools.
Existing user dictionary will be automatically backed up and migrated
however old tools will not be able to manipulate the dictionary directly.
Use the new chewing-cli tool to inspect/dump/build user dictionaries.
Updated documentation will be provided in final release.

• Added

  • A new command line tool chewing-cli is included in this release. It can
    create or inspect dictionary files.
  • Reintroduce DictionaryMut as a separate trait.

• Changed

  • CMake default preset switched to build with rust.
  • CMake minimum required version bumped to 3.24.0.
  • Improved TrieBufDictionary::from ergonomics.
  • Rename DictEntries to Entries.
  • Ensure all fallible API returns Error.
  • Improved algorithm so using Tab key to find alternative phrasing is more
    responsive.
  • Easy symbol input now accepts arbitrary long "symbols". (rust)

• Removed

  • HashMap implemented Dictionary was removed.

• Documentation

  • Clarify the accepted value of chewing_handle_Default method.

Full Changelog: v0.7.0...v0.8.0-rc.1

v0.7.0

What's New in libchewing 0.7.0 (Apr 7, 2024)

This is the first beta release of the Rust rewrite. The library has been tested
with the AFL++ fuzzer and most of the obvious bugs have been fixed so it should
not crash for normal use. Now we invite power users to daily drive the beta
release in order to find more subtle bugs.

The Rust rewrite is ABI compatiable with previous releases. The most notable
difference with the C version includes a new portable dictionary format that can
be used both as system dictionary and user dictionary. Users with existing user
phrase dictionary will be automatically migrated. Old files will be backed up
automatically. With these change we will be able to support dictionary sharing
and loading multiple dictionaries in future releases.

• Future roadmap

  • 0.8.x after bug bashing this version will be the first public release with
    rust implementation as the default.
  • 0.9.x will focus on the stablization of the Rust crate API.
  • 1.0.0 will be the first public release of the Rust API. C implementation
    will be removed from the repository.

• Added

  • Added Colemak-DH ANSI/Orth layout support
  • Replace bespoke RIFF+TLV file format with standard DER format (rust)
  • Supporting migrating user dictionary file to new format. (rust)
  • Allow creating in memory user dictionary (rust)
  • New chewing-cli tool can be used to create or inspect dictionary files. (rust)
  • New chewing-cli tool can dump dictionary as or import from CSV file. (rust)
  • Audit dependencies using cargo-vet (rust)

• Changed

  • Buffer user dictionary changes and flush in background without blocking main
    thread (rust)

• Bug fixed

  • Some unsigned underflow issue found by afl++ (rust)
  • Skip symbol selection if preedit buffer is empty (rust)
  • Allow numlock in selection mode (rust)
  • Avoid infinite recursion (rust)
  • Ensure to return KeyBehavior::Commit when we push to commit buffer (rust)
  • Adjust selection offset after delete symbols (rust)
  • Disable trace logging if no logger is enabled (rust)
  • Should not start selection in English mode (rust)
  • Use offset to select from paginated phrase list (rust)

• Documentation

  • Added document for fuzzers (rust)
  • Describe how to compile dictionary files
  • Update build instruction and Rust MSRV info

• Testing

  • Added a fuzzer harness with afl++ (rust)

New Contributors

• @mlouielu made their first contribution in #441

Full Changelog: v0.6.0...v0.7.0

v0.7.0-rc.2

[0.7.0-rc.2] - 2024-03-31

🐛 Bug Fixes

• (editor) Ensure to return KeyBehavior::Commit when we push to commit buffer
• (conv) Adjust selection offset after delete symbols
• (logging) Disable trace logging if no logger is enabled
• (editor) Should not start selection in English mode

🚜 Refactor

• Fix more clippy warnings

🧪 Testing

• (fuzzing) Disable checksum for fuzzing build
• (fuzzing) Test that pre-edit buffer does not suddenly disappear
• (fuzzing) Use more bits from input randomness

⚙️ Miscellaneous Tasks

• Bump pre-release version

Build

• Rename chewing-testhelper to chewing_testhelper
• (vet) Add cargo-vet configs
• (vet) First round of audits
• Change source archive to use zstd
• (vet) Another round of audits

v0.7.0-rc.1

What's New in libchewing 0.7.0-rc.1 (Mar 25, 2024)

This is the first beta release of the Rust rewrite. The library has been tested
with the AFL++ fuzzer and most of the obvious bugs have been fixed so it should
not crash for normal use. Now we invite power users to daily drive the beta
release in order to find more subtle bugs.

The Rust rewrite is ABI compatiable with previous releases. The most notable
difference with the C version includes a new portable system dictionary format
and two new user dictionary format. Users with existing user phrase dictionary
will be automatically migrated to either a new sqlite3 schema or CDB based
format depending on compile time configuration. Old files will be backed up
automatically. With these change we will be able to support dictionary sharing
and loading multiple dictionaries in future releases.

• Future roadmap

  • 0.8.x after bug bashing this version will be the first public release with
    rust implementation as the default.
  • 0.9.x will focus on the stablization of the Rust crate API.
  • 1.0.0 will be the first public release of the Rust API. C implementation
    will be removed from the repository.

• Added

  • Added Colemak-DH ANSI/Orth layout support
  • Supporting migrating user dictionary file to new format. (rust)
  • Allow creating in memory user dictionary (rust)
  • New chewing-cli tool can be used to create or inspect dictionary files. (rust)
  • New chewing-cli tool can dump dictionary as or import from CSV file. (rust)

• Changed

  • New trie dictionary format now has built-in checksum. (rust)

• Bug fixed

  • Some unsigned underflow issue found by afl++ (rust)
  • Skip symbol selection if preedit buffer is empty (rust)
  • Allow numlock in selection mode (rust)
  • Avoid infinite recursion (rust)

• Documentation

  • Added document for fuzzers (rust)
  • Describe how to compile dictionary files
  • Update build instruction and Rust MSRV info

• Testing

  • Added a fuzzer harness with afl++ (rust)

v0.6.0

What's New in libchewing 0.6.0 (Jan 27, 2024)

This release contains many improvements and bug fixes. It's the first
release since 2016. We have started a major rewrite in Rust so we expect
to have more frequent releases in the following months.

• Added

  • New API chewing_phone_to_bopomofo to convert syllable in u16 to bopomofo
    string [ShengYenPeng]. #166
  • New API chewing_set_autoLearn and chewing_get_autoLearn can be used to
    disable and enable new phrase auto learning. [Hiunn-hué] #261
  • Add default config test for autoLearn option [Billy Su].
  • Add Android platform support [hiroshiyui]. #408
  • New chewing Rust crate can be downloaded from
    crates.io [Kan-ru Chen]. #423
  • Add several CMake presets for supported configurations [Kan-ru Chen]. #424

• Changed

  • Implement has_userphrase in testhelper with the public
    chewing_userphrase_lookup API. [ShengYenPeng]. #223
  • Updated HSU layout's JVC conversion to be more intuitive [Hiunn-hué]. #259
  • Phrase auto learning now only triggers after committing [Hiunn-hué]. #260
  • chewing_handle_CtrlNum now handles cursor better and has clearer error
    message if there is nothing to be added to user phrase. [Hiunn-hué] #277
  • chewing_handle_ShiftLeft and chewing_handle_ShiftRight now also displays
    aux message for new user phrases added. [Hiunn-hué] #278
  • Symbol used for full-width colon changed from ︰ (U+FE30) to ： (U+FF1A)
    [school510587] #304
  • Support cross compiling with qemu. [sgn] #333
  • CMake minimum version changed to 3.21.0
  • Rewrite chewing internal in Rust [Kan-ru Chen] #347 #378

• Dictionary

  • Add new syllable ㄒㄩㄢ for 亘 [yan12125] #252 #253

• Removed

  • Autotools build tools are removed. CMake is the recommended way to build
    libchewing.

• Bug fixed

  • Fix ChoiceEndChoice expression always true [Billy Su].
  • Fix strncpy warning in hash.c [Bo-Cheng Jhan].
  • Fix unable to input some Bopomofos with ET26 layout [Hiunn-hué]. #235
  • Fix double free in userphrase-hash's TerminateUserphrase [cwlin]. #254
  • Fix hash item offsett calculation in uhash [cwlin]. #255
  • Fix ChoiceEndChoice expression always true [Billy Su]. #258
  • Fix clearing bopomofo buffer after switching Chi-Eng mode [Hiunn-hué]. #268
  • Fix chewing_handle_Space cannot input space with empty buffer [Hiunn-hué]. #274
  • Fix a strncpy warning and use ARRAY_SIZE [school510587] #296
  • Enhance chewing.py to support platforms with 64-bit pointer [cheese] #310
  • Fix init_database to correctly read phrase frequency for sorting [yan12125] #334 #337
  • Fix installation of data files [yan12125] #350
  • Fix building shared library on all supported platforms [Kan-ru Chen] #375 #376
  • Fix dvorak-qwerty symbol mapping [hiroshiyui] #407
  • prevent memory leak in error case of GetDefaultUserPhrasePath [Kuang-che Wu].
  • Fix PinYinData keySeq buffer overflow in PinYinInput() [Kuang-che Wu]. #238
  • Fix trying to allocate buffer of negative size [Kuang-che Wu].
  • Fix access violation due to negative cursor position [Kuang-che Wu].
  • Fix several buffer overflow. [ChangZhuo Chen]. #241 #242

• Platform-specific

  • Install instructions for macOS [Chocobo1].

New Contributors

• @cwlin made their first contribution in #254
• @Billy4195 made their first contribution in #258
• @splasky made their first contribution in #280
• @school510587 made their first contribution in #296
• @cheese made their first contribution in #310
• @sgn made their first contribution in #333
• @ianchen-tw made their first contribution in #396
• @hiroshiyui made their first contribution in #408

Full Changelog: v0.5.1...v0.6.0

v0.6.0-rc.2

What's New in libchewing 0.6.0-rc.2 (UNRELEASED)

• Added

  • Initial documentation to the Rust chewing crate.

• Changed

  • Many documentation improvements.

• Fixed

  • Build texinfo on macOS.

What's New in libchewing 0.6.0-rc.1 (UNRELEASED)

This release contains many improvements and bug fixes. It's the first
release since 2016. We have started a major rewrite in Rust so we expect
to have more frequent releases in the following months.

• Added

  • New API chewing_phone_to_bopomofo to convert syllable in u16 to bopomofo
    string [ShengYenPeng]. #166
  • New API chewing_set_autoLearn and chewing_get_autoLearn can be used to
    disable and enable new phrase auto learning. [Hiunn-hué] #261
  • Add default config test for autoLearn option [Billy Su].
  • Add Android platform support [hiroshiyui]. #408
  • New chewing Rust crate can be downloaded from
    crates.io [Kan-ru Chen]. #423
  • Add several CMake presets for supported configurations [Kan-ru Chen]. #424

• Changed

  • Implement has_userphrase in testhelper with the public
    chewing_userphrase_lookup API. [ShengYenPeng]. #223
  • Updated HSU layout's JVC conversion to be more intuitive [Hiunn-hué]. #259
  • Phrase auto learning now only triggers after committing [Hiunn-hué]. #260
  • chewing_handle_CtrlNum now handles cursor better and has clearer error
    message if there is nothing to be added to user phrase. [Hiunn-hué] #277
  • chewing_handle_ShiftLeft and chewing_handle_ShiftRight now also displays
    aux message for new user phrases added. [Hiunn-hué] #278
  • Symbol used for full-width colon changed from ︰ (U+FE30) to ： (U+FF1A)
    [school510587] #304
  • Support cross compiling with qemu. [sgn] #333
  • CMake minimum version changed to 3.21.0
  • Rewrite chewing internal in Rust [Kan-ru Chen] #347 #378

• Dictionary

  • Add new syllable ㄒㄩㄢ for 亘 [yan12125] #252 #253

• Bug fixed

  • Fix ChoiceEndChoice expression always true [Billy Su].
  • Fix strncpy warning in hash.c [Bo-Cheng Jhan].
  • Fix unable to input some Bopomofos with ET26 layout [Hiunn-hué]. #235
  • Fix double free in userphrase-hash's TerminateUserphrase [cwlin]. #254
  • Fix hash item offsett calculation in uhash [cwlin]. #255
  • Fix ChoiceEndChoice expression always true [Billy Su]. #258
  • Fix clearing bopomofo buffer after switching Chi-Eng mode [Hiunn-hué]. #268
  • Fix chewing_handle_Space cannot input space with empty buffer [Hiunn-hué]. #274
  • Fix a strncpy warning and use ARRAY_SIZE [school510587] #296
  • Enhance chewing.py to support platforms with 64-bit pointer [cheese] #310
  • Fix init_database to correctly read phrase frequency for sorting [yan12125] #334 #337
  • Fix installation of data files [yan12125] #350
  • Fix building shared library on all supported platforms [Kan-ru Chen] #375 #376
  • Fix dvorak-qwerty symbol mapping [hiroshiyui] #407
  • prevent memory leak in error case of GetDefaultUserPhrasePath [Kuang-che Wu].
  • Fix PinYinData keySeq buffer overflow in PinYinInput() [Kuang-che Wu]. #238
  • Fix trying to allocate buffer of negative size [Kuang-che Wu].
  • Fix access violation due to negative cursor position [Kuang-che Wu].
  • Fix several buffer overflow. [ChangZhuo Chen]. #241 #242

• Platform-specific

  • Install instructions for Mac OS [Chocobo1].

New Contributors

• @cwlin made their first contribution in #254
• @Billy4195 made their first contribution in #258
• @splasky made their first contribution in #280
• @school510587 made their first contribution in #296
• @cheese made their first contribution in #310
• @sgn made their first contribution in #333
• @ianchen-tw made their first contribution in #396
• @hiroshiyui made their first contribution in #408

Full Changelog: v0.5.1...v0.6.0-rc.2