This dynamic module serves as an interface to invoke macOS’s built-in NLP tokenizer, specifically the NLTokenizer
.
This module is written in Swift, and requires macOS 10.15 or later.
# For release
swift build -c release
# For debug
swift build
If you enconter the folloing error:
No such module “PackageDescription”
run the following command and try again:
sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer
The module provides two functions:
emt--do-split-helper
This function takes a string and return an array of bounds. A bound is a cons with the starting position and the ending position of a word.
emt--word-at-point-or-forward
This functions takes a string and a position, and returns the bound of the word at the position. If the position is at bound of two words, it returns the word at the right side of that position. This function does not tokenize the whole string, so it is faster in some cases.
This module uses emacs-swift-module, which provides an interface for writing Emacs dynamic modules in Swift.