Skip to content

Latest commit

 

History

History
83 lines (55 loc) · 2.68 KB

README.md

File metadata and controls

83 lines (55 loc) · 2.68 KB
Raw

Zemberek-NLP

!!!! Important Notice: I have stopped working with the project indefinitely. Feel free to fork the project !!!!

Here is the the new home of the Zemberek project.
Zemberek-NLP is a Natural Language Processing library. Some modules are specifically developed for Turkish language.

Usage

Maven

Add this to pom.xml file

<repositories>
    <repository>
        <id>ahmetaa-repo</id>
        <name>ahmetaa Maven Repo on Github</name>
        <url>https://raw.github.com/ahmetaa/maven-repo/master</url>
    </repository>
</repositories>

And dependecies (For example morphology):

<dependencies>
    <dependency>
        <groupId>zemberek-nlp</groupId>
        <artifactId>morphology</artifactId>
        <version>0.9.0</version>
    </dependency>
</dependencies>

Jar distributions

[Zemberek-NLP-Distributions] (https://github.com/ahmetaa/zemberek-nlp-distributions) page has versions and separate module and dependent jars.

Alternatively there is a public [Google docs page] (https://drive.google.com/#folders/0B9TrB39LQKZWSjNKdVcwWUxxUm8) for distributions downloads.

Examples

[Turkish-nlp-examples] (https://github.com/ahmetaa/turkish-nlp-examples) contains a maven java project with small usage examples.

Known Issues and Limitations

  • Project requires Java 7.
  • Currently word and sentence parse module operations generates parse graph with each initialization. So each run in the system takes some seconds. We will fix it in the next version with fast serialization of the parse graph.
  • Morphological parsing does not work for some obvious and frequent words.
  • Morphological disambiguation is working less accurate then expected (Not very usable).
  • Morphological generation may not work for some obvious Stem-Suffix combinations.
  • Please see issues section for further issues and feel free to create new ones.

Modules

Core

Core classes such as special Collection classes, Hash functions and helpers.

Morphology

Turkish morphological parsing, disambiguation and generation. [Morphology Documentation] (https://github.com/ahmetaa/zemberek-nlp/tree/master/morphology)

Tokenization

Turkish Tokenization and sentence boundary detection. So far only rule based algorithms.

Hyphenation

Turkish syllabification and hyphenation.

Language modelling

[Language model compression] (https://github.com/ahmetaa/zemberek-nlp/tree/master/lm)

Acknowledgements

Please refer to contributors.txt file.

Portions of this code has been developed in Tübitak BİLGEM's Speech and Language Technologies Laboratory.

Bilgem Logo