transition-joint-tagger

This is a code based on the model proposed by Meishan Zhang.

Installation

For training, a GPU is strongly recommended for speed. CPU is supported but training could be extremely slow.

PyTorch

The code is based on PyTorch 0.3.0. You can find installation instructions here.

Data

We mainly focus on 5 datasets, including CTB5, CTB6, CTB7, PKU and NCC.

Training

To train a tagger model, simpliy run train.py with the following parameters:

--rand_embedding      # use this if you want to randomly initialize the embeddings
--char_emb_file       # file dir for character embedding
--bichar_emb_file     # file dir for bi-character embedding
--word_file     	  # file dir for word embedding
--train_file		  # path to training file
--dev_file		  	  # path to development file
--test_file		  	  # path to test file
--gpu 				  # gpu id, set to -1 if use cpu mode
--batch_size  		  # batch size, default=16')
--checkpoint 		  # path to checkpoint and saved model

Decoding

To tag a raw file, simpliy run predict.py with the following parameters:

--load_arg      	  # path to saved json file with all args
--load_check_point    # path to saved model
--test_file     	  # path to test file 
--test_file_out   	  # path to test file output
--batch_size  		  # batch size, default=16')

Performance

Meishan's paper

dataset	SEG	POS
CTB50	98.50	94.95
CTB60	96.36	92.51
CTB70	96.25	91.87
PKU	96.35	94.14
NCC	95.30	90.42

Ours results

dataset	SEG	POS
CTB50
CTB60
CTB70
PKU
NCC