Can handle non-English text with bonito ?

[zhanglt]

I found bontio only chandle English context。
thanks！

[nanyyyyyy]

Thank you for your interest in our work. But, Bonito doesn't support languages besides English. thanks

[zhanglt]

I tried to train bointo to make it support Chinese and acceptable.
kitsdk/bonito-chinese-v1

[nihalnayak]

Hi @zhanglt

This is awesome! Could you briefly link the dataset/recipe you used to fine-tune this new model on the huggingface repo card?

We'd be happy to list this model on the repo. If we see more interest, we'd even fine-tune a chinese-variant of Bonito.

[zhanglt]

Sorry, I'm just a rookie.
At first, I wanted to fine-tune bonito-v1 with the Chinese data collection to make bonito-vi support Chinese, but the effect was not very ideal.
Later, I simply switched to the Chinese basic model (Qwen2.5), using the original training code of bonito, and using the Chinese data collection in ctga format for rebuilding bonito. It initially seems that the effect is better than the first way。

[nihalnayak]

That's great! The second approach makes more sense. Could you upload the Chinese CTGA dataset on Hugging Face? It would be really helpful for the community in general.