Unexpected behaviour of hi_IN dictionary

[KernelYoda]

Tried hi_IN dictionary with Hunspell (v1.7.0) on macOS 10.13. It is producing incorrect results for words within and outside the dictionary.

Hunspell 1.7.0
शब्द
# शब्द 0

द्वारा
# द्व 0
# र 4

ढूंढेगा
# ढूंढेग 0

आवश्यकता
# आवश्यकत 0

कोशिश
# क 0
# श 2
# श 4

बिंदु
# ब 0
# ंदु 2

दुनिया
# ब 0
# ंदुन 2
# य 7

बनाना
# बन 0
# न 3

अपरिमाप्य
# अपर 0
# म 4
# प्य 6

I have also noticed that the words are incorrectly split where ा, ि, ी, ो, ौ mAtra occurs. This may contribute to the incorrect results.

[Shreeshrii]

I had created the files over two years ago with the latest hunspell version at the time. I have not tested it since.

Thanks for bringing the error to notice. I will take a look.

[Shreeshrii]

I get different results:

ubuntu@tesseract-ocr:~/hindi-hunspell/Hindi$ hunspell -d hi_IN
Hunspell 1.7.0
हम्
& हम् 15 0: हम, म्, अहम्, हम्द, हम्ल, हमे, अम्, आम्, ओम्, गम्, चम्, दम्, पम्, हमल, हल्
मरा
*
*

शब्द
*

द्वारा
& द्व 15 0: द्, दव, द्वा, द्वि, द्वै, द्वौ, द्वज, द्वत, द्वध, पद्व, द्वय, द्रव, दर्व, दाव, दिव
*

ढूंढेगा
& ढूंढेग 4 0: ढूंढे, ढूंढेगा, ढूंढे ग, ढूंढते

आवश्यकता
& आवश्यकत 5 0: आवश्यक, आवश्यकता, आवश्य कत, आवश्यक त, आवश्य

अपरिमाप्य
*
*
& प्य 15 6: प्, पय, प्यू, प्यो, आप्य, पच्य, जप्य, पठ्य, पण्य, पत्य, तप्य, पथ्य, पद्य, प्लय, पव्य

[KernelYoda]

I can see that even you are facing the issue of incorrect word split. I have filed an issue with Hunspell for this behaviour: hunspell/hunspell#634

Regarding the suggestions produced, I am using the same dictionary as the one in dict-hi_IN.zip.

[KernelYoda]

@Shreeshrii Do you have a fix for the incorrect word split? (It is present in the output you have shared)

[Shreeshrii]

No. I think it may require change in how hunspell does the processing.

…

On Wed, Jul 17, 2019 at 3:59 PM Anudeep Vishwakarma < ***@***.***> wrote: @Shreeshrii <https://github.com/Shreeshrii> Do you have a fix for the incorrect word split? (It is present in the output you have shared) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6?email_source=notifications&email_token=ABG37I37DML3BQR7XGYDTNTP73YBXA5CNFSM4H7ZDRL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2DYKLQ#issuecomment-512197934>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABG37I4VB4ZI5X5GFO4IVHTP73YBXANCNFSM4H7ZDRLQ> .

--

____________________________________________________________ भजन - कीर्तन - आरती @ http://bhajans.ramparivar.com

[KernelYoda]

Got it. Can you please upvote this issue which I raised with Hunspell: hunspell/hunspell#634

Given the above bug/issue, is there a way to check this Hindi dictionary's effectiveness? I don't think it would give correct results, let alone integrating it in an application as a spell checker solution.