pipx installed vosk not recognized by nerd-dictation, nor installable using pipx!?

[zenny]

Hi,

There cannot be a better solution for STT in command line linux than nerd-dictation. Kudos to @ideasman42 for developing a nifty tool.

1. Meanwhile there seems to be a dilemma that >python3.11 has moved from pip3 to pipx which nerd-dictation failed to detect:

$ ./nerd-dictation begin --vosk-model-dir=./model &
[1] 31046
[ zenny@MACHINE-GA-970A-D3 ~/Downloads/STT/nerd-dictation ]$ Traceback (most recent call last):
  File "/home/zenny/Downloads/STT/nerd-dictation/./nerd-dictation", line 1974, in <module>
    main()
  File "/home/zenny/Downloads/STT/nerd-dictation/./nerd-dictation", line 1970, in main
    args.func(args)
  File "/home/zenny/Downloads/STT/nerd-dictation/./nerd-dictation", line 1835, in <lambda>
    func=lambda args: main_begin(
                      ^^^^^^^^^^^
  File "/home/zenny/Downloads/STT/nerd-dictation/./nerd-dictation", line 1437, in main_begin
    found_any = text_from_vosk_pipe(
                ^^^^^^^^^^^^^^^^^^^^
  File "/home/zenny/Downloads/STT/nerd-dictation/./nerd-dictation", line 957, in text_from_vosk_pipe
    import vosk  # type: ignore
    ^^^^^^^^^^^
ModuleNotFoundError: No module named 'vosk'

[1]  + exit 1     ./nerd-dictation begin --vosk-model-dir=./model

Whereas pipx list shows the existence of vosk module:

   package vosk 0.3.45, installed using Python 3.12.2                                 │
    - vosk-transcriber

2. I have also tried with the solution provided in Pipx #103 without success:

$ pipx install https://github.com/ideasman42/nerd-dictation.git
  ERROR: Cannot unpack file /tmp/pip-unpack-389h1kou/nerd-dictation.git (downloaded from /tmp/pip-req-build-8wu4622r, content-type: text/html; charset=utf-8); cannot detect
archive format
ERROR: Cannot determine archive format of /tmp/pip-req-build-8wu4622r
Cannot determine package name from spec
'https://github.com/ideasman42/nerd-dictation.git'. Check package spec for errors.

Any input appreciated.

Cheers,
/z

[sunberrysolar]

Same here on manjaro, any help?

[sunberrysolar]

Solved on manjaro by installing python-vosk-api from AUR instead of using pip3, then follow the install instructions

[DovieW]

for me what fixed it was yay -S python-vosk-bin instead of using pip3 install vosk

I had to uninstall a package due to conflict with a package I guess I picked up from trying other commands: sudo pacman -R vosk-api-bin-debug

[DovieW]

For some reason, when i tried this on a different machine, i also had to add to the PYTHONPATH the location of the installed python-vosk-bin

get path with: sudo updatedb && locate vosk

[zahlman]

First, a quick detail about how open source projects work and how GitHub works:

I have also tried with the solution provided in #103 without success:

That is a pull request, meaning that someone else proposed a change to the project. If the change were accepted, it should make it possible to use the project that way. But it is not a "provided solution", because the change hasn't actually been made. It's still under discussion.

With that out of the way:

Meanwhile there seems to be a dilemma that >python3.11 has moved from pip3 to pipx which nerd-dictation failed to detect:

This is a misunderstanding. Pip (pip3) and Pipx (pipx) are separate programs with separate purposes (although Pipx uses Pip under the hood). Pip is still usable in Python 3.11 and onward and there has been no such "movement".

However, Python 3.11 introduces a new security feature specifically for the system Python installation that comes with Linux, in cooperation with the distro maintainers. (It can be disabled, but this is generally a bad idea.) You can still use Pip normally with any other copy of Python - for example, if you build it from source, or use a tool like PyEnv to install a separate copy, or - well, keep reading.

The goal, as described in this YouTube video, is to allow your system package manager to handle all changes to the system Python's libraries, without Pip interfering. It's important to understand that even when you install just for the current user (without sudo rights and using --user with Pip), the packages you install are still seen by Python as part of the system Python installation - and they can potentially interfere with important parts of the system which are written in Python. (These "important parts" can include the package manager itself, so in rare cases it can be extremely difficult to recover.)

Because of this, we need to choose a different approach when we install Python packages for the system Python. One way, of course, is to use the system package manager, if it makes the package available. If you have the option, go ahead and take it. But there is way more ready-packaged Python code out there than your distro maintainers can possibly oversee. (PyPI currently hosts over half a million projects.)

Pipx is another approach. But it's intended for installing applications, not libraries. In our case, we want to install Vosk, which is a library that nerd-dictation will use. Pipx is able to install Vosk "as an application", because it also comes with a vosk-transcriber driver script. But when you do this, it's installed into a virtual environment that Pipx creates and manages. You can find this easily enough (if you haven't messed with Pipx config, it should be at ~/.local/pipx/venvs/vosk); but in order for nerd-dictation to use the library that was installed in the virtual environment, it needs to run in the virtual environment.

Since the main nerd-dictation script is an executable Python script, with #!/usr/bin/env python as a shebang, we therefore must arrange things so that /usr/bin/env finds the virtual environment's Python, rather than the system Python.

The easiest way to do that is to activate the virtual environment, by sourcing its activate script first: source ~/.local/pipx/venvs/vosk/bin/activate. Of course, this is easier when the virtual environment is somewhere more convenient - but virtual environments cannot safely be moved. This is why I say that Pipx is not suitable for libraries. It becomes much easier if you create and manage the virtual environment yourself.

Please also refer to this Linux Mint forum thread where I help out with installing this project.