Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-enable TTS (bark.cpp) Inference #359

Merged
merged 5 commits into from
Jan 27, 2025
Merged

Re-enable TTS (bark.cpp) Inference #359

merged 5 commits into from
Jan 27, 2025

Conversation

xsxszab
Copy link
Collaborator

@xsxszab xsxszab commented Jan 24, 2025

Description

This PR re-enables TTS (bark.cpp) components in the SDK by addressing previously known issues. The effect can be verified by running tests/test_tts_generation.py.


Previous Problem with bark.cpp

The issues with bark.cpp were caused by a shared library version mismatch:

  1. Submodule llama.cpp generates a libggml shared library and uses it as a dependency.
  2. bark.cpp relies on encodec.cpp, which includes a forked version of GGML. When bark.cpp compilation is enabled, the forked version of libggml overwrites the one generated by llama.cpp.
  3. The forked version of libggml is outdated and incompatible with llama.cpp, causing runtime errors when loaded.

Main Changes

  1. Updated NexaAI's Forked bark.cpp:

    • Renamed the ggml library carried by bark.cpp from libggml to libggml-bark (see this PR).
    • Updated bark.cpp in Nexa SDK accordingly .
  2. Isolated Shared Libraries:

    • Organized all shared libraries (e.g., stable-diffusion, llama, bark.cpp) into separate subfolders to prevent future naming conflicts.
  3. Shared Library Loading:

    • Updated shared library loading functions in nexa/gguf/lib_utils.py and nexa/gguf/bark/bark_cpp.py to include an additional argument for specifying subdirectories mentioned in Change 2. The new argument carries a placeholder default value, ensuring backward compatibility.
  4. Fixed Broken Test Cases:

    • Resolved an import issue in test_image_generation.py.

Testing

  • All tests available under tests/ are passed.
  • The change is only tested on MacOS since this is the only device I have. Please test it on other platforms as well before merge.

@Davidqian123 Davidqian123 requested a review from vinovo January 24, 2025 19:50
@Davidqian123 Davidqian123 merged commit 50239ba into main Jan 27, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants