Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add F5 TTS #162

Open
pabloortegaa opened this issue Oct 22, 2024 · 4 comments
Open

Add F5 TTS #162

pabloortegaa opened this issue Oct 22, 2024 · 4 comments

Comments

@pabloortegaa
Copy link

I just saw this TTS model(https://github.com/SWivid/F5-TTS), which works very well for English. Are you planning on including it on the project? Thanks!!

@KoljaB
Copy link
Owner

KoljaB commented Oct 22, 2024

Absolutely thinking about that, I love that TTS system. Things that keep me off currently:

  1. Not on pypi - without pip install I need to copy their whole repo, don't want that
  2. No streaming support so bigger latency than for other engines
    Both are kind of bummers but 1 is the bigger one. I need a solution to gracefully install it with RealtimeTTS

@aadr1024
Copy link

aadr1024 commented Nov 11, 2024

Hi! @KoljaB, would pip install git+https://github.com/SWivid/F5-TTS.git not suffice as a pip install?

@KoljaB
Copy link
Owner

KoljaB commented Nov 11, 2024

Sadly not, pypi would not accept this within a setup file. So I can't integrate it to be installed with "pip install RealtimeTTS[F5]" currently. Also no streaming support is still a bummer.
Still my first pick TTS system that I would love to integrate (together with GPT-SoVITS).

@KoljaB
Copy link
Owner

KoljaB commented Nov 11, 2024

Looked into F5-TTS code.

Implementing real-time streaming is not trivial because the model processes entire sequences at once using full-sequence attention and ODE integration. This doesn't support incremental output. So it would require significant architectural changes to support incremental, real-time computation needed for streaming.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants