Coqui Engine Error - 'int' object has no attribute 'device'

[RandomUser27]

I tried running the coqui test script and can seem to get around this error. Any suggestions? This seems like the perfect library for one of my projects, so I'm hoping there is something obvious that I'm missing.

Starting to play stream
⚡ synthesizing → 'Hey guys! These here are realtime spoken sentences based on local text synthesis.'
C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\TTS\tts\layers\xtts\stream_generator.py:138: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation)
warnings.warn(

C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\generation\configuration_utils.py:774: UserWarning: return_dict_in_generate is NOT set to True, but output_hidden_states is. When return_dict_in_generate is not True, output_hidden_states is ignored.
warnings.warn(

CoquiEngine: General synthesis error: 'int' object has no attribute 'device' occured in synthesize worker thread of coqui engine.

ERROR:root:Error synthesizing text: Hey guys! These here are realtime spoken sentences based on local text synthesis.
Traceback: Traceback (most recent call last):
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\RealtimeTTS\engines\coqui_engine.py", line 718, in _synthesize_worker
for i, chunk in enumerate(chunks):
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\utils_contextlib.py", line 36, in generator_context
response = gen.send(None)
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\TTS\tts\models\xtts.py", line 652, in inference_stream
gpt_generator = self.gpt.get_generator(
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\TTS\tts\layers\xtts\gpt.py", line 603, in get_generator
return self.gpt_inference.generate_stream(
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\utils_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\TTS\tts\layers\xtts\stream_generator.py", line 186, in generate
model_kwargs["attention_mask"] = self._prepare_attention_mask_for_generation(
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\generation\utils.py", line 588, in _prepare_attention_mask_for_generation
isin_mps_friendly(elements=eos_token_id, test_elements=pad_token_id).any()
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\pytorch_utils.py", line 324, in isin_mps_friendly
if elements.device.type == "mps" and not is_torch_greater_or_equal_than_2_4:
AttributeError: 'int' object has no attribute 'device'
ERROR:root:Error: 'int' object has no attribute 'device'

WARNING:root:engine coqui failed to synthesize sentence "Hey guys! These here are realtime spoken sentences based on local text synthesis.", unknown error
Error: 'int' object has no attribute 'device'
WARNING:root:engine coqui is the only engine available, can't switch to another engine
⚡ synthesizing → 'With a local, neuronal, cloned voice.'
WARNING:root:engine coqui failed to synthesize sentence "With a local, neuronal, cloned voice." with error:
Traceback: Traceback (most recent call last):
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\multiprocessing\connection.py", line 317, in _recv_bytes
nread, err = ov.GetOverlappedResult(True)
BrokenPipeError: [WinError 109] The pipe has been ended

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\RealtimeTTS\text_to_stream.py", line 407, in synthesize_worker
success = self.engine.synthesize(sentence)
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\RealtimeTTS\engines\coqui_engine.py", line 941, in synthesize
status, result = self.parent_synthesize_pipe.recv()
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\multiprocessing\connection.py", line 255, in recv
buf = self._recv_bytes()
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\multiprocessing\connection.py", line 326, in _recv_bytes
raise EOFError
EOFError

Error:
WARNING:root:engine coqui is the only engine available, can't switch to another engine
⚡ synthesizing → 'So every spoken sentence sounds unique.'
WARNING:root:engine coqui failed to synthesize sentence "So every spoken sentence sounds unique." with error: [WinError 232] The pipe is being closed
Traceback: Traceback (most recent call last):
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\RealtimeTTS\text_to_stream.py", line 407, in synthesize_worker
success = self.engine.synthesize(sentence)
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\RealtimeTTS\engines\coqui_engine.py", line 939, in synthesize
self.send_command("synthesize", data)
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\RealtimeTTS\engines\coqui_engine.py", line 808, in send_command
self.parent_synthesize_pipe.send(message)
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\multiprocessing\connection.py", line 211, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\multiprocessing\connection.py", line 285, in _send_bytes
ov, err = _winapi.WriteFile(self._handle, buf, overlapped=True)
BrokenPipeError: [WinError 232] The pipe is being closed

Error: [WinError 232] The pipe is being closed
WARNING:root:engine coqui is the only engine available, can't switch to another engine
Traceback (most recent call last):
File "c:\Users\xxxxxx\Documents\Python Scripts\xxxxxxbot\talk.py", line 1066, in
user_chatbot_conversation() # Start the conversation
File "c:\Users\xxxxxx\Documents\Python Scripts\xxxxxxbot\talk.py", line 1029, in user_chatbot_conversation
process_and_play(tts_approach, chatbot_response, speaker_style, audio_ref_file, talk_speed)
File "c:\Users\xxxxxx\Documents\Python Scripts\xxxxxxbot\talk.py", line 555, in process_and_play
engine.shutdown()
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\RealtimeTTS\engines\coqui_engine.py", line 1064, in shutdown
self.send_command("shutdown", {})
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\RealtimeTTS\engines\coqui_engine.py", line 808, in send_command
self.parent_synthesize_pipe.send(message)
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\multiprocessing\connection.py", line 211, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\multiprocessing\connection.py", line 285, in _send_bytes
ov, err = _winapi.WriteFile(self._handle, buf, overlapped=True)
BrokenPipeError: [WinError 232] The pipe is being closed

[KoljaB]

Never saw this one before. Which torch and transformers versions are you using, I'll try to reproduce it.

[RandomUser27]

Torch 2.5.1+cu124
Transformers 4.46.2
Thank you for looking into it!

[RandomUser27]

Just in case it's a useful datapoint - I tried downgrading to CUDA 12.1 and torch 2.5.1+cu121 and seem to be getting the same error.

[RandomUser27]

I uninstalled coqui-tts before reinstalling RealtimeTTS and that seemed to fix the issue - so I think this can be closed out, but thank you anyways!

[KoljaB]

Wow, that's strange because RealtimeTTS imports coqui-tts, so it shouldn't make a difference if it was installed or not. Mysterious...

[RandomUser27]

Perhaps there was something wrong with my original installation of coqui-tts was resolved when it was reinstalled through the RealtimeTTS install? Or perhaps I changed something else along the way that resolved the issue - I'm not sure.

I did run into another issue with the DeepSpeed wheel returning as invalid - any suggestions there? I can open a new issue thread to keep that separate if preferred.

pip install https://github.com/KoljaB/RealtimeTTS/blob/master/deepspeed/deepspeed-0.15.2%2Bcuda121-cp310-cp310-win_amd64.whl
Collecting deepspeed==0.15.2+cuda121
Downloading https://github.com/KoljaB/RealtimeTTS/blob/master/deepspeed/deepspeed-0.15.2%2Bcuda121-cp310-cp310-win_amd64.whl
| 230.3 kB 1.3 MB/s 0:00:00
ERROR: Wheel 'deepspeed' located at C:\Users\xxxxx\AppData\Local\Temp\pip-unpack-fdjh5m5r\deepspeed-0.15.2+cuda121-cp310-cp310-win_amd64.whl is invalid.

[KoljaB]

If this wheel doesn't work please try:

	pip install torch==2.1.2+cu121 torchaudio==2.1.2 --index-url https://download.pytorch.org/whl/cu121
	pip install https://github.com/daswer123/deepspeed-windows-wheels/releases/download/11.2/deepspeed-0.11.2+cuda121-cp310-cp310-win_amd64.whl

[RandomUser27]

The wheel installed just fine after downgrading torch to 2.1.2 - but now I'm getting this error on import.

Maybe a mismatch between my transformers version and my torch version?
This (huggingface/diffusers#8958) seems to be a similar issue, and they recommend commenting out the register_fake line in the _meta_registrations.py file so I may try that.

Traceback (most recent call last):
File "C:\Users\xxxxxx\Documents\Python Scripts\xxxxxxbot\talk.py", line 29, in
from sentence_transformers import SentenceTransformer, util
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\sentence_transformers_init_.py", line 14, in
from sentence_transformers.cross_encoder.CrossEncoder import CrossEncoder
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\sentence_transformers\cross_encoder_init_.py", line 3, in
from .CrossEncoder import CrossEncoder
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\sentence_transformers\cross_encoder\CrossEncoder.py", line 18, in
from sentence_transformers.evaluation.SentenceEvaluator import SentenceEvaluator
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\sentence_transformers\evaluation_init_.py", line 9, in
from .NanoBEIREvaluator import NanoBEIREvaluator
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\sentence_transformers\evaluation\NanoBEIREvaluator.py", line 11, in
from sentence_transformers import SentenceTransformer
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\sentence_transformers\SentenceTransformer.py", line 33, in
from sentence_transformers.model_card import SentenceTransformerModelCardData, generate_model_card
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\sentence_transformers\model_card.py", line 25, in
from transformers.integrations import CodeCarbonCallback
File "", line 1075, in _handle_fromlist
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\utils\import_utils.py", line 1766, in getattr
module = self._get_module(self._class_to_module[name])
File "C:\Users\xxxxxx\AppData\Local\Programs\Python\Python310\lib\site-packages\transformers\utils\import_utils.py", line 1780, in _get_module
raise RuntimeError(
RuntimeError: Failed to import transformers.integrations.integration_utils because of the following error (look up to see its traceback):
Failed to import transformers.modeling_utils because of the following error (look up to see its traceback):
module 'torch.library' has no attribute 'register_fake'

[KoljaB]

Hard to give support here, this is no direct RealtimeTTS issue. Feels like sentence_transformers needs more recent torch and or transformers library. This is a bit out of my scope. Deepspeed is really hard to install manually on windows but if you want to give it a try here are step-by-step instructions how to do it. Maybe you find a working windows wheel for a more recent torch version somewhere in the net, sorry my attempt to provide a recent wheel doesn't seem to work.

[RandomUser27]

You've already provided more than enough - thank you!
I may try those step-by-step instructions, but it sounds like the Deepspeed maintainers are working on updated windows install support so I may just wait for that update. In any case - RealtimeTTS works perfectly outside of my Deepspeed issue so thank you again for the support and the amazing library!