Dimension in acoustic transformer

[mohit2b]

Hi, I am trying to run your Trimodal-BART-driver.py file. In it in line 416 I believe the dimension of acoustic input is batch x max_acoustic_len x acoustic_dim. The output returned by self.acoustic_transformer(acoustic_input) is of shape batch x max_acoustic_len x acoustic_dim. So the output of self.acoustic_transformer(acoustic_input)[-1] will be of shape max_acoustic_len x acoustic_dim. And you are passing this output of shape (max_acoustic_len x acoustic_dim) to MAF_layer in line 418. Therefore the dimension of acoustic_context in MAF_layer is (max_acoustic_len x acoustic_dim).

So my question is since the dimension of acoustic_context in MAF_layer is (max_acoustic_len x acoustic_dim) in line 242 it is throwing error. Please help.

[Aeron00]

Hii, @mohit2b did you able to solve the problem, i am getting the same issue, any help appreciated. Thanks

[mohit2b]

@Aeron00 please check ACL paper link (software), they have different python script for acoustic transformer. (https://aclanthology.org/2022.acl-long.411/)

[Aeron00]

Thank you😊 @mohit2b i was able to solve the error.