Is it feasible to combine diffusion models and language models to mimic divergent thinking?

[win10ogod]

Does using a diffusion model in a language model increase the generality of the language model?

[Ph0rk0z]

well there is stuff like this:

https://github.com/haotian-liu/LLaVA

https://github.com/Vision-CAIR/MiniGPT-4

[win10ogod]

The goal of diffusion models is to learn the latent structure of a dataset by modeling the way in which data points diffuse through the latent space.
When used in text generation, it means learning language structure and semantics. I think it should be able to be used in text generation.
But I'm not an industry person, just a hobbyist.