question about training

[Guardian-in-the-WF]

Great work. Does unidiffuser need to use gpt2(llm) for training? And at what stage should the llm be fine-tuned?

[NieShenRuc]

Thank you for your interest.
Before training UViT, we first need to fine-tune GPT-2 for text reconstruction. For specific fine-tuning methods, please refer to Appendix E. During the UViT training process, there's no need to update the parameters of GPT-2.