Generate Text with Embedding as Input

[Sappique]

Is there a way to use an embedding as input to generate text?

I'm working on a research project for which I would like to obtain the embedding of a prompt using Llama.embed(), modify it in some way, and then use this modified embedding to generate further text.

As I understand it, the embedding is just the state of the last hidden layer of the network. So in theory this should be possible to do.

I'm currently using mistral-7b-openorca.Q4_K_M.