Tips for prompting Vicuna-13b

[rlancemartin]

I'm not sure this is the correct place for the ticket, but wanted to add it here anyway b/c @daanelson has been very helpful.

We're using the Replicate Vicuna-13b API.

We're evaluating it for question-answering using an app we developed w/ LangChain here.

I notice the question responses for Vicuna are quite verbose: all are here.

We are using GPT4 as the grader, and currently Vicuna is failing each question :-P (vs GPT-3.5-turbo 70% pass rate).

We are using this prompt template:

template = """Use the following pieces of context to answer the question at the end. Use three sentences maximum. 
{context}
Question: {question}
Answer: Think step by step """

Perhaps Vicuna benefits from an alternative prompt?

[joehoover]

Thanks for raising this @rlancemartin!

In general, I believe the preferred prompt format for this model is:

### Human
{input}
### Assistant
{output}

However, even with that format, I still observed some pretty verbose responses. Setting top_p=0.25 appeared to mitigate that, though.

I ran some tests with with one of your examples and settled on the following:

### Human:
Use the following information to answer the question at the end. Your answer should not be longer than three sentences:

Context: First, the need for a large dataset of labeled examples for every new task limits the applicability of language models. Second, high capacity models tend to over-fit on narrow fine-tuning datasets and do not generalize well outside of them. Third, humans do not require large supervised datasets to learn most language tasks. To be broadly useful, we would someday like our NLP systems to have this same fluidity and generality.

Question: What are the limitations of task-specific fine-tuning?
 
### Assistant:
Let's think step by step.

Which yielded:

First,task-specific fine-tuning requires a large dataset of labeled examples for each new task, which can be difficult and time-consuming to obtain. Second, high capacity models tend to over-fit on narrow fine-tuning datasets and do not generalize well outside of them. Finally, humans do not require large supervised datasets to learn most language tasks, so it would be ideal if our NLP systems could have the same fluidity and generality.

[rlancemartin]

Cool, very helpful! Thanks.

[mtfarsani]

White Classical Watch, Cyberpunk, sweat drops, neon hubs, 80s aesthetics, insane, intricate, highly detailed, digital painting, artstation, concept art, smooth, sharp focus, illustration, unreal engine 5, 8k, art by artgerm and greg rutkowski and alphonse mucha

[mael-ha]

Human:

Use the following information to answer the question at the end. Your answer should not be longer than three sentences:

Context: First, the need for a large dataset of labeled examples for every new task limits the applicability of language models. Second, high capacity models tend to over-fit on narrow fine-tuning datasets and do not generalize well outside of them. Third, humans do not require large supervised datasets to learn most language tasks. To be broadly useful, we would someday like our NLP systems to have this same fluidity and generality.

Question: What are the limitations of task-specific fine-tuning?

Assistant:

Let's think step by step.

@joehoover thanks mate, playing a bit with Vircuna13 and was struggling to get consistent completion, it has been very helpful !! ✌️