[QUESTION] What is the retrieval datasets when evaluating downstream tasks?

[ZihaoLin0123]

I read the InstructRetro paper and it said

For downstream task evaluation, we follow Retro (Borgeaud et al., 2022) and use task-specific corpus and state-of-the-art retrievers and DRAGON+ (Lin et al., 2023)) to retrieve the most relevant and high-quality information for the task. For NQ and TriviaQA, we use DPR to retrieve from Wikipedia. For doc2dial and other long-form QA datasets, we use DRAGON+.

Does this mean that for each QA dataset, you only use its corpus as retrieval data? For example, when you evaluate NewsQA, you first collect all the articles of NewsQA without considering other tasks' corpus such as SQuAD and Wikipedia, chunk them, and embed them. Then you input a question, and the retriever will retrieve related article chunks from the corpus. Finally, concatenate the retrieved chunks with the input and feed them into LLM. Is that correct?

[github-actions[bot]]

Marking as stale. No activity in 60 days.