Retriever-Augmented Generation, or RAG, is a type of language generation model that combines pre-trained parametric and non-parametric memory for language generation. Specifically, the parametric memory is a pre-trained seq2seq model and the non-parametric memory is a dense vector index of Wikipedia, accessed with a pre-trained neural retriever. For query $x$, Maximum Inner Product Search (MIPS) is used to find the top-K documents $z_{i}$. For final prediction $y$, we treat $z$ as a latent variable and marginalize over seq2seq predictions given different documents.
Source: Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPaper | Code | Results | Date | Stars |
---|
Task | Papers | Share |
---|---|---|
Retrieval | 138 | 32.09% |
Question Answering | 56 | 13.02% |
Language Modelling | 29 | 6.74% |
Information Retrieval | 23 | 5.35% |
Large Language Model | 21 | 4.88% |
Open-Domain Question Answering | 12 | 2.79% |
Text Generation | 12 | 2.79% |
Benchmarking | 9 | 2.09% |
Chatbot | 6 | 1.40% |