RAG RAG Model The RAG model is a retrieval-augmented generation model that can be leveraged for question-answering tasks using RagTokenForGeneration or RagSequenceForGeneration as proposed in Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks by Patrick Lewis, Ethan Perez, Aleksandara Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, Douwe Kiela. It was added to the library in PyTorch with the following checkpoints: facebook/rag-token-nq facebook/rag-sequence-nq facebook/rag-token-base facebook/rag-sequence-base Contributions: RAG #6813 (@ola13) [RAG] Add attention_mask to RAG generate #7373 (@patrickvonplaten) [RAG] Add missing doc and attention_mask to rag #7382 (@patrickvonplaten) [Rag] Fix wrong usage of num_beams and bos_token_id in Rag Sequence generation #7386 (@patrickvonplaten) [RAG] Fix retrieval offset in RAG's HfIndex and better integration tests #7372 (@lhoestq) [RAG] Remove dependency on examples/seq2seq from rag #7395 (@ola13) [Rag] fix rag retriever save_pretrained method #7399 (@patrickvonplaten) [RAG] Clean Rag readme in examples #7413 (@ola13) [RAG] Model cards - clean cards #7420 (@patrickvonplaten) Document RAG again #7377 (@sgugger) Bug fixes and improvements Mark big downloads slow #7325 (@sgugger) [Bug Fix] The actual batch_size is inconsistent with the settings. #7235 (@HuangLianzhe) Fixed results of SQuAD-FR evaluation #7313 (@psorianom) [s2s] add supported architecures to MD #7252 (@sshleifer) Add num workers cli arg #7322 (@chadykamar) [s2s] add src_lang kwarg for distributed eval #7300 (@sshleifer) [s2s] only save metrics.json from rank zero #7331 (@sshleifer) [code quality] fix confused flake8 #7309 (@stas00) [testing] skip decorators: docs, tests, bugs #7334 (@stas00) Fixed evaluation_strategy on epoch end bug #7340 (@WissamAntoun) Models doc #7345 (@sgugger) Ensure that integrations are imported before transformers or ml libs #7330 (@dsblank) [Benchmarks] Change all args to from no_... to their positive form #7075 (@fmcurti) Remove reference to args in XLA check #7344 (@ZeroCool2u) wip: Code to add lang tags to marian model cards #6586 (@sshleifer) Expand a bit the documentation doc #7350 (@sgugger) Check decorator order #7326 (@sgugger) Update modeling_tf_longformer.py #7359 (@Line290) Updata tokenization_auto.py #6870 (@hjptriplebee) Update the TF models to remove their interdependencies #7238 (@jplu) Make PyTorch model files independent from each other #7352 (@sgugger) Clean RAG docs and template docs #7348 (@sgugger) Fixing case in which Trainer hung while saving model in distributed training #7365 (@TevenLeScao) Formatter #7368 (@LysandreJik) [seq2seq] make it easier to run the scripts #7274 (@stas00) Remove mentions of RAG from the docs #7376 (@sgugger) [fsmt] build/test scripts #7257 (@stas00) [s2s] distributed eval allows num_return_sequences > 1 #7254 (@sshleifer) Seq2SeqTrainer #6769 (@patil-suraj) modeling_bart: 3 small cleanups that dont change outputs #7381 (@sshleifer) Check config type using type instead of isinstance #7363 (@LysandreJik) [s2s, examples] minor doc changes #7385 (@patil-suraj) Remove unhelpful bart warning #7391 (@sshleifer) [code quality] new make target that combines style and quality targets #7310 (@stas00) Speedup check_copies script #7394 (@sgugger) Fix BartModel output documentation #7390 (@sgugger) Fix FP16 and attention masks in FunnelTransformer #7374 (@sgugger) [Longformer, Bert, Roberta, ...] Fix multi gpu training #7272 (@patrickvonplaten) [s2s] add create student script #7290 (@patil-suraj) [s2s] rougeLSum expects \n between sentences #7410 (@sshleifer) [T5] allow config.decoder_layers to control decoer size #7409 (@sshleifer) Flos fix #7384 (@marrrcin) Catch PyTorch warning when saving/loading scheduler #7401 (@sgugger) Pull request template #7392 (@LysandreJik) Reorganize documentation navbar #7423 (@sgugger)