This is a limited proof of concept to search for research data, not a production system.

Search the MIT Libraries

Title: huggingface/transformers: RAG

Type Software Thomas Wolf, Lysandre Debut, Julien Chaumond, Patrick von Platen, Sam Shleifer, Victor SANH, Sylvain Gugger, Manuel Romero, Funtowicz Morgan, Stas Bekman, Aymeric Augustin, Rémi Louf, Stefan Schweter, Denis, Suraj Patil, erenup, Matt, Grégory Châtel, Julien Plu, Piero Molino, Anthony MOI, Bram Vanroy, Clement, Teven, Joe Davison, Gunnlaugur Thor Briem, Kevin Canwen Xu, Tim Rault, Catalin Voss, Lorenzo Ampil (2020): huggingface/transformers: RAG. Zenodo. Software. https://zenodo.org/record/4056444

Authors: Thomas Wolf (@huggingface) ; Lysandre Debut (Hugging Face) ; Julien Chaumond (Hugging Face) ; Patrick von Platen ; Sam Shleifer (Huggingface) ; Victor SANH (@huggingface) ; Sylvain Gugger ; Manuel Romero ; Funtowicz Morgan (HuggingFace) ; Stas Bekman (Stasosphere Online Inc.) ; Aymeric Augustin (@qonto) ; Rémi Louf (Freelance) ; Stefan Schweter ; Denis ; Suraj Patil (Wynum) ; erenup ; Matt ; Grégory Châtel (DisAItek & Intel AI Innovators) ; Julien Plu (Hugging Face) ; Piero Molino ; Anthony MOI (Hugging Face) ; Bram Vanroy (@UGent) ; Clement (@huggingface) ; Teven (HuggingFace) ; Joe Davison (Hugging Face) ; Gunnlaugur Thor Briem (Qlik) ; Kevin Canwen Xu ; Tim Rault (@huggingface) ; Catalin Voss (Stanford University) ; Lorenzo Ampil (@thinkingmachines) ;

Links

Summary

RAG RAG Model

The RAG model is a retrieval-augmented generation model that can be leveraged for question-answering tasks using RagTokenForGeneration or RagSequenceForGeneration as proposed in Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks by Patrick Lewis, Ethan Perez, Aleksandara Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, Douwe Kiela.

It was added to the library in PyTorch with the following checkpoints:

facebook/rag-token-nq facebook/rag-sequence-nq facebook/rag-token-base facebook/rag-sequence-base Contributions: RAG #6813 (@ola13) [RAG] Add attention_mask to RAG generate #7373 (@patrickvonplaten) [RAG] Add missing doc and attention_mask to rag #7382 (@patrickvonplaten) [Rag] Fix wrong usage of num_beams and bos_token_id in Rag Sequence generation #7386 (@patrickvonplaten) [RAG] Fix retrieval offset in RAG's HfIndex and better integration tests #7372 (@lhoestq) [RAG] Remove dependency on examples/seq2seq from rag #7395 (@ola13) [Rag] fix rag retriever save_pretrained method #7399 (@patrickvonplaten) [RAG] Clean Rag readme in examples #7413 (@ola13) [RAG] Model cards - clean cards #7420 (@patrickvonplaten) Document RAG again #7377 (@sgugger) Bug fixes and improvements Mark big downloads slow #7325 (@sgugger) [Bug Fix] The actual batch_size is inconsistent with the settings. #7235 (@HuangLianzhe) Fixed results of SQuAD-FR evaluation #7313 (@psorianom) [s2s] add supported architecures to MD #7252 (@sshleifer) Add num workers cli arg #7322 (@chadykamar) [s2s] add src_lang kwarg for distributed eval #7300 (@sshleifer) [s2s] only save metrics.json from rank zero #7331 (@sshleifer) [code quality] fix confused flake8 #7309 (@stas00) [testing] skip decorators: docs, tests, bugs #7334 (@stas00) Fixed evaluation_strategy on epoch end bug #7340 (@WissamAntoun) Models doc #7345 (@sgugger) Ensure that integrations are imported before transformers or ml libs #7330 (@dsblank) [Benchmarks] Change all args to from no_... to their positive form #7075 (@fmcurti) Remove reference to args in XLA check #7344 (@ZeroCool2u) wip: Code to add lang tags to marian model cards #6586 (@sshleifer) Expand a bit the documentation doc #7350 (@sgugger) Check decorator order #7326 (@sgugger) Update modeling_tf_longformer.py #7359 (@Line290) Updata tokenization_auto.py #6870 (@hjptriplebee) Update the TF models to remove their interdependencies #7238 (@jplu) Make PyTorch model files independent from each other #7352 (@sgugger) Clean RAG docs and template docs #7348 (@sgugger) Fixing case in which Trainer hung while saving model in distributed training #7365 (@TevenLeScao) Formatter #7368 (@LysandreJik) [seq2seq] make it easier to run the scripts #7274 (@stas00) Remove mentions of RAG from the docs #7376 (@sgugger) [fsmt] build/test scripts #7257 (@stas00) [s2s] distributed eval allows num_return_sequences > 1 #7254 (@sshleifer) Seq2SeqTrainer #6769 (@patil-suraj) modeling_bart: 3 small cleanups that dont change outputs #7381 (@sshleifer) Check config type using type instead of isinstance #7363 (@LysandreJik) [s2s, examples] minor doc changes #7385 (@patil-suraj) Remove unhelpful bart warning #7391 (@sshleifer) [code quality] new make target that combines style and quality targets #7310 (@stas00) Speedup check_copies script #7394 (@sgugger) Fix BartModel output documentation #7390 (@sgugger) Fix FP16 and attention masks in FunnelTransformer #7374 (@sgugger) [Longformer, Bert, Roberta, ...] Fix multi gpu training #7272 (@patrickvonplaten) [s2s] add create student script #7290 (@patil-suraj) [s2s] rougeLSum expects \n between sentences #7410 (@sshleifer) [T5] allow config.decoder_layers to control decoer size #7409 (@sshleifer) Flos fix #7384 (@marrrcin) Catch PyTorch warning when saving/loading scheduler #7401 (@sgugger) Pull request template #7392 (@LysandreJik) Reorganize documentation navbar #7423 (@sgugger)

More information

  • DOI: 10.5281/zenodo.4056444

Dates

  • Publication date: 2020
  • Issued: September 28, 2020

Rights

  • info:eu-repo/semantics/openAccess Open Access

Much of the data past this point we don't have good examples of yet. Please share in #rdi slack if you have good examples for anything that appears below. Thanks!

Format

electronic resource

Relateditems

DescriptionItem typeRelationshipUri
IsSupplementTohttps://github.com/huggingface/transformers/tree/v3.3.0
IsVersionOfhttps://doi.org/10.5281/zenodo.3385997
IsPartOfhttps://zenodo.org/communities/zenodo