This is a limited proof of concept to search for research data, not a production system.

Search the MIT Libraries

Title: huggingface/transformers: v4.8.0 Integration with the Hub and Flax/JAX support

Type Software Thomas Wolf, Lysandre Debut, Sylvain Gugger, Patrick von Platen, Julien Chaumond, Stas Bekman, Sam Shleifer, Victor SANH, Manuel Romero, Funtowicz Morgan, Julien Plu, Suraj Patil, Aymeric Augustin, Rémi Louf, Stefan Schweter, Denis, erenup, Matt, Nicolas Patry, Joe Davison, Anthony MOI, Philipp Schmid, Teven, Piero Molino, Grégory Châtel, Bram Vanroy, Clement, Kevin Canwen Xu, Daniel Stancl, Philip May (2021): huggingface/transformers: v4.8.0 Integration with the Hub and Flax/JAX support. Zenodo. Software. https://zenodo.org/record/5020016

Authors: Thomas Wolf (@huggingface) ; Lysandre Debut (Hugging Face) ; Sylvain Gugger ; Patrick von Platen ; Julien Chaumond (Hugging Face) ; Stas Bekman (Stasosphere Online Inc.) ; Sam Shleifer (@facebookresearch) ; Victor SANH (@huggingface) ; Manuel Romero ; Funtowicz Morgan (HuggingFace) ; Julien Plu (Hugging Face) ; Suraj Patil (Huggingface) ; Aymeric Augustin (@qonto) ; Rémi Louf (Freelance) ; Stefan Schweter ; Denis ; erenup ; Matt (@huggingface) ; Nicolas Patry (@huggingface) ; Joe Davison (Hugging Face) ; Anthony MOI (@huggingface) ; Philipp Schmid (@huggingface) ; Teven (HuggingFace) ; Piero Molino ; Grégory Châtel (DisAItek & Intel AI Innovators) ; Bram Vanroy (@UGent) ; Clement (@huggingface) ; Kevin Canwen Xu ; Daniel Stancl (Seznam.cz) ; Philip May (@t-systems-on-site-services-gmbh) ;

Links

Summary

v4.8.0 Integration with the Hub and Flax/JAX support Integration with the Hub

Uploading Transformers object to the Hub has never been easier, with all models, tokenizers and configurations having a revamp push_to_hub() method as well as a push_to_hub argument in their save_pretrained() method. The workflow of this method is changed a bit to be more like git, with a local clone of the repo in a folder of the working directory, to make it easier to apply patches (use use_temp_dir=True to clone in temporary folders for the same behavior as the experimental API).

The Trainer also has a push_to_hub API that you need to activate by passing push_to_hub=True in its TrainingArguments. You can control the repo name and organization with push_to_hub_model_id and push_to_hub_organization. This will upload the model, its configuration and the associated tokenizer, the TensorBoard runs (if tensorboard is installed) as well as a draft of model card with the training metrics results on the evaluation set.

Clean push to hub API #12187 (@sgugger) Flax/JAX support

Flax/JAX is becoming a fully supported backend of the Transformers library with more models having an implementation in it. BART, CLIP and T5 join the already existing models, find the whole list here.

[Flax] FlaxAutoModelForSeq2SeqLM #12228 (@patil-suraj) [FlaxBart] few small fixes #12247 (@patil-suraj) [FlaxClip] fix test from/save pretrained test #12284 (@patil-suraj) [Flax] [WIP] allow loading head model with base model weights #12255 (@patil-suraj) [Flax] Fix flax test save pretrained #12256 (@patrickvonplaten) [Flax] Add jax flax to env command #12251 (@patrickvonplaten) add FlaxAutoModelForImageClassification in main init #12298 (@patil-suraj) [Flax T5] Fix weight initialization and fix docs #12327 (@patrickvonplaten) General improvements and bug fixes AutoTokenizer: infer the class from the tokenizer config if possible #12208 (@sgugger) update desc for map in all examples #12226 (@bhavitvyamalik) Depreciate pythonic Mish and support PyTorch 1.9 version of Mish #12240 (@digantamisra98) [t5 doc] make the example work out of the box #12239 (@stas00) Better CI feedback #12279 (@LysandreJik) Fix for making student ProphetNet for Seq2Seq Distillation #12130 (@vishal-burman) [DeepSpeed] don't ignore --adafactor #12257 (@stas00) Tensorflow QA example #12252 (@Rocketknight1) [tests] reset report_to to none, avoid deprecation warning #12293 (@stas00) [trainer + examples] set log level from CLI #12276 (@stas00) [tests] multiple improvements #12294 (@stas00) Trainer: adjust wandb installation example #12291 (@stefan-it) Fix and improve documentation for LEDForConditionalGeneration #12303 (@ionicsolutions) [Flax] Main doc for event orga #12305 (@patrickvonplaten) [trainer] 2 bug fixes and a rename #12309 (@stas00) FlaxBartPretrainedModel -> FlaxBartPreTrainedModel #12313 (@sgugger) [docs] performance #12258 (@stas00) Add CodeCarbon Integration #12304 (@JetRunner) Optimizing away the fill-mask pipeline. #12113 (@Narsil) Add output in a dictionary for TF generate method #12139 (@stancld) Flax summarization script #12230 (@patil-suraj) Rewrite ProphetNet to adapt converting ONNX friendly #11981 (@jiafatom) Flax T5 #12150 (@vasudevgupta7) Add mention of the huggingface_hub methods for offline mode #12320 (@LysandreJik) [Flax/JAX] Add how to propose projects markdown #12311 (@patrickvonplaten) [TFWav2Vec2] Fix docs #12283 (@chenht2010) Add all XxxPreTrainedModel to the main init #12314 (@sgugger) Conda build #12323 (@LysandreJik) Changed modeling_fx_utils.py to utils/fx.py for clarity #12326 (@michaelbenayoun)

More information

  • DOI: 10.5281/zenodo.5020016

Dates

  • Publication date: 2021
  • Issued: June 23, 2021

Rights

  • info:eu-repo/semantics/openAccess Open Access

Much of the data past this point we don't have good examples of yet. Please share in #rdi slack if you have good examples for anything that appears below. Thanks!

Format

electronic resource

Relateditems

DescriptionItem typeRelationshipUri
IsSupplementTohttps://github.com/huggingface/transformers/tree/v4.8.0
IsVersionOfhttps://doi.org/10.5281/zenodo.3385997
IsPartOfhttps://zenodo.org/communities/zenodo