v4.8.0 Integration with the Hub and Flax/JAX support Integration with the Hub Uploading Transformers object to the Hub has never been easier, with all models, tokenizers and configurations having a revamp push_to_hub() method as well as a push_to_hub argument in their save_pretrained() method. The workflow of this method is changed a bit to be more like git, with a local clone of the repo in a folder of the working directory, to make it easier to apply patches (use use_temp_dir=True to clone in temporary folders for the same behavior as the experimental API). The Trainer also has a push_to_hub API that you need to activate by passing push_to_hub=True in its TrainingArguments. You can control the repo name and organization with push_to_hub_model_id and push_to_hub_organization. This will upload the model, its configuration and the associated tokenizer, the TensorBoard runs (if tensorboard is installed) as well as a draft of model card with the training metrics results on the evaluation set. Clean push to hub API #12187 (@sgugger) Flax/JAX support Flax/JAX is becoming a fully supported backend of the Transformers library with more models having an implementation in it. BART, CLIP and T5 join the already existing models, find the whole list here. [Flax] FlaxAutoModelForSeq2SeqLM #12228 (@patil-suraj) [FlaxBart] few small fixes #12247 (@patil-suraj) [FlaxClip] fix test from/save pretrained test #12284 (@patil-suraj) [Flax] [WIP] allow loading head model with base model weights #12255 (@patil-suraj) [Flax] Fix flax test save pretrained #12256 (@patrickvonplaten) [Flax] Add jax flax to env command #12251 (@patrickvonplaten) add FlaxAutoModelForImageClassification in main init #12298 (@patil-suraj) [Flax T5] Fix weight initialization and fix docs #12327 (@patrickvonplaten) General improvements and bug fixes AutoTokenizer: infer the class from the tokenizer config if possible #12208 (@sgugger) update desc for map in all examples #12226 (@bhavitvyamalik) Depreciate pythonic Mish and support PyTorch 1.9 version of Mish #12240 (@digantamisra98) [t5 doc] make the example work out of the box #12239 (@stas00) Better CI feedback #12279 (@LysandreJik) Fix for making student ProphetNet for Seq2Seq Distillation #12130 (@vishal-burman) [DeepSpeed] don't ignore --adafactor #12257 (@stas00) Tensorflow QA example #12252 (@Rocketknight1) [tests] reset report_to to none, avoid deprecation warning #12293 (@stas00) [trainer + examples] set log level from CLI #12276 (@stas00) [tests] multiple improvements #12294 (@stas00) Trainer: adjust wandb installation example #12291 (@stefan-it) Fix and improve documentation for LEDForConditionalGeneration #12303 (@ionicsolutions) [Flax] Main doc for event orga #12305 (@patrickvonplaten) [trainer] 2 bug fixes and a rename #12309 (@stas00) FlaxBartPretrainedModel -> FlaxBartPreTrainedModel #12313 (@sgugger) [docs] performance #12258 (@stas00) Add CodeCarbon Integration #12304 (@JetRunner) Optimizing away the fill-mask pipeline. #12113 (@Narsil) Add output in a dictionary for TF generate method #12139 (@stancld) Flax summarization script #12230 (@patil-suraj) Rewrite ProphetNet to adapt converting ONNX friendly #11981 (@jiafatom) Flax T5 #12150 (@vasudevgupta7) Add mention of the huggingface_hub methods for offline mode #12320 (@LysandreJik) [Flax/JAX] Add how to propose projects markdown #12311 (@patrickvonplaten) [TFWav2Vec2] Fix docs #12283 (@chenht2010) Add all XxxPreTrainedModel to the main init #12314 (@sgugger) Conda build #12323 (@LysandreJik) Changed modeling_fx_utils.py to utils/fx.py for clarity #12326 (@michaelbenayoun)