Title: huggingface/transformers: Tokenizer fixes

Type Software Thomas Wolf, Lysandre Debut, Julien Chaumond, Patrick von Platen, Victor SANH, Sam Shleifer, Funtowicz Morgan, Aymeric Augustin, Rémi Louf, Manuel Romero, Sylvain Gugger, Stefan Schweter, Denis, erenup, Matt, Grégory Châtel, Piero Molino, Anthony MOI, Bram Vanroy, Suraj Patil, Gunnlaugur Thor Briem, Tim Rault, Bilal Khan, Catalin Voss, Malte Pietsch, Julien Plu, Lorenzo Ampil, Davide Fiocco, Louis Martin, Fei Wang (2020): huggingface/transformers: Tokenizer fixes. Zenodo. Software. https://zenodo.org/record/3932369

Authors: Thomas Wolf (@huggingface) ; Lysandre Debut (Hugging Face) ; Julien Chaumond (Hugging Face) ; Patrick von Platen ; Victor SANH (@huggingface) ; Sam Shleifer (Huggingface) ; Funtowicz Morgan (HuggingFace) ; Aymeric Augustin (@qonto) ; Rémi Louf ; Manuel Romero ; Sylvain Gugger ; Stefan Schweter ; Denis ; erenup ; Matt ; Grégory Châtel (DisAItek & Intel AI Innovators) ; Piero Molino ; Anthony MOI (Hugging Face) ; Bram Vanroy (@UGent) ; Suraj Patil (Wynum) ; Gunnlaugur Thor Briem (Qlik) ; Tim Rault (@huggingface) ; Bilal Khan ; Catalin Voss (Stanford University) ; Malte Pietsch (deepset) ; Julien Plu (Leboncoin Lab) ; Lorenzo Ampil (@thinkingmachines) ; Davide Fiocco (@frontiersin) ; Louis Martin ; Fei Wang (University of Southern California) ;

Links

• Item record in Zenodo
• Digital object URL

Summary

Tokenizer fixes

Fixes bugs introduced by v3.0.0 and v3.0.1 in tokenizers.

More information

• DOI: 10.5281/zenodo.3932369

Dates

• Publication date: 2020
• Issued: July 06, 2020

Rights

• info:eu-repo/semantics/openAccess Open Access

Format

electronic resource

Relateditems