utilities for applying these metrics at evaluation time. T5.evaluation contains two core components: We provide some predefined metrics in t5.trics.įinally, t5.data contains a Mixture class that can be instantiated to combine multiple Task datasets for multi-task training using various functions for specifying the mixture rates. You may also define a postprocess function to convert the target and prediction text to another format before calling the metric. The metric function returns a score given the target and prediction from the model. If you create your own, you must use the flags -pad_id=0 -eos_id=1 -unk_id=2 -bos_id=-1 with spm_train to be compatible with our model code. You can create your own model with the google/sentencepiece library, or use our default one at t5.data.DEFAULT_SPM_PATH. The SentencePiece model is used to tokenize the input strings and decode the output tokens. We provide many predefined preprocessors in t5.data.preprocessors, but you may also define your own. We implemented our unsupervised pre-training objectives using these token preprocessors. In addition to text preprocessing, you can also use one or more token preprocessors to modify the inputs post-tokenization. For example, the predefined t5. preprocessor converts inputs in the form The text preprocessor converts the examples in the source dataset into the appropriate format for a text-to-text model with fields for inputs and targets. The data source can be an arbitrary function that provides a tf.data.Dataset, but we also provide simpler wrappers for datasets available in TensorFlow Datasets (TFDS) (a TfdsTask) or stored as text files with one example per line (a TextLineTask). T5.data is a package for defining Task objects that provide tf.data.Datasets.Īdditionally, you may optionally provide: The t5 library can be used for future model development by providing useful modules for training and fine-tuning (potentially huge) models on mixtures of text-to-text tasks. 
It also provides a way to fine-tune the pre-trained models released alongside the publication.
The bulk of the code in this repository is used for loading, preprocessing, mixing, and evaluating datasets. In the paper, we demonstrate how to achieve state-of-the-art results on multiple NLP tasks using a text-to-text transformer pre-trained on a large text corpus. The t5 library serves primarily as code for reproducing the experiments in Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.
0 kommentar(er)
