---
license: gpl-3.0
language:
- en
widget:
- text: "Sentence: We tried 4 different styles of donuts. The distribution of numerals in the sentence is equal to"
  example_title: "Example 1"
- text: "Sentence: No one's going to take you seriously if they're full of typos. The distribution of subordinates in the sentence is equal to"
  example_title: "Example 2"
---

# Li-IT5 Small

<p align="center">
    <img src="lit5.png" alt="Linguistically-Informed T5" width="500"/>
</p>


This model is released as part of the paper "Linguistic Knowledge Can Enhance Encoder-Decoder Models (*If You Let It*)" (Miaschi et al., 2024). 
If you use this model in your work, we kindly ask you to cite our paper:

```bibtex
@inproceedings{miaschi_linguistic_knowledge,
    title = "Linguistic Knowledge Can Enhance Encoder-Deocer Models (If You Let It)",
    author = "Miaschi, Alessio and Dell'Orletta Felice and Venturi, Giulia",
}
```

Other information can be found in the original [GitHub repository](https://github.com/alemiaschi/linguistically_informed_t5/tree/main).

## Model Description

The model is based on a T5 model fine-tuned in a multitask fashion to solve a set of raw, morpho-syntactic and syntactic tasks (i.e. predictions of linguistic properties). 
The full list of the 10 linguistic properties used as intermediate tasks can be found in the original paper.

This model is based on the English version of t5-small, [t5-small](https://huggingface.co/google-t5/t5-small).

## Model variations

The other fine-tuned models presented in the original study are the following:

- [li-it5-small](https://huggingface.co/alemiaschi/li-it5-small)
- [li-it5-base](https://huggingface.co/alemiaschi/li-it5-base)