File size: 8,600 Bytes

---
language:
- en
tags:
- t5
- analysis
- book
- notes
datasets:
- kmfoda/booksum
metrics:
- rouge
widget:
- text: I'm just a girl standing in front of a boy asking him to love her.
  example_title: Notting Hill
- text: Son, your ego is writing checks your body can't cash.
  example_title: top gun
- text: I really love to eat beans.
  example_title: beans
- text: >-
    The ledge, where I placed my candle, had a few mildewed books piled up in
    one corner; and it was covered with writing scratched on the paint. This
    writing, however, was nothing but a name repeated in all kinds of
    characters, large and small—Catherine Earnshaw, here and there varied to
    Catherine Heathcliff, and then again to Catherine Linton. In vapid
    listlessness I leant my head against the window, and continued spelling over
    Catherine Earnshaw—Heathcliff—Linton, till my eyes closed; but they had not
    rested five minutes when a glare of white letters started from the dark, as
    vivid as spectres—the air swarmed with Catherines; and rousing myself to
    dispel the obtrusive name, I discovered my candle wick reclining on one of
    the antique volumes, and perfuming the place with an odour of roasted
    calf-skin.
  example_title: Wuthering Heights
- text: >-
    Did you ever hear the tragedy of Darth Plagueis The Wise? I thought not.
    It’s not a story the Jedi would tell you. It’s a Sith legend. Darth Plagueis
    was a Dark Lord of the Sith, so powerful and so wise he could use the Force
    to influence the midichlorians to create life… He had such a knowledge of
    the dark side that he could even keep the ones he cared about from dying.
    The dark side of the Force is a pathway to many abilities some consider to
    be unnatural. He became so powerful… the only thing he was afraid of was
    losing his power, which eventually, of course, he did. Unfortunately, he
    taught his apprentice everything he knew, then his apprentice killed him in
    his sleep. Ironic. He could save others from death, but not himself.
  example_title: Darth Plagueis The Wise
inference:
  parameters:
    no_repeat_ngram_size: 2
    max_length: 64
    early_stopping: true
license: bsd-3-clause
library_name: transformers
pipeline_tag: text2text-generation
---


# T5-large for Lexical Analysis 

- This model was trained a text-to-text task with input text as a summary of a chapter, and the output text as the analysis of that chapter on the [booksum](https://arxiv.org/abs/2105.08209) dataset.
- it has somewhat learned how to complete literary analysis on an arbitrary input text.
- **NOTE: this is fairly intensive computationally and recommended to be run on GPU. please see example usage in [this demo notebook](https://colab.research.google.com/gist/pszemraj/8e9cc5bee5cac7916ef9241b66e01b05/demo-t5-large-for-lexical-analysis.ipynb)**
  - The API is set to return max 64 tokens to avoid timeouts on CPU.

## Example

- PROMPT: 

> Carmen: We're kids, not monsters. Dr. Romero: What's the difference?

- OUTPUT:

> Commentary on Act IV, scenes i-ii In these scenes, we see Dracula's transformation of the children into "monstrous" creatures. Doctor Romero says, "We're kidnapped, but not monsters." This is an important question for the audience to ask: Is there a difference between childhood and adulthood?

_longer examples are available in both [the demo notebook](https://colab.research.google.com/gist/pszemraj/8e9cc5bee5cac7916ef9241b66e01b05/demo-t5-large-for-lexical-analysis.ipynb) and at the bottom of this README._

# checkpoints

- a fine-tuned version of [google/t5-v1_1-large](https://huggingface.co/google/t5-v1_1-large)
- 16 epochs of training total

## Model description

- automatic literary analysis on arbitrary text
- `booksum` is a dataset created primarily for long-range summarization. It's excellent at that, and usually, this is done with the `chapter` and `summary_text` columns. However, there is a `summary_analysis` column as well, containing literary analysis on the passage in question :eyes:
- this model was trained on text-to-text with `summary_text` as the input and `summary_analysis` as the output, so it will analyze whatever text for _deeper meaning_.

## Intended uses & limitations

- given the absence of more specific info, the analysis will reference authors/titles from the training data in the context of the analysis.

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- distributed_type: multi-GPU
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- num_epochs: 4

### Training results



### Framework versions

- Transformers 4.16.2
- Pytorch 1.10.0+cu111
- Datasets 1.18.3
- Tokenizers 0.11.0


## Extended Examples

the below responses from the model are unedited.

### A prose passage from _The Lord of the Rings_

> Far above the Ephel Duath in the West the night-sky was still dim and pale. There, peering among the cloud-wrack above a dark tor high up in the mountains, Sam saw a white star twinkle for a while. The beauty of it smote his heart, as he looked up out of the forsaken land, and hope returned to him. For like a shaft, clear and cold, the thought pierced him that in the end the Shadow was only a small and passing thing: there was light and high beauty for ever beyond its reach.

analysis:

> Notes Sam's encounter with the white star is one of the most memorable scenes in the novel. It was a moment of great relief for him to see that there was light beyond his reach, and he had been longing for it all along. He looked up into the heavens at the sight of it: "It was as if I were looking out through the window of my own soul" . In this chapter, however, we are reminded of another scene from the Book of Eliot, when Sam saw the White Star twinkled for the first time since the book began. The night-sky still shrouded by clouds and cloud-wreck on the mountain tops, but now the darkness has begun to darken the whole world. When Sam looks up above the dusk over the mountains, which pierces his heart, Sam realizes that the Shadow will not last forever.
 
### the rick and morty copypasta

> Notes This is a very long post, but it's important to remember that this is the first of many posts on this site. It will take some time for me to get over the fact that I don't have an incredibly high iq to appreciate all of the humor in the show. In other words, people who dislike the series are idiots- they have no idea what these jokes mean or why they're funny at all. The only way to truly appreciate them is to be one of their diehard fans. If you do not like them, you'll end up making fools out of your own stupidity. For example, Dan Harmon writes: "I hated him because I loved him so much" . He was also extremely intelligent, and he would never have been able to fully appreciate the depth of his humour. That said, however, is that if you did not love him, then you should go ahead and make fun of him. And yet another reason why Morty dislikes him is partly due to his lack of narcissism rather than any kind of self-delusion. But there is something special about Mr. Moriarty himself- despite his lowly wittedness, which makes him seem almost superstitious. His attitude towards life seems to stem from his belief that nothing can ever be good enough to save the world. However, as noted above, Dickens says, "Life is full of paradoxes and contradictions... Life is more complex than anything else." Indeed, most critics have pointed out that even those with lower IQ points could possibly be seen as being subversive; indeed, readers might find it hard to sympathize with such simpletons. Of course, Stevenson has made it clear that we need to look beyond the surface level of normalcy in order to understand the absurdity of modern society. There are several examples of this sort of hypocrisy going on in contemporary literature. One of my favorite books is Fathers Sons, written by Alexander Nevsky, published in 1897. These books were published around 18 years before the novel was published. They were serialised in serial format, meaning that they were produced in 1921. Their publication dates back to 1864, when they appeared in London during the late eighteenth century England. At the time of its publication date, it was released in November 1793. When it came out in December, the book had already been published after 1859.