Removed unnecessary parameter

get_input_embeddings() directly from base model

Missing 'from_pretrained'

config didn't need to be a property

There's a simpler way of doing this, I hope

Might be simpler to inherit from RobertaModel rather than PreTrainedModel

Removed a base model that was causing a loop in model initialisation

Problems with config

Fixed typo

Removed line that would have failed

Fixed import

Typo

Further changes for compatibility with HuggingFace Pytorch implementation

Minor error in setting up tokenizer

PyTorch implementation of HugggingFace PreTrainedModel class does not allow direct setting of base_model. Rejig constructors accordingly

Removed superfluous ()

Removed superfluous ()

Corrected inheritance

Updated requitements.txt

Modified CombinedCorpus to use PyTorch

Modified training scripts to use PyTorch

Converted QaracTrainerModel to use PyTorch

Converted QaracDecoderModel to use PyTorch

Converted QaracEncoderModel to use PyTorch

Converted GlobalAttentionPoolingHead to use PyTorch

items, not values

Removed diagnostics

Removed unnecessary files

Use dictionary coprehensions to do padding and return dictionaries instead of default dictionaries

Implement max_lengths for CorpusRepeater

Forgot a len

TPUs need constant batch shapes

Completed script for testing consistency

Trainable => trainable

Ensure weights are trainable

Fixed name of argument

Tensor is logits

Removed extraneous self

The other layer returned a tuple as well

Low level RoBERTa layers don't necessarily return what I expect them to

Fixed typo

Needed more arguments

Arguments to Concatenate layer should be in a list

Fixed arguments to decoder head

Incoplete testing script for consistency. Fixed typo

Testing script for reasoning

Testing scripts

Attention masks, generation, and testing script

Attention masks are only necessary for inputs

Making sure RoBERTa layers have all required arguments