Ensure tokenizer is on GPU

Diagnostics

Diagnostics

Diagnostics

Diagnostics

Ensure attention mask is assigned to GPU

Ensure consistency of device assignment when training

Fixed import

Fixed import

Factorized the weight matrix in the GlobalAttentionPoolingHead, thus reducing the number of parameters in this layer by a factor of 48

Reduced batch size

Created HuggingFace Space

Ensure consitency value is 32 bit

Correct dimension of consistency cosine

Using torch.nn.CosineSimilarity to simplify code

Removed unnecessary parameters

Attention mask in decoder

Set use_cache argument

Fix Einstein summation notation

Use keepdim option when normalising vectors

Make EPSILON a tensor

torch.maximum, not torch.max

Unsqueeze attention mask

Unpack BatchEncoding

Create BatchEncoding in pad

Use BatchEncoding for training, not batch_encoding

Use BatchEncoding for training

Removed unnecessary on_epoch_end

input_embeddings not needed

Removed unnecessary parameter

get_input_embeddings() directly from base model

Missing 'from_pretrained'

config didn't need to be a property

There's a simpler way of doing this, I hope

Might be simpler to inherit from RobertaModel rather than PreTrainedModel

Removed a base model that was causing a loop in model initialisation

Problems with config

Removed line that would have failed

Fixed import

Typo

Further changes for compatibility with HuggingFace Pytorch implementation

PyTorch implementation of HugggingFace PreTrainedModel class does not allow direct setting of base_model. Rejig constructors accordingly

Removed superfluous ()

Removed superfluous ()

Corrected inheritance

Modified CombinedCorpus to use PyTorch

Converted QaracTrainerModel to use PyTorch

Converted QaracDecoderModel to use PyTorch

Converted QaracEncoderModel to use PyTorch

Converted GlobalAttentionPoolingHead to use PyTorch