metadata
license: cc-by-nc-sa-4.0
pipeline_tag: fill-mask
language: en
datasets:
- OpenSubtitles
Model description
This model is based on An Exploration of Hierarchical Attention Transformers for Efficient Long Document Classification. Ilias Chalkidis, Xiang Dai, Manos Fergadiotis, Prodromos Malakasiotis, and Desmond Elliott. 2022. arXiv:2210.05529 (Preprint).
Initial weights were taken from google/bert_uncased_L-8_H-256_A-4.