Longformer 8192

Longformer model with 8192 tokens sequence length. Initialized from FaceboookAI/roberta-base. No additional pre-training has been done.

Model can be loaded and run using AutoModel and AutoTokenizer.

Downloads last month: 67

Safetensors

Model size

152M params

Tensor type

F32

Inference Examples

Feature Extraction

This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for capemox/longformer-8192

Base model

FacebookAI/roberta-base

Finetuned

(1267)

this model