gmongaras
/

Softmax_Attention_GPT_1.2B

Feature Extraction

Model card Files Files and versions Community

This repository contains the GPT-Soft-1.2b model described in Cottention: Linear Transformers With Cosine Attention.

Downloads last month: 5

Inference Providers NEW

Feature Extraction

This model is not currently available via any of the supported Inference Providers.

The model cannot be deployed to the HF Inference API: The model has no library tag.

Collection including gmongaras/Softmax_Attention_GPT_1.2B

Cosine Attention (Cottention)

Models for the paper Cottention: Linear Transformers With Cosine Attention https://arxiv.org/abs/2409.18747 • 6 items • Updated Oct 7, 2024