mosaicml
/

mosaic-bert-base-seqlen-2048

Model card Files Files and versions Community

mosaic-bert-base-seqlen-2048 / config.json

Commit History

Change attention_probs_dropout_prob to 0.1 so that triton FlashAttention dependencies are avoided

2e90941

jacobfulano commited on Jan 3

Update config so that alibi_starting_size is explicitly set to 2048

97f26fe

jacobfulano commited on May 31, 2023

Upload BertForMaskedLM

b7a0389

jacobfulano commited on May 1, 2023