NovaSearch

/

stella_en_400M_v5

like 176

Follow

NovaSearch 9

512 is the Sequence Length, whereas the numbers in 2_Dense_... refer to the dimensionality, i.e. the number of values that make up the text embeddings.
All of the different dimensionalities use a sequence length of 512 tokens.

• Tom Aarsen

Thanks

But 512 is mentioned in The models are finally trained by MRL, so they have multiple dimensions: 512, 768, 1024, 2048, 4096, 6144 and 8192.
256 is not mentioned but present in the repository. And it makes sense to have 512 between 256 and 768.