Text Generation
Transformers
PyTorch
llama
text-generation-inference
Inference Endpoints

Should the "Max Positional Embeddings" be 4096?

#10
by barleyspectacular - opened

Noticed this in the config.json

"max_position_embeddings": 2048,

Should this be 4096? Or perhaps this is because the dataset is 2048 context length?

barleyspectacular changed discussion title from Should the "Max Positional Encoding" be 4096? to Should the "Max Positional Embeddings" be 4096?

This is blocking vllm usage as well https://github.com/vllm-project/vllm/issues/525

Sign up or log in to comment