Update README.md
Browse files
README.md
CHANGED
@@ -59,7 +59,7 @@ LongLLaMA Code is built upon the foundation of [Code Llama](https://huggingface.
|
|
59 |
|
60 |
|
61 |
**LongLLaMA** is an [OpenLLaMA](https://github.com/openlm-research/open_llama) model finetuned with the FoT method,
|
62 |
-
with three layers used for context extension. **Crucially, LongLLaMA is able to extrapolate much beyond the context length seen in training:
|
63 |
**LongLLaMA Code** is a [Code Llama](https://huggingface.co/codellama/CodeLlama-7b-hf) model finetuned with the FoT method.
|
64 |
|
65 |
|
|
|
59 |
|
60 |
|
61 |
**LongLLaMA** is an [OpenLLaMA](https://github.com/openlm-research/open_llama) model finetuned with the FoT method,
|
62 |
+
with three layers used for context extension. **Crucially, LongLLaMA is able to extrapolate much beyond the context length seen in training: 8k. E.g., in the passkey retrieval task, it can handle inputs of length 256k**.
|
63 |
**LongLLaMA Code** is a [Code Llama](https://huggingface.co/codellama/CodeLlama-7b-hf) model finetuned with the FoT method.
|
64 |
|
65 |
|