PramaLLC/BEN
Image Segmentation
•
Updated
•
246
•
78
Vadim Karpenko
jrell
·
AI & ML interests
None yet
Recent Activity
new activity
about 1 month ago
ArliAI/Qwen2.5-32B-ArliAI-RPMax-v1.3-GGUF:32k context bug
liked
a model
about 1 month ago
PramaLLC/BEN
Organizations
None yet
jrell's activity
32k context bug
#1 opened about 1 month ago
by
jrell
liked
a
model
about 1 month ago
LoLCATs models.
1
#369 opened 2 months ago
by
jrell
LM Studio produces gibberish (GGUF)
4
#1 opened 2 months ago
by
jrell
liked
a
model
3 months ago
llama.cpp error: 'done_getting_tensors: wrong number of tensors; expected 292, got 291'
2
#1 opened 5 months ago
by
jrell
"llama.cpp error: 'error loading model vocabulary: unknown pre-tokenizer type: 'dolphin12b''"
23
#1 opened 5 months ago
by
jrell
reacted to
singhsidhukuldeep's
post with ❤️
7 months ago
Post
1457
You are all happy 😊 that
@meta-llama
released Llama 3.
Then you are sad 😔 that it only has a context length of 8k.
Then you are happy 😄 that you can just scale llama-3 PoSE to 96k without training, only needing to modify max_position_embeddings and rope_theta.
But then you are sad 😢 it only improves the model's long-context retrieval performance (i.e., finding needles) while hardly improving its long-context utilization capability (doing QA and summarization).
But then you are happy 😁 that the
@GradientsTechnologies community has released the long-context Llama-3-8B-Instruct-262K with long context (262k-1M+).
Now we have another paper "Extending Llama-3's Context Ten-Fold Overnight" 📜.
The context length of Llama-3-8B-Instruct is extended from 8K to 80K using QLoRA fine-tuning⚙️.
The training cycle is highly efficient, taking "only" 😂 8 hours on a single 8xA800 (80G) GPU machine.
The model also preserves its original capability over short contexts. ✁
The dramatic context extension is mainly attributed to merely 3.5K synthetic training samples generated by GPT-4.📊
The paper suggests that the context length could be extended far beyond 80K with more computation resources (😅 GPU-poor).
The team plans to publicly release all resources, including data, model, data generation pipeline, and training code, to facilitate future research from the ❤️ community.
Paper: https://arxiv.org/abs/2404.19553
This is where we are... until next time... 🌟
Extending Llama-3's Context Ten-Fold Overnight (2404.19553)
Then you are sad 😔 that it only has a context length of 8k.
Then you are happy 😄 that you can just scale llama-3 PoSE to 96k without training, only needing to modify max_position_embeddings and rope_theta.
But then you are sad 😢 it only improves the model's long-context retrieval performance (i.e., finding needles) while hardly improving its long-context utilization capability (doing QA and summarization).
But then you are happy 😁 that the
@GradientsTechnologies community has released the long-context Llama-3-8B-Instruct-262K with long context (262k-1M+).
Now we have another paper "Extending Llama-3's Context Ten-Fold Overnight" 📜.
The context length of Llama-3-8B-Instruct is extended from 8K to 80K using QLoRA fine-tuning⚙️.
The training cycle is highly efficient, taking "only" 😂 8 hours on a single 8xA800 (80G) GPU machine.
The model also preserves its original capability over short contexts. ✁
The dramatic context extension is mainly attributed to merely 3.5K synthetic training samples generated by GPT-4.📊
The paper suggests that the context length could be extended far beyond 80K with more computation resources (😅 GPU-poor).
The team plans to publicly release all resources, including data, model, data generation pipeline, and training code, to facilitate future research from the ❤️ community.
Paper: https://arxiv.org/abs/2404.19553
This is where we are... until next time... 🌟
Extending Llama-3's Context Ten-Fold Overnight (2404.19553)
LM Studio crash
9
#2 opened about 1 year ago
by
jrell
liked
2
models
about 1 year ago
liked
2
models
over 1 year ago
How to run this Model ?
8
#1 opened over 1 year ago
by
deleted