Fusho/Style_LoRA_XL
Updated
β’
16
later
fjall
AI & ML interests
GhosTShell
Recent Activity
liked
a model
2 days ago
Fusho/Style_LoRA_XL
liked
a model
about 1 month ago
hhks/PonyXL_Notes_Backup
liked
a model
about 2 months ago
dantion/loras
Organizations
None yet
fjall's activity
liked
a
model
2 days ago
liked
a
model
about 1 month ago
liked
a
model
about 2 months ago
liked
7
models
3 months ago
PsySpy/New3DSDXL
Updated
β’
5
hskWih/tmp_model
Updated
β’
6
SenY/LoRA
Updated
β’
152
deadman44/Flux_Photoreal_LoRA
Text-to-Image
β’
Updated
β’
313
β’
15
JujoHotaru/lora
Text-to-Image
β’
Updated
β’
10.6k
β’
297
WarriorMama777/GLaDOS_TTS
Text-to-Speech
β’
Updated
β’
13
Kijai/flux-fp8
Updated
β’
711
liked
a
dataset
3 months ago
liked
3
models
3 months ago
liked
a
model
4 months ago
liked
2
models
5 months ago
reacted to
Lewdiculous's
post with β€οΈ
7 months ago
Post
42452
More context for your Pascal GPU or older!
Update: Now available in the official releases of KoboldCpp!
[releases] https://github.com/LostRuins/koboldcpp/releases/latest
These are great news for all the users with GTX 10XX, P40...
Flash Attention implementation for older NVIDIA GPUs without requiring Tensor Cores has come to llama.cpp in the last few days, and should be merged in the next version of KoboldCpp, you can already try it with another fork or by building it.
[Mentioned KCPP fork] https://github.com/Nexesenex/kobold.cpp/releases/latest
[PR] https://github.com/ggerganov/llama.cpp/pull/7188
You should expect less VRAM usage for the same context, allowing you to experience higher contexts with your current GPU.
There have also been reported final tokens/second speed improvements for inference, so that's also grand!
If you have tried it, I'd like to hear your experiences with --flashattention so far, especially for this implementation and for the large number of Pascal (GTX 10XX, P40...) cards.
Discussion linked bellow, with more links to relevant information:
https://huggingface.co/LWDCLS/LLM-Discussions/discussions/11
Cheers!
Update: Now available in the official releases of KoboldCpp!
[releases] https://github.com/LostRuins/koboldcpp/releases/latest
These are great news for all the users with GTX 10XX, P40...
Flash Attention implementation for older NVIDIA GPUs without requiring Tensor Cores has come to llama.cpp in the last few days, and should be merged in the next version of KoboldCpp, you can already try it with another fork or by building it.
[Mentioned KCPP fork] https://github.com/Nexesenex/kobold.cpp/releases/latest
[PR] https://github.com/ggerganov/llama.cpp/pull/7188
You should expect less VRAM usage for the same context, allowing you to experience higher contexts with your current GPU.
There have also been reported final tokens/second speed improvements for inference, so that's also grand!
If you have tried it, I'd like to hear your experiences with --flashattention so far, especially for this implementation and for the large number of Pascal (GTX 10XX, P40...) cards.
Discussion linked bellow, with more links to relevant information:
https://huggingface.co/LWDCLS/LLM-Discussions/discussions/11
Cheers!
- 24 replies
liked
2
models
9 months ago
