DevQuasar/Rombo-Org.Rombo-LLM-V3.1-QWQ-32b-GGUF
Text Generation
•
Updated
DevQuasar
community
Verified
AI & ML interests
Open-Source LLMs, Local AI Projects: https://pypi.org/project/llm-predictive-router/
Recent Activity
View all activity
DevQuasar's activity
csabakecskemeti
updated
a
model
about 2 hours ago
csabakecskemeti
updated
a
collection
about 3 hours ago
csabakecskemeti
updated
a
model
about 3 hours ago
csabakecskemeti
updated
a
model
about 5 hours ago
csabakecskemeti
published
a
model
about 5 hours ago
csabakecskemeti
published
a
model
about 6 hours ago
csabakecskemeti
updated
a
model
about 9 hours ago
csabakecskemeti
updated
a
collection
about 9 hours ago
csabakecskemeti
updated
3
models
about 9 hours ago
csabakecskemeti
updated
a
collection
about 9 hours ago
csabakecskemeti
updated
a
model
about 9 hours ago
csabakecskemeti
published
a
model
about 9 hours ago
csabakecskemeti
updated
a
collection
about 10 hours ago
csabakecskemeti
published
a
model
about 10 hours ago
csabakecskemeti
updated
a
model
about 10 hours ago
csabakecskemeti
posted
an
update
5 days ago
Post
1879
-UPDATED-
4bit inference is working! The blogpost is updated with code snippet and requirements.txt
https://devquasar.com/uncategorized/all-about-amd-and-rocm/
-UPDATED-
I've played around with an MI100 and ROCm and collected my experience in a blogpost:
https://devquasar.com/uncategorized/all-about-amd-and-rocm/
Unfortunately I've could not make inference or training work with model loaded in 8bit or use BnB, but did everything else and documented my findings.
4bit inference is working! The blogpost is updated with code snippet and requirements.txt
https://devquasar.com/uncategorized/all-about-amd-and-rocm/
-UPDATED-
I've played around with an MI100 and ROCm and collected my experience in a blogpost:
https://devquasar.com/uncategorized/all-about-amd-and-rocm/
Unfortunately I've could not make inference or training work with model loaded in 8bit or use BnB, but did everything else and documented my findings.
-
- 4 replies
csabakecskemeti
posted
an
update
11 days ago
Post
2743
Testing Training on AMD/ROCm the first time!
I've got my hands on an AMD Instinct MI100. It's about the same price used as a V100 but on paper has more TOPS (V100 14TOPS vs MI100 23TOPS) also the HBM has faster clock so the memory bandwidth is 1.2TB/s.
For quantized inference it's a beast (MI50 was also surprisingly fast)
For LORA training with this quick test I could not make the bnb config works so I'm running the FT on the fill size model.
Will share all the install, setup and setting I've learned in a blog post, together with the cooling shroud 3D design.
I've got my hands on an AMD Instinct MI100. It's about the same price used as a V100 but on paper has more TOPS (V100 14TOPS vs MI100 23TOPS) also the HBM has faster clock so the memory bandwidth is 1.2TB/s.
For quantized inference it's a beast (MI50 was also surprisingly fast)
For LORA training with this quick test I could not make the bnb config works so I'm running the FT on the fill size model.
Will share all the install, setup and setting I've learned in a blog post, together with the cooling shroud 3D design.
-
- 8 replies
csabakecskemeti
posted
an
update
20 days ago
Post
1620
I found if we apply the reasoning system prompt (that has been published on the
NousResearch/DeepHermes-3-Llama-3-8B-Preview model card) other models are also react to it and start mimicking reasoning. Some better some worse. I've seen internal monologue and self questioning.
Here's a blogpost about it:
http://devquasar.com/ai/reasoning-system-prompt/
Here's a blogpost about it:
http://devquasar.com/ai/reasoning-system-prompt/