21 3 110

Csaba Kecskemeti PRO

csabakecskemeti

https://devquasar.com/

csabakecskemeti

AI & ML interests

None yet

Recent Activity

updated a model 2 minutes ago

DevQuasar/LatitudeGames.Wayfarer-Large-70B-Llama-3.3-GGUF

updated a model about 3 hours ago

DevQuasar/openai-community.openai-gpt-GGUF

published a model about 3 hours ago

DevQuasar/openai-community.openai-gpt-GGUF

View all activity

Organizations

Posts 19

Post

1465

-UPDATED-
4bit inference is working! The blogpost is updated with code snippet and requirements.txt
https://devquasar.com/uncategorized/all-about-amd-and-rocm/
-UPDATED-
I've played around with an MI100 and ROCm and collected my experience in a blogpost:
https://devquasar.com/uncategorized/all-about-amd-and-rocm/
Unfortunately I've could not make inference or training work with model loaded in 8bit or use BnB, but did everything else and documented my findings.

Post

2729

Testing Training on AMD/ROCm the first time!

I've got my hands on an AMD Instinct MI100. It's about the same price used as a V100 but on paper has more TOPS (V100 14TOPS vs MI100 23TOPS) also the HBM has faster clock so the memory bandwidth is 1.2TB/s.
For quantized inference it's a beast (MI50 was also surprisingly fast)

For LORA training with this quick test I could not make the bnb config works so I'm running the FT on the fill size model.

Will share all the install, setup and setting I've learned in a blog post, together with the cooling shroud 3D design.

View all Posts

models 1

csabakecskemeti/bert-base-case-yelp5-tuned-experiment

Text Classification • Updated Apr 5, 2024 • 14

datasets

None public yet