Supported Models

HUGS supports a wide range of open AI models, including LLMs, Multimodal Models, and Embedding Models. Below is a matrix of all the models supported by HUGS and the hardware they are supported on.

15 Models Supported

Model	1x NVIDIA A10G	2x NVIDIA A10G	4x NVIDIA A10G	8x NVIDIA A10G	1x NVIDIA L4	2x NVIDIA L4	4x NVIDIA L4	8x NVIDIA L4	1x NVIDIA L40S	2x NVIDIA L40S	4x NVIDIA L40S	8x NVIDIA L40S	1x NVIDIA A100 80GB	2x NVIDIA A100 80GB	4x NVIDIA A100 80GB	8x NVIDIA A100 80GB	1x NVIDIA H100	2x NVIDIA H100	4x NVIDIA H100	8x NVIDIA H100	8x AMD Instinct MI300X	2x inf2	8x inf2	24x inf2
meta-llama/Meta-Llama-3.1-8B-Instruct	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅
meta-llama/Meta-Llama-3.1-70B-Instruct	❌	❌	❌	✅	❌	❌	❌	✅	❌	❌	✅	✅	❌	❌	✅	✅	❌	❌	✅	✅	✅	❌	❌	✅
meta-llama/Meta-Llama-3.1-405B-Instruct-FP8	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	✅	❌	❌	❌	❌
NousResearch/Hermes-3-Llama-3.1-8B	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅
NousResearch/Hermes-3-Llama-3.1-70B	❌	❌	❌	✅	❌	❌	❌	✅	❌	❌	✅	✅	❌	❌	✅	✅	❌	❌	✅	✅	✅	❌	❌	✅
NousResearch/Hermes-3-Llama-3.1-405B-FP8	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	✅	❌	❌	❌	❌
NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO	❌	❌	❌	✅	❌	❌	❌	✅	❌	❌	✅	✅	❌	✅	✅	✅	❌	✅	✅	✅	✅	❌	❌	✅
mistralai/Mixtral-8x7B-Instruct-v0.1	❌	❌	❌	✅	❌	❌	❌	✅	❌	❌	✅	✅	❌	✅	✅	✅	❌	✅	✅	✅	✅	❌	❌	✅
mistralai/Mistral-7B-Instruct-v0.3	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅
mistralai/Mixtral-8x22B-Instruct-v0.1	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	✅	❌	❌	❌	✅	❌	❌	❌	✅	✅	❌	❌	✅
google/gemma-2-27b-it	❌	❌	✅	✅	❌	❌	✅	✅	❌	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	❌	❌	❌
google/gemma-2-9b-it	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	❌	❌	❌
Qwen/Qwen2.5-7B-Instruct	✅	✅	✅	❌	✅	✅	✅	❌	✅	✅	✅	❌	✅	✅	✅	❌	✅	✅	✅	❌	❌	❌	❌	❌
meta-llama/Llama-3.2-11B-Vision-Instruct	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	❌	❌	❌	❌
meta-llama/Llama-3.2-90B-Vision-Instruct	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	❌	✅	❌	❌	✅	✅	❌	❌	✅	✅	❌	❌	❌	❌

Last Updated: 2024-11-28

< > Update on GitHub