Hugging Face's logo
Michael Goin's picture
TFLOPS icon 82.6 TFLOPS
72 15 20

Michael Goin PRO

mgoin
rahulsinghal's profile picture heli-stand's profile picture imjliao's profile picture
·

AI & ML interests

LLM inference optimization, compression, quantization, pruning, distillation

Recent Activity

updated a model about 12 hours ago
nm-testing/pixtral-12b-FP8-dynamic-all
updated a model 3 days ago
mistralai/Pixtral-Large-Instruct-2411
New activity 3 days ago
mistralai/Pixtral-Large-Instruct-2411

Organizations

Neural Magic's profile picture garage-bAInd's profile picture Blog-explorers's profile picture Revel Labs's profile picture ZeroGPU Explorers's profile picture NM Testing's profile picture MLX Community's profile picture Social Post Explorers's profile picture

mgoin's activity

New activity in mistralai/Pixtral-Large-Instruct-2411 3 days ago

Add config_format and load_format to vLLM args
#5 opened 3 days ago by mgoin

Update config.json to use null for sliding_window
#4 opened 3 days ago by mgoin
New activity in mgoin/nemotron-3-8b-chat-4k-sft-hf 9 days ago

Adding `safetensors` variant of this model
#1 opened 9 days ago by SFconvertbot
New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a16 17 days ago

Is this the standard GPTQ quantization?

1
#5 opened 20 days ago by molereddy
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w8a16 17 days ago

Model weights are not loaded

4
#3 opened 3 months ago by MarvelousMouse
New activity in neuralmagic/pixtral-12b-FP8-dynamic 21 days ago

Update model card
#1 opened 21 days ago by nm-research
New activity in nm-testing/llava-1.5-7b-hf-FP8-dynamic 21 days ago

Add chat_template to tokenizer_config.json
#1 opened 22 days ago by nm-research
New activity in neuralmagic/Mistral-Nemo-Instruct-2407-FP8 22 days ago

7900xtx torch._scaled_mm is only supported on CUDA devices with compute capability >= 9.0 or 8.9, or ROCm MI300+

1
#3 opened 22 days ago by aaaaaaaaaasdf
New activity in mistral-community/pixtral-12b about 1 month ago

Why is the Pixtral activation function "gelu" when the reference code uses "silu"?

2
#10 opened about 1 month ago by mgoin

Update tokenizer_config.json with chat_template

2
#11 opened about 1 month ago by mgoin
New activity in neuralmagic/Llama-3.2-90B-Vision-Instruct-FP8-dynamic about 1 month ago

Any chance your team is working on a 4-bit Llama-3.2-90B-Vision-Instruct-quantized.w4a16 version?

1
#1 opened about 2 months ago by mrhendrey
New activity in neuralmagic/Llama-3.2-11B-Vision-Instruct-FP8-dynamic about 2 months ago

Oom with 24g vram

3
#1 opened about 2 months ago by Klopez
New activity in neuralmagic/Phi-3.5-mini-instruct-FP8-KV about 2 months ago

latest vllm docker (v0.6.2) fail to load

2
#1 opened about 2 months ago by choronz333
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w4a16 3 months ago

Issue with loading model

1
#1 opened 3 months ago by xSumukhax
New activity in neuralmagic/DeepSeek-Coder-V2-Instruct-FP8 3 months ago

Can it run on A100/A800 with VLLM?

3
#1 opened 4 months ago by Parkerlambert123
New activity in neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w4a16 3 months ago

weights does not exist when trying to deploy in sagemaker endpoint

1
#1 opened 3 months ago by LorenzoCevolaniAXA
New activity in meta-llama/Llama-3.1-405B-Instruct 4 months ago

8-kv-heads

4
#17 opened 4 months ago by ArthurZ
New activity in meta-llama/Llama-3.1-405B 4 months ago

8-kv-heads

3
#21 opened 4 months ago by ArthurZ
New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a16 4 months ago

run with vllm

8
#4 opened 4 months ago by kuliev-vitaly
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8 4 months ago

Not able to run Model using VLLM

1
#3 opened 4 months ago by Pchaudhary