Hugging Face's logo

nvidia
/
Llama-3.1-Nemotron-70B-Instruct-HF

Text Generation
Transformers
Safetensors
English
llama
nvidia
llama3.1
conversational
text-generation-inference
Model card Files Community
11

the model is not optimize in term of inference

#7
by Imran1 - opened
Discussion
Imran1

i load the model for inference using 2H100 gpu. but the model is very slow with Flash Attention.

arthrod

Can you share your code?

Sign up or log in to comment