best open source model atm
This is the best open source model at the moment imo
This is the best open source model at the moment imo
I presume you were able to run it? If so, did you face any problems? I have not been able to run the model on 2xH100. (See https://huggingface.co/nvidia/Llama-3.1-Nemotron-70B-Instruct/discussions/10 for the problems I faced.) If you did something non-standard, please do let me know. Thanks.
This is the best open source model at the moment imo
I presume you were able to run it? If so, did you face any problems? I have not been able to run the model on 2xH100. (See https://huggingface.co/nvidia/Llama-3.1-Nemotron-70B-Instruct/discussions/10 for the problems I faced.) If you did something non-standard, please do let me know. Thanks.
I was able to use the q8 version with 4090, best cpu at the moment and a lot of ram, it's slow but workable for what i use it, i haven't tested the fp16 version