Outstanding Model
I tried most models that are coming in the recent days and this is the best one to run locally, fater than gpt4all and way more accurate.
This is my start script with llama.cpp
title llama.cpp
:start
main -i --interactive-first -r "### Human:" --temp 0 -c 2048 -n -1 --ignore-eos --repeat_penalty 1.2 --instruct -m ggml-vicuna-7b-4bit-rev1.bin
pause
goto start
I tried most models that are coming in the recent days and this is the best one to run locally, fater than gpt4all and way more accurate. This is my start script with llama.cpp
main -i --interactive-first -r "### Human:" --temp 0 -c 2048 -n -1 --ignore-eos --repeat_penalty 1.2 --instruct -m ggml-vicuna-7b-4bit-rev1.bin
You are my hero! Thanks to your comment, I was able to compile llama.cpp from https://github.com/ggerganov/llama.cpp
under the Windows 10, then run ggml-vicuna-7b-4bit-rev1.bin with your cmd line that I cited above. It works on laptop with 16 Gb RAM and rather fast! I agree that it may be the best LLM to run locally!
And it seems that it can write much more correct and longer program code than gpt4all! It's just amazing!
But sometimes, after a few answers, it just freezes while continuing to load the CPU. Has anyone noticed this?
How can i use ggml-vicuna-7b-4bit-rev1.bin model to run web ui?
I tried most models that are coming in the recent days and this is the best one to run locally, fater than gpt4all and way more accurate. This is my start script with llama.cpp
main -i --interactive-first -r "### Human:" --temp 0 -c 2048 -n -1 --ignore-eos --repeat_penalty 1.2 --instruct -m ggml-vicuna-7b-4bit-rev1.binYou are my hero! Thanks to your comment, I was able to compile llama.cpp from https://github.com/ggerganov/llama.cpp
under the Windows 10, then run ggml-vicuna-7b-4bit-rev1.bin with your cmd line that I cited above. It works on laptop with 16 Gb RAM and rather fast! I agree that it may be the best LLM to run locally!And it seems that it can write much more correct and longer program code than gpt4all! It's just amazing!
But sometimes, after a few answers, it just freezes while continuing to load the CPU. Has anyone noticed this?
@phew @ai2p - I just saw your comments here .... hey I'd reached the exact same conclusion - I've also experienced the same performance on my 16GB RAM laptop .... the vicuna 7b model seems to give the best results overall.
Y'all make sure to update to v1.1