Feature request of quantization of starchat-alpha
#2
by
delas
- opened
Hi!
Firstly, thank you for your work implementing starcoder for ggml. Working fine with the 4 bit quantized version.
I would like to make a feature request related with the implementation of the 4 bit quantize version of starchat-alpha for enabling the usage with ggml.
Best Regards,
Jordi
Hello
How to run that starcoder ggml? I tried with llama.cpp and koboldcpp ... no working
This comment has been hidden
Thank you .
Is working so bad for you like for me?
I'm asking for c++ Fibonacci code and getting always for python.... and other ask also is giving me python code....
I suppose starcoder is not optimized for chat interaction but for autocompletion, that's why the finetuned variation starchat-alpha probably is more useful for interaction purposes.
Oh I see . Thanks .