Pythia 12b encoded into 4 bit as named. The first one I am uploading has true sequential and act order but no group size (full rank)

You can likely use it with textgen-ui or have a go at my v1/v2 supporting version

https://github.com/Ph0rk0z/text-generation-webui-testing/tree/DualModel

get tokenizers from pythia-12b or lotus-12b here on HF

Downloads last month: 13

Inference Examples

Text Generation

This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.