NousResearch

/

Genstruct-7B

like 364

I want to run this with Augmentoolkit. For local model usage it usually uses the aphrodite engine, which takes awq or gptq quants (I mean I could quant it myself using lcpp and run a server with that but that's slower).

Are there quants available somewhere?