Spaces:

Intel
/

low_bit_open_llm_leaderboard

Running

Add support for AQLM

by BlackSamorez - opened May 11

May 11

AQLM is a SOTA 2-bit LLM quantization algorithm, that shows incredible precision for its compression ratio. It's fully integrated with transformers and there are quite a few models prequantized.
Adding it to the leaderboard would shed light at what 2-bit quantization is really capable of.

lvkaokao

Intel org May 13

•

edited May 13

hi, @BlackSamorez , we will support AQLM as soon as possible! Thanks~

wenhuach

Intel org May 14

•

edited May 14

@BlackSamorez please kindly consider to compare your method with AutoRound which have already shown remarkable results at W2G128 and W2G32, as presented in https://github.com/intel/auto-round/blob/main/docs/acc.md, without introducing any extra overhead at inference,

lvkaokao

Intel org May 15

hi @BlackSamorez we add AQLM, we evaluate 2 models now and we will add more models results.

BlackSamorez changed discussion status to closed May 23

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment