awq4nvila / README.md
Louym's picture
[MAJOR] Rename and Add 3B models
b861479
metadata
license: apache-2.0

Some scales for deployment of NVILA.

'nvila2-video' is for the 7B video nvila model.

Files ending with smooth-scale are the act scale for smoothquant.
Files ending with w4-g128-v2 are the real quantized LLM.
The other files with only model names are the awq scales.

If you merely want to run the quantized NVILA models, the awq scales are of no use. You only need the smooth-scale file to quantize the visiontower and the real quantized files to run LLM.