mat1 and mat2 shapes cannot be multiplied (38x5120 and 1x2560)
#13
by
LaferriereJC
- opened
RuntimeError: mat1 and mat2 shapes cannot be multiplied (38x5120 and 1x2560)
Output generated in 0.36 seconds (0.00 tokens/s, 0 tokens, context 38, seed 1942857980)
LaferriereJC
changed discussion title from
error when attempting to run model quantized
to error when attempting to run model quantized mat1 and mat2 shapes cannot be multiplied (38x5120 and 1x2560)
LaferriereJC
changed discussion title from
error when attempting to run model quantized mat1 and mat2 shapes cannot be multiplied (38x5120 and 1x2560)
to mat1 and mat2 shapes cannot be multiplied (38x5120 and 1x2560)
In the file config.json, change the value of "pretraining_tp" (currently on line 18) from 2 to 1. This should fix it.