chaispace/chai-small

Chai-small is an experimental model. It demonstrates notable improvements on the ARC Challenge benchmark, matching modern 3B model performance.

For users seeking multilingual capabilities and broad general knowledge on simple tasks, qwen or llama may be more suitable. However, if you need a compact model specifically optimized for basic English language tasks at half the size, chai-small is the ideal choice.

prompt

The model is designed to run without any specific prompt template.

benchmarks

Zero-shot performance comparison with leading compact language models:

Parameters	Model	MMLU	ARC-C	HellaSwag	PIQA	Winogrande	Average
0.5b	qwen 2	44.13	28.92	49.05	69.31	56.99	49.68
0.3b	smollm2	25.52	37.71	56.41	71.93	59.27	50.17
0.5b	qwen 2.5	47.29	31.83	52.17	70.29	57.06	51.72
1.24b	llama 3.2	36.75	36.18	63.70	74.54	60.54	54.34
0.5b	chai-small	25.51	38.82	63.02	74.70	61.25	52.66