---
language:
- ms
library_name: transformers
---

# Full Parameter Finetuning 0.5B 8192 context length Qwen 2 on Malaysian text 12.86B tokens

Continue pretraining https://huggingface.co/Qwen/Qwen2-0.5B on 12.86B tokens Malaysian dataset using 8192 context length.

WanDB at https://wandb.ai/huseinzol05/Qwen2-0.5B-fpf/workspace?nw=nwuserhuseinzol05