rwitz/cat0.2

Cat0.2

Cat0.2 is a model further fine-tuned from Cat0.1 using an iterative dataset approach. The process involves iterating over the dataset three times, using the previous generation as the next iteration's dataset. This model is based on an 8 billion parameter Llama 3.1 architecture, further quantized to 4 bits for ease of upload. It has been iteratively trained on several hundred examples of data.

Iterative Process

1. Generate examples with the base model (Llama 3.1 8B).
2. Fine-tune on those examples.
3. Generate new examples with the updated model.
4. Repeat the process.

Note: The model may exhibit some hallucinations in its output due to the iterative training approach and may be prone to repetition.