---
library_name: transformers
license: llama3.1
datasets:
- Locutusque/ApolloRP-2.0-SFT
language:
- en
pipeline_tag: text-generation
tags:
- not-for-all-audiences
---

# Model Card for Locutusque/Apollo-2.0-Llama-3.1-8B

<!-- Provide a quick summary of what the model is/does. -->

SFT of Llama-3.1 8B. I was going to use DPO, but it made the model worse.

~50 point elo increase on the chaiverse leaderboard over preview versions.

## Model Details


### Model Description

<!-- Provide a longer summary of what this model is. -->

Fine-tuned Llama-3.1-8B on Locutusque/ApolloRP-2.0-SFT. Results in a good roleplaying language model, that isn't dumb.

- **Developed by:** Locutusque
- **Model type:** Llama3.1
- **Language(s) (NLP):** English
- **License:** Llama 3.1 Community License Agreement

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Demo:** https://huggingface.co/spaces/Locutusque/Locutusque-Models
### Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

RP/ERP, instruction following, conversation, etc


## Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

This model is completely uncensored - use at your own risk.

### Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.


## Training Details

### Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

[Locutusque/ApolloRP-2.0-SFT](https://huggingface.co/datasets/Locutusque/ApolloRP-2.0-SFT)

The training data is cleaned from refusals, and "slop".

#### Training Hyperparameters

- **Training regime:** bf16 non-mixed precision