---
license: apache-2.0
datasets:
- M4-ai/Rhino
language:
- en
base_model: mistralai/Mistral-7B-v0.1
co2_eq_emissions:
  emissions: 3.8
---

# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->
This model aims to be a high-performance chatbot. During training, examples that have a quality score of less than 0.03 are skipped.

## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->

This model is to be used as a general-purpose chatbot/assistant. Trained on about 300,000 examples of M4-ai/Rhino, examples with a quality score lower than 0.03 are removed. During validation, this model achieved a loss of 0.55

- **Developed by:** Locutusque
- **Model type:** mistral
- **Language(s) (NLP):** English
- **License:** cc-by-nc-4.0
- **Finetuned from model:** mistralai/Mistral-7B-v0.1


## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

This model is to be used as a general-purpose assistant, and may need to be further fine-tuned on DPO to detoxify the model or SFT for a more specific task.

### Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

This model should be used as a general assistant. This model is capable of writing code, answering questions, and following instructions.

### Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.


## Training Details


#### Training Hyperparameters

- **Training regime:** bf16 non-mixed precision <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

## Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

### Testing Data, Factors & Metrics

#### Testing Data

<!-- This should link to a Dataset Card if possible. -->

First 100 examples of M4-ai/Rhino. Training data does not include these examples.


### Results

Test loss - 0.55

#### Summary


## Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).

- **Hardware Type:** 8 TPU V3s
- **Hours used:** 3
- **Cloud Provider:** Kaggle
- **Compute Region:** [More Information Needed]
- **Carbon Emitted:** 3.8