---
license: cc-by-4.0
datasets:
- Aiwensile2/Minecraft_QA-pairs_Instruction_Dataset
---
# MineMA: Fine-Tuned Models for Minecraft Q&A

## Overview

In this repository, we present the MineMA series of models, fine-tuned specifically for Minecraft-related Q&A tasks. Utilizing the LoRA method for efficient model fine-tuning, we have adapted pre-trained LLaMA models to respond accurately and effectively to Minecraft-related instructions and queries. Our fine-tuning process leverages the specially generated Minecraft dataset to ensure relevance and accuracy in the Q&A responses.

## Models

The MineMA series includes several models fine-tuned on different base models from the LLaMA series. Below is the list of the fine-tuned models provided in this repository:

- **MineMA-8B**(v1, v2, v3, v4), derived from the base model LLaMA-3-8B-Instruct.
- **MineMA-13B**(v1, v2), derived from the base model LLaMA-2-13B-Chat.
- **MineMA-70B**, derived from the base model LLaMA-3-70B-Instruct.

These models have been fine-tuned by using the **Minecraft_QA-pairs_Instruction_Dataset**. We have only released four models of MineMA-8B for the time being, and we will supplement more models in the future.

## Fine-Tuning Methodology

### LoRA Method for Fine-Tuning

We employed the **LoRA (Low-Rank Adaptation)** method for fine-tuning our models. LoRA is a parameter-efficient training technique that introduces small, trainable low-rank matrices to adapt a pre-trained neural network, allowing for targeted updates without the need for retraining the entire model. This method strikes a balance between computational efficiency and training effectiveness.

### Training Parameters

Here are the specific training parameters:

| Model        | lora\_r | lora\_alpha | lora\_dropout | learning\_rate | weight\_decay | Single Round? |
|--------------|---------|-------------|---------------|----------------|---------------|---------------|
| MineMA-13B-v1| 64      | 128         | 0.1           | 1E-04          | 1E-04         | False         |
| MineMA-13B-v2| 128     | 256         | 0.1           | 1E-04          | 1E-04         | False         |
| MineMA-8B-v1 | 64      | 128         | 0.1           | 1E-04          | 1E-04         | True          |
| MineMA-8B-v2 | 32      | 64          | 0.1           | 1E-04          | 1E-04         | False         |
| MineMA-8B-v3 | 64      | 128         | 0.1           | 1E-04          | 1E-04         | False         |
| MineMA-8B-v4 | 128     | 256         | 0.1           | 1E-04          | 1E-04         | False         |
| MineMA-70B   | 16      | 32          | 0.1           | 1E-04          | 1E-04         | True          |

## Dataset

We used the **Minecraft_QA-pairs_Instruction_Dataset** for fine-tuning all the models in the MineMA series. This dataset has 390,317 instruction entries specifically designed for Minecraft-related Q&A tasks. You can access the dataset via the following link:

[Minecraft_QA-pairs_Instruction_Dataset](https://huggingface.co/datasets/Aiwensile2/Minecraft_QA-pairs_Instruction_Dataset)

## Details
### License
These models are made available under the Creative Commons Attribution 4.0 International License.

### DOI
10.57967/hf/2488