Update README.md

README.md

@@ -1,3 +1,46 @@
----
-license: cc-by-4.0
----
+---
+license: cc-by-4.0
+datasets:
+- Aiwensile2/Minecraft_QA-pairs_Instruction_Dataset
+---
+# MineMA: Fine-Tuned Models for Minecraft Q&A
+
+## Overview
+
+In this repository, we present the MineMA series of models, fine-tuned specifically for Minecraft-related Q&A tasks. Utilizing the LoRA method for efficient model fine-tuning, we have adapted pre-trained LLaMA models to respond accurately and effectively to Minecraft-related instructions and queries. Our fine-tuning process leverages the specially generated Minecraft dataset to ensure relevance and accuracy in the Q&A responses.
+
+## Models
+
+The MineMA series includes several models fine-tuned on different base models from the LLaMA series. Below is the list of the fine-tuned models provided in this repository:
+
+- **MineMA-8B**(v1, v2, v3, v4), derived from the base model LLaMA-3-8B-Instruct.
+- **MineMA-13B**(v1, v2), derived from the base model LLaMA-2-13B-Chat.
+- **MineMA-70B**, derived from the base model LLaMA-3-70B-Instruct.
+
+These models have been fine-tuned by using the **Minecraft_QA-pairs_Instruction_Dataset**. We have only released four models of MineMA-8B for the time being, and we will supplement more models in the future.
+
+## Fine-Tuning Methodology
+
+### LoRA Method for Fine-Tuning
+
+We employed the **LoRA (Low-Rank Adaptation)** method for fine-tuning our models. LoRA is a parameter-efficient training technique that introduces small, trainable low-rank matrices to adapt a pre-trained neural network, allowing for targeted updates without the need for retraining the entire model. This method strikes a balance between computational efficiency and training effectiveness.
+
+### Training Parameters
+
+Here are the specific training parameters:
+
+| Model        | lora\_r | lora\_alpha | lora\_dropout | learning\_rate | weight\_decay | Single Round? |
+|--------------|---------|-------------|---------------|----------------|---------------|---------------|
+| MineMA-13B-v1| 64      | 128         | 0.1           | 1E-04          | 1E-04         | False         |
+| MineMA-13B-v2| 128     | 256         | 0.1           | 1E-04          | 1E-04         | False         |
+| MineMA-8B-v1 | 64      | 128         | 0.1           | 1E-04          | 1E-04         | True          |
+| MineMA-8B-v2 | 32      | 64          | 0.1           | 1E-04          | 1E-04         | False         |
+| MineMA-8B-v3 | 64      | 128         | 0.1           | 1E-04          | 1E-04         | False         |
+| MineMA-8B-v4 | 128     | 256         | 0.1           | 1E-04          | 1E-04         | False         |
+| MineMA-70B   | 16      | 32          | 0.1           | 1E-04          | 1E-04         | True          |
+
+## Dataset
+
+We used the **Minecraft_QA-pairs_Instruction_Dataset** for fine-tuning all the models in the MineMA series. This dataset has 390,317 instruction entries specifically designed for Minecraft-related Q&A tasks. You can access the dataset via the following link:
+
+[Minecraft_QA-pairs_Instruction_Dataset](https://huggingface.co/datasets/Aiwensile2/Minecraft_QA-pairs_Instruction_Dataset)