jisx commited on
Commit
cefcc59
1 Parent(s): 870d20a

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +46 -0
README.md ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ ---
3
+ language:
4
+ - bg
5
+ - cs
6
+ - zh
7
+ - de
8
+ - fi
9
+ - fr
10
+ - ru
11
+ - es
12
+ tags:
13
+ - generation
14
+ - question answering
15
+ - instruction tuning
16
+ license: cc-by-nc-4.0
17
+ ---
18
+
19
+ ### Model Description
20
+
21
+ This HF repository contains LLMs instruction tuned with full-parameter fine-tuning and then used to study whether monolingual or multilingual instruction tuning is more favourable.
22
+ * [GitHub](https://github.com/hplt-project/monolingual-multilingual-instruction-tuning/tree/main)
23
+ * [Paper](https://arxiv.org/abs/2309.08958)
24
+
25
+ #### Instruction tuning details
26
+ * Base model: [bloom-1b7](https://huggingface.co/bloom-1b7)
27
+ * Instruction tuning language: multilingual downsampled (Bulgarian, Czech, Chinese, German, Finnish, French, Russian, and Spanish)
28
+ * Training method: full-parameter fine-tuning.
29
+ * Best checkpoint: best cross-entropy on a validation set, trained for 3 epochs.
30
+ * Dataset: machine-translated from [yahma/alpaca-cleaned](https://huggingface.co/datasets/yahma/alpaca-cleaned). You can download our data [HERE](https://github.com/hplt-project/monolingual-multilingual-instruction-tuning/tree/main/training-data).
31
+
32
+ #### Usage
33
+ The model checkpoint should be loaded using `transformers` library.
34
+
35
+ Please refer to our Github repository [HERE](https://github.com/hplt-project/monolingual-multilingual-instruction-tuning/tree/main/fpft) for inference and training instructions.
36
+
37
+ #### Citation
38
+ ```
39
+ @inproceedings{chen-etal-2024-monolingual,
40
+ title="Monolingual or multilingual instruction tuning: Which makes a better {Alpaca}",
41
+ author="Pinzhen Chen and Shaoxiong Ji and Nikolay Bogoychev and Andrey Kutuzov and Barry Haddow and Kenneth Heafield",
42
+ year="2024",
43
+ booktitle = "Findings of the Association for Computational Linguistics: EACL 2024",
44
+ }
45
+ ```
46
+