|
--- |
|
license: apache-2.0 |
|
datasets: |
|
- QingyiSi/Alpaca-CoT |
|
language: |
|
- zh |
|
- en |
|
--- |
|
|
|
This is a QLoRa adapter model to [Falcon-40b](https://huggingface.co/tiiuae/falcon-40b). |
|
|
|
Though Falcon is not specifically trained on Chinese corpus, it exhibits strong performance in Chinese Language Understanding in our experiment. We would like to explore out of curiosity whether a |
|
small amount of Chinese instruction data can push it further and make it better at speaking.The LoRa model is trained with the [QLoRa repo](https://github.com/artidoro/qlora) on a subset of bilingual instruction data from [Alpaca-CoT dataset](https://huggingface.co/datasets/QingyiSi/Alpaca-CoT). |
|
|
|
The result is suprisingly good considering the number of Chinese tokens it has ever seen, though it shows clear limitions. Please check the examples below. |
|
|
|
``` |
|
Question: 序列1,4,9,16的后一个数是什么? |
|
Answer: 序列1,4,9,16的后一个数是25。因为序列中的每个数字都是前一个数字加上一个递增的差值,第一个差值是3,第二个差值是5,第三个差值是7,第四个差值是9,所以最后一个数字应该是第四个数字16加上第四个差值9,即16+9=25。因此,序列1,4,9,16的后一个数是25。 |
|
|
|
Question: 已知全班所有同学的姓名存储于列表names中,所有同学的身高存储于列表heights中,它们的关系是一一对应的。请写一段Python代码,将所有同学的姓名按身高由低到高排序。 |
|
Answer: names = ['小明', '小红', '小张', '小李'] |
|
heights = [170, 165, 175, 180] |
|
|
|
# 使用zip函数将姓名和身高对应起来,然后根据身高排序 |
|
sorted_names = sorted(names, key=lambda x: heights[names.index(x)]) |
|
|
|
# 输出排序后的姓名列表 |
|
print(sorted_names) |
|
|
|
Question: 中国的“东北三省”指的是哪里? |
|
Answer: 中国的东北三省指的是吉林省、黑龙江省和辽宁省。这些省份位于中国东北地区,北临俄罗斯和乌克兰,东临日本海。它们是中国最大的省份之一,占中国陆地面积的七分之一。东北三省以其自然风光、丰富的民族文化和历史遗产而闻名。它们是中国旅游业的重要目的地之一。您还有其他问题吗? |
|
|
|
Question: “但愿人长久,千里共婵娟”,其中婵娟指的是什么? A、月亮 B、姻缘 |
|
Answer: 这道题目考察的是对于古诗词的理解和文化常识。这句话出自唐代诗人杜甫的《月夜》一诗,其中婵娟是指月亮。这句话的意思是希望人和月亮能够长久存在,共同见证世间美好的事物。婵娟是古代诗词中常用的一个词语,指的是月亮的光辉、美丽和媚态。因此,答案是A,月亮。 |
|
``` |
|
|
|
|
|
## Evalutions |
|
|
|
We evaluate on [C-Eval](https://cevalbenchmark.com/). More results will be added later. |
|
|
|
Result on C-Eval test set with 5-shot and no CoT, which is better than LLaMA-series models. |
|
|
|
| Average | Avg(Hard) | STEM | Social Science | Humanities | Others | |
|
| - | - | - | - | - | - | |
|
| 42.0 | 32.4 | 38.5 | 50.3 | 42.5 | 40.5 | |