Update README.md

22f3f7d verified 7 months ago

7.05 kB

	---
	language:
	- fi
	license: apache-2.0
	tags:
	- text-generation-inference
	- transformers
	- unsloth
	- llama
	- trl
	- sft
	base_model: LumiOpen/Viking-7B
	datasets:
	- mpasila/Finnish-Alpaca-Tiny
	---
	This is an ExLlamaV2 quantized model in 4bpw of [mpasila/Finnish-Alpaca-Tiny-V2-7B](https://huggingface.co/mpasila/Finnish-Alpaca-Tiny-V2-7B) using the default calibration dataset.

	# Original Model card:

	This is a merge of [mpasila/Finnish-Alpaca-Tiny-V2-LoRA-7B](https://huggingface.co/mpasila/Finnish-Alpaca-Tiny-V2-LoRA-7B).

	LoRA trained in 4-bit with 2k context using [LumiOpen/Viking-7B](https://huggingface.co/LumiOpen/Viking-7B/) as the base model for 1 epoch.

	Dataset used is [mpasila/Finnish-Alpaca-Tiny](https://huggingface.co/datasets/mpasila/Finnish-Alpaca-Tiny).

	It works relatively well for question and answering. I will make a bigger dataset for the next fine-tune.

	### Prompt format: Alpaca
	It uses Alpaca format but with a translated instruction at the start:
	```
	{
	"instruction,output": "Alla on ohje, jossa kuvataan tehtävä. Kirjoita vastaus, joka täyttää pyynnön asianmukaisesti.\n\n### Instruction:\n%instruction%\n\n### Response:\n%output%",
	"instruction,input,output": "Alla on ohje, jossa kuvataan tehtävä ja joka on yhdistetty kontekstia lisäävään syötteeseen. Kirjoita vastaus, joka täyttää pyynnön asianmukaisesti.\n\n### Instruction:\n%instruction%\n\n### Input:\n%input%\n\n### Response:\n%output%"
	}

	```

	## Evaluation

	\| Model \| Size \| Type \| FIN-bench (score) \|
	\|-------\|------\|------\|-------\|
	\| mpasila/Finnish-Alpaca-Tiny-V2-7B \| 7B \| Instruct \| 0.4654 \|
	\| [mpasila/Finnish-Alpaca-Small-7B](https://huggingface.co/mpasila/Finnish-Alpaca-Small-7B) \| 7B \| Instruct \| 0.3586 \|
	\| [mpasila/Alpacazord-Viking-7B](https://huggingface.co/mpasila/Alpacazord-Viking-7B) \| 7B \| Instruct \| 0.4123 \|
	\| [mpasila/NordicAlpaca-Finnish-V1-7B](https://huggingface.co/mpasila/NordicAlpaca-Finnish-V1-7B) \| 7B \| Instruct \| 0.3891 \|
	\| [mpasila/Finnish-Viking-Alpaca-V1-7B](https://huggingface.co/mpasila/Finnish-Viking-Alpaca-V1-7B) \| 7B \| Instruct \| 0.3943 \|
	\| [Finnish-NLP/llama-7b-finnish-instruct-v0.1](https://huggingface.co/Finnish-NLP/llama-7b-finnish-instruct-v0.1) \| 7B \| Instruct \| 0.4365 \|
	\| [Finnish-NLP/llama-7b-finnish-instruct-v0.2](https://huggingface.co/Finnish-NLP/llama-7b-finnish-instruct-v0.2) \| 7B \| Instruct \| 0.3993 \|
	\| [Finnish-NLP/llama-7b-finnish](https://huggingface.co/Finnish-NLP/llama-7b-finnish) \| 7B \| Base \| 0.2350 \|
	\| [LumiOpen/Viking-7B (1000B)](https://huggingface.co/LumiOpen/Viking-7B) \| 7B \| Base \| 0.3721 \|
	\| [HPLT/gpt-7b-nordic-prerelease](https://huggingface.co/HPLT/gpt-7b-nordic-prerelease) \| 7B \| Base \| 0.3169 \|

	[Source](https://docs.google.com/spreadsheets/d/1rqJb9dQVihg-Z1_Ras1L_-wuzPg9xNzpdmM2x5HueeY/edit?usp=sharing)

	#### FIN-bench scores:

	\| Task \|Version\| Metric \|Value \| \|Stderr\|
	\|------------------------------------------------\|------:\|---------------------\|-----:\|---\|-----:\|
	\|bigbench_analogies \| 0\|multiple_choice_grade\|0.6385\|± \|0.0423\|
	\|bigbench_arithmetic_1_digit_addition \| 0\|multiple_choice_grade\|0.7200\|± \|0.0451\|
	\|bigbench_arithmetic_1_digit_division \| 0\|multiple_choice_grade\|0.7391\|± \|0.0936\|
	\|bigbench_arithmetic_1_digit_multiplication \| 0\|multiple_choice_grade\|0.4800\|± \|0.0502\|
	\|bigbench_arithmetic_1_digit_subtraction \| 0\|multiple_choice_grade\|0.6300\|± \|0.0485\|
	\|bigbench_arithmetic_2_digit_addition \| 0\|multiple_choice_grade\|0.4000\|± \|0.0492\|
	\|bigbench_arithmetic_2_digit_division \| 0\|multiple_choice_grade\|0.5000\|± \|0.0503\|
	\|bigbench_arithmetic_2_digit_multiplication \| 0\|multiple_choice_grade\|0.2800\|± \|0.0451\|
	\|bigbench_arithmetic_2_digit_subtraction \| 0\|multiple_choice_grade\|0.4300\|± \|0.0498\|
	\|bigbench_arithmetic_3_digit_addition \| 0\|multiple_choice_grade\|0.5800\|± \|0.0496\|
	\|bigbench_arithmetic_3_digit_division \| 0\|multiple_choice_grade\|0.3100\|± \|0.0465\|
	\|bigbench_arithmetic_3_digit_multiplication \| 0\|multiple_choice_grade\|0.2900\|± \|0.0456\|
	\|bigbench_arithmetic_3_digit_subtraction \| 0\|multiple_choice_grade\|0.5100\|± \|0.0502\|
	\|bigbench_arithmetic_4_digit_addition \| 0\|multiple_choice_grade\|0.5300\|± \|0.0502\|
	\|bigbench_arithmetic_4_digit_division \| 0\|multiple_choice_grade\|0.3900\|± \|0.0490\|
	\|bigbench_arithmetic_4_digit_multiplication \| 0\|multiple_choice_grade\|0.3100\|± \|0.0465\|
	\|bigbench_arithmetic_4_digit_subtraction \| 0\|multiple_choice_grade\|0.6200\|± \|0.0488\|
	\|bigbench_arithmetic_5_digit_addition \| 0\|multiple_choice_grade\|0.6500\|± \|0.0479\|
	\|bigbench_arithmetic_5_digit_division \| 0\|multiple_choice_grade\|0.3200\|± \|0.0469\|
	\|bigbench_arithmetic_5_digit_multiplication \| 0\|multiple_choice_grade\|0.3000\|± \|0.0461\|
	\|bigbench_arithmetic_5_digit_subtraction \| 0\|multiple_choice_grade\|0.6400\|± \|0.0482\|
	\|bigbench_cause_and_effect_one_sentence \| 0\|multiple_choice_grade\|0.5686\|± \|0.0700\|
	\|bigbench_cause_and_effect_one_sentence_no_prompt\| 0\|multiple_choice_grade\|0.6471\|± \|0.0676\|
	\|bigbench_cause_and_effect_two_sentences \| 0\|multiple_choice_grade\|0.4314\|± \|0.0700\|
	\|bigbench_emotions \| 0\|multiple_choice_grade\|0.2250\|± \|0.0331\|
	\|bigbench_empirical_judgments \| 0\|multiple_choice_grade\|0.2525\|± \|0.0439\|
	\|bigbench_general_knowledge \| 0\|multiple_choice_grade\|0.3429\|± \|0.0571\|
	\|bigbench_hhh_alignment_harmless \| 0\|multiple_choice_grade\|0.3793\|± \|0.0643\|
	\|bigbench_hhh_alignment_helpful \| 0\|multiple_choice_grade\|0.3390\|± \|0.0622\|
	\|bigbench_hhh_alignment_honest \| 0\|multiple_choice_grade\|0.3729\|± \|0.0635\|
	\|bigbench_hhh_alignment_other \| 0\|multiple_choice_grade\|0.5349\|± \|0.0770\|
	\|bigbench_intent_recognition \| 0\|multiple_choice_grade\|0.2153\|± \|0.0156\|
	\|bigbench_misconceptions \| 0\|multiple_choice_grade\|0.5224\|± \|0.0433\|
	\|bigbench_paraphrase \| 0\|multiple_choice_grade\|0.4750\|± \|0.0354\|
	\|bigbench_sentence_ambiguity \| 0\|multiple_choice_grade\|0.4833\|± \|0.0651\|
	\|bigbench_similarities_abstraction \| 0\|multiple_choice_grade\|0.6974\|± \|0.0530\|

	# Uploaded model

	- Developed by: mpasila
	- License: apache-2.0
	- Finetuned from model : LumiOpen/Viking-7B

	This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

	[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)

	---
	language:
	- fi
	license: apache-2.0
	tags:
	- text-generation-inference
	- transformers
	- unsloth
	- llama
	- trl
	- sft
	base_model: LumiOpen/Viking-7B
	datasets:
	- mpasila/Finnish-Alpaca-Tiny
	---
	This is an ExLlamaV2 quantized model in 4bpw of [mpasila/Finnish-Alpaca-Tiny-V2-7B](https://huggingface.co/mpasila/Finnish-Alpaca-Tiny-V2-7B) using the default calibration dataset.

	# Original Model card:

	This is a merge of [mpasila/Finnish-Alpaca-Tiny-V2-LoRA-7B](https://huggingface.co/mpasila/Finnish-Alpaca-Tiny-V2-LoRA-7B).

	LoRA trained in 4-bit with 2k context using [LumiOpen/Viking-7B](https://huggingface.co/LumiOpen/Viking-7B/) as the base model for 1 epoch.

	Dataset used is [mpasila/Finnish-Alpaca-Tiny](https://huggingface.co/datasets/mpasila/Finnish-Alpaca-Tiny).

	It works relatively well for question and answering. I will make a bigger dataset for the next fine-tune.

	### Prompt format: Alpaca
	It uses Alpaca format but with a translated instruction at the start:
	```
	{
	"instruction,output": "Alla on ohje, jossa kuvataan tehtävä. Kirjoita vastaus, joka täyttää pyynnön asianmukaisesti.\n\n### Instruction:\n%instruction%\n\n### Response:\n%output%",
	"instruction,input,output": "Alla on ohje, jossa kuvataan tehtävä ja joka on yhdistetty kontekstia lisäävään syötteeseen. Kirjoita vastaus, joka täyttää pyynnön asianmukaisesti.\n\n### Instruction:\n%instruction%\n\n### Input:\n%input%\n\n### Response:\n%output%"
	}

	```

	## Evaluation

	\| Model \| Size \| Type \| FIN-bench (score) \|
	\|-------\|------\|------\|-------\|
	\| mpasila/Finnish-Alpaca-Tiny-V2-7B \| 7B \| Instruct \| 0.4654 \|
	\| [mpasila/Finnish-Alpaca-Small-7B](https://huggingface.co/mpasila/Finnish-Alpaca-Small-7B) \| 7B \| Instruct \| 0.3586 \|
	\| [mpasila/Alpacazord-Viking-7B](https://huggingface.co/mpasila/Alpacazord-Viking-7B) \| 7B \| Instruct \| 0.4123 \|
	\| [mpasila/NordicAlpaca-Finnish-V1-7B](https://huggingface.co/mpasila/NordicAlpaca-Finnish-V1-7B) \| 7B \| Instruct \| 0.3891 \|
	\| [mpasila/Finnish-Viking-Alpaca-V1-7B](https://huggingface.co/mpasila/Finnish-Viking-Alpaca-V1-7B) \| 7B \| Instruct \| 0.3943 \|
	\| [Finnish-NLP/llama-7b-finnish-instruct-v0.1](https://huggingface.co/Finnish-NLP/llama-7b-finnish-instruct-v0.1) \| 7B \| Instruct \| 0.4365 \|
	\| [Finnish-NLP/llama-7b-finnish-instruct-v0.2](https://huggingface.co/Finnish-NLP/llama-7b-finnish-instruct-v0.2) \| 7B \| Instruct \| 0.3993 \|
	\| [Finnish-NLP/llama-7b-finnish](https://huggingface.co/Finnish-NLP/llama-7b-finnish) \| 7B \| Base \| 0.2350 \|
	\| [LumiOpen/Viking-7B (1000B)](https://huggingface.co/LumiOpen/Viking-7B) \| 7B \| Base \| 0.3721 \|
	\| [HPLT/gpt-7b-nordic-prerelease](https://huggingface.co/HPLT/gpt-7b-nordic-prerelease) \| 7B \| Base \| 0.3169 \|

	[Source](https://docs.google.com/spreadsheets/d/1rqJb9dQVihg-Z1_Ras1L_-wuzPg9xNzpdmM2x5HueeY/edit?usp=sharing)

	#### FIN-bench scores:

	\| Task \|Version\| Metric \|Value \| \|Stderr\|
	\|------------------------------------------------\|------:\|---------------------\|-----:\|---\|-----:\|
	\|bigbench_analogies \| 0\|multiple_choice_grade\|0.6385\|± \|0.0423\|
	\|bigbench_arithmetic_1_digit_addition \| 0\|multiple_choice_grade\|0.7200\|± \|0.0451\|
	\|bigbench_arithmetic_1_digit_division \| 0\|multiple_choice_grade\|0.7391\|± \|0.0936\|
	\|bigbench_arithmetic_1_digit_multiplication \| 0\|multiple_choice_grade\|0.4800\|± \|0.0502\|
	\|bigbench_arithmetic_1_digit_subtraction \| 0\|multiple_choice_grade\|0.6300\|± \|0.0485\|
	\|bigbench_arithmetic_2_digit_addition \| 0\|multiple_choice_grade\|0.4000\|± \|0.0492\|
	\|bigbench_arithmetic_2_digit_division \| 0\|multiple_choice_grade\|0.5000\|± \|0.0503\|
	\|bigbench_arithmetic_2_digit_multiplication \| 0\|multiple_choice_grade\|0.2800\|± \|0.0451\|
	\|bigbench_arithmetic_2_digit_subtraction \| 0\|multiple_choice_grade\|0.4300\|± \|0.0498\|
	\|bigbench_arithmetic_3_digit_addition \| 0\|multiple_choice_grade\|0.5800\|± \|0.0496\|
	\|bigbench_arithmetic_3_digit_division \| 0\|multiple_choice_grade\|0.3100\|± \|0.0465\|
	\|bigbench_arithmetic_3_digit_multiplication \| 0\|multiple_choice_grade\|0.2900\|± \|0.0456\|
	\|bigbench_arithmetic_3_digit_subtraction \| 0\|multiple_choice_grade\|0.5100\|± \|0.0502\|
	\|bigbench_arithmetic_4_digit_addition \| 0\|multiple_choice_grade\|0.5300\|± \|0.0502\|
	\|bigbench_arithmetic_4_digit_division \| 0\|multiple_choice_grade\|0.3900\|± \|0.0490\|
	\|bigbench_arithmetic_4_digit_multiplication \| 0\|multiple_choice_grade\|0.3100\|± \|0.0465\|
	\|bigbench_arithmetic_4_digit_subtraction \| 0\|multiple_choice_grade\|0.6200\|± \|0.0488\|
	\|bigbench_arithmetic_5_digit_addition \| 0\|multiple_choice_grade\|0.6500\|± \|0.0479\|
	\|bigbench_arithmetic_5_digit_division \| 0\|multiple_choice_grade\|0.3200\|± \|0.0469\|
	\|bigbench_arithmetic_5_digit_multiplication \| 0\|multiple_choice_grade\|0.3000\|± \|0.0461\|
	\|bigbench_arithmetic_5_digit_subtraction \| 0\|multiple_choice_grade\|0.6400\|± \|0.0482\|
	\|bigbench_cause_and_effect_one_sentence \| 0\|multiple_choice_grade\|0.5686\|± \|0.0700\|
	\|bigbench_cause_and_effect_one_sentence_no_prompt\| 0\|multiple_choice_grade\|0.6471\|± \|0.0676\|
	\|bigbench_cause_and_effect_two_sentences \| 0\|multiple_choice_grade\|0.4314\|± \|0.0700\|
	\|bigbench_emotions \| 0\|multiple_choice_grade\|0.2250\|± \|0.0331\|
	\|bigbench_empirical_judgments \| 0\|multiple_choice_grade\|0.2525\|± \|0.0439\|
	\|bigbench_general_knowledge \| 0\|multiple_choice_grade\|0.3429\|± \|0.0571\|
	\|bigbench_hhh_alignment_harmless \| 0\|multiple_choice_grade\|0.3793\|± \|0.0643\|
	\|bigbench_hhh_alignment_helpful \| 0\|multiple_choice_grade\|0.3390\|± \|0.0622\|
	\|bigbench_hhh_alignment_honest \| 0\|multiple_choice_grade\|0.3729\|± \|0.0635\|
	\|bigbench_hhh_alignment_other \| 0\|multiple_choice_grade\|0.5349\|± \|0.0770\|
	\|bigbench_intent_recognition \| 0\|multiple_choice_grade\|0.2153\|± \|0.0156\|
	\|bigbench_misconceptions \| 0\|multiple_choice_grade\|0.5224\|± \|0.0433\|
	\|bigbench_paraphrase \| 0\|multiple_choice_grade\|0.4750\|± \|0.0354\|
	\|bigbench_sentence_ambiguity \| 0\|multiple_choice_grade\|0.4833\|± \|0.0651\|
	\|bigbench_similarities_abstraction \| 0\|multiple_choice_grade\|0.6974\|± \|0.0530\|

	# Uploaded model

	- Developed by: mpasila
	- License: apache-2.0
	- Finetuned from model : LumiOpen/Viking-7B

	This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

	[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)