Update README.md

README.md

@@ -11,6 +11,53 @@ base_model: rinna/llama-3-youko-8b
 pipeline_tag: translation
 ---
 
-# Model Card
+# VNTL
 
-TBD
+This is an [LLaMA 3 Youko](https://huggingface.co/rinna/llama-3-youko-8b) qlora created using a slightly modified version of the [VNTL-v3.1-1k](https://huggingface.co/datasets/lmg-anon/VNTL-v3.1-1k) dataset, concatenated with the [VNTL-Chat](https://huggingface.co/datasets/lmg-anon/VNTL-Chat) dataset. 
+
+This was trained mostly with the same hyperparameters as the [VNTL 7B v0.3.1 lora](https://huggingface.co/lmg-anon/vntl-7b-v0.3.1-lora), the differences are:
+- Added \["<\<METADATA>>", "<\<TRANSLATE>>", "<\<JAPANESE>>", "<\<ENGLISH>>", "<\<CHAT>>", "<\<HUMAN>>", "<\<LLM>>"\] as special tokens.
+- Trained the \["embed_tokens", "lm_head"\] layers.
+- 10x smaller learning rate, 0.00065 -> 0.000065.
+
+This version also includes a new "chat mode", which was lazily trained just to find out how that would impact the end result. I think this ended up quite good for breaking down or explaining Japanese sentences, but it is terrible for most other things, which is expected given the nature of the VNTL-Chat dataset. To be honest, I wasn't aiming for it to be good, so the fact that it works at all is very nice.
+
+**Eval Loss**: 0.8
+
+## Translation Prompt
+
+This is an prompt example for translation:
+```
+<<METADATA>>
+[character] Name: Uryuu Shingo (瓜生 新吾) | Gender: Male | Aliases: Onii-chan (お兄ちゃん)
+[character] Name: Uryuu Sakuno (瓜生 桜乃) | Gender: Female
+<<TRANSLATE>>
+<<JAPANESE>>
+[桜乃]: 『……ごめん』
+<<ENGLISH>>
+[Sakuno]: 『... Sorry.』<|end_of_text|>
+<<JAPANESE>>
+[新吾]: 「ううん、こう言っちゃなんだけど、迷子でよかったよ。桜乃は可愛いから、いろいろ心配しちゃってたんだぞ俺」
+<<ENGLISH>>
+```
+
+The generated translation for that prompt, with temperature 0, is:
+```
+[Shingo]: 「No, I know this isn't the best way to say it, but I'm glad you got lost. You're so cute that I was worried about you.」
+```
+
+## Chat Prompt
+
+This is an prompt example for chat:
+```
+<<CHAT>>
+<<HUMAN>>
+Hi! Can you break this down to me? Like, what is the meaning of "桜乃は可愛いから"?
+"ううん、こう言っちゃなんだけど、迷子でよかったよ。桜乃は可愛いから、いろいろ心配しちゃってたんだぞ俺」"
+<<LLM>>
+```
+
+The generated response for that prompt, with temperature 0, is:
+```
+Certainly! The phrase "桜乃は可愛いから" literally translates to "Because Sakura-no is cute." In Japanese culture, there is a strong emphasis on physical attractiveness and cuteness, especially when it comes to young girls or women. This phrase suggests that the speaker finds Sakura-no (presumably a girl) to be very attractive and endearing in appearance.
+```