|
--- |
|
license: apache-2.0 |
|
language: |
|
- ja |
|
tags: |
|
- mistral |
|
- mixtral |
|
- not-for-all-audiences |
|
- nsfw |
|
base_model: |
|
- NTQAI/chatntq-ja-7b-v1.0 |
|
- stabilityai/japanese-stablelm-instruct-gamma-7b |
|
- Elizezen/Antler-7B |
|
- Elizezen/Hameln-japanese-mistral-7B |
|
--- |
|
|
|
# LightChatAssistant-4x7B |
|
[GGUF版はこちら/Click here for the GGUF version](https://huggingface.co/Aratako/LightChatAssistant-4x7B-GGUF) |
|
|
|
## 概要 |
|
[Sdff-Ltba/LightChatAssistant-2x7B](https://huggingface.co/Sdff-Ltba/LightChatAssistant-2x7B)と全く同じ手法を用いつつ、対象を4モデルに拡張してそれらをMoEしたモデルです。素晴らしい手法を提案いただいた[@Sdff-Ltba](https://huggingface.co/Sdff-Ltba)さんに感謝します。 |
|
|
|
モデルサイズは大きくなっていますが、推論に同時に使われるエキスパートは2つなので、GPUにオフロードさえできれば元の[Sdff-Ltba/LightChatAssistant-2x7B](https://huggingface.co/Sdff-Ltba/LightChatAssistant-2x7B)とほぼ同じような速度で動きます。 |
|
|
|
|
|
以下の4モデルをChat Vectorの加算によって対話強化して素材に使用しています。 |
|
- [NTQAI/chatntq-ja-7b-v1.0](https://huggingface.co/NTQAI/chatntq-ja-7b-v1.0) |
|
- [stabilityai/japanese-stablelm-instruct-gamma-7b](https://huggingface.co/stabilityai/japanese-stablelm-instruct-gamma-7b) |
|
- [Elizezen/Antler-7B](https://huggingface.co/Elizezen/Antler-7B) |
|
- [Elizezen/Hameln-japanese-mistral-7B](https://huggingface.co/Elizezen/Hameln-japanese-mistral-7B) |
|
|
|
Chat Vectorの適用方法は[Sdff-Ltba/LightChatAssistant-2x7B](https://huggingface.co/Sdff-Ltba/LightChatAssistant-2x7B)と同じで、Mistral-7B-Instruct-v0.2とMistral-7B-v0.1の差分を0.8倍してそれぞれのモデルに足しています。 |
|
|
|
その後、[mergekit](https://github.com/cg123/mergekit)を使い、4モデルをMoEしました。MoEのconfigは以下の通りです。 |
|
|
|
```yaml |
|
base_model: ./chatntq-chatvector |
|
gate_mode: hidden |
|
dtype: bfloat16 |
|
experts: |
|
- source_model: ./chatntq-chatvector |
|
positive_prompts: |
|
- "roleplay" |
|
- "question" |
|
- "answer" |
|
- "chat" |
|
- "companion" |
|
- "character" |
|
- "math" |
|
- "code" |
|
- "[Mode: Roleplay]" |
|
- "[Mode: Chat]" |
|
negative_prompts: |
|
- "storywriting" |
|
- "book" |
|
- "story" |
|
- "chapter" |
|
- "tale" |
|
- "history" |
|
- "write" |
|
- "novel" |
|
- "[Mode: Writing]" |
|
- source_model: ./japanese-stablelm-instruct-gamma-chatvector |
|
positive_prompts: |
|
- "roleplay" |
|
- "question" |
|
- "answer" |
|
- "chat" |
|
- "companion" |
|
- "character" |
|
- "math" |
|
- "code" |
|
- "[Mode: Roleplay]" |
|
- "[Mode: Chat]" |
|
negative_prompts: |
|
- "storywriting" |
|
- "book" |
|
- "story" |
|
- "chapter" |
|
- "tale" |
|
- "history" |
|
- "write" |
|
- "novel" |
|
- "[Mode: Writing]" |
|
- source_model: ./Hameln-japanese-mistral-chatvector |
|
positive_prompts: |
|
- "sex" |
|
- "storywriting" |
|
- "erotic" |
|
- "fuck" |
|
- "orgasm" |
|
- "uncensored" |
|
- "book" |
|
- "story" |
|
- "chapter" |
|
- "tale" |
|
- "history" |
|
- "write" |
|
- "novel" |
|
- "[Mode: Writing]" |
|
- source_model: ./Antler-chatvector |
|
positive_prompts: |
|
- "sex" |
|
- "storywriting" |
|
- "erotic" |
|
- "fuck" |
|
- "orgasm" |
|
- "uncensored" |
|
- "book" |
|
- "story" |
|
- "chapter" |
|
- "tale" |
|
- "history" |
|
- "write" |
|
- "novel" |
|
- "[Mode: Writing]" |
|
tokenizer_source: union |
|
``` |
