File size: 3,412 Bytes
82a3049 ffee704 0e6f1b7 82a3049 ffee704 74e7581 ffee704 27a8c8b ffee704 27a8c8b ffee704 27a8c8b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 |
---
license: apache-2.0
language:
- ja
tags:
- mistral
- mixtral
- not-for-all-audiences
- nsfw
base_model:
- NTQAI/chatntq-ja-7b-v1.0
- stabilityai/japanese-stablelm-instruct-gamma-7b
- Elizezen/Antler-7B
- Elizezen/Hameln-japanese-mistral-7B
---
# LightChatAssistant-4x7B
[GGUF版はこちら/Click here for the GGUF version](https://huggingface.co/Aratako/LightChatAssistant-4x7B-GGUF)
## 概要
[Sdff-Ltba/LightChatAssistant-2x7B](https://huggingface.co/Sdff-Ltba/LightChatAssistant-2x7B)と全く同じ手法を用いつつ、対象を4モデルに拡張してそれらをMoEしたモデルです。素晴らしい手法を提案いただいた[@Sdff-Ltba](https://huggingface.co/Sdff-Ltba)さんに感謝します。
モデルサイズは大きくなっていますが、推論に同時に使われるエキスパートは2つなので、GPUにオフロードさえできれば元の[Sdff-Ltba/LightChatAssistant-2x7B](https://huggingface.co/Sdff-Ltba/LightChatAssistant-2x7B)とほぼ同じような速度で動きます。
以下の4モデルをChat Vectorの加算によって対話強化して素材に使用しています。
- [NTQAI/chatntq-ja-7b-v1.0](https://huggingface.co/NTQAI/chatntq-ja-7b-v1.0)
- [stabilityai/japanese-stablelm-instruct-gamma-7b](https://huggingface.co/stabilityai/japanese-stablelm-instruct-gamma-7b)
- [Elizezen/Antler-7B](https://huggingface.co/Elizezen/Antler-7B)
- [Elizezen/Hameln-japanese-mistral-7B](https://huggingface.co/Elizezen/Hameln-japanese-mistral-7B)
Chat Vectorの適用方法は[Sdff-Ltba/LightChatAssistant-2x7B](https://huggingface.co/Sdff-Ltba/LightChatAssistant-2x7B)と同じで、Mistral-7B-Instruct-v0.2とMistral-7B-v0.1の差分を0.8倍してそれぞれのモデルに足しています。
その後、[mergekit](https://github.com/cg123/mergekit)を使い、4モデルをMoEしました。MoEのconfigは以下の通りです。
```yaml
base_model: ./chatntq-chatvector
gate_mode: hidden
dtype: bfloat16
experts:
- source_model: ./chatntq-chatvector
positive_prompts:
- "roleplay"
- "question"
- "answer"
- "chat"
- "companion"
- "character"
- "math"
- "code"
- "[Mode: Roleplay]"
- "[Mode: Chat]"
negative_prompts:
- "storywriting"
- "book"
- "story"
- "chapter"
- "tale"
- "history"
- "write"
- "novel"
- "[Mode: Writing]"
- source_model: ./japanese-stablelm-instruct-gamma-chatvector
positive_prompts:
- "roleplay"
- "question"
- "answer"
- "chat"
- "companion"
- "character"
- "math"
- "code"
- "[Mode: Roleplay]"
- "[Mode: Chat]"
negative_prompts:
- "storywriting"
- "book"
- "story"
- "chapter"
- "tale"
- "history"
- "write"
- "novel"
- "[Mode: Writing]"
- source_model: ./Hameln-japanese-mistral-chatvector
positive_prompts:
- "sex"
- "storywriting"
- "erotic"
- "fuck"
- "orgasm"
- "uncensored"
- "book"
- "story"
- "chapter"
- "tale"
- "history"
- "write"
- "novel"
- "[Mode: Writing]"
- source_model: ./Antler-chatvector
positive_prompts:
- "sex"
- "storywriting"
- "erotic"
- "fuck"
- "orgasm"
- "uncensored"
- "book"
- "story"
- "chapter"
- "tale"
- "history"
- "write"
- "novel"
- "[Mode: Writing]"
tokenizer_source: union
``` |