---
license: apache-2.0
language:
- ja
tags:
- mistral
- mixtral
- not-for-all-audiences
- nsfw
base_model:
- NTQAI/chatntq-ja-7b-v1.0
- stabilityai/japanese-stablelm-instruct-gamma-7b
- Elizezen/Antler-7B
- Elizezen/Hameln-japanese-mistral-7B
---

# LightChatAssistant-4x7B
[GGUF版はこちら/Click here for the GGUF version](https://huggingface.co/Aratako/LightChatAssistant-4x7B-GGUF)

## 概要
[Sdff-Ltba/LightChatAssistant-2x7B](https://huggingface.co/Sdff-Ltba/LightChatAssistant-2x7B)と全く同じ手法を用いつつ、対象を4モデルに拡張してそれらをMoEしたモデルです。素晴らしい手法を提案いただいた[@Sdff-Ltba](https://huggingface.co/Sdff-Ltba)さんに感謝します。

モデルサイズは大きくなっていますが、推論に同時に使われるエキスパートは2つなので、GPUにオフロードさえできれば元の[Sdff-Ltba/LightChatAssistant-2x7B](https://huggingface.co/Sdff-Ltba/LightChatAssistant-2x7B)とほぼ同じような速度で動きます。


以下の4モデルをChat Vectorの加算によって対話強化して素材に使用しています。
- [NTQAI/chatntq-ja-7b-v1.0](https://huggingface.co/NTQAI/chatntq-ja-7b-v1.0)
- [stabilityai/japanese-stablelm-instruct-gamma-7b](https://huggingface.co/stabilityai/japanese-stablelm-instruct-gamma-7b)
- [Elizezen/Antler-7B](https://huggingface.co/Elizezen/Antler-7B)
- [Elizezen/Hameln-japanese-mistral-7B](https://huggingface.co/Elizezen/Hameln-japanese-mistral-7B)

Chat Vectorの適用方法は[Sdff-Ltba/LightChatAssistant-2x7B](https://huggingface.co/Sdff-Ltba/LightChatAssistant-2x7B)と同じで、Mistral-7B-Instruct-v0.2とMistral-7B-v0.1の差分を0.8倍してそれぞれのモデルに足しています。

その後、[mergekit](https://github.com/cg123/mergekit)を使い、4モデルをMoEしました。MoEのconfigは以下の通りです。

```yaml
base_model: ./chatntq-chatvector
gate_mode: hidden
dtype: bfloat16
experts:
  - source_model: ./chatntq-chatvector
    positive_prompts:
    - "roleplay"
    - "question"
    - "answer"
    - "chat"
    - "companion"
    - "character"
    - "math"
    - "code"
    - "[Mode: Roleplay]"
    - "[Mode: Chat]"
    negative_prompts:
    - "storywriting"
    - "book"
    - "story"
    - "chapter"
    - "tale"
    - "history"
    - "write"
    - "novel"
    - "[Mode: Writing]"
  - source_model: ./japanese-stablelm-instruct-gamma-chatvector
    positive_prompts:
    - "roleplay"
    - "question"
    - "answer"
    - "chat"
    - "companion"
    - "character"
    - "math"
    - "code"
    - "[Mode: Roleplay]"
    - "[Mode: Chat]"
    negative_prompts:
    - "storywriting"
    - "book"
    - "story"
    - "chapter"
    - "tale"
    - "history"
    - "write"
    - "novel"
    - "[Mode: Writing]"
  - source_model: ./Hameln-japanese-mistral-chatvector
    positive_prompts:
    - "sex"
    - "storywriting"
    - "erotic"
    - "fuck"
    - "orgasm"
    - "uncensored"
    - "book"
    - "story"
    - "chapter"
    - "tale"
    - "history"
    - "write"
    - "novel"
    - "[Mode: Writing]"
  - source_model: ./Antler-chatvector
    positive_prompts:
    - "sex"
    - "storywriting"
    - "erotic"
    - "fuck"
    - "orgasm"
    - "uncensored"
    - "book"
    - "story"
    - "chapter"
    - "tale"
    - "history"
    - "write"
    - "novel"
    - "[Mode: Writing]"
tokenizer_source: union
```