Model Card for Model ID

image

Merged model using mergekit

This model aimed to act like visual novel character.

Merge Format

models:
  - model: mistralai/Mistral-Small-Instruct-2409_SFT
    layer_range: [0, 56]
  - model: mistralai/Mistral-Small-Instruct-2409
    layer_range: [0, 56]
merge_method: slerp
base_model: mistralai/Mistral-Small-Instruct-2409_SFT
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5 # fallback for rest of tensors
dtype: bfloat16

WaifuModel Collections

Unified demo

WaifuAssistant

Update 2.0

  • 2024.09.23 Update 22B, Ver 2.0

Model Details

Model Description

Currently, chatbot has below personality.

character visual_novel
ムラサメ Senren*Banka
茉子 Senren*Banka
芳乃 Senren*Banka
レナ Senren*Banka
千咲 Senren*Banka
芦花 Senren*Banka
愛衣 Café Stella and the Reaper's Butterflies
栞那 Café Stella and the Reaper's Butterflies
ナツメ Café Stella and the Reaper's Butterflies
Café Stella and the Reaper's Butterflies
涼音 Café Stella and the Reaper's Butterflies
あやせ Riddle Joker
七海 Riddle Joker
羽月 Riddle Joker
茉優 Riddle Joker
小春 Riddle Joker

But you can chat your own Character with persona text.

Feel free to test.

Your feedback will be helpful for improving model.

Dataset

Riddle Joker(Prviate)

Café Stella and the Reaper's Butterflies(Private)

Senren*Banka(Private)

roleplay4fun/aesir-v1.1

kalomaze/Opus_Instruct_3k

Gryphe/Sonnet3.5-SlimOrcaDedupCleaned

Aratako/Synthetic-JP-EN-Coding-Dataset-567k (only using 50000 sample)

Aratako/Synthetic-Japanese-Roleplay-gpt-4o-mini-39.6k-formatted

Aratako/Synthetic-Japanese-Roleplay-NSFW-Claude-3.5s-15.3k-formatted

SkunkworksAI/reasoning-0.01

Feature

  • Fluent Chat performance
  • Reduce repetition problem when generate with many turn(over 20~30)
  • Zero Shot character persona using description of character.
  • 128k context window
  • Memory ability that does not forget even after long-context generation

Demo

You can use Demo in google colab.

Check Here

Bias, Risks, and Limitations

This model can generate NSFW content.

Use & Credit

This model is currently available for non-commercial & Research purpose only.

Also, since I'm not detailed in licensing, I hope you use this model responsibly.

By sharing this model, I hope to contribute to the research efforts of our community (the open-source community and Waifu Lovers).

Citation

@misc {ChatWaifu_22B_v2.0
    author       = { YoungWoo Nam },
    title        = { ChatWaifu_22B_v2.0_preview },
    year         = 2024,
    url          = { https://huggingface.co/spow12/ChatWaifu_22B_v2.0_preview },
    publisher    = { Hugging Face }
}

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric Value
Avg. 29.12
IFEval (0-Shot) 67.45
BBH (3-Shot) 45.49
MATH Lvl 5 (4-Shot) 16.31
GPQA (0-shot) 8.72
MuSR (0-shot) 3.53
MMLU-PRO (5-shot) 33.20
Downloads last month
25
Safetensors
Model size
22.2B params
Tensor type
BF16
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for spow12/ChatWaifu_22B_v2.0_preview

Finetuned
(23)
this model
Quantizations
3 models

Datasets used to train spow12/ChatWaifu_22B_v2.0_preview

Collection including spow12/ChatWaifu_22B_v2.0_preview

Evaluation results