Virt-io/Helen-v1_7B

Information

Uses ChatML, but Alpaca seems to work fine.

SillyTavern Presets

This is meant to be a smart RP model.

I did some unholy things to make the model load. (I removed all configs and replaced them with the ones from Hercules)

Important: When using this model it is necessary to include example messages, or it may sound bland. With examples it does pretty well.

Reasoning behind the selected values

To my understanding layers in the middle have a lower effect than later layers on the final output.

So keeping this in mind, I only increase the % of the smart model, at the middle. As to increase its reasoning abilities, but keep it uncensored.

Helen-v1

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the SLERP merge method.

Models Merged

The following models were included in the merge:

• l3utterfly/mistral-7b-v0.2-layla-v4
• Locutusque/Hercules-4.0-Mistral-v0.2-7B
• Weyaxi/Einstein-v5-v0.2-7B

Configuration

The following YAML configuration was used to produce this model:

slices:
  - sources:
      - model: l3utterfly/mistral-7b-v0.2-layla-v4
        layer_range: [0, 32]
      - model: Mergekit/Hercules-Einstein_low
        layer_range: [0, 32]
merge_method: slerp
base_model: l3utterfly/mistral-7b-v0.2-layla-v4
parameters:
  t:
    - filter: self_attn
      value: [0.15, 0.20, 0.35, 0.10, 0.15]
    - filter: mlp
      value: [0.15, 0.20, 0.35, 0.10, 0.15]
    - value: 0.15 # fallback for rest of tensors
dtype: float16