28/04/2024- UPDATE:
Fixed tokenizer / vocab issues. Verified operation, conversion to GGUF now works too. GGUF uploaded, with Imatrix Plus GGUFs to follow shortly.
Imatrix Plus GGUFs are here
This includes all Imatrix compressions as well as regular "Qs" which have also been "Imatrixed" too.
"Imatrix Plus" is an upgraded form of Imatrix which using full precision for specific parts of the compression.
This results in a higher quality model, especially at lower compressions. This method is applied across all compressions from IQ1 to Q8.
This merge was an experiment to test already established Roleplay, Fiction and Story generation of "Tiefighter" with a some of "Orca 2"'s qualities.
A blank or standard Alpaca Template for text generation will work. Currently "CHATML" is untested.
Context length: 4096.
This is a merge of pre-trained language models created using mergekit.
This model was merged using the SLERP merge method.
The following models were included in the merge:
The following YAML configuration was used to produce this model:
slices:
- sources:
- model: KoboldAI/LLaMA2-13B-Tiefighter
layer_range: [0, 40]
- model: microsoft/Orca-2-13b
layer_range: [0, 40]
merge_method: slerp
base_model: microsoft/Orca-2-13b
parameters:
t:
- filter: self_attn
value: [0, 0.5, 0.3, 0.7, 1]
- filter: mlp
value: [1, 0.5, 0.7, 0.3, 0]
- value: 0.5
dtype: bfloat16