Hugging Face's logo

nbeerbower
/
bruphin-lambda

Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
1
bruphin-lambda
File size: 2,269 Bytes 
2b63d8f
 
 
 
 
 
 
 
 
 
 
276d1c0
2b63d8f
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
 
---
license: apache-2.0
base_model:
- chihoonlee10/T3Q-Mistral-Orca-Math-DPO
- nbeerbower/bruphin-kappa
library_name: transformers
tags:
- mergekit
- merge

---
![image/png](https://cdn-lfs-us-1.huggingface.co/repos/2b/ed/2bed4d865015eea76d66e4abb9ecb64d6eec6bbbb05d183eaadee021dcf35b53/8e4f6b05ab037981807f86da66ec1a54ce2cda0dab6c57addcbffec2b7e3bf75?response-content-disposition=inline%3B+filename*%3DUTF-8%27%27bruphin.png%3B+filename%3D%22bruphin.png%22%3B&response-content-type=image%2Fpng&Expires=1712112316&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTcxMjExMjMxNn19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy11cy0xLmh1Z2dpbmdmYWNlLmNvL3JlcG9zLzJiL2VkLzJiZWQ0ZDg2NTAxNWVlYTc2ZDY2ZTRhYmI5ZWNiNjRkNmVlYzZiYmJiMDVkMTgzZWFhZGVlMDIxZGNmMzViNTMvOGU0ZjZiMDVhYjAzNzk4MTgwN2Y4NmRhNjZlYzFhNTRjZTJjZGEwZGFiNmM1N2FkZGNiZmZlYzJiN2UzYmY3NT9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSomcmVzcG9uc2UtY29udGVudC10eXBlPSoifV19&Signature=g7fhO-yfpN9sqZx--XTySUK%7EeWP48iAs4Ec5tm5%7Es2j78A5-sFA8IqmgFm9Ndv6UGu2tCDWNSJ-qx7lQ-5PadWZn-F%7E420lLr0-KzJ-ERjiAFpNXkFjxpjH4h3AvHqJ4GrL9K85PdCttaq4-KaKNEmtJOVfVSKLAmp4Z2uBpIMvcFyoh1b7l62LR9NeK4cRA9m9fdoEkZR-RxwhXl43gpTOOCpKE0wMgAGDNJdyppDd86s7lReV6eH2qFNGJyNAfvWaO-eMsGc5Jv-AJ9srt6p6-8%7EqqUxfl9pby-2yzbFqFOtxLbPVWWb-v-Uvg6PJ2CIQs2CeouOF0uwgT3W-oXA__&Key-Pair-Id=KCD77M1F0VK2B)
# bruphin-lambda

This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).

## Merge Details
### Merge Method

This model was merged using the SLERP merge method.

### Models Merged

The following models were included in the merge:
* [chihoonlee10/T3Q-Mistral-Orca-Math-DPO](https://huggingface.co/chihoonlee10/T3Q-Mistral-Orca-Math-DPO)
* [nbeerbower/bruphin-kappa](https://huggingface.co/nbeerbower/bruphin-kappa)

### Configuration

The following YAML configuration was used to produce this model:

```yaml
models:
  - model: chihoonlee10/T3Q-Mistral-Orca-Math-DPO
    layer_range: [0, 32]
  - model: nbeerbower/bruphin-kappa
    layer_range: [0, 32]
merge_method: slerp
base_model: nbeerbower/bruphin-kappa
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
dtype: bfloat16

```