rudrashah commited on
Commit
98935f8
1 Parent(s): 09b0691

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -22
README.md CHANGED
@@ -3,35 +3,15 @@ license: apache-2.0
3
  tags:
4
  - merge
5
  - mergekit
6
- - lazymergekit
7
  - EmbeddedLLM/Mistral-7B-Merge-14-v0.1
8
  - OpenPipe/mistral-ft-optimized-1218
9
  ---
10
 
11
  # RLM-mini
12
 
13
- RLM-mini is a merge of the following models using [mergekit](https://github.com/cg123/mergekit):
14
  * [EmbeddedLLM/Mistral-7B-Merge-14-v0.1](https://huggingface.co/EmbeddedLLM/Mistral-7B-Merge-14-v0.1)
15
  * [OpenPipe/mistral-ft-optimized-1218](https://huggingface.co/OpenPipe/mistral-ft-optimized-1218)
16
 
17
- ## 🧩 Configuration
18
 
19
- ```yaml
20
- slices:
21
- - sources:
22
- - model: EmbeddedLLM/Mistral-7B-Merge-14-v0.1
23
- layer_range: [0, 32]
24
- - model: OpenPipe/mistral-ft-optimized-1218
25
- layer_range: [0, 32]
26
- merge_method: slerp
27
- base_model: EmbeddedLLM/Mistral-7B-Merge-14-v0.1
28
- parameters:
29
- t:
30
- - filter: self_attn
31
- value: [0, 0.5, 0.3, 0.7, 1]
32
- - filter: mlp
33
- value: [1, 0.5, 0.7, 0.3, 0]
34
- - value: 0.5
35
- dtype: bfloat16
36
-
37
- ```
 
3
  tags:
4
  - merge
5
  - mergekit
 
6
  - EmbeddedLLM/Mistral-7B-Merge-14-v0.1
7
  - OpenPipe/mistral-ft-optimized-1218
8
  ---
9
 
10
  # RLM-mini
11
 
12
+ RLM-mini is a 7.2 Billion parameter model, created by merge and fine tune of Two models,
13
  * [EmbeddedLLM/Mistral-7B-Merge-14-v0.1](https://huggingface.co/EmbeddedLLM/Mistral-7B-Merge-14-v0.1)
14
  * [OpenPipe/mistral-ft-optimized-1218](https://huggingface.co/OpenPipe/mistral-ft-optimized-1218)
15
 
16
+ # Usage
17