ArkaAbacus commited on
Commit
540ef8d
1 Parent(s): 54a8c35

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -3
README.md CHANGED
@@ -8,9 +8,16 @@ base_model: moreh/MoMo-72B-lora-1.8.7-DPO
8
 
9
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/64c14f6b02e1f8f67c73bd05/e4u8VYfDBh11u60rFYJHF.png)
10
 
11
- This model is a finetune of moreh's [MoMo-72B](https://huggingface.co/moreh/MoMo-72B-lora-1.8.7-DPO) model.
12
- It has been trained with new datasets and a new technique, which we will share to the community soon.
13
- This model has not utilised any form of merging.
 
 
 
 
 
 
 
14
 
15
  ### Evaluation Results
16
 
 
8
 
9
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/64c14f6b02e1f8f67c73bd05/e4u8VYfDBh11u60rFYJHF.png)
10
 
11
+ Smaug arrives!
12
+
13
+ We recently released Smaug-72B-v0.1 which has taken first place on the Open LLM Leaderboard by HuggingFace. It is the first open-source model to have an average score more than 80.
14
+
15
+ Smaug-72B is finetuned directly from [moreh/MoMo-72B-lora-1.8.7-DPO](https://huggingface.co/moreh/MoMo-72B-lora-1.8.7-DPO), and is ultimately based on [Qwen-72B](https://huggingface.co/Qwen/Qwen-72B),.
16
+
17
+ To do so we built on techniques and datasets used in our previous model efforts, with the addition of some new datasets and a new approach. We believe this new approach is generally useful in training across a wide range of model types and downstream use cases, and it powers both our Smaug-34B and 72B models.
18
+
19
+ We are currently working on writing up this new technique in the form of a technical report which we aim to release on arXiv soon (we may also be releasing a new member of the Smaug lineup at that time!). We are excited to release the details to the open source community for them to build on and improve Smaug and spawn more dragons to dominate the LLM space. Keep watching this space for our announcements!
20
+
21
 
22
  ### Evaluation Results
23