sarath-shekkizhar
commited on
Commit
•
75d20ad
1
Parent(s):
602f82f
adding model card
Browse files
README.md
CHANGED
@@ -1,8 +1,6 @@
|
|
1 |
---
|
2 |
-
model_type: Fine-tuned 7B model for chat.
|
3 |
license: {apache-2.0}
|
4 |
base_model: {openchat/openchat_3.5}
|
5 |
-
demo: [Hugging Face Spaces](https://huggingface.co/spaces/tenyx/TenyxChat-7B-v1)
|
6 |
---
|
7 |
|
8 |
# TenyxChat: Language Model Alignment using Tenyx Fine-tuning
|
@@ -11,6 +9,12 @@ Introducing TenyxChat, a series of ChatGPT-like models trained to function as us
|
|
11 |
|
12 |
We fine-tune [Openchat-3.5](https://arxiv.org/pdf/2309.11235.pdf) with our proprietary approach ([blog](https://www.tenyx.com/post/forgetting-and-toxicity-in-llms-a-deep-dive-on-fine-tuning-methods), [service](https://www.tenyx.com/fine-tuning)), which shows an increase in [MT-Bench](https://arxiv.org/abs/2306.05685), without a drop in performance of the model on other benchmarks. Our approach aims to mitigate forgetting in LLMs in a computationally efficient manner, thereby enabling continual fine-tuning capabilities without altering the pre-trained output distribution. TenyxChat-7B-v1 was trained using eight A100s (80GB) for two hours, with a training setup obtained from HuggingFaceH4 ([GitHub](https://github.com/huggingface/alignment-handbook)).
|
13 |
|
|
|
|
|
|
|
|
|
|
|
|
|
14 |
|
15 |
## Usage
|
16 |
|
|
|
1 |
---
|
|
|
2 |
license: {apache-2.0}
|
3 |
base_model: {openchat/openchat_3.5}
|
|
|
4 |
---
|
5 |
|
6 |
# TenyxChat: Language Model Alignment using Tenyx Fine-tuning
|
|
|
9 |
|
10 |
We fine-tune [Openchat-3.5](https://arxiv.org/pdf/2309.11235.pdf) with our proprietary approach ([blog](https://www.tenyx.com/post/forgetting-and-toxicity-in-llms-a-deep-dive-on-fine-tuning-methods), [service](https://www.tenyx.com/fine-tuning)), which shows an increase in [MT-Bench](https://arxiv.org/abs/2306.05685), without a drop in performance of the model on other benchmarks. Our approach aims to mitigate forgetting in LLMs in a computationally efficient manner, thereby enabling continual fine-tuning capabilities without altering the pre-trained output distribution. TenyxChat-7B-v1 was trained using eight A100s (80GB) for two hours, with a training setup obtained from HuggingFaceH4 ([GitHub](https://github.com/huggingface/alignment-handbook)).
|
11 |
|
12 |
+
# Model details
|
13 |
+
|
14 |
+
- Model type: Fine-tuned 7B model for chat.
|
15 |
+
- License: Apache 2.0
|
16 |
+
- Base model: OpenChat 3.5 ([https://huggingface.co/openchat/openchat_3.5](https://huggingface.co/openchat/openchat_3.5))
|
17 |
+
- Demo: Hugging face space
|
18 |
|
19 |
## Usage
|
20 |
|