Update README.md

<!-- coding=utf-8
Copyright 2024 Jingze Shi, Bingheng Wu and the HuggingFace Inc. team. All rights reserved.

This code is based on the Wonderful Matrices paper implementation.

https://arxiv.org/abs/2407.16958

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. -->

# **Doge 197M**

Doge is an ongoing research project where we aim to train a series of small language models to further explore whether the Transformer framework allows for more complex feedforward network structures, enabling the model to have fewer cache states and larger knowledge capacity.

In addition, Doge uses Inner Function Attention with Dynamic Mask as sequence transformation and Cross Domain Mixture of Experts as state transformation. This model is trained by Jingze Shi, it only allows text input and text generation, for detailed algorithm and model architecture, please refer to [Wonderful Matrices](https://arxiv.org/abs/2407.16958), the ongoing research repository is [Doge](https://github.com/LoserCheems/Doge).

# Fine-tue
We selected an open-source Chinese medical question answering dataset for fine-tuning.

# Uses
```python
from transformers import AutoTokenizer, AutoModelForCausalLM,GenerationConfig, TextStreamer

tokenizer = AutoTokenizer.from_pretrained("wubingheng/Doge_medical_chat-197M")
model = AutoModelForCausalLM.from_pretrained("wubingheng/Doge_medical_chat-197M", trust_remote_code=True)

generation_config = GenerationConfig(
max_new_tokens=256,
min_new_tokens=1,
num_beams=1,
eos_token_id=[tokenizer.eos_token_id],
stop_strings=[tokenizer.eos_token],
early_stopping=False,
use_cache=True,
do_sample=True,
temperature=0.8,
repetition_penalty=1.0,
)
steamer = TextStreamer(tokenizer=tokenizer, skip_prompt=True)

system_prompt = """
你是医疗助手Doge, 按照用户的问题回复帮助性的答案.

以下是你可以参考的文档:

昨天发烧38度多今天又35度5是怎么回事啊是不是低烧啊还要继续喝退烧药吗

您好，如果您昨天的体温确实达到了38度多，那么这个体温是算作发烧的。今天的体温虽然比昨天低，但是也属于正常体温范围内。这种情况可能是由于身体抵抗力增强，病情好转所致。如果您没有其他不适症状，建议您可以暂停使用退烧药，注意休息，多喝水，适当增加营养，保持良好的作息习惯。如果您出现其他不适症状或体温再次升高，建议您及时就医

""".strip()

prompt = "我现在发烧了该怎么办？"
conversation = [
{"role": "system", "content": system_prompt},
{"role": "user", "content": prompt},
]
inputs = tokenizer.apply_chat_template(
conversation=conversation,
tokenize=True,
return_tensors="pt",
)
print(prompt)
print()
output = model.generate(
inputs,
tokenizer=tokenizer,
generation_config=generation_config,
streamer=steamer
)
## Citation

```bibtex


@misc
{shi2024wonderfulmatrices,
title={Wonderful Matrices: More Efficient and Effective Architecture for Language Modeling Tasks},
author={Jingze Shi and Bingheng Wu and Lu He and Luchang Jiang},
year={2024},
eprint={2407.16958},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2407.16958},
}
```

README.md

@@ -1,5 +1,13 @@
----
-license: other
-license_name: doge
-license_link: LICENSE
----
+---
+license: apache-2.0
+license_name: doge
+license_link: LICENSE
+datasets:
+- michaelwzhu/ChatMed_Consult_Dataset
+metrics:
+- code_eval
+base_model:
+- JingzeShi/Doge-76M
+pipeline_tag: text-generation
+library_name: transformers
+---