Update README.md

README.md

@@ -38,39 +38,21 @@ pipeline_tag: text-generation
    <img src="https://styles.redditmedia.com/t5_6075m3/styles/profileIcon_71syco7c5lt81.png?width=256&height=256&frame=1&auto=webp&crop=256:256,smart&s=24bd3c71dc11edc5d4f88d0cbc1da72ed7ae1969" alt="RunPod Logo" style="width:30px; vertical-align: middle; display: inline-block; margin-right: 5px; margin-left: 5px; margin-top: 0px; margin-bottom: 0px;"/>
 </p>
 
-<div style="background-color: white; padding: 0.7em; border-radius: 0.5em; color: black; display: flex; flex-direction: column; justify-content: center; text-align: center; ont-size: 0.5em; border: 0.8em solid #864AF9;">
+<div style="background-color: white; padding: 0.7em; border-radius: 0.5em; color: black; display: flex; flex-direction: column; justify-content: center; text-align: center">
   <a href="https://huggingface.co/openchat/openchat-3.5-0106" style="text-decoration: none; color: black;">
-    <span style="font-size: 1.7em; font-family: 'Helvetica'; letter-spacing: 0.1em; font-weight: bold; color: black;">OPENCHAT</span><span style="font-size: 1.8em; font-family: 'Helvetica'; color: #3c72db; ">3.5</span>
-        <span style="font-size: 1.0em;  font-family: 'Helvetica'; color:  white; background-color: #864AF9; vertical-align: top; border-radius: 6em; padding: 0.066em 0.4em; letter-spacing: 0.1em; font-weight: bold;">0106</span>
+    <span style="font-size: 1.7em; font-family: 'Helvetica'; letter-spacing: 0.1em; font-weight: bold; color: black;">Llama 3 Version: OPENCHAT</span><span style="font-size: 1.8em; font-family: 'Helvetica'; color: #3c72db; ">3.6</span>
+        <span style="font-size: 1.0em;  font-family: 'Helvetica'; color:  white; background-color: #03045e; vertical-align: top; border-radius: 6em; padding: 0.066em 0.4em; letter-spacing: 0.1em; font-weight: bold;">20240522</span>
     <span style="font-size: 0.85em; font-family: 'Helvetica'; color: black;">
-      <br> 🏆 The Overall Best Performing Open Source 7B Model 🏆
-    <br> 🤖 Outperforms <span style="font-weight: bold;">ChatGPT</span> (March) and <span style="font-weight: bold;">Grok-1</span> 🤖
-      <br> 🚀<span style="font-size: 1em; font-family: 'Helvetica'; color: black; font-weight: bold;">15</span>-point improvement in Coding over <span style="font-size: 0.9em;
-      font-family: 'Helvetica'; color: black; font-weight: bold;">OpenChat-3.5🚀</span>
-      <br><br><span style="font-size: 1em; font-family: 'Helvetica'; color: #3c72db; font-weight: bold;">New Features</span>
-      <br> 💡 2 Modes: Coding + Generalist, Mathematical Reasoning 💡
-      <br> 🧑‍⚖️ Experimental support for Evaluator and Feedback capabilities 🧑‍⚖️
+      <br> 🏆 The Overall Best Performing Open Source 8B Model 🏆
+      <br> 🚀 Outperforms Llama-3-8B-Instruct and open-source finetunes 🚀
     </span>
   </a>
 </div>
 
 <div style="display: flex; justify-content: center; align-items: center">
-  <img src="https://raw.githubusercontent.com/imoneoi/openchat/master/assets/openchat-bench-0106.png" style="width: 100%; border-radius: 1em">
+  <img src="" style="width: 100%; border-radius: 1em">
 </div>
 
-
-<div>
-<h3> Table of Contents</h3>
-</div>
-
-1. [Usage](#usage)
-2. [Benchmarks](#benchmarks)
-3. [Limitations](#limitations)
-4. [License](#license)
-6. [Citation](#citation)
-7. [Acknowledgements](#acknowledgements)
-
-
 <div align="center">
 <h2> Usage </h2>
 </div>
@@ -81,56 +63,35 @@ Once started, the server listens at `localhost:18888` for requests and is compat
 
 If you want to deploy the server as an online service, you can use `--api-keys sk-KEY1 sk-KEY2 ...` to specify allowed API keys and `--disable-log-requests --disable-log-stats --log-file openchat.log` for logging only to a file. For security purposes, we recommend using an [HTTPS gateway](https://fastapi.tiangolo.com/es/deployment/concepts/#security-https) in front of the server.
 
-| Model             | Size | Context | Weights                                                          | Serving                                                                                                          |
-|-------------------|------|---------|------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------|
-| OpenChat-3.5-0106 | 7B   | 8192    | [Huggingface](https://huggingface.co/openchat/openchat-3.5-0106) | `python -m ochat.serving.openai_api_server --model openchat/openchat-3.5-0106 --engine-use-ray --worker-use-ray` |
+| Model                 | Size | Context | Weights                                                                 | Serving                                                                               |
+|-----------------------|------|---------|-------------------------------------------------------------------------|---------------------------------------------------------------------------------------|
+| OpenChat-3.6-20240522 | 8B   | 8192    | [Huggingface](https://huggingface.co/openchat/openchat-3.6-8b-20240522) | `python -m ochat.serving.openai_api_server --model openchat/openchat-3.6-8b-20240522` |
 
 <details>
   <summary>Example request (click to expand)</summary>
 
-💡 **Default Mode (GPT4 Correct)**: Best for coding, chat and general tasks
-
 ```bash
 curl http://localhost:18888/v1/chat/completions \
   -H "Content-Type: application/json" \
   -d '{
-    "model": "openchat_3.5",
+    "model": "openchat_3.6",
     "messages": [{"role": "user", "content": "You are a large language model named OpenChat. Write a poem to describe yourself"}]
   }'
 ```
 
-🧮 **Mathematical Reasoning Mode**: Tailored for solving math problems
-
-```bash
-curl http://localhost:18888/v1/chat/completions \
-  -H "Content-Type: application/json" \
-  -d '{
-    "model": "openchat_3.5",
-    "condition": "Math Correct",
-    "messages": [{"role": "user", "content": "10.3 − 7988.8133 = "}]
-  }'
-```
-
 </details>
 
 ### Conversation templates
 
-💡 **Default Mode (GPT4 Correct)**: Best for coding, chat and general tasks
+💡 **Default Mode**: Best for coding, chat and general tasks
 
 ```
 GPT4 Correct User: Hello<|end_of_turn|>GPT4 Correct Assistant: Hi<|end_of_turn|>GPT4 Correct User: How are you today?<|end_of_turn|>GPT4 Correct Assistant:
 ```
 
-🧮 **Mathematical Reasoning Mode**: Tailored for solving math problems
-
-```
-Math Correct User: 10.3 − 7988.8133=<|end_of_turn|>Math Correct Assistant:
-```
-
 ⚠️ **Notice:** Remember to set `<|end_of_turn|>` as end of generation token.
 
-The default (GPT4 Correct) template is also available as the integrated `tokenizer.chat_template`,
-which can be used instead of manually specifying the template:
+The default template is also available as the integrated `tokenizer.chat_template`, which can be used instead of manually specifying the template:
 
 ```python
 messages = [
@@ -139,98 +100,7 @@ messages = [
     {"role": "user", "content": "How are you today?"}
 ]
 tokens = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
-assert tokens == [1, 420, 6316, 28781, 3198, 3123, 1247, 28747, 22557, 32000, 420, 6316, 28781, 3198, 3123, 21631, 28747, 15359, 32000, 420, 6316, 28781, 3198, 3123, 1247, 28747, 1602, 460, 368, 3154, 28804, 32000, 420, 6316, 28781, 3198, 3123, 21631, 28747]
-```
-
-<div align="center">
-<h2> (Experimental) Evaluator / Feedback Capabilities </h2>
-</div>
-
-We've included evaluator capabilities in this release to advance open-source models as evaluators. You can use `Default Mode (GPT4 Correct)` with the following prompt (same as [Prometheus](https://huggingface.co/datasets/kaist-ai/Feedback-Collection)) to evaluate a response.
-
 ```
-###Task Description:
-An instruction (might include an Input inside it), a response to evaluate, a reference answer that gets a score of 5, and a score rubric representing a evaluation criteria are given.
-1. Write a detailed feedback that assess the quality of the response strictly based on the given score rubric, not evaluating in general.
-2. After writing a feedback, write a score that is an integer between 1 and 5. You should refer to the score rubric.
-3. The output format should look as follows: "Feedback: (write a feedback for criteria) [RESULT] (an integer number between 1 and 5)"
-4. Please do not generate any other opening, closing, and explanations.
-
-###The instruction to evaluate:
-{orig_instruction}
-
-###Response to evaluate:
-{orig_response}
-
-###Reference Answer (Score 5):
-{orig_reference_answer}
-
-###Score Rubrics:
-[{orig_criteria}]
-Score 1: {orig_score1_description}
-Score 2: {orig_score2_description}
-Score 3: {orig_score3_description}
-Score 4: {orig_score4_description}
-Score 5: {orig_score5_description}
-
-###Feedback: 
-```
-<div align="center">
-<h2> Benchmarks </h2>
-</div>
-
-| Model                 | # Params | Average  | MT-Bench | HumanEval | BBH MC   | AGIEval  | TruthfulQA | MMLU     | GSM8K    | BBH CoT  |
-|-----------------------|----------|----------|----------|-----------|----------|----------|------------|----------|----------|----------|
-| **OpenChat-3.5-0106** | **7B**   | **64.5** | 7.8      | **71.3**  | **51.5** | **49.1** | 61.0       | 65.8     | **77.4** | 62.2     |
-| OpenChat-3.5-1210     | **7B**   | 63.8     | 7.76     | 68.9      | 49.5     | 48.0     | **61.8**   | 65.3     | 77.3     | 61.8     |
-| OpenChat-3.5          | **7B**   | 61.6     | 7.81     | 55.5      | 47.6     | 47.4     | 59.1       | 64.3     | 77.3     | 63.5     |
-| ChatGPT (March)*      | ???B     | 61.5     | **7.94** | 48.1      | 47.6     | 47.1     | 57.7       | **67.3** | 74.9     | **70.1** |
-|                       |          |          |          |           |          |          |            |          |          |          |
-| OpenHermes 2.5        | 7B       | 59.3     | 7.54     | 48.2      | 49.4     | 46.5     | 57.5       | 63.8     | 73.5     | 59.9     |
-| OpenOrca Mistral      | 7B       | 52.7     | 6.86     | 38.4      | 49.4     | 42.9     | 45.9       | 59.3     | 59.1     | 58.1     |
-| Zephyr-β^             | 7B       | 34.6     | 7.34     | 22.0      | 40.6     | 39.0     | 40.8       | 39.8     | 5.1      | 16.0     |
-| Mistral               | 7B       | -        | 6.84     | 30.5      | 39.0     | 38.0     | -          | 60.1     | 52.2     | -        |
-
-<details>
-  <summary>Evaluation Details(click to expand)</summary>
-  
-*: ChatGPT (March) results are from [GPT-4 Technical Report](https://arxiv.org/abs/2303.08774), [Chain-of-Thought Hub](https://github.com/FranxYao/chain-of-thought-hub), and our evaluation. Please note that ChatGPT is not a fixed baseline and evolves rapidly over time.
-
-^: Zephyr-β often fails to follow few-shot CoT instructions, likely because it was aligned with only chat data but not trained on few-shot data.
-
-**: Mistral and Open-source SOTA results are taken from reported results in instruction-tuned model papers and official repositories.
-
-All models are evaluated in chat mode (e.g. with the respective conversation template applied). All zero-shot benchmarks follow the same setting as in the AGIEval paper and Orca paper. CoT tasks use the same configuration as Chain-of-Thought Hub, HumanEval is evaluated with EvalPlus, and MT-bench is run using FastChat. To reproduce our results, follow the instructions in [our repository](https://github.com/imoneoi/openchat/#benchmarks).
-
-
-</details>
-<div>
-<h3>HumanEval+</h3>
-</div>
-
-| Model                       | Size   | HumanEval+ pass@1 |
-|-----------------------------|--------|-------------------|
-| **OpenChat-3.5-0106**       | **7B** | **65.9**          |
-| ChatGPT (December 12, 2023) | ???B   | 64.6              |
-| WizardCoder-Python-34B-V1.0 | 34B    | 64.6              |
-| OpenChat 3.5 1210           | 7B     | 63.4              |
-| OpenHermes 2.5              | 7B     | 41.5              |
-
-<div>
-<h3>OpenChat-3.5 vs. Grok</h3>
-</div>
-
-🔥 OpenChat-3.5-0106 (7B) now outperforms Grok-0 (33B) on **all 4 benchmarks** and Grok-1 (???B) on average and **3/4 benchmarks**.
-
-|                       | License     | # Param | Average  | MMLU   | HumanEval | MATH     | GSM8k    |
-|-----------------------|-------------|---------|----------|--------|-----------|----------|----------|
-| **OpenChat-3.5-0106** | Apache-2.0  | **7B**  | **61.0** | 65.8   | **71.3**  | **29.3** | **77.4** |
-| OpenChat-3.5-1210     | Apache-2.0  | **7B**  | 60.1     | 65.3   | 68.9      | 28.9     | 77.3     |
-| OpenChat-3.5          | Apache-2.0  | **7B**  | 56.4     | 64.3   | 55.5      | 28.6     | 77.3     |
-| Grok-0                | Proprietary | 33B     | 44.5     | 65.7   | 39.7      | 15.7     | 56.8     |
-| Grok-1                | Proprietary | ???B    | 55.8     | **73** | 63.2      | 23.9     | 62.9     |
-
-*: Grok results are reported by [X.AI](https://x.ai/).
 
 <div align="center">
 <h2> Limitations </h2>
@@ -250,10 +120,14 @@ OpenChat may sometimes generate information that does not exist or is not accura
 OpenChat may sometimes generate harmful, hate speech, biased responses, or answer unsafe questions. It's crucial to apply additional AI safety measures in use cases that require safe and moderated responses.
 
 <div align="center">
-<h2> License </h2>
+<h2> 💌 Contact </h2>
 </div>
 
-Our OpenChat 3.5 code and models are distributed under the Apache License 2.0.
+We look forward to hearing you and collaborating on this exciting project!
+
+**Project Lead:**
+- Guan Wang [imonenext at gmail dot com]
+- [Alpay Ariyak](https://github.com/alpayariyak) [aariyak at wpi dot edu]
 
 <div align="center">
 <h2> Citation </h2>
@@ -266,14 +140,4 @@ Our OpenChat 3.5 code and models are distributed under the Apache License 2.0.
   journal={arXiv preprint arXiv:2309.11235},
   year={2023}
 }
-```
-
-<div align="center">
-<h2> 💌 Contact </h2>
-</div>
-
-We look forward to hearing you and collaborating on this exciting project!
-
-**Project Lead:**
-- Guan Wang [imonenext at gmail dot com]
-- [Alpay Ariyak](https://github.com/alpayariyak) [aariyak at wpi dot edu]
+```