Update README.md

README.md

@@ -8,9 +8,20 @@ datasets:
 
 # Leeroo Dedidcated Math Expert 🤗
 
-The model is built by applying [Orchestration of Expert](https://arxiv.org/abs/2401.13979) for the math domain. It either generates solutions or, when necessary, utilizes GPT-4 to fill in gaps in its knowledge base.
+The model is built by applying [Orchestration of Expert](https://arxiv.org/abs/2401.13979) for the math domain. The dedicated model either generates solutions or, when necessary, utilizes GPT-4 (or similar performing LLM) to fill in gaps in its knowledge base. Specifically, when given an input, the dedicated model first determines if the input question is solvable with the base model. If solvable, the orchestrator is detached, and token generation is invoked using the base LLM expert. If the problem is difficult and requires a larger model like GPT-4, it produces `<GPT4>` token (i.e., token_id=32000).  
 
-In evaluations using the GSM8k dataset of [OpenLLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard), Leeroo Math 7B model achieved an accuracy of 84.77% in 5-shot setting, positioning it among the top performers in its class and notably surpassing the MetaMath 7B model (its base model), which scores 68.84% on the same dataset. This was accomplished while relying on GPT-4 for responses to half of the questions posed by GSM8k.
+The Orchestrator is trained to estimate the knowledge of the base model, for any given query, then we marge into the base model (MetaMath7b, here).  
+In general for **any domain**, you can build it by:
+- Choose a base LLM expert 🤗
+- Train a domain-specific Orchestrator
+- Merge Orchestrator with the base expert
+  
+<p align="center">
+  <img src="train_orch.png" width="300"/>
+</p>
+
+
+✅ In evaluations using the [GSM8k dataset](https://huggingface.co/datasets/gsm8k) of [OpenLLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard), Leeroo Math 7b model achieves an accuracy of 84.77% in 5-shot setting, positioning it among the top performers in its class and notably surpassing its base model, which scores 68.84% on the same dataset. This was accomplished while relying on GPT-4 for responses to half of the questions posed by GSM8k.
 
 ## Sample Usage
 
@@ -41,6 +52,60 @@ print(decoded[0])
 # <GPT4></s>
 ```
 
+You can also add your OpenAI API to get the full answer when `<GPT4>` token is generated:
+
+```python
+from openai import OpenAI
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained("leeroo/LeerooDedicated-Math-7B-OOE", trust_remote_code=True)
+tokenizer = AutoTokenizer.from_pretrained("leeroo/LeerooDedicated-Math-7B-OOE")
+openai_client = OpenAI(
+    api_key=  "OPENAI_API_KEY",
+    base_url= "https://api.openai.com/v1"
+)
+
+def generate(prompt, tokenizer, model, openai_client, max_new_tokens=100, verbose=True):
+    inputs = tokenizer(prompt, return_tensors="pt")
+    inputs = {k:v.to(model.device) for k,v in inputs.items()}
+    gen_tokens = model.generate( **inputs , max_new_tokens=max_new_tokens, do_sample=False, pad_token_id= tokenizer.pad_token_id)
+    if gen_tokens[0, inputs['input_ids'].shape[1]] != tokenizer.unk_token_id:
+        if verbose: print("\033[94mGenerating using MetaMath7b.\033[0m")
+        gen_text = tokenizer.decode(
+            gen_tokens[0, inputs['input_ids'].shape[1]:].tolist() )
+    else:
+        if verbose: print("\033[94mGenerating using gpt4.\033[0m")
+        gen_text = openai_client.completions.create(
+            model = "gpt-4-1106-preview", # NOTE you can use any bigger mode here having performance similar to gpt4
+            prompt = prompt,
+            max_tokens = max_new_tokens,
+            temperature = 0.0
+        ).choices[0].text
+    return gen_text
+
+# the following question is answered by the leeroo expert
+prompt = "Question: Natalia sold clips to 48 of her friends in April,and then she sold half as many clips in May.How many clips did Natalia sell altogether in April and May?\nAnswer:"
+generation = generate(prompt, tokenizer, model, openai_client, max_new_tokens=500)
+print(generation)
+#> Generating using MetaMath7b.
+# Natalia sold 48 clips in April.\nIn May, she sold half as many clips as in April,
+# so she sold 48/2 = 24 clips.\nAltogether, Natalia sold 48 + 24 = 72 clips in April and May.\n#### 72\nThe answer is: 72</s>
+
+# sends the following question to GPT4
+prompt = "James loves to go swimming and has to swim across a 40-mile lake.  He can swim at a pace of 2 miles per hour.  He swims 60% of the distance.  After that, he stops on an island and rests for half as long as the swimming time.  He then finishes the remaining distance while going half the speed.  How many hours did it take him to get across the lake?"
+generation = generate(prompt, tokenizer, model, openai_client, max_new_tokens=500)
+print(generation)
+#> Generating using gpt4.
+#   He swam 40*.6=24 miles
+# So he swam for 24/2=12 hours
+# He rested for 12/2=6 hours
+# He had 40-24=16 miles left to swim
+# He swam at 2/2=1 mile per hour
+# So he swam for 16/1=16 hours
+# So in total, it took him 12+6+16=34 hours
+# 34
+```
+
 ## Learn More
 
 🔍 To a deeper dive into our method and results, refer to [HF blog 🤗](https://huggingface.co/blog/alirezamsh/leeroo-multi-model-system), [publication](https://arxiv.org/abs/2401.13979), and [repository](https://github.com/Leeroo-AI/leeroo_orchestrator).