Update README.md

README.md

@@ -191,7 +191,41 @@ Covering 12 Major Languages including English, Chinese, French, Hindi, Spanish,
 
 
    </details>
+## Model Download and Inference
+   We take Apollo-MoE-0.5B as an example
+   1. Login Huggingface
+      
+       ```
+       huggingface-cli login --token $HUGGINGFACE_TOKEN
+       ```
+       
+   2. Download model to local dir
+        
+       ```
+       from huggingface_hub import snapshot_download
+       import os
+
+       local_model_dir=os.path.join('/path/to/models/dir','Apollo-MoE-0.5B')
+       snapshot_download(repo_id="FreedomIntelligence/Apollo-MoE-0.5B", local_dir=local_model_dir)
+       ```
+       
+   3. Inference Example
 
+      ```
+      from transformers import AutoTokenizer, AutoModelForCausalLM, GenerationConfig
+      import os
+      
+      local_model_dir=os.path.join('/path/to/models/dir','Apollo-MoE-0.5B')
+      
+      model=AutoModelForCausalLM.from_pretrained(local_model_dir,trust_remote_code=True)
+      tokenizer = AutoTokenizer.from_pretrained(local_model_dir,trust_remote_code=True)
+      generation_config = GenerationConfig.from_pretrained(local_model_dir, pad_token_id=tokenizer.pad_token_id, num_return_sequences=1, max_new_tokens=7, min_new_tokens=2, do_sample=False, temperature=1.0, top_k=50, top_p=1.0)
+      
+      inputs = tokenizer('Answer direclty.\nThe capital of Mongolia is Ulaanbaatar.\nThe capital of Iceland is Reykjavik.\nThe capital of Australia is', return_tensors='pt')
+      inputs = inputs.to(model.device)
+      pred = model.generate(**inputs,generation_config=generation_config)
+      print(tokenizer.decode(pred.cpu()[0], skip_special_tokens=True))
+      ```
 
 ## Results reproduction
    <details><summary>Click to expand</summary>