Update README.md

README.md

@@ -11,50 +11,58 @@ datasets:
 
 ## Model Description
 
-LLaMA-2-7B-32K-Chat is an open-source, long-context chat model finetuned from [Llama-2-7B-32K](https://huggingface.co/togethercomputer/LLaMA-2-7B-32K), over high-quality instructions and chat data.
+LLaMA-2-7B-32K-Chat is an open-source, long-context chat model finetuned from [LLaMA-2-7B-32K](https://huggingface.co/togethercomputer/LLaMA-2-7B-32K), over high-quality instructions and chat data.
 We build Llama-2-7B-32K-Chat with less than 200 lines of Python script using [Together API](https://together.ai/blog/api-announcement), and we also make the recipe fully available. 
-We hope that this can enable everyone to finetune their own version of [Llama-2-7B-32K](https://huggingface.co/togethercomputer/LLaMA-2-7B-32K) — play with [Together API](https://together.ai/blog/api-announcement) and give us feedback! 
+We hope that this can enable everyone to finetune their own version of [LLaMA-2-7B-32K](https://huggingface.co/togethercomputer/LLaMA-2-7B-32K) — play with [Together API](https://together.ai/blog/api-announcement) and give us feedback! 
 
-Llama-2-7B-32K-Chat is fine-tuned over 19K single- and multi-round conversations generated by human instructions and Llama-2-70B-Chat outputs,
-The dataset is also released [here](https://huggingface.co/datasets/togethercomputer/llama-instruct).
 
-## Inference
+## Data Collection
+
+LLaMA-2-7B-32K-Chat is fine-tuned over two datasets: (1) 19K single- and multi-round conversations generated by human instructions and 
+Llama-2-70B-Chat outputs, along with (2) 4K instructions of summarization from the BookSum datasets. 
+We collected dataset (1) following the distillation paradigm that is used by Alpaca, Vicuna, WizardLM, Orca — producing instructions by querying a powerful LLM.
+We release such dataset [here](https://huggingface.co/datasets/togethercomputer/llama-instruct). For more details, please refer to our [Github repo](https://github.com/togethercomputer/LLaMA-2-32K-Chat).
+
+
+## Model Usage
 
 You can use the [Together API](https://together.ai/blog/api-announcement) to try out LLaMA-2-7B-32K-Chat for inference. 
 The updated inference stack allows for efficient inference.
+The model will also be hosted on [Together Playground](https://api.together.xyz/playground). You can simply play with the model by using prompt formatted by 
 
-To run the model locally, we strongly recommend to install Flash Attention V2, which is necessary to obtain the best performance:
 ```
-# Please update the path of `CUDA_HOME`
-export CUDA_HOME=/usr/local/cuda-11.8
-pip install transformers==4.31.0
-pip install sentencepiece
-pip install ninja
-pip install flash-attn --no-build-isolation
-pip install git+https://github.com/HazyResearch/flash-attention.git#subdirectory=csrc/rotary
+[INST] <your instruction here> [\INST].
 ```
 
-You can use this model directly from the Hugging Face Model Hub or fine-tune it on your own data using the OpenChatKit.
-
-```python
-from transformers import AutoTokenizer, AutoModelForCausalLM
+For example, if we query the model with 
 
-tokenizer = AutoTokenizer.from_pretrained("togethercomputer/LLaMA-2-7B-32K")
-model = AutoModelForCausalLM.from_pretrained("togethercomputer/LLaMA-2-7B-32K", trust_remote_code=True, torch_dtype=torch.float16)
-
-input_context = "Your text here"
-input_ids = tokenizer.encode(input_context, return_tensors="pt")
-output = model.generate(input_ids, max_length=128, temperature=0.7)
-output_text = tokenizer.decode(output[0], skip_special_tokens=True)
-print(output_text)
+```
+[INST] Write a poem about cats [\INST],
 ```
 
-Alternatively, you can set `trust_remote_code=False` if you prefer not to use flash attention.
-
-To chat with the model, the prompt is in the format of 
-
+the model will return
 ```
-[INST] Write a song about elepants [\INST]
+Sure, here's a short poem about cats:
+
+Cats, cats, oh how they enthrall,
+With their soft fur and gentle grace,
+They purr with joy, their eyes aglow,
+And steal our hearts with their charm and grace.
+
+In their own time, they move about,
+With graceful strides, they tiptoe out,
+Their whiskers twitching, as they go,
+Their curiosity piqued, they want to know.
+
+They climb, they jump, they play, they chase,
+Their agility, a sight to behold,
+Their naps, a treat, a break from the chase,
+A time to rest, a time to be told.
+
+Cats, cats, they're unique and special,
+A part of our lives, a treasure to hold,
+Their personalities, so diverse,
+Their charm, a gift, that's forever told.
 ```
 
 ## Limitations and Bias