Update README.md
Browse files
README.md
CHANGED
|
@@ -3,6 +3,10 @@ license: apache-2.0
|
|
| 3 |
inference: false
|
| 4 |
---
|
| 5 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
# MegaBeam-Mistral-7B-300k Model
|
| 7 |
|
| 8 |
MegaBeam-Mistral-7B-300k is a fine-tuned [Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) language model that supports input contexts up to 320k tokens. MegaBeam-Mistral-7B-300k can be deployed on a single AWS `g5.48xlarge` instance using serving frameworks such as [vLLM](https://github.com/vllm-project/vllm), Sagemaker [DJL](https://docs.aws.amazon.com/sagemaker/latest/dg/deploy-models-frameworks-djl-serving.html) endpoint, and others. Similarities and differences beween MegaBeam-Mistral-7B-300k and [Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) are summarized below:
|
|
|
|
| 3 |
inference: false
|
| 4 |
---
|
| 5 |
|
| 6 |
+
# Mistral-7b-300k-gguf models
|
| 7 |
+
|
| 8 |
+
Since only two formats are useful, I have converted model into those formats only.
|
| 9 |
+
|
| 10 |
# MegaBeam-Mistral-7B-300k Model
|
| 11 |
|
| 12 |
MegaBeam-Mistral-7B-300k is a fine-tuned [Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) language model that supports input contexts up to 320k tokens. MegaBeam-Mistral-7B-300k can be deployed on a single AWS `g5.48xlarge` instance using serving frameworks such as [vLLM](https://github.com/vllm-project/vllm), Sagemaker [DJL](https://docs.aws.amazon.com/sagemaker/latest/dg/deploy-models-frameworks-djl-serving.html) endpoint, and others. Similarities and differences beween MegaBeam-Mistral-7B-300k and [Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) are summarized below:
|