| Quantization made by Richard Erkhov. | |
| [Github](https://github.com/RichardErkhov) | |
| [Discord](https://discord.gg/pvy7H8DZMG) | |
| [Request more models](https://github.com/RichardErkhov/quant_request) | |
| Mistral-7B-v0.1-sharded - bnb 8bits | |
| - Model creator: https://huggingface.co/alexsherstinsky/ | |
| - Original model: https://huggingface.co/alexsherstinsky/Mistral-7B-v0.1-sharded/ | |
| Original model description: | |
| --- | |
| license: apache-2.0 | |
| pipeline_tag: text-generation | |
| tags: | |
| - pretrained | |
| inference: | |
| parameters: | |
| temperature: 0.7 | |
| --- | |
| # Note: Sharded Version of the Original "Mistral 7B" Model | |
| This is just a version of https://huggingface.co/mistralai/Mistral-7B-v0.1 which is sharded to 2GB maximum parts in order to reduce the RAM required when loading. | |
| # Model Card for Mistral-7B-v0.1 | |
| The Mistral-7B-v0.1 Large Language Model (LLM) is a pretrained generative text model with 7 billion parameters. | |
| Mistral-7B-v0.1 outperforms Llama 2 13B on all benchmarks we tested. | |
| For full details of this model please read our [Release blog post](https://mistral.ai/news/announcing-mistral-7b/) | |
| ## Model Architecture | |
| Mistral-7B-v0.1 is a transformer model, with the following architecture choices: | |
| - Grouped-Query Attention | |
| - Sliding-Window Attention | |
| - Byte-fallback BPE tokenizer | |
| ## Troubleshooting | |
| - If you see the following error: | |
| ``` | |
| Traceback (most recent call last): | |
| File "", line 1, in | |
| File "/transformers/models/auto/auto_factory.py", line 482, in from_pretrained | |
| config, kwargs = AutoConfig.from_pretrained( | |
| File "/transformers/models/auto/configuration_auto.py", line 1022, in from_pretrained | |
| config_class = CONFIG_MAPPING[config_dict["model_type"]] | |
| File "/transformers/models/auto/configuration_auto.py", line 723, in getitem | |
| raise KeyError(key) | |
| KeyError: 'mistral' | |
| ``` | |
| Installing transformers from source should solve the issue: | |
| ``` | |
| pip install git+https://github.com/huggingface/transformers | |
| ``` | |
| This should not be required after transformers-v4.33.4. | |
| ## Notice | |
| Mistral 7B is a pretrained base model and therefore does not have any moderation mechanisms. | |
| ## The Mistral AI Team | |
| Albert Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, William El Sayed. | |