|
--- |
|
inference: false |
|
library_name: transformers |
|
language: |
|
- en |
|
- fr |
|
- de |
|
- es |
|
- it |
|
- pt |
|
- ja |
|
- ko |
|
- zh |
|
- ar |
|
- el |
|
- fa |
|
- pl |
|
- id |
|
- cs |
|
- he |
|
- hi |
|
- nl |
|
- ro |
|
- ru |
|
- tr |
|
- uk |
|
- vi |
|
license: cc-by-nc-4.0 |
|
extra_gated_prompt: By submitting this form, you agree to the [License Agreement](https://cohere.com/c4ai-cc-by-nc-license) and |
|
acknowledge that the information you provide will be collected, used, and shared |
|
in accordance with Cohere’s [Privacy Policy]( https://cohere.com/privacy). You’ll |
|
receive email updates about C4AI and Cohere research, events, products and services. |
|
You can unsubscribe at any time. |
|
extra_gated_fields: |
|
Name: text |
|
Affiliation: text |
|
Country: |
|
type: select |
|
options: |
|
- Aruba |
|
- Afghanistan |
|
- Angola |
|
- Anguilla |
|
- Åland Islands |
|
- Albania |
|
- Andorra |
|
- United Arab Emirates |
|
- Argentina |
|
- Armenia |
|
- American Samoa |
|
- Antarctica |
|
- French Southern Territories |
|
- Antigua and Barbuda |
|
- Australia |
|
- Austria |
|
- Azerbaijan |
|
- Burundi |
|
- Belgium |
|
- Benin |
|
- Bonaire Sint Eustatius and Saba |
|
- Burkina Faso |
|
- Bangladesh |
|
- Bulgaria |
|
- Bahrain |
|
- Bahamas |
|
- Bosnia and Herzegovina |
|
- Saint Barthélemy |
|
- Belarus |
|
- Belize |
|
- Bermuda |
|
- Plurinational State of Bolivia |
|
- Brazil |
|
- Barbados |
|
- Brunei-Darussalam |
|
- Bhutan |
|
- Bouvet-Island |
|
- Botswana |
|
- Central African Republic |
|
- Canada |
|
- Cocos (Keeling) Islands |
|
- Switzerland |
|
- Chile |
|
- China |
|
- Côte-dIvoire |
|
- Cameroon |
|
- Democratic Republic of the Congo |
|
- Cook Islands |
|
- Colombia |
|
- Comoros |
|
- Cabo Verde |
|
- Costa Rica |
|
- Cuba |
|
- Curaçao |
|
- Christmas Island |
|
- Cayman Islands |
|
- Cyprus |
|
- Czechia |
|
- Germany |
|
- Djibouti |
|
- Dominica |
|
- Denmark |
|
- Dominican Republic |
|
- Algeria |
|
- Ecuador |
|
- Egypt |
|
- Eritrea |
|
- Western Sahara |
|
- Spain |
|
- Estonia |
|
- Ethiopia |
|
- Finland |
|
- Fiji |
|
- Falkland Islands (Malvinas) |
|
- France |
|
- Faroe Islands |
|
- Federated States of Micronesia |
|
- Gabon |
|
- United Kingdom |
|
- Georgia |
|
- Guernsey |
|
- Ghana |
|
- Gibraltar |
|
- Guinea |
|
- Guadeloupe |
|
- Gambia |
|
- Guinea Bissau |
|
- Equatorial Guinea |
|
- Greece |
|
- Grenada |
|
- Greenland |
|
- Guatemala |
|
- French Guiana |
|
- Guam |
|
- Guyana |
|
- Hong Kong |
|
- Heard Island and McDonald Islands |
|
- Honduras |
|
- Croatia |
|
- Haiti |
|
- Hungary |
|
- Indonesia |
|
- Isle of Man |
|
- India |
|
- British Indian Ocean Territory |
|
- Ireland |
|
- Islamic Republic of Iran |
|
- Iraq |
|
- Iceland |
|
- Israel |
|
- Italy |
|
- Jamaica |
|
- Jersey |
|
- Jordan |
|
- Japan |
|
- Kazakhstan |
|
- Kenya |
|
- Kyrgyzstan |
|
- Cambodia |
|
- Kiribati |
|
- Saint-Kitts-and-Nevis |
|
- South Korea |
|
- Kuwait |
|
- Lao-Peoples-Democratic-Republic |
|
- Lebanon |
|
- Liberia |
|
- Libya |
|
- Saint-Lucia |
|
- Liechtenstein |
|
- Sri Lanka |
|
- Lesotho |
|
- Lithuania |
|
- Luxembourg |
|
- Latvia |
|
- Macao |
|
- Saint Martin (French-part) |
|
- Morocco |
|
- Monaco |
|
- Republic of Moldova |
|
- Madagascar |
|
- Maldives |
|
- Mexico |
|
- Marshall Islands |
|
- North Macedonia |
|
- Mali |
|
- Malta |
|
- Myanmar |
|
- Montenegro |
|
- Mongolia |
|
- Northern Mariana Islands |
|
- Mozambique |
|
- Mauritania |
|
- Montserrat |
|
- Martinique |
|
- Mauritius |
|
- Malawi |
|
- Malaysia |
|
- Mayotte |
|
- Namibia |
|
- New Caledonia |
|
- Niger |
|
- Norfolk Island |
|
- Nigeria |
|
- Nicaragua |
|
- Niue |
|
- Netherlands |
|
- Norway |
|
- Nepal |
|
- Nauru |
|
- New Zealand |
|
- Oman |
|
- Pakistan |
|
- Panama |
|
- Pitcairn |
|
- Peru |
|
- Philippines |
|
- Palau |
|
- Papua New Guinea |
|
- Poland |
|
- Puerto Rico |
|
- North Korea |
|
- Portugal |
|
- Paraguay |
|
- State of Palestine |
|
- French Polynesia |
|
- Qatar |
|
- Réunion |
|
- Romania |
|
- Russia |
|
- Rwanda |
|
- Saudi Arabia |
|
- Sudan |
|
- Senegal |
|
- Singapore |
|
- South Georgia and the South Sandwich Islands |
|
- Saint Helena Ascension and Tristan da Cunha |
|
- Svalbard and Jan Mayen |
|
- Solomon Islands |
|
- Sierra Leone |
|
- El Salvador |
|
- San Marino |
|
- Somalia |
|
- Saint Pierre and Miquelon |
|
- Serbia |
|
- South Sudan |
|
- Sao Tome and Principe |
|
- Suriname |
|
- Slovakia |
|
- Slovenia |
|
- Sweden |
|
- Eswatini |
|
- Sint Maarten (Dutch-part) |
|
- Seychelles |
|
- Syrian Arab Republic |
|
- Turks and Caicos Islands |
|
- Chad |
|
- Togo |
|
- Thailand |
|
- Tajikistan |
|
- Tokelau |
|
- Turkmenistan |
|
- Timor Leste |
|
- Tonga |
|
- Trinidad and Tobago |
|
- Tunisia |
|
- Turkey |
|
- Tuvalu |
|
- Taiwan |
|
- United Republic of Tanzania |
|
- Uganda |
|
- Ukraine |
|
- United States Minor Outlying Islands |
|
- Uruguay |
|
- United-States |
|
- Uzbekistan |
|
- Holy See (Vatican City State) |
|
- Saint Vincent and the Grenadines |
|
- Bolivarian Republic of Venezuela |
|
- Virgin Islands British |
|
- Virgin Islands U.S. |
|
- VietNam |
|
- Vanuatu |
|
- Wallis and Futuna |
|
- Samoa |
|
- Yemen |
|
- South Africa |
|
- Zambia |
|
- Zimbabwe |
|
I agree to use this model for non-commercial use ONLY: checkbox |
|
base_model: CohereForAI/aya-23-35B |
|
tags: |
|
- TensorBlock |
|
- GGUF |
|
--- |
|
|
|
<div style="width: auto; margin-left: auto; margin-right: auto"> |
|
<img src="https://i.imgur.com/jC7kdl8.jpeg" alt="TensorBlock" style="width: 100%; min-width: 400px; display: block; margin: auto;"> |
|
</div> |
|
<div style="display: flex; justify-content: space-between; width: 100%;"> |
|
<div style="display: flex; flex-direction: column; align-items: flex-start;"> |
|
<p style="margin-top: 0.5em; margin-bottom: 0em;"> |
|
Feedback and support: TensorBlock's <a href="https://x.com/tensorblock_aoi">Twitter/X</a>, <a href="https://t.me/TensorBlock">Telegram Group</a> and <a href="https://x.com/tensorblock_aoi">Discord server</a> |
|
</p> |
|
</div> |
|
</div> |
|
|
|
## CohereForAI/aya-23-35B - GGUF |
|
|
|
This repo contains GGUF format model files for [CohereForAI/aya-23-35B](https://huggingface.co/CohereForAI/aya-23-35B). |
|
|
|
The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d). |
|
|
|
<div style="text-align: left; margin: 20px 0;"> |
|
<a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;"> |
|
Run them on the TensorBlock client using your local machine ↗ |
|
</a> |
|
</div> |
|
|
|
## Prompt template |
|
|
|
``` |
|
<BOS_TOKEN><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>{system_prompt}<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>{prompt}<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|> |
|
``` |
|
|
|
## Model file specification |
|
|
|
| Filename | Quant type | File Size | Description | |
|
| -------- | ---------- | --------- | ----------- | |
|
| [aya-23-35B-Q2_K.gguf](https://huggingface.co/tensorblock/aya-23-35B-GGUF/blob/main/aya-23-35B-Q2_K.gguf) | Q2_K | 13.817 GB | smallest, significant quality loss - not recommended for most purposes | |
|
| [aya-23-35B-Q3_K_S.gguf](https://huggingface.co/tensorblock/aya-23-35B-GGUF/blob/main/aya-23-35B-Q3_K_S.gguf) | Q3_K_S | 15.862 GB | very small, high quality loss | |
|
| [aya-23-35B-Q3_K_M.gguf](https://huggingface.co/tensorblock/aya-23-35B-GGUF/blob/main/aya-23-35B-Q3_K_M.gguf) | Q3_K_M | 17.618 GB | very small, high quality loss | |
|
| [aya-23-35B-Q3_K_L.gguf](https://huggingface.co/tensorblock/aya-23-35B-GGUF/blob/main/aya-23-35B-Q3_K_L.gguf) | Q3_K_L | 19.149 GB | small, substantial quality loss | |
|
| [aya-23-35B-Q4_0.gguf](https://huggingface.co/tensorblock/aya-23-35B-GGUF/blob/main/aya-23-35B-Q4_0.gguf) | Q4_0 | 20.229 GB | legacy; small, very high quality loss - prefer using Q3_K_M | |
|
| [aya-23-35B-Q4_K_S.gguf](https://huggingface.co/tensorblock/aya-23-35B-GGUF/blob/main/aya-23-35B-Q4_K_S.gguf) | Q4_K_S | 20.378 GB | small, greater quality loss | |
|
| [aya-23-35B-Q4_K_M.gguf](https://huggingface.co/tensorblock/aya-23-35B-GGUF/blob/main/aya-23-35B-Q4_K_M.gguf) | Q4_K_M | 21.527 GB | medium, balanced quality - recommended | |
|
| [aya-23-35B-Q5_0.gguf](https://huggingface.co/tensorblock/aya-23-35B-GGUF/blob/main/aya-23-35B-Q5_0.gguf) | Q5_0 | 24.340 GB | legacy; medium, balanced quality - prefer using Q4_K_M | |
|
| [aya-23-35B-Q5_K_S.gguf](https://huggingface.co/tensorblock/aya-23-35B-GGUF/blob/main/aya-23-35B-Q5_K_S.gguf) | Q5_K_S | 24.340 GB | large, low quality loss - recommended | |
|
| [aya-23-35B-Q5_K_M.gguf](https://huggingface.co/tensorblock/aya-23-35B-GGUF/blob/main/aya-23-35B-Q5_K_M.gguf) | Q5_K_M | 25.008 GB | large, very low quality loss - recommended | |
|
| [aya-23-35B-Q6_K.gguf](https://huggingface.co/tensorblock/aya-23-35B-GGUF/blob/main/aya-23-35B-Q6_K.gguf) | Q6_K | 28.707 GB | very large, extremely low quality loss | |
|
| [aya-23-35B-Q8_0.gguf](https://huggingface.co/tensorblock/aya-23-35B-GGUF/blob/main/aya-23-35B-Q8_0.gguf) | Q8_0 | 37.179 GB | very large, extremely low quality loss - not recommended | |
|
|
|
|
|
## Downloading instruction |
|
|
|
### Command line |
|
|
|
Firstly, install Huggingface Client |
|
|
|
```shell |
|
pip install -U "huggingface_hub[cli]" |
|
``` |
|
|
|
Then, downoad the individual model file the a local directory |
|
|
|
```shell |
|
huggingface-cli download tensorblock/aya-23-35B-GGUF --include "aya-23-35B-Q2_K.gguf" --local-dir MY_LOCAL_DIR |
|
``` |
|
|
|
If you wanna download multiple model files with a pattern (e.g., `*Q4_K*gguf`), you can try: |
|
|
|
```shell |
|
huggingface-cli download tensorblock/aya-23-35B-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf' |
|
``` |
|
|
