Edit Models filters

Inference status

Misc

arxiv: 2309.00071

Inference Endpoints

AutoTrain Compatible

text-generation-inference

4-bit precision

8-bit precision

Misc with no match

text-embeddings-inference

Carbon Emissions

Mixture of Experts

Models

448

Full-text search

Active filters: 2309.00071

starsy/Llama-3-70B-Instruct-Gradient-1048k-AWQ

Text Generation • Updated May 17, 2024 • 20 • 1

RichardErkhov/gradientai_-_Llama-3-8B-Instruct-Gradient-4194k-4bits

Text Generation • Updated May 15, 2024 • 21

RichardErkhov/gradientai_-_Llama-3-8B-Instruct-Gradient-4194k-8bits

Text Generation • Updated May 15, 2024 • 21

RichardErkhov/gradientai_-_Llama-3-8B-Instruct-Gradient-4194k-gguf

Updated May 15, 2024 • 11

RichardErkhov/gradientai_-_Llama-3-70B-Instruct-Gradient-1048k-gguf

Updated May 29, 2024 • 104

RichardErkhov/yanismiraoui_-_Yarn-Mistral-7b-128k-sharded-gguf

Updated Jun 1, 2024 • 21

Qwen/Qwen2-72B-Instruct-GPTQ-Int4

Text Generation • Updated Aug 21, 2024 • 4.83k • 31

Qwen/Qwen2-72B-Instruct-GPTQ-Int8

Text Generation • Updated Aug 21, 2024 • 28k • 14

Qwen/Qwen2-72B-Instruct-AWQ

Text Generation • Updated Aug 21, 2024 • 15.6k • 37

Qwen/Qwen2-7B-Instruct-GPTQ-Int4

Text Generation • Updated Aug 21, 2024 • 7.34k • 23

Qwen/Qwen2-7B-Instruct-GPTQ-Int8

Text Generation • Updated Aug 21, 2024 • 956 • 16

Qwen/Qwen2-7B-Instruct-AWQ

Text Generation • Updated Aug 21, 2024 • 597 • 19

LoneStriker/Qwen2-7B-Instruct-3.0bpw-h6-exl2

Text Generation • Updated Jun 6, 2024 • 17

LoneStriker/Qwen2-7B-Instruct-4.0bpw-h6-exl2

Text Generation • Updated Jun 6, 2024 • 16 • 1

LoneStriker/Qwen2-7B-Instruct-5.0bpw-h6-exl2

Text Generation • Updated Jun 6, 2024 • 18

LoneStriker/Qwen2-7B-Instruct-6.0bpw-h6-exl2

Text Generation • Updated Jun 6, 2024 • 16

LoneStriker/Qwen2-7B-Instruct-8.0bpw-h8-exl2

Text Generation • Updated Jun 6, 2024 • 19

Zoyd/Qwen_Qwen2-7B-Instruct-2_2bpw_exl2

Text Generation • Updated Jun 6, 2024 • 17

Zoyd/Qwen_Qwen2-7B-Instruct-3_0bpw_exl2

Text Generation • Updated Jun 6, 2024 • 16

Zoyd/Qwen_Qwen2-7B-Instruct-2_5bpw_exl2

Text Generation • Updated Jun 6, 2024 • 20

Zoyd/Qwen_Qwen2-7B-Instruct-4_0bpw_exl2

Text Generation • Updated Jun 6, 2024 • 18

Zoyd/Qwen_Qwen2-7B-Instruct-3_5bpw_exl2

Text Generation • Updated Jun 6, 2024 • 19

Zoyd/Qwen_Qwen2-7B-Instruct-3_75bpw_exl2

Text Generation • Updated Jun 6, 2024 • 22

Zoyd/Qwen_Qwen2-7B-Instruct-6_0bpw_exl2

Text Generation • Updated Jun 6, 2024 • 22

Zoyd/Qwen_Qwen2-7B-Instruct-4_25bpw_exl2

Text Generation • Updated Jun 6, 2024 • 17

Zoyd/Qwen_Qwen2-7B-Instruct-5_0bpw_exl2

Text Generation • Updated Jun 6, 2024 • 18

Zoyd/Qwen_Qwen2-7B-Instruct-6_5bpw_exl2

Text Generation • Updated Jun 6, 2024 • 14

Zoyd/Qwen_Qwen2-7B-Instruct-8_0bpw_exl2

Text Generation • Updated Jun 6, 2024 • 20

bullerwins/Qwen2-72B-Instruct_exl2_8.0bpw

Text Generation • Updated Jun 7, 2024 • 20 • 1

bullerwins/Qwen2-72B-Instruct_exl2_6.0bpw

Text Generation • Updated Jun 7, 2024 • 19