Edit Models filters

Inference status

Misc

4-bit precision

AutoTrain Compatible

text-generation-inference

Inference Endpoints

8-bit precision

Mixture of Experts

text-embeddings-inference

Carbon Emissions

Models

15,934

Full-text search

Active filters: 4-bit

unsloth/Llama-3.1-Nemotron-70B-Instruct-bnb-4bit

Text Generation • Updated 5 days ago • 21.1k • 14

mlx-community/nvidia_Llama-3.1-Nemotron-70B-Instruct-HF_4bit

Text Generation • Updated 6 days ago • 650 • 10

Qwen/Qwen2-VL-72B-Instruct-AWQ

Image-Text-to-Text • Updated 27 days ago • 2.18M • 23

Qwen/Qwen2.5-32B-Instruct-AWQ

Text Generation • Updated 13 days ago • 17.7k • 31

xmadai/Llama-3.1-405B-Instruct-xMADai-4bit

Text Generation • Updated 5 days ago • 88 • 5

Qwen/Qwen2-VL-7B-Instruct-AWQ

Image-Text-to-Text • Updated 27 days ago • 56.4k • 33

unsloth/Llama-3.2-11B-Vision-Instruct-bnb-4bit

Updated 27 days ago • 20k • 32

xmadai/Llama-3.2-3B-Instruct-xMADai-4bit

Text Generation • Updated 5 days ago • 46 • 4

xmadai/Llama-3.2-1B-Instruct-xMADai-4bit

Text Generation • Updated 5 days ago • 37 • 4

unsloth/Llama-3.2-1B-Instruct-bnb-4bit

Text Generation • Updated 22 days ago • 75.4k • 8

hugging-quants/gemma-2-9b-it-AWQ-INT4

Text Generation • Updated 5 days ago • 240 • 3

mlx-community/Ministral-8B-Instruct-2410-4bit

Updated 5 days ago • 172 • 3

MaziyarPanahi/Llama-3.1-Nemotron-70B-Instruct-HF-GGUF

Text Generation • Updated 5 days ago • 1k • 3

WisPerMed/Llama-3.1-SauerkrautLM-70b-Instruct-AWQ

Text Generation • Updated 5 days ago • 106 • 3

TheBloke/Mixtral-8x7B-Instruct-v0.1-GPTQ

Text Generation • Updated Dec 14, 2023 • 85k • 133

neuralmagic/Mistral-7B-Instruct-v0.3-GPTQ-4bit

Text Generation • Updated Jun 10 • 130k • 14

hugging-quants/Meta-Llama-3.1-405B-Instruct-AWQ-INT4

Text Generation • Updated Sep 13 • 9.1k • 36

hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4

Text Generation • Updated Aug 7 • 167k • 81

unsloth/Meta-Llama-3.1-70B-Instruct-bnb-4bit

Text Generation • Updated Sep 3 • 13.1k • 20

shuyuej/bge-multilingual-gemma2-GPTQ

Feature Extraction • Updated Jul 29 • 3.75k • 3

PrunaAI/THUDM-LongWriter-glm4-9b-bnb-4bit-smashed

Updated Aug 16 • 7 • 2

Qwen/Qwen2-VL-2B-Instruct-GPTQ-Int4

Image-Text-to-Text • Updated Sep 21 • 6.17k • 13

Qwen/Qwen2-VL-72B-Instruct-GPTQ-Int4

Image-Text-to-Text • Updated 28 days ago • 11.6k • 6

Qwen/Qwen2.5-72B-Instruct-AWQ

Text Generation • Updated 13 days ago • 23.2k • 24

SeanScripts/pixtral-12b-nf4

Image-Text-to-Text • Updated 26 days ago • 2.24k • 10

mlx-community/Llama-3.2-3B-Instruct-4bit

Text Generation • Updated 27 days ago • 9.67k • 9

unsloth/Llama-3.2-3B-bnb-4bit

Text Generation • Updated 22 days ago • 32.7k • 4

unsloth/Llama-3.2-3B-Instruct-bnb-4bit

Text Generation • Updated 22 days ago • 185k • 12

John6666/llama-joycaption-alpha-two-hf-llava-nf4

Updated 6 days ago • 156 • 6

TheBloke/Wizard-Vicuna-13B-Uncensored-GPTQ

Text Generation • Updated Sep 27, 2023 • 6.54k • 317