Quantized Models (GGUF, IQ, Imatrix)
Collection
Various quantizations of models in the GGUF format. Models with a "checkmark" are personal favorites. An "orange arrow" means it's being uploaded.
•
88 items
•
Updated
•
47
My GGUF-IQ-Imatrix quants for Sao10K/MN-BackyardAI-Party-12B-v1.
"For best results, set both <|im_end|>
and [INST]
as stopping strings. Recommended Temperature is <1 , min_p of at least 0.1."
"This does require a lot of tinkering to fit within SillyTavern / other frontends."
Prompting:
- Similar to Mistral for group chats (please read the original model page for information on this)
- ChatML for one-on-one chats
Base model
Sao10K/MN-BackyardAI-Party-12B-v1