File size: 429 Bytes
1eaabd2 |
1 2 3 4 5 6 7 8 9 10 11 12 |
Commands to make this:
```bash
cd llama.cpp
./convert.py --outfile Karasu-Mixtral-8x22B-v0.1-q3_k_m --outtype f16 /workspace/llm_training/axolotl/mixtral_8x22B_training/merged_model_multiling
./quantize /workspace/Karasu-Mixtral-8x22B-v0.1.gguf /workspace/Karasu-Mixtral-8x22B-v0.1_q3_k_m.gguf Q3_K_M
./gguf-split --split --split-max-size 5G /workspace/Karasu-Mixtral-8x22B-v0.1_q3_k_m.gguf /workspace/somewhere-sensible
``` |