Magician-MoE-4x7B
Magician-MoE-4x7B is a Mixure of Experts (MoE) made with the following models:
- deepseek-ai/deepseek-coder-6.7b-instruct
- ise-uiuc/Magicoder-S-CL-7B
- WizardLM/WizardMath-7B-V1.0
- WizardLM/WizardCoder-Python-7B-V1.0
𧩠Configuration
base_model: ise-uiuc/Magicoder-S-CL-7B
gate_mode: cheap_embed
experts:
- source_model: deepseek-ai/deepseek-coder-6.7b-instruct
positive_prompts: ["You are an AI coder","coding","Java expert"]
- source_model: ise-uiuc/Magicoder-S-CL-7B
positive_prompts: ["You are an AI programmer","programming","C++ expert"]
- source_model: WizardLM/WizardMath-7B-V1.0
positive_prompts: ["Math problem solving","Think step by step","Math expert"]
- source_model: WizardLM/WizardCoder-Python-7B-V1.0
positive_prompts: ["Great at Deep learning","Algorithm and Data Structure","Python expert"]
π» Usage
!pip install -qU transformers bitsandbytes accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "FelixChao/Magician-MoE-4x7B"
tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
"text-generation",
model=model,
model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)
messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
- Downloads last month
- 1,197
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.