README.md

---
license: cc-by-nc-4.0
tags:
- moe
- frankenmoe
- merge
- mergekit
- lazymergekit
base_model:
- mlabonne/NeuralDaredevil-7B
- BioMistral/BioMistral-7B
- mistralai/Mathstral-7B-v0.1
- FPHam/Writing_Partner_Mistral_7B
library_name: transformers
pipeline_tag: text-generation
---

# EduMixtral-4x7B

<img src="https://cdn-uploads.huggingface.co/production/uploads/65ba68a15d2ef0a4b2c892b4/1hvgYltQRmbkzHMSXvGYh.jpeg" width=400>

EduMixtral-4x7B is an experimental model that combines different educational focused language models intended for downstream human/ai student/teacher application research.
Intended to cover: general knowledge, medical field, math, and writing assistance. 

## 🤏 Models Merged

EduMixtral-4x7B is a Mixture of Experts (MoE) made with the following models using [Mergekit](https://github.com/arcee-ai/mergekit):
* [mlabonne/NeuralDaredevil-7B](https://huggingface.co/mlabonne/NeuralDaredevil-7B) <- Base Model
* [BioMistral/BioMistral-7B](https://huggingface.co/BioMistral/BioMistral-7B)
* [mistralai/Mathstral-7B-v0.1](https://huggingface.co/mistralai/Mathstral-7B-v0.1)
* [FPHam/Writing_Partner_Mistral_7B](https://huggingface.co/FPHam/Writing_Partner_Mistral_7B)

## 🧩 Configuration

```yaml
base_model: mlabonne/NeuralDaredevil-7B
gate_mode: hidden
experts:
  - source_model: mlabonne/NeuralDaredevil-7B
    positive_prompts:
    - "hello"
    - "help"
    - "question"
    - "explain"
    - "information"
  - source_model: BioMistral/BioMistral-7B
    positive_prompts:
    - "medical"
    - "health"
    - "biomedical"
    - "clinical"
    - "anatomy"
  - source_model: mistralai/Mathstral-7B-v0.1
    positive_prompts:
    - "math"
    - "calculation"
    - "equation"
    - "geometry"
    - "algebra"
  - source_model: FPHam/Writing_Partner_Mistral_7B
    positive_prompts:
    - "writing"
    - "creative process"
    - "story structure"
    - "character development"
    - "plot"
```

## 💻 Usage

It is reccomended to load in 8bit or 4bit quantization

```python
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig

# Load the tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("AdamLucek/EduMixtral-4x7B")
model = AutoModelForCausalLM.from_pretrained(
    "AdamLucek/EduMixtral-4x7B",
    device_map="cuda",
    quantization_config=BitsAndBytesConfig(load_in_8bit=True)
)

# Prepare the input text
input_text = "Math problem: Xiaoli reads a 240-page story book. She reads (1/8) of the whole book on the first day and (1/5) of the whole book on the second day. How many pages did she read in total in two days?"
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

# Generate the output with specified parameters
outputs = model.generate(
    **input_ids,
    max_new_tokens=256,
    num_return_sequences=1
)

# Decode and print the generated text
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

**Output:**

>Solution:  
>To find the total number of pages Xiaoli read in two days, we need to add the number of pages she read on the first day and the second day.  
>On the first day, Xiaoli read 1/8 of the book. Since the book has 240 pages, the number of pages she read on the first day is:  
>\[ \frac{1}{8} \times 240 = 30 \text{ pages} \]  
>On the second day, Xiaoli read 1/5 of the book. The number of pages she read on the second day is:  
>\[ \frac{1}{5} \times 240 = 48 \text{ pages} \]  
>To find the total number of pages she read in two days, we add the pages she read on the first day and the second day:  
>\[ 30 \text{ pages} + 48 \text{ pages} = 78 \text{ pages} \]  
>Therefore, Xiaoli read a total of 78 pages in two days.  
>Final answer: Xiaoli read 78 pages in total