AdamLucek commited on
Commit
cb2ebbb
1 Parent(s): e23f7b7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -5
README.md CHANGED
@@ -26,28 +26,27 @@ EduMixtral is a Mixture of Experts (MoE) made with the following models using [M
26
 
27
  ## Usage
28
 
 
 
29
  ```python
30
  from transformers import AutoTokenizer, AutoModelForCausalLM
31
- import torch
32
 
33
  # Load the tokenizer and model
34
  tokenizer = AutoTokenizer.from_pretrained("AdamLucek/EduMixtral-4x7B")
35
  model = AutoModelForCausalLM.from_pretrained(
36
  "AdamLucek/EduMixtral-4x7B",
37
  device_map="cuda",
38
- torch_dtype=torch.bfloat16
39
  )
40
 
41
  # Prepare the input text
42
- input_text = "The length of a rectangular vegetable field is 120m, the length is more than the width (2/3), calculate how many meters is the width of this vegetable field?"
43
  input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
44
 
45
  # Generate the output with specified parameters
46
  outputs = model.generate(
47
  **input_ids,
48
  max_new_tokens=256,
49
- temperature=0.7,
50
- top_p=0.9,
51
  num_return_sequences=1
52
  )
53
 
@@ -55,6 +54,19 @@ outputs = model.generate(
55
  print(tokenizer.decode(outputs[0], skip_special_tokens=True))
56
  ```
57
 
 
 
 
 
 
 
 
 
 
 
 
 
 
58
  ## 🧩 Configuration
59
 
60
  ```yaml
 
26
 
27
  ## Usage
28
 
29
+ It is reccomended to load in 8bit or 4bit quantization
30
+
31
  ```python
32
  from transformers import AutoTokenizer, AutoModelForCausalLM
 
33
 
34
  # Load the tokenizer and model
35
  tokenizer = AutoTokenizer.from_pretrained("AdamLucek/EduMixtral-4x7B")
36
  model = AutoModelForCausalLM.from_pretrained(
37
  "AdamLucek/EduMixtral-4x7B",
38
  device_map="cuda",
39
+ quantization_config=BitsAndBytesConfig(load_in_8bit=True)
40
  )
41
 
42
  # Prepare the input text
43
+ input_text = "Math problem: Xiaoli reads a 240-page story book. She reads (1/8) of the whole book on the first day and (1/5) of the whole book on the second day. How many pages did she read in total in two days?"
44
  input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
45
 
46
  # Generate the output with specified parameters
47
  outputs = model.generate(
48
  **input_ids,
49
  max_new_tokens=256,
 
 
50
  num_return_sequences=1
51
  )
52
 
 
54
  print(tokenizer.decode(outputs[0], skip_special_tokens=True))
55
  ```
56
 
57
+ **Output:**
58
+
59
+ >Solution:
60
+ >To find the total number of pages Xiaoli read in two days, we need to add the number of pages she read on the first day and the second day.
61
+ >On the first day, Xiaoli read 1/8 of the book. Since the book has 240 pages, the number of pages she read on the first day is:
62
+ >\[ \frac{1}{8} \times 240 = 30 \text{ pages} \]
63
+ >On the second day, Xiaoli read 1/5 of the book. The number of pages she read on the second day is:
64
+ >\[ \frac{1}{5} \times 240 = 48 \text{ pages} \]
65
+ >To find the total number of pages she read in two days, we add the pages she read on the first day and the second day:
66
+ >\[ 30 \text{ pages} + 48 \text{ pages} = 78 \text{ pages} \]
67
+ >Therefore, Xiaoli read a total of 78 pages in two days.
68
+ >Final answer: Xiaoli read 78 pages in total
69
+
70
  ## 🧩 Configuration
71
 
72
  ```yaml