ZifanScale
commited on
Commit
•
04cee05
1
Parent(s):
6661ab7
Update README.md
Browse files
README.md
CHANGED
@@ -81,4 +81,21 @@ performance but retention of general capabilities (MMLU)
|
|
81 |
|
82 |
The following picture shows LLaMA-3-8B-instruct multiple choice benchmark accuracies before and after RMU.
|
83 |
|
84 |
-
![](rmu_result.png)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
81 |
|
82 |
The following picture shows LLaMA-3-8B-instruct multiple choice benchmark accuracies before and after RMU.
|
83 |
|
84 |
+
![](rmu_result.png)
|
85 |
+
|
86 |
+
## Model Use
|
87 |
+
|
88 |
+
```
|
89 |
+
import transformers
|
90 |
+
import torch
|
91 |
+
|
92 |
+
model_id = "ScaleAI/mhj-llama3-8b-rmu"
|
93 |
+
|
94 |
+
pipeline = transformers.pipeline(
|
95 |
+
"text-generation",
|
96 |
+
model=model_id,
|
97 |
+
model_kwargs={"torch_dtype": torch.bfloat16},
|
98 |
+
device_map="auto",
|
99 |
+
)
|
100 |
+
|
101 |
+
```
|