Update README.md
Browse files
README.md
CHANGED
@@ -27,7 +27,7 @@ This model has been quantized using [GPTQModel](https://github.com/ModelCloud/GP
|
|
27 |
- **meta**:
|
28 |
- **quantizer**: "gptqmodel:0.9.9-dev0"
|
29 |
|
30 |
-
|
31 |
```python
|
32 |
from transformers import AutoTokenizer
|
33 |
from gptqmodel import GPTQModel
|
@@ -45,4 +45,95 @@ outputs = model.generate(input_ids=input_tensor.to(model.device), max_new_tokens
|
|
45 |
result = tokenizer.decode(outputs[0][input_tensor.shape[1]:], skip_special_tokens=True)
|
46 |
|
47 |
print(result)
|
48 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
27 |
- **meta**:
|
28 |
- **quantizer**: "gptqmodel:0.9.9-dev0"
|
29 |
|
30 |
+
## Example:
|
31 |
```python
|
32 |
from transformers import AutoTokenizer
|
33 |
from gptqmodel import GPTQModel
|
|
|
45 |
result = tokenizer.decode(outputs[0][input_tensor.shape[1]:], skip_special_tokens=True)
|
46 |
|
47 |
print(result)
|
48 |
+
```
|
49 |
+
|
50 |
+
## lm-eval benchmark:
|
51 |
+
|
52 |
+
| Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr|
|
53 |
+
|---------------------------------------|------:|------|-----:|----------|---|-----:|---|-----:|
|
54 |
+
|arc_challenge | 1|none | 0|acc |↑ |0.6186|± |0.0142|
|
55 |
+
| | |none | 0|acc_norm |↑ |0.6297|± |0.0141|
|
56 |
+
|arc_easy | 1|none | 0|acc |↑ |0.8628|± |0.0071|
|
57 |
+
| | |none | 0|acc_norm |↑ |0.8338|± |0.0076|
|
58 |
+
|boolq | 2|none | 0|acc |↑ |0.8761|± |0.0058|
|
59 |
+
|hellaswag | 1|none | 0|acc |↑ |0.6463|± |0.0048|
|
60 |
+
| | |none | 0|acc_norm |↑ |0.8389|± |0.0037|
|
61 |
+
|lambada_openai | 1|none | 0|acc |↑ |0.7561|± |0.0060|
|
62 |
+
| | |none | 0|perplexity|↓ |3.0311|± |0.0639|
|
63 |
+
|mmlu | 1|none | |acc |↑ |0.8100|± |0.0032|
|
64 |
+
| - humanities | 1|none | |acc |↑ |0.7981|± |0.0057|
|
65 |
+
| - formal_logic | 0|none | 0|acc |↑ |0.6349|± |0.0431|
|
66 |
+
| - high_school_european_history | 0|none | 0|acc |↑ |0.8545|± |0.0275|
|
67 |
+
| - high_school_us_history | 0|none | 0|acc |↑ |0.9412|± |0.0165|
|
68 |
+
| - high_school_world_history | 0|none | 0|acc |↑ |0.9198|± |0.0177|
|
69 |
+
| - international_law | 0|none | 0|acc |↑ |0.9008|± |0.0273|
|
70 |
+
| - jurisprudence | 0|none | 0|acc |↑ |0.8796|± |0.0315|
|
71 |
+
| - logical_fallacies | 0|none | 0|acc |↑ |0.8650|± |0.0268|
|
72 |
+
| - moral_disputes | 0|none | 0|acc |↑ |0.8266|± |0.0204|
|
73 |
+
| - moral_scenarios | 0|none | 0|acc |↑ |0.8559|± |0.0117|
|
74 |
+
| - philosophy | 0|none | 0|acc |↑ |0.8360|± |0.0210|
|
75 |
+
| - prehistory | 0|none | 0|acc |↑ |0.8827|± |0.0179|
|
76 |
+
| - professional_law | 0|none | 0|acc |↑ |0.6675|± |0.0120|
|
77 |
+
| - world_religions | 0|none | 0|acc |↑ |0.9181|± |0.0210|
|
78 |
+
| - other | 1|none | |acc |↑ |0.8304|± |0.0064|
|
79 |
+
| - business_ethics | 0|none | 0|acc |↑ |0.7900|± |0.0409|
|
80 |
+
| - clinical_knowledge | 0|none | 0|acc |↑ |0.8566|± |0.0216|
|
81 |
+
| - college_medicine | 0|none | 0|acc |↑ |0.7630|± |0.0324|
|
82 |
+
| - global_facts | 0|none | 0|acc |↑ |0.5800|± |0.0496|
|
83 |
+
| - human_aging | 0|none | 0|acc |↑ |0.8206|± |0.0257|
|
84 |
+
| - management | 0|none | 0|acc |↑ |0.8835|± |0.0318|
|
85 |
+
| - marketing | 0|none | 0|acc |↑ |0.9231|± |0.0175|
|
86 |
+
| - medical_genetics | 0|none | 0|acc |↑ |0.9400|± |0.0239|
|
87 |
+
| - miscellaneous | 0|none | 0|acc |↑ |0.9144|± |0.0100|
|
88 |
+
| - nutrition | 0|none | 0|acc |↑ |0.8660|± |0.0195|
|
89 |
+
| - professional_accounting | 0|none | 0|acc |↑ |0.6454|± |0.0285|
|
90 |
+
| - professional_medicine | 0|none | 0|acc |↑ |0.8971|± |0.0185|
|
91 |
+
| - virology | 0|none | 0|acc |↑ |0.5602|± |0.0386|
|
92 |
+
| - social sciences | 1|none | |acc |↑ |0.8736|± |0.0059|
|
93 |
+
| - econometrics | 0|none | 0|acc |↑ |0.7018|± |0.0430|
|
94 |
+
| - high_school_geography | 0|none | 0|acc |↑ |0.9242|± |0.0189|
|
95 |
+
| - high_school_government_and_politics| 0|none | 0|acc |↑ |0.9741|± |0.0115|
|
96 |
+
| - high_school_macroeconomics | 0|none | 0|acc |↑ |0.8410|± |0.0185|
|
97 |
+
| - high_school_microeconomics | 0|none | 0|acc |↑ |0.8992|± |0.0196|
|
98 |
+
| - high_school_psychology | 0|none | 0|acc |↑ |0.9229|± |0.0114|
|
99 |
+
| - human_sexuality | 0|none | 0|acc |↑ |0.8779|± |0.0287|
|
100 |
+
| - professional_psychology | 0|none | 0|acc |↑ |0.8497|± |0.0145|
|
101 |
+
| - public_relations | 0|none | 0|acc |↑ |0.7273|± |0.0427|
|
102 |
+
| - security_studies | 0|none | 0|acc |↑ |0.8163|± |0.0248|
|
103 |
+
| - sociology | 0|none | 0|acc |↑ |0.9154|± |0.0197|
|
104 |
+
| - us_foreign_policy | 0|none | 0|acc |↑ |0.9300|± |0.0256|
|
105 |
+
| - stem | 1|none | |acc |↑ |0.7456|± |0.0075|
|
106 |
+
| - abstract_algebra | 0|none | 0|acc |↑ |0.6300|± |0.0485|
|
107 |
+
| - anatomy | 0|none | 0|acc |↑ |0.7926|± |0.0350|
|
108 |
+
| - astronomy | 0|none | 0|acc |↑ |0.8947|± |0.0250|
|
109 |
+
| - college_biology | 0|none | 0|acc |↑ |0.9444|± |0.0192|
|
110 |
+
| - college_chemistry | 0|none | 0|acc |↑ |0.5800|± |0.0496|
|
111 |
+
| - college_computer_science | 0|none | 0|acc |↑ |0.6700|± |0.0473|
|
112 |
+
| - college_mathematics | 0|none | 0|acc |↑ |0.5400|± |0.0501|
|
113 |
+
| - college_physics | 0|none | 0|acc |↑ |0.6275|± |0.0481|
|
114 |
+
| - computer_security | 0|none | 0|acc |↑ |0.8200|± |0.0386|
|
115 |
+
| - conceptual_physics | 0|none | 0|acc |↑ |0.7830|± |0.0269|
|
116 |
+
| - electrical_engineering | 0|none | 0|acc |↑ |0.7862|± |0.0342|
|
117 |
+
| - elementary_mathematics | 0|none | 0|acc |↑ |0.7593|± |0.0220|
|
118 |
+
| - high_school_biology | 0|none | 0|acc |↑ |0.9194|± |0.0155|
|
119 |
+
| - high_school_chemistry | 0|none | 0|acc |↑ |0.7143|± |0.0318|
|
120 |
+
| - high_school_computer_science | 0|none | 0|acc |↑ |0.9200|± |0.0273|
|
121 |
+
| - high_school_mathematics | 0|none | 0|acc |↑ |0.5185|± |0.0305|
|
122 |
+
| - high_school_physics | 0|none | 0|acc |↑ |0.6556|± |0.0388|
|
123 |
+
| - high_school_statistics | 0|none | 0|acc |↑ |0.7361|± |0.0301|
|
124 |
+
| - machine_learning | 0|none | 0|acc |↑ |0.7054|± |0.0433|
|
125 |
+
|openbookqa | 1|none | 0|acc |↑ |0.3660|± |0.0216|
|
126 |
+
| | |none | 0|acc_norm |↑ |0.4620|± |0.0223|
|
127 |
+
|piqa | 1|none | 0|acc |↑ |0.8264|± |0.0088|
|
128 |
+
| | |none | 0|acc_norm |↑ |0.8319|± |0.0087|
|
129 |
+
|rte | 1|none | 0|acc |↑ |0.7184|± |0.0271|
|
130 |
+
|truthfulqa_mc1 | 2|none | 0|acc |↑ |0.3917|± |0.0171|
|
131 |
+
|winogrande | 1|none | 0|acc |↑ |0.7924|± |0.0114|
|
132 |
+
|
133 |
+
| Groups |Version|Filter|n-shot|Metric| |Value | |Stderr|
|
134 |
+
|------------------|------:|------|------|------|---|-----:|---|-----:|
|
135 |
+
|mmlu | 1|none | |acc |↑ |0.8100|± |0.0032|
|
136 |
+
| - humanities | 1|none | |acc |↑ |0.7981|± |0.0057|
|
137 |
+
| - other | 1|none | |acc |↑ |0.8304|± |0.0064|
|
138 |
+
| - social sciences| 1|none | |acc |↑ |0.8736|± |0.0059|
|
139 |
+
| - stem | 1|none | |acc |↑ |0.7456|± |0.0075|
|