Upload README.md
Browse files
README.md
CHANGED
@@ -99,7 +99,7 @@ Models are released as sharded safetensors files.
|
|
99 |
|
100 |
| Branch | Bits | GS | AWQ Dataset | Seq Len | Size |
|
101 |
| ------ | ---- | -- | ----------- | ------- | ---- |
|
102 |
-
| [main](https://huggingface.co/TheBloke/Synatra-7B-v0.3-RP-AWQ/tree/main) | 4 | 128 |
|
103 |
|
104 |
<!-- README_AWQ.md-provided-files end -->
|
105 |
|
@@ -439,3 +439,16 @@ print(decoded[0])
|
|
439 |
# Why It's benchmark score is lower than preview version?
|
440 |
|
441 |
**Apparently**, Preview model uses Alpaca Style prompt which has no pre-fix. But ChatML do.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
99 |
|
100 |
| Branch | Bits | GS | AWQ Dataset | Seq Len | Size |
|
101 |
| ------ | ---- | -- | ----------- | ------- | ---- |
|
102 |
+
| [main](https://huggingface.co/TheBloke/Synatra-7B-v0.3-RP-AWQ/tree/main) | 4 | 128 | [korean](https://huggingface.co/datasets/beomi/KoAlpaca-v1.1a/viewer/) | 4096 | 4.15 GB
|
103 |
|
104 |
<!-- README_AWQ.md-provided-files end -->
|
105 |
|
|
|
439 |
# Why It's benchmark score is lower than preview version?
|
440 |
|
441 |
**Apparently**, Preview model uses Alpaca Style prompt which has no pre-fix. But ChatML do.
|
442 |
+
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
|
443 |
+
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_maywell__Synatra-7B-v0.3-RP)
|
444 |
+
|
445 |
+
| Metric | Value |
|
446 |
+
|-----------------------|---------------------------|
|
447 |
+
| Avg. | 57.38 |
|
448 |
+
| ARC (25-shot) | 62.2 |
|
449 |
+
| HellaSwag (10-shot) | 82.29 |
|
450 |
+
| MMLU (5-shot) | 60.8 |
|
451 |
+
| TruthfulQA (0-shot) | 52.64 |
|
452 |
+
| Winogrande (5-shot) | 76.48 |
|
453 |
+
| GSM8K (5-shot) | 21.15 |
|
454 |
+
| DROP (3-shot) | 46.06 |
|