TheBloke commited on
Commit
97472aa
1 Parent(s): 1c836fb

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -1
README.md CHANGED
@@ -99,7 +99,7 @@ Models are released as sharded safetensors files.
99
 
100
  | Branch | Bits | GS | AWQ Dataset | Seq Len | Size |
101
  | ------ | ---- | -- | ----------- | ------- | ---- |
102
- | [main](https://huggingface.co/TheBloke/Synatra-7B-v0.3-RP-AWQ/tree/main) | 4 | 128 | japanese | 4096 | 4.15 GB
103
 
104
  <!-- README_AWQ.md-provided-files end -->
105
 
@@ -439,3 +439,16 @@ print(decoded[0])
439
  # Why It's benchmark score is lower than preview version?
440
 
441
  **Apparently**, Preview model uses Alpaca Style prompt which has no pre-fix. But ChatML do.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
99
 
100
  | Branch | Bits | GS | AWQ Dataset | Seq Len | Size |
101
  | ------ | ---- | -- | ----------- | ------- | ---- |
102
+ | [main](https://huggingface.co/TheBloke/Synatra-7B-v0.3-RP-AWQ/tree/main) | 4 | 128 | [korean](https://huggingface.co/datasets/beomi/KoAlpaca-v1.1a/viewer/) | 4096 | 4.15 GB
103
 
104
  <!-- README_AWQ.md-provided-files end -->
105
 
 
439
  # Why It's benchmark score is lower than preview version?
440
 
441
  **Apparently**, Preview model uses Alpaca Style prompt which has no pre-fix. But ChatML do.
442
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
443
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_maywell__Synatra-7B-v0.3-RP)
444
+
445
+ | Metric | Value |
446
+ |-----------------------|---------------------------|
447
+ | Avg. | 57.38 |
448
+ | ARC (25-shot) | 62.2 |
449
+ | HellaSwag (10-shot) | 82.29 |
450
+ | MMLU (5-shot) | 60.8 |
451
+ | TruthfulQA (0-shot) | 52.64 |
452
+ | Winogrande (5-shot) | 76.48 |
453
+ | GSM8K (5-shot) | 21.15 |
454
+ | DROP (3-shot) | 46.06 |