SicariusSicariiStuff commited on
Commit
7acff0e
β€’
1 Parent(s): 999ba8a

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +122 -0
README.md ADDED
@@ -0,0 +1,122 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: apache-2.0
5
+ ---
6
+
7
+ <div align="center">
8
+ <b style="font-size: 40px;">Zion_Alpha_Instruction_Tuned_SLERP</b>
9
+
10
+
11
+ </div>
12
+
13
+
14
+ <img src="https://i.imgur.com/e1LEQ18.png" alt="Zion_Alpha_Instruction_Tuned_SLERP" style="width: 50%; min-width: 400px; display: block; margin: auto;">
15
+
16
+
17
+ # Model Details
18
+
19
+ Zion_Alpha is the first **REAL** Hebrew model in the world. This version WAS fine tuned for tasks. I did the finetune using SOTA techniques and using my insights from years of underwater basket weaving. If you wanna offer me a job, just add me on Facebook.
20
+
21
+ # Future Plans
22
+ I plan to perform a SLERP merge with one of my other fine-tuned models, which has a bit more knowledge about Israeli topics. Additionally, I might create a larger model using MergeKit, but we'll see how it goes.
23
+
24
+ # Looking for Sponsors
25
+ Since all my work is done on-premises, I am constrained by my current hardware. I would greatly appreciate any support in acquiring an A6000, which would enable me to train significantly larger models much faster.
26
+
27
+ # Papers?
28
+ Maybe. We'll see. No promises here πŸ€“
29
+
30
+ # Contact Details
31
+ I'm not great at self-marketing (to say the least) and don't have any social media accounts. If you'd like to reach out to me, you can email me at [email protected]. Please note that this email might receive more messages than I can handle, so I apologize in advance if I can't respond to everyone.
32
+
33
+ # Versions and QUANTS
34
+ - Base model: [FP16](https://huggingface.co/SicariusSicariiStuff/Zion_Alpha)
35
+ - Instruction tuned: [FP16](https://huggingface.co/SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned) | [GGUF](https://huggingface.co/SicariusSicariiStuff/Zion_Alpha_Instruction_Tuned_GGUF)
36
+
37
+
38
+ # Model architecture
39
+ Based on Mistral 7B. I didn't even bother to alter the tokenizer.
40
+
41
+ # The recommended prompt setting is Debug-deterministic:
42
+ ```
43
+ temperature: 1
44
+ top_p: 1
45
+ top_k: 1
46
+ typical_p: 1
47
+ min_p: 1
48
+ repetition_penalty: 1
49
+ ```
50
+
51
+ # The recommended instruction template is Mistral:
52
+ ```
53
+ {%- for message in messages %}
54
+ {%- if message['role'] == 'system' -%}
55
+ {{- message['content'] -}}
56
+ {%- else -%}
57
+ {%- if message['role'] == 'user' -%}
58
+ {{-'[INST] ' + message['content'].rstrip() + ' [/INST]'-}}
59
+ {%- else -%}
60
+ {{-'' + message['content'] + '</s>' -}}
61
+ {%- endif -%}
62
+ {%- endif -%}
63
+ {%- endfor -%}
64
+ {%- if add_generation_prompt -%}
65
+ {{-''-}}
66
+ {%- endif -%}
67
+ ```
68
+ # English to hebrew example:
69
+
70
+
71
+ <div align="center">
72
+ <b style="font-size: 40px;">Zion_Alpha English to Hebrew example</b>
73
+
74
+
75
+ </div>
76
+
77
+
78
+ <img src="https://i.imgur.com/JnTuawF.png" alt="Zion_Alpha" style="width: 40%; min-width: 600px; display: block; margin: auto;">
79
+
80
+
81
+ # English to hebrew example:
82
+
83
+
84
+ <div align="center">
85
+ <b style="font-size: 40px;">Zion_Alpha Hebrew to English example</b>
86
+
87
+
88
+ </div>
89
+
90
+
91
+ <img src="https://i.imgur.com/Wm2igLJ.png" alt="Zion_Alpha" style="width: 40%; min-width: 600px; display: block; margin: auto;">
92
+
93
+
94
+ <div align="center">
95
+ <b style="font-size: 30px;">Unscripted video: live zero shot demonstration at story writing capabilities in Hebrew</b>
96
+
97
+ [![Zion_Alpha Story writing](https://img.youtube.com/vi/YYKeovnS0do/0.jpg)](https://www.youtube.com/watch?v=YYKeovnS0do)
98
+ </div>
99
+
100
+ <div align="center">
101
+ <b style="font-size: 30px;">Zion_Alpha VS Mistral 'Hebrew' Live & unscripted in real time</b>
102
+
103
+ [![Zion_Alpha Story writing](https://img.youtube.com/vi/YYKeovnS0do/0.jpg)](https://www.youtube.com/watch?v=DQFtx8M2txc)
104
+ </div>
105
+
106
+ <div align="center">
107
+ <b style="font-size: 30px;">Zion_Alpha VS Mistral 'Hebrew' Live & unscripted in real time Long text translation</b>
108
+
109
+ [![Zion_Alpha Story writing](https://img.youtube.com/vi/YYKeovnS0do/0.jpg)](https://www.youtube.com/watch?v=w5fz3Ot6tH8)
110
+ </div>
111
+
112
+ ### History
113
+ The model was originally trained about 2 month after Mistral (v0.1) was released.
114
+ As of 04 June 2024, Zion_Alpha got the **Highest SNLI score in the world** among open source models in Hebrew, surpassing most of the models by a huge margin. (**84.05** score)
115
+ <img src="https://i.imgur.com/7HokS5w.png" alt="Zion_Alpha SNLI Score" style="width: 80%; min-width: 700px; display: block; margin: auto;">
116
+
117
+ ### Support
118
+ <img src="https://i.imgur.com/0lHHN95.png" alt="GPUs too expensive" style="width: 10%; min-width: 100px; display: block; margin: left;">
119
+
120
+ - [My Ko-fi page](https://ko-fi.com/sicarius) ALL donations will go for research resources and compute, every bit counts πŸ™πŸ»
121
+ - [My Patreon](https://patreon.com/TenebraAI) ALL donations will go for research resources and compute, every bit counts πŸ™πŸ»
122
+