Severian commited on
Commit
4b41142
1 Parent(s): 492f2e4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +172 -5
README.md CHANGED
@@ -6,10 +6,7 @@ datasets:
6
  - Severian/Internal-Knowledge-Map
7
  ---
8
 
9
- # New Fixed Version with extended training being uploaded by end of day 3/5!
10
-
11
-
12
- ## Unfortunately there are some issues with this current model in how it was fused during training, leading to bad outputs. I am retraining and will reupload ASAP. In the meantime you can still use the Q8 GGUF version which works great.
13
 
14
  ## GGUF Q8 Version: https://huggingface.co/Severian/Nexus-IKM-Mistral-7B-GGUF
15
 
@@ -18,6 +15,176 @@ datasets:
18
 
19
  This model is the second trained with experimental 'Internal Knowledge Map' dataset. Developed with an aim to go beyond the scope of usual data processing capabilities, this model gets trained to build comprehensive understanding and reasoning in a wide range of knowledge domains with elaborate guidelines. It bases its reasoning on a specially selected dataset emphasizing the interrelations of the diverse disciplines which aim to synthesize, integrate, and apply complex information in ways that mimic humanly abstract reasoning and creative thought processes.
20
 
21
- At the very core of the development of this model is the desire to make sure that LLMs engage in a kind of cognitive activity not limited to memory but actually taking on abstract reasoning, problem-solving, and generation of new insights. To achieve this, 'Nexus-IKM-Mistral-7B' has been fine-tuned until 10 Epochs on this unique dataset, which resulted in the model demonstrating greater capability for giving rise to insights and problem-solving in complex, multi-disciplinary settings. This involves improved ability in drawing links between different pieces of knowledge, reasoning through complex scenarios, and proposing innovative solutions that cut across various domains, including science, technology, environmental studies, and humanities.
22
 
23
  Test this out and see if you find anything interesting or intriguing. I will keep iterating more versions but this one seems like a fun and useful way to start.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
  - Severian/Internal-Knowledge-Map
7
  ---
8
 
9
+ # New Fixed Version with extended training being uploaded right now!
 
 
 
10
 
11
  ## GGUF Q8 Version: https://huggingface.co/Severian/Nexus-IKM-Mistral-7B-GGUF
12
 
 
15
 
16
  This model is the second trained with experimental 'Internal Knowledge Map' dataset. Developed with an aim to go beyond the scope of usual data processing capabilities, this model gets trained to build comprehensive understanding and reasoning in a wide range of knowledge domains with elaborate guidelines. It bases its reasoning on a specially selected dataset emphasizing the interrelations of the diverse disciplines which aim to synthesize, integrate, and apply complex information in ways that mimic humanly abstract reasoning and creative thought processes.
17
 
18
+ At the very core of the development of this model is the desire to make sure that LLMs engage in a kind of cognitive activity not limited to memory but actually taking on abstract reasoning, problem-solving, and generation of new insights. To achieve this, 'Nexus-IKM-Mistral-7B' has been fine-tuned until convergance at 15 Epochs on this unique dataset, which resulted in the model demonstrating greater capability for giving rise to insights and problem-solving in complex, multi-disciplinary settings. This involves improved ability in drawing links between different pieces of knowledge, reasoning through complex scenarios, and proposing innovative solutions that cut across various domains, including science, technology, environmental studies, and humanities.
19
 
20
  Test this out and see if you find anything interesting or intriguing. I will keep iterating more versions but this one seems like a fun and useful way to start.
21
+
22
+ # Training Snaphot
23
+
24
+ ```
25
+ Step Training Loss
26
+ 1 3.223000
27
+ 2 3.221300
28
+ 3 3.215900
29
+ 4 3.210600
30
+ 5 3.203000
31
+ 6 3.193500
32
+ 7 3.184000
33
+ 8 3.173400
34
+ 9 3.162400
35
+ 10 3.151500
36
+ 11 3.140500
37
+ 12 3.128800
38
+ 13 3.117600
39
+ 14 3.106700
40
+ 15 3.095500
41
+ 16 3.084700
42
+ 17 3.073700
43
+ 18 3.062700
44
+ 19 3.052300
45
+ 20 3.041800
46
+
47
+
48
+ 201 1.273200
49
+ 202 1.257600
50
+ 203 1.241900
51
+ 204 1.226100
52
+ 205 1.210800
53
+ 206 1.195500
54
+ 207 1.180800
55
+ 208 1.166000
56
+ 209 1.151200
57
+ 210 1.136900
58
+ 211 1.122000
59
+ 212 1.106600
60
+ 213 1.091200
61
+ 214 1.075200
62
+ 215 1.059200
63
+ 216 1.042900
64
+ 217 1.026600
65
+ 218 1.010300
66
+ 219 0.994200
67
+ 416 0.041700
68
+ 417 0.041700
69
+ 418 0.041600
70
+ 419 0.041600
71
+ 420 0.041600
72
+ 421 0.041600
73
+ 422 0.041500
74
+ 423 0.041500
75
+ 424 0.041500
76
+ 425 0.041400
77
+ 426 0.041400
78
+ 427 0.041400
79
+
80
+ 668 0.035200
81
+ 669 0.035100
82
+ 670 0.035100
83
+ 671 0.035100
84
+ 672 0.035100
85
+ 673 0.035000
86
+ 674 0.035000
87
+ 675 0.035000
88
+ 676 0.035000
89
+ 677 0.034900
90
+ 678 0.034900
91
+ 679 0.034900
92
+ 680 0.034800
93
+ 681 0.034800
94
+ 682 0.034800
95
+ 683 0.034800
96
+ 684 0.034800
97
+ 685 0.034700
98
+ 686 0.034700
99
+
100
+ 1209 0.006600
101
+ 1210 0.006500
102
+ 1211 0.006300
103
+ 1212 0.006200
104
+ 1213 0.006100
105
+ 1214 0.006000
106
+ 1215 0.005800
107
+ 1216 0.005700
108
+ 1217 0.005600
109
+ 1218 0.005500
110
+ 1219 0.005400
111
+ 1220 0.005300
112
+ 1221 0.005100
113
+ 1222 0.004900
114
+ 1223 0.004800
115
+ 1224 0.004700
116
+ 1225 0.004600
117
+ 1226 0.004500
118
+ 1227 0.004400
119
+ 1228 0.004300
120
+ 1229 0.004200
121
+ 1230 0.004000
122
+ 1231 0.003900
123
+ 1232 0.003800
124
+ 1233 0.003700
125
+ 1234 0.003500
126
+ 1235 0.003400
127
+ 1236 0.003300
128
+ 1237 0.003200
129
+ 1238 0.003000
130
+ 1239 0.003000
131
+ 1240 0.002900
132
+ 1241 0.002800
133
+ 1242 0.002700
134
+ 1243 0.002600
135
+ 1244 0.002500
136
+ 1245 0.002400
137
+ 1246 0.002300
138
+ 1247 0.002200
139
+ 1248 0.002100
140
+ 1249 0.002000
141
+ 1250 0.001900
142
+ 1251 0.001800
143
+ 1252 0.001800
144
+ 1253 0.001700
145
+ 1254 0.001600
146
+ 1255 0.001600
147
+ 1256 0.001500
148
+ 1257 0.001400
149
+ 1258 0.001300
150
+ 1259 0.001300
151
+ 1260 0.001200
152
+ 1261 0.001200
153
+ 1262 0.001100
154
+ 1263 0.001100
155
+ 1264 0.001000
156
+ 1265 0.001000
157
+ 1266 0.000900
158
+ 1267 0.000900
159
+ 1268 0.000800
160
+ 1269 0.000800
161
+ 1270 0.000800
162
+ 1271 0.000800
163
+ 1272 0.000700
164
+ 1273 0.000700
165
+ 1274 0.000700
166
+ 1275 0.000600
167
+ 1276 0.000600
168
+ 1277 0.000600
169
+ 1278 0.000600
170
+ 1279 0.000500
171
+ 1280 0.000500
172
+ 1281 0.000500
173
+ 1282 0.000500
174
+ 1283 0.000500
175
+ 1284 0.000500
176
+ 1285 0.000500
177
+ 1286 0.000400
178
+ 1287 0.000400
179
+ 1288 0.000400
180
+ 1289 0.000400
181
+ 1290 0.000400
182
+ 1291 0.000400
183
+ 1292 0.000400
184
+ 1293 0.000400
185
+ 1294 0.000400
186
+ 1295 0.000400
187
+ 1296 0.000400
188
+ 1297 0.000300
189
+ 1298 0.000300
190
+ ```