aashish1904 commited on
Commit
0b1ea19
1 Parent(s): 8c46a22

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +760 -0
README.md ADDED
@@ -0,0 +1,760 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ ---
3
+
4
+ tags:
5
+ - merge
6
+ - mergekit
7
+ - lazymergekit
8
+ - not-for-all-audiences
9
+ - nsfw
10
+ - rp
11
+ - roleplay
12
+ - role-play
13
+ license: llama3
14
+ language:
15
+ - en
16
+ pipeline_tag: text-generation
17
+ base_model:
18
+ - Sao10K/L3-8B-Stheno-v3.2
19
+ - ChaoticNeutrals/Poppy_Porpoise-1.0-L3-8B
20
+ - Nitral-AI/Hathor_Stable-v0.2-L3-8B
21
+ - NeverSleep/Llama-3-Lumimaid-8B-v0.1-OAS
22
+ - Hastagaras/Jamet-8B-L3-MK.V-Blackroot
23
+ - openlynn/Llama-3-Soliloquy-8B-v2
24
+ - NousResearch/Meta-Llama-3-8B-Instruct
25
+ - turboderp/llama3-turbcat-instruct-8b
26
+ - VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
27
+ - TIGER-Lab/MAmmoTH2-8B-Plus
28
+ - jondurbin/bagel-8b-v1.0
29
+ - abacusai/Llama-3-Smaug-8B
30
+ - failspy/Meta-Llama-3-8B-Instruct-abliterated-v3
31
+ - AwanLLM/Awanllm-Llama-3-8B-Cumulus-v1.0
32
+ - lodrick-the-lafted/Limon-8B
33
+ - vicgalle/Configurable-Llama-3-8B-v0.3
34
+ - Undi95/Llama3-Unholy-8B-OAS
35
+ - Undi95/Unholy-8B-DPO-OAS
36
+ - WhiteRabbitNeo/Llama-3-WhiteRabbitNeo-8B-v2.0
37
+ - migtissera/Tess-2.0-Llama-3-8B
38
+ - defog/llama-3-sqlcoder-8b
39
+ - HPAI-BSC/Llama3-Aloe-8B-Alpha
40
+ - maldv/llama-3-fantasy-writer-8b
41
+ - lodrick-the-lafted/Olethros-8B
42
+ - Magpie-Align/Llama-3-8B-ShareGPT-112K
43
+ - Magpie-Align/Llama-3-8B-WildChat
44
+ - Magpie-Align/Llama-3-8B-Tulu-330K
45
+ - Magpie-Align/Llama-3-8B-OpenHermes-243K
46
+ - Magpie-Align/Llama-3-8B-WizardLM-196K
47
+ - Magpie-Align/Llama-3-8B-Ultrachat-200K
48
+ - refuelai/Llama-3-Refueled
49
+ - Danielbrdz/Barcenas-Llama3-8b-ORPO
50
+ - migtissera/Llama-3-8B-Synthia-v3.5
51
+ - chujiezheng/Llama-3-Instruct-8B-SimPO-ExPO
52
+ - chujiezheng/LLaMA3-iterative-DPO-final-ExPO
53
+ - chargoddard/prometheus-2-llama-3-8b
54
+
55
+ ---
56
+
57
+ ![](https://cdn.discordapp.com/attachments/791342238541152306/1264099835221381251/image.png?ex=669ca436&is=669b52b6&hm=129f56187c31e1ed22cbd1bcdbc677a2baeea5090761d2f1a458c8b1ec7cca4b&)
58
+
59
+ # QuantFactory/L3-Deluxe-Scrambled-Eggs-On-Toast-8B-GGUF
60
+ This is quantized version of [Casual-Autopsy/L3-Deluxe-Scrambled-Eggs-On-Toast-8B](https://huggingface.co/Casual-Autopsy/L3-Deluxe-Scrambled-Eggs-On-Toast-8B) created using llama.cpp
61
+
62
+ # Original Model Card
63
+
64
+
65
+ # L3-Deluxe-Scrambled-Eggs-On-Toast-8B
66
+
67
+ **L3-Deluxe-Scrambled-Eggs-On-Toast-8B** is a role-play model merger **using 36 models** that was made **in 23 merging steps.**
68
+
69
+ The goal is to create both a creative and smart model by using gradients.
70
+ Each model has their own section in the gradient where they have a larger weight to promote intelligence whereas the rest of the models in the section of the gradient have a small weight to promote creativity.
71
+
72
+ The following models were used as inspiration:
73
+ * [grimjim/kunoichi-lemon-royale-v3-32K-7B](https://huggingface.co/grimjim/kunoichi-lemon-royale-v3-32K-7B)
74
+ * [invisietch/EtherealRainbow-v0.3-8B](https://huggingface.co/invisietch/EtherealRainbow-v0.3-8B)
75
+ * [PJMixers/LLaMa-3-CursedStock-v2.0-8B](https://huggingface.co/PJMixers/LLaMa-3-CursedStock-v2.0-8B)
76
+
77
+ ## Instruct Format
78
+
79
+ Llama 3
80
+
81
+ ## Settings/Presets
82
+
83
+ ### Instruct/Context
84
+
85
+ Virt-io's [SillyTavern Presets](https://huggingface.co/Virt-io/SillyTavern-Presets/tree/main/Prompts/LLAMA-3/v1.9) is recommended.
86
+
87
+ ### Sampler Settings
88
+
89
+ Here are the current recommended settings for **more creativity**.
90
+ ```
91
+ Top K: 60
92
+ Min P: 0.035
93
+ Rep Pen: 1.05
94
+ Rep Pen Range: 2048
95
+ Pres Pen: 0.15
96
+ Smoothing Factor: 0.25
97
+ Dyna Temp:
98
+ Min Temp: 0.75
99
+ Max Temp: 1.5
100
+ Expo: 0.85
101
+ ```
102
+
103
+ No known presets for **more adherencey**. Please recommend some if you can!
104
+
105
+ ## Quants
106
+
107
+ Weighted quants by:
108
+ - [mradermacher](https://huggingface.co/mradermacher/L3-Deluxe-Scrambled-Eggs-On-Toast-8B-i1-GGUF)
109
+
110
+ Static quants by:
111
+ - [mradermacher](https://huggingface.co/mradermacher/L3-Deluxe-Scrambled-Eggs-On-Toast-8B-GGUF)
112
+
113
+ # Secret Sauce
114
+
115
+ ## Models Used
116
+
117
+ L3-Scrambled-Eggs-On-Toast-8B is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
118
+ * [Sao10K/L3-8B-Stheno-v3.2](https://huggingface.co/Sao10K/L3-8B-Stheno-v3.2)
119
+ * [ChaoticNeutrals/Poppy_Porpoise-1.0-L3-8B](https://huggingface.co/ChaoticNeutrals/Poppy_Porpoise-1.0-L3-8B)
120
+ * [Nitral-AI/Hathor_Stable-v0.2-L3-8B](https://huggingface.co/Nitral-AI/Hathor_Stable-v0.2-L3-8B)
121
+ * [NeverSleep/Llama-3-Lumimaid-8B-v0.1-OAS](https://huggingface.co/NeverSleep/Llama-3-Lumimaid-8B-v0.1-OAS)
122
+ * [Hastagaras/Jamet-8B-L3-MK.V-Blackroot](https://huggingface.co/Hastagaras/Jamet-8B-L3-MK.V-Blackroot)
123
+ * [openlynn/Llama-3-Soliloquy-8B-v2](https://huggingface.co/openlynn/Llama-3-Soliloquy-8B-v2)
124
+ * [NousResearch/Meta-Llama-3-8B-Instruct](https://huggingface.co/NousResearch/Meta-Llama-3-8B-Instruct)
125
+ * [turboderp/llama3-turbcat-instruct-8b](https://huggingface.co/turboderp/llama3-turbcat-instruct-8b)
126
+ * [VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct](https://huggingface.co/VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct)
127
+ * [TIGER-Lab/MAmmoTH2-8B-Plus](https://huggingface.co/TIGER-Lab/MAmmoTH2-8B-Plus)
128
+ * [jondurbin/bagel-8b-v1.0](https://huggingface.co/jondurbin/bagel-8b-v1.0)
129
+ * [abacusai/Llama-3-Smaug-8B](https://huggingface.co/abacusai/Llama-3-Smaug-8B)
130
+ * [failspy/Meta-Llama-3-8B-Instruct-abliterated-v3](https://huggingface.co/failspy/Meta-Llama-3-8B-Instruct-abliterated-v3)
131
+ * [AwanLLM/Awanllm-Llama-3-8B-Cumulus-v1.0](https://huggingface.co/AwanLLM/Awanllm-Llama-3-8B-Cumulus-v1.0)
132
+ * [lodrick-the-lafted/Limon-8B](https://huggingface.co/lodrick-the-lafted/Limon-8B)
133
+ * [vicgalle/Configurable-Llama-3-8B-v0.3](https://huggingface.co/vicgalle/Configurable-Llama-3-8B-v0.3)
134
+ * [Undi95/Llama3-Unholy-8B-OAS](https://huggingface.co/Undi95/Llama3-Unholy-8B-OAS)
135
+ * [Undi95/Unholy-8B-DPO-OAS](https://huggingface.co/Undi95/Unholy-8B-DPO-OAS)
136
+ * [WhiteRabbitNeo/Llama-3-WhiteRabbitNeo-8B-v2.0](https://huggingface.co/WhiteRabbitNeo/Llama-3-WhiteRabbitNeo-8B-v2.0)
137
+ * [migtissera/Tess-2.0-Llama-3-8B](https://huggingface.co/migtissera/Tess-2.0-Llama-3-8B)
138
+ * [defog/llama-3-sqlcoder-8b](https://huggingface.co/defog/llama-3-sqlcoder-8b)
139
+ * [HPAI-BSC/Llama3-Aloe-8B-Alpha](https://huggingface.co/HPAI-BSC/Llama3-Aloe-8B-Alpha)
140
+ * [maldv/llama-3-fantasy-writer-8b](https://huggingface.co/maldv/llama-3-fantasy-writer-8b)
141
+ * [lodrick-the-lafted/Olethros-8B](https://huggingface.co/lodrick-the-lafted/Olethros-8B)
142
+ * [Magpie-Align/Llama-3-8B-ShareGPT-112K](https://huggingface.co/Magpie-Align/Llama-3-8B-ShareGPT-112K)
143
+ * [Magpie-Align/Llama-3-8B-WildChat](https://huggingface.co/Magpie-Align/Llama-3-8B-WildChat)
144
+ * [Magpie-Align/Llama-3-8B-Tulu-330K](https://huggingface.co/Magpie-Align/Llama-3-8B-Tulu-330K)
145
+ * [Magpie-Align/Llama-3-8B-OpenHermes-243K](https://huggingface.co/Magpie-Align/Llama-3-8B-OpenHermes-243K)
146
+ * [Magpie-Align/Llama-3-8B-WizardLM-196K](https://huggingface.co/Magpie-Align/Llama-3-8B-WizardLM-196K)
147
+ * [Magpie-Align/Llama-3-8B-Ultrachat-200K](https://huggingface.co/Magpie-Align/Llama-3-8B-Ultrachat-200K)
148
+ * [refuelai/Llama-3-Refueled](https://huggingface.co/refuelai/Llama-3-Refueled)
149
+ * [Danielbrdz/Barcenas-Llama3-8b-ORPO](https://huggingface.co/Danielbrdz/Barcenas-Llama3-8b-ORPO)
150
+ * [migtissera/Llama-3-8B-Synthia-v3.5](https://huggingface.co/migtissera/Llama-3-8B-Synthia-v3.5)
151
+ * [chujiezheng/Llama-3-Instruct-8B-SimPO-ExPO](https://huggingface.co/chujiezheng/Llama-3-Instruct-8B-SimPO-ExPO)
152
+ * [chujiezheng/LLaMA3-iterative-DPO-final-ExPO](https://huggingface.co/chujiezheng/LLaMA3-iterative-DPO-final-ExPO)
153
+ * [chargoddard/prometheus-2-llama-3-8b](https://huggingface.co/chargoddard/prometheus-2-llama-3-8b)
154
+
155
+ ## YAML Configs Used
156
+
157
+ The following YAML configs were used to make this mode
158
+
159
+ ### Eggs-and-Bread-RP-pt.1
160
+
161
+ ```yaml
162
+ models:
163
+ - model: Sao10K/L3-8B-Stheno-v3.2
164
+ - model: ChaoticNeutrals/Poppy_Porpoise-1.0-L3-8B
165
+ parameters:
166
+ density: 0.5
167
+ weight: [0.33, 0.0825, 0.0825, 0.0825, 0.0825]
168
+ - model: Nitral-AI/Hathor_Stable-v0.2-L3-8B
169
+ parameters:
170
+ density: 0.5
171
+ weight: [0.0825, 0.33, 0.0825, 0.0825, 0.0825]
172
+ - model: NeverSleep/Llama-3-Lumimaid-8B-v0.1-OAS
173
+ parameters:
174
+ density: 0.5
175
+ weight: [0.0825, 0.0825, 0.33, 0.0825, 0.0825]
176
+ - model: Hastagaras/Jamet-8B-L3-MK.V-Blackroot
177
+ parameters:
178
+ density: 0.5
179
+ weight: [0.0825, 0.0825, 0.0825, 0.33, 0.0825]
180
+ - model: openlynn/Llama-3-Soliloquy-8B-v2
181
+ parameters:
182
+ density: 0.5
183
+ weight: [0.0825, 0.0825, 0.0825, 0.0825, 0.33]
184
+ merge_method: dare_ties
185
+ base_model: Sao10K/L3-8B-Stheno-v3.2
186
+ parameters:
187
+ normalize: false
188
+ int8_mask: true
189
+ dtype: bfloat16
190
+ ```
191
+
192
+ ### Eggs-and-Bread-RP-pt.2
193
+
194
+ ```yaml
195
+ models:
196
+ - model: Sao10K/L3-8B-Stheno-v3.2
197
+ - model: ChaoticNeutrals/Poppy_Porpoise-1.0-L3-8B
198
+ parameters:
199
+ gamma: 0.01
200
+ density: 0.9
201
+ weight: [0.0825, 0.0825, 0.0825, 0.0825, 0.33]
202
+ - model: Nitral-AI/Hathor_Stable-v0.2-L3-8B
203
+ parameters:
204
+ gamma: 0.01
205
+ density: 0.9
206
+ weight: [0.0825, 0.0825, 0.0825, 0.33, 0.0825]
207
+ - model: NeverSleep/Llama-3-Lumimaid-8B-v0.1-OAS
208
+ parameters:
209
+ gamma: 0.01
210
+ density: 0.9
211
+ weight: [0.0825, 0.0825, 0.33, 0.0825, 0.0825]
212
+ - model: Hastagaras/Jamet-8B-L3-MK.V-Blackroot
213
+ parameters:
214
+ gamma: 0.01
215
+ density: 0.9
216
+ weight: [0.0825, 0.33, 0.0825, 0.0825, 0.0825]
217
+ - model: openlynn/Llama-3-Soliloquy-8B-v2
218
+ parameters:
219
+ gamma: 0.01
220
+ density: 0.9
221
+ weight: [0.33, 0.0825, 0.0825, 0.0825, 0.0825]
222
+ merge_method: breadcrumbs_ties
223
+ base_model: Sao10K/L3-8B-Stheno-v3.2
224
+ parameters:
225
+ normalize: false
226
+ int8_mask: true
227
+ dtype: bfloat16
228
+ ```
229
+
230
+ ### Egg-and-Bread-RP
231
+
232
+ ```yaml
233
+ models:
234
+ - model: Casual-Autopsy/Eggs-and-Bread-RP-pt.1
235
+ - model: Casual-Autopsy/Eggs-and-Bread-RP-pt.2
236
+ merge_method: slerp
237
+ base_model: Casual-Autopsy/Eggs-and-Bread-RP-pt.1
238
+ parameters:
239
+ t:
240
+ - filter: self_attn
241
+ value: [0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5]
242
+ - filter: mlp
243
+ value: [0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5]
244
+ - value: 0.5
245
+ dtype: bfloat16
246
+ ```
247
+
248
+ ### Eggs-and-Bread-IQ-pt.1
249
+
250
+ ```yaml
251
+ models:
252
+ - model: NousResearch/Meta-Llama-3-8B-Instruct
253
+ - model: turboderp/llama3-turbcat-instruct-8b
254
+ parameters:
255
+ density: 0.5
256
+ weight: [0.33, 0.0825, 0.0825, 0.0825, 0.0825]
257
+ - model: VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
258
+ parameters:
259
+ density: 0.5
260
+ weight: [0.0825, 0.33, 0.0825, 0.0825, 0.0825]
261
+ - model: TIGER-Lab/MAmmoTH2-8B-Plus
262
+ parameters:
263
+ density: 0.5
264
+ weight: [0.0825, 0.0825, 0.33, 0.0825, 0.0825]
265
+ - model: jondurbin/bagel-8b-v1.0
266
+ parameters:
267
+ density: 0.5
268
+ weight: [0.0825, 0.0825, 0.0825, 0.33, 0.0825]
269
+ - model: abacusai/Llama-3-Smaug-8B
270
+ parameters:
271
+ density: 0.5
272
+ weight: [0.0825, 0.0825, 0.0825, 0.0825, 0.33]
273
+ merge_method: dare_ties
274
+ base_model: NousResearch/Meta-Llama-3-8B-Instruct
275
+ parameters:
276
+ normalize: false
277
+ int8_mask: true
278
+ dtype: bfloat16
279
+ ```
280
+
281
+ ### Eggs-and-Bread-IQ-pt.2
282
+
283
+ ```yaml
284
+ models:
285
+ - model: NousResearch/Meta-Llama-3-8B-Instruct
286
+ - model: turboderp/llama3-turbcat-instruct-8b
287
+ parameters:
288
+ gamma: 0.01
289
+ density: 0.9
290
+ weight: [0.0825, 0.0825, 0.0825, 0.0825, 0.33]
291
+ - model: VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
292
+ parameters:
293
+ gamma: 0.01
294
+ density: 0.9
295
+ weight: [0.0825, 0.0825, 0.0825, 0.33, 0.0825]
296
+ - model: TIGER-Lab/MAmmoTH2-8B-Plus
297
+ parameters:
298
+ gamma: 0.01
299
+ density: 0.9
300
+ weight: [0.0825, 0.0825, 0.33, 0.0825, 0.0825]
301
+ - model: jondurbin/bagel-8b-v1.0
302
+ parameters:
303
+ gamma: 0.01
304
+ density: 0.9
305
+ weight: [0.0825, 0.33, 0.0825, 0.0825, 0.0825]
306
+ - model: abacusai/Llama-3-Smaug-8B
307
+ parameters:
308
+ gamma: 0.01
309
+ density: 0.9
310
+ weight: [0.33, 0.0825, 0.0825, 0.0825, 0.0825]
311
+ merge_method: breadcrumbs_ties
312
+ base_model: NousResearch/Meta-Llama-3-8B-Instruct
313
+ parameters:
314
+ normalize: false
315
+ int8_mask: true
316
+ dtype: bfloat16
317
+ ```
318
+
319
+ ### Eggs-and-Bread-IQ
320
+
321
+ ```yaml
322
+ models:
323
+ - model: Casual-Autopsy/Eggs-and-Bread-IQ-pt.1
324
+ - model: Casual-Autopsy/Eggs-and-Bread-IQ-pt.2
325
+ merge_method: slerp
326
+ base_model: Casual-Autopsy/Eggs-and-Bread-IQ-pt.1
327
+ parameters:
328
+ t:
329
+ - filter: self_attn
330
+ value: [0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5]
331
+ - filter: mlp
332
+ value: [0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5]
333
+ - value: 0.5
334
+ dtype: bfloat16
335
+ ```
336
+
337
+ ### Eggs-and-Bread-Uncen-pt.1
338
+
339
+ ```yaml
340
+ models:
341
+ - model: failspy/Meta-Llama-3-8B-Instruct-abliterated-v3
342
+ - model: AwanLLM/Awanllm-Llama-3-8B-Cumulus-v1.0
343
+ parameters:
344
+ density: 0.5
345
+ weight: [0.33, 0.0825, 0.0825, 0.0825, 0.0825]
346
+ - model: lodrick-the-lafted/Limon-8B
347
+ parameters:
348
+ density: 0.5
349
+ weight: [0.0825, 0.33, 0.0825, 0.0825, 0.0825]
350
+ - model: vicgalle/Configurable-Llama-3-8B-v0.3
351
+ parameters:
352
+ density: 0.5
353
+ weight: [0.0825, 0.0825, 0.33, 0.0825, 0.0825]
354
+ - model: Undi95/Llama3-Unholy-8B-OAS
355
+ parameters:
356
+ density: 0.5
357
+ weight: [0.0825, 0.0825, 0.0825, 0.33, 0.0825]
358
+ - model: Undi95/Unholy-8B-DPO-OAS
359
+ parameters:
360
+ density: 0.5
361
+ weight: [0.0825, 0.0825, 0.0825, 0.0825, 0.33]
362
+ merge_method: dare_ties
363
+ base_model: failspy/Meta-Llama-3-8B-Instruct-abliterated-v3
364
+ parameters:
365
+ normalize: false
366
+ int8_mask: true
367
+ dtype: bfloat16
368
+ ```
369
+
370
+ ### Eggs-and-Bread-Uncen-pt.2
371
+
372
+ ```yaml
373
+ models:
374
+ - model: failspy/Meta-Llama-3-8B-Instruct-abliterated-v3
375
+ - model: AwanLLM/Awanllm-Llama-3-8B-Cumulus-v1.0
376
+ parameters:
377
+ gamma: 0.01
378
+ density: 0.9
379
+ weight: [0.0825, 0.0825, 0.0825, 0.0825, 0.33]
380
+ - model: lodrick-the-lafted/Limon-8B
381
+ parameters:
382
+ gamma: 0.01
383
+ density: 0.9
384
+ weight: [0.0825, 0.0825, 0.0825, 0.33, 0.0825]
385
+ - model: vicgalle/Configurable-Llama-3-8B-v0.3
386
+ parameters:
387
+ gamma: 0.01
388
+ density: 0.9
389
+ weight: [0.0825, 0.0825, 0.33, 0.0825, 0.0825]
390
+ - model: Undi95/Llama3-Unholy-8B-OAS
391
+ parameters:
392
+ gamma: 0.01
393
+ density: 0.9
394
+ weight: [0.0825, 0.33, 0.0825, 0.0825, 0.0825]
395
+ - model: Undi95/Unholy-8B-DPO-OAS
396
+ parameters:
397
+ gamma: 0.01
398
+ density: 0.9
399
+ weight: [0.33, 0.0825, 0.0825, 0.0825, 0.0825]
400
+ merge_method: breadcrumbs_ties
401
+ base_model: failspy/Meta-Llama-3-8B-Instruct-abliterated-v3
402
+ parameters:
403
+ normalize: false
404
+ int8_mask: true
405
+ dtype: bfloat16
406
+ ```
407
+
408
+ ### Eggs-and-Bread-Uncen
409
+
410
+ ```yaml
411
+ models:
412
+ - model: Casual-Autopsy/Eggs-and-Bread-Uncen-pt.1
413
+ - model: Casual-Autopsy/Eggs-and-Bread-Uncen-pt.2
414
+ merge_method: slerp
415
+ base_model: Casual-Autopsy/Eggs-and-Bread-Uncen-pt.1
416
+ parameters:
417
+ t:
418
+ - filter: self_attn
419
+ value: [0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5]
420
+ - filter: mlp
421
+ value: [0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5]
422
+ - value: 0.5
423
+ dtype: bfloat16
424
+ ```
425
+
426
+ ### Scrambled-Eggs-On-Toast-1
427
+
428
+ ```yaml
429
+ models:
430
+ - model: Casual-Autopsy/Eggs-and-Bread-RP
431
+ - model: Casual-Autopsy/Eggs-and-Bread-Uncen
432
+ merge_method: slerp
433
+ base_model: Casual-Autopsy/Eggs-and-Bread-RP
434
+ parameters:
435
+ t:
436
+ - value: [0.1, 0.15, 0.2, 0.4, 0.6, 0.4, 0.2, 0.15, 0.1]
437
+ dtype: bfloat16
438
+ ```
439
+
440
+ ### L3-Scrambled-Eggs-On-Toast-8B
441
+
442
+ ```yaml
443
+ models:
444
+ - model: Casual-Autopsy/Scrambled-Eggs-On-Toast-1
445
+ - model: Casual-Autopsy/Eggs-and-Bread-IQ
446
+ merge_method: slerp
447
+ base_model: Casual-Autopsy/Scrambled-Eggs-On-Toast-1
448
+ parameters:
449
+ t:
450
+ - value: [0.7, 0.5, 0.3, 0.25, 0.2, 0.25, 0.3, 0.5, 0.7]
451
+ dtype: bfloat16
452
+ ```
453
+ ### Eggs-and-Bread-Misc1-pt.1
454
+
455
+ ```yaml
456
+ models:
457
+ - model: WhiteRabbitNeo/Llama-3-WhiteRabbitNeo-8B-v2.0
458
+ - model: migtissera/Tess-2.0-Llama-3-8B
459
+ parameters:
460
+ density: 0.5
461
+ weight: [0.33, 0.0825, 0.0825, 0.0825, 0.0825]
462
+ - model: defog/llama-3-sqlcoder-8b
463
+ parameters:
464
+ density: 0.5
465
+ weight: [0.0825, 0.33, 0.0825, 0.0825, 0.0825]
466
+ - model: HPAI-BSC/Llama3-Aloe-8B-Alpha
467
+ parameters:
468
+ density: 0.5
469
+ weight: [0.0825, 0.0825, 0.33, 0.0825, 0.0825]
470
+ - model: maldv/llama-3-fantasy-writer-8b
471
+ parameters:
472
+ density: 0.5
473
+ weight: [0.0825, 0.0825, 0.0825, 0.33, 0.0825]
474
+ - model: lodrick-the-lafted/Olethros-8B
475
+ parameters:
476
+ density: 0.5
477
+ weight: [0.0825, 0.0825, 0.0825, 0.0825, 0.33]
478
+ merge_method: dare_ties
479
+ base_model: WhiteRabbitNeo/Llama-3-WhiteRabbitNeo-8B-v2.0
480
+ parameters:
481
+ normalize: false
482
+ int8_mask: true
483
+ dtype: bfloat16
484
+ ```
485
+
486
+ ### Eggs-and-Bread-Misc1-pt.2
487
+
488
+ ```yaml
489
+ models:
490
+ - model: WhiteRabbitNeo/Llama-3-WhiteRabbitNeo-8B-v2.0
491
+ - model: migtissera/Tess-2.0-Llama-3-8B
492
+ parameters:
493
+ gamma: 0.01
494
+ density: 0.9
495
+ weight: [0.0825, 0.0825, 0.0825, 0.0825, 0.33]
496
+ - model: defog/llama-3-sqlcoder-8b
497
+ parameters:
498
+ gamma: 0.01
499
+ density: 0.9
500
+ weight: [0.0825, 0.0825, 0.0825, 0.33, 0.0825]
501
+ - model: HPAI-BSC/Llama3-Aloe-8B-Alpha
502
+ parameters:
503
+ gamma: 0.01
504
+ density: 0.9
505
+ weight: [0.0825, 0.0825, 0.33, 0.0825, 0.0825]
506
+ - model: maldv/llama-3-fantasy-writer-8b
507
+ parameters:
508
+ gamma: 0.01
509
+ density: 0.9
510
+ weight: [0.0825, 0.33, 0.0825, 0.0825, 0.0825]
511
+ - model: lodrick-the-lafted/Olethros-8B
512
+ parameters:
513
+ gamma: 0.01
514
+ density: 0.9
515
+ weight: [0.33, 0.0825, 0.0825, 0.0825, 0.0825]
516
+ merge_method: breadcrumbs_ties
517
+ base_model: WhiteRabbitNeo/Llama-3-WhiteRabbitNeo-8B-v2.0
518
+ parameters:
519
+ normalize: false
520
+ int8_mask: true
521
+ dtype: bfloat16
522
+ ```
523
+
524
+ ### Eggs-and-Bread-Misc1
525
+
526
+ ```yaml
527
+ models:
528
+ - model: Casual-Autopsy/Eggs-and-Bread-Misc1-pt.1
529
+ - model: Casual-Autopsy/Eggs-and-Bread-Misc1-pt.2
530
+ merge_method: slerp
531
+ base_model: Casual-Autopsy/Eggs-and-Bread-Misc1-pt.1
532
+ parameters:
533
+ t:
534
+ - filter: self_attn
535
+ value: [0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5]
536
+ - filter: mlp
537
+ value: [0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5]
538
+ - value: 0.5
539
+ dtype: bfloat16
540
+ ```
541
+
542
+ ### Eggs-and-Bread-FFT-pt.1
543
+
544
+ ```yaml
545
+ models:
546
+ - model: Magpie-Align/Llama-3-8B-ShareGPT-112K
547
+ - model: Magpie-Align/Llama-3-8B-WildChat
548
+ parameters:
549
+ density: 0.5
550
+ weight: [0.33, 0.0825, 0.0825, 0.0825, 0.0825]
551
+ - model: Magpie-Align/Llama-3-8B-Tulu-330K
552
+ parameters:
553
+ density: 0.5
554
+ weight: [0.0825, 0.33, 0.0825, 0.0825, 0.0825]
555
+ - model: Magpie-Align/Llama-3-8B-OpenHermes-243K
556
+ parameters:
557
+ density: 0.5
558
+ weight: [0.0825, 0.0825, 0.33, 0.0825, 0.0825]
559
+ - model: Magpie-Align/Llama-3-8B-WizardLM-196K
560
+ parameters:
561
+ density: 0.5
562
+ weight: [0.0825, 0.0825, 0.0825, 0.33, 0.0825]
563
+ - model: Magpie-Align/Llama-3-8B-Ultrachat-200K
564
+ parameters:
565
+ density: 0.5
566
+ weight: [0.0825, 0.0825, 0.0825, 0.0825, 0.33]
567
+ merge_method: dare_ties
568
+ base_model: Magpie-Align/Llama-3-8B-ShareGPT-112K
569
+ parameters:
570
+ normalize: false
571
+ int8_mask: true
572
+ dtype: bfloat16
573
+ ```
574
+
575
+ ### Eggs-and-Bread-FFT-pt.2
576
+
577
+ ```yaml
578
+ models:
579
+ - model: Magpie-Align/Llama-3-8B-ShareGPT-112K
580
+ - model: Magpie-Align/Llama-3-8B-WildChat
581
+ parameters:
582
+ gamma: 0.01
583
+ density: 0.9
584
+ weight: [0.0825, 0.0825, 0.0825, 0.0825, 0.33]
585
+ - model: Magpie-Align/Llama-3-8B-Tulu-330K
586
+ parameters:
587
+ gamma: 0.01
588
+ density: 0.9
589
+ weight: [0.0825, 0.0825, 0.0825, 0.33, 0.0825]
590
+ - model: Magpie-Align/Llama-3-8B-OpenHermes-243K
591
+ parameters:
592
+ gamma: 0.01
593
+ density: 0.9
594
+ weight: [0.0825, 0.0825, 0.33, 0.0825, 0.0825]
595
+ - model: Magpie-Align/Llama-3-8B-WizardLM-196K
596
+ parameters:
597
+ gamma: 0.01
598
+ density: 0.9
599
+ weight: [0.0825, 0.33, 0.0825, 0.0825, 0.0825]
600
+ - model: Magpie-Align/Llama-3-8B-Ultrachat-200K
601
+ parameters:
602
+ gamma: 0.01
603
+ density: 0.9
604
+ weight: [0.33, 0.0825, 0.0825, 0.0825, 0.0825]
605
+ merge_method: breadcrumbs_ties
606
+ base_model: Magpie-Align/Llama-3-8B-ShareGPT-112K
607
+ parameters:
608
+ normalize: false
609
+ int8_mask: true
610
+ dtype: bfloat16
611
+ ```
612
+
613
+ ### Eggs-and-Bread-FFT
614
+
615
+ ```yaml
616
+ models:
617
+ - model: Casual-Autopsy/Eggs-and-Bread-FFT-pt.1
618
+ - model: Casual-Autopsy/Eggs-and-Bread-FFT-pt.2
619
+ merge_method: slerp
620
+ base_model: Casual-Autopsy/Eggs-and-Bread-FFT-pt.1
621
+ parameters:
622
+ t:
623
+ - filter: self_attn
624
+ value: [0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5]
625
+ - filter: mlp
626
+ value: [0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5]
627
+ - value: 0.5
628
+ dtype: bfloat16
629
+ ```
630
+
631
+ ### Eggs-and-Bread-Misc2-pt.1
632
+
633
+ ```yaml
634
+ models:
635
+ - model: refuelai/Llama-3-Refueled
636
+ - model: Danielbrdz/Barcenas-Llama3-8b-ORPO
637
+ parameters:
638
+ density: 0.5
639
+ weight: [0.33, 0.0825, 0.0825, 0.0825, 0.0825]
640
+ - model: migtissera/Llama-3-8B-Synthia-v3.5
641
+ parameters:
642
+ density: 0.5
643
+ weight: [0.0825, 0.33, 0.0825, 0.0825, 0.0825]
644
+ - model: chujiezheng/Llama-3-Instruct-8B-SimPO-ExPO
645
+ parameters:
646
+ density: 0.5
647
+ weight: [0.0825, 0.0825, 0.33, 0.0825, 0.0825]
648
+ - model: chujiezheng/LLaMA3-iterative-DPO-final-ExPO
649
+ parameters:
650
+ density: 0.5
651
+ weight: [0.0825, 0.0825, 0.0825, 0.33, 0.0825]
652
+ - model: chargoddard/prometheus-2-llama-3-8b
653
+ parameters:
654
+ density: 0.5
655
+ weight: [0.0825, 0.0825, 0.0825, 0.0825, 0.33]
656
+ merge_method: dare_ties
657
+ base_model: refuelai/Llama-3-Refueled
658
+ parameters:
659
+ normalize: false
660
+ int8_mask: true
661
+ dtype: bfloat16
662
+ ```
663
+
664
+ ### Eggs-and-Bread-Misc2-pt.2
665
+
666
+ ```yaml
667
+ models:
668
+ - model: refuelai/Llama-3-Refueled
669
+ - model: Danielbrdz/Barcenas-Llama3-8b-ORPO
670
+ parameters:
671
+ gamma: 0.01
672
+ density: 0.9
673
+ weight: [0.0825, 0.0825, 0.0825, 0.0825, 0.33]
674
+ - model: migtissera/Llama-3-8B-Synthia-v3.5
675
+ parameters:
676
+ gamma: 0.01
677
+ density: 0.9
678
+ weight: [0.0825, 0.0825, 0.0825, 0.33, 0.0825]
679
+ - model: chujiezheng/Llama-3-Instruct-8B-SimPO-ExPO
680
+ parameters:
681
+ gamma: 0.01
682
+ density: 0.9
683
+ weight: [0.0825, 0.0825, 0.33, 0.0825, 0.0825]
684
+ - model: chujiezheng/LLaMA3-iterative-DPO-final-ExPO
685
+ parameters:
686
+ gamma: 0.01
687
+ density: 0.9
688
+ weight: [0.0825, 0.33, 0.0825, 0.0825, 0.0825]
689
+ - model: chargoddard/prometheus-2-llama-3-8b
690
+ parameters:
691
+ gamma: 0.01
692
+ density: 0.9
693
+ weight: [0.33, 0.0825, 0.0825, 0.0825, 0.0825]
694
+ merge_method: breadcrumbs_ties
695
+ base_model: refuelai/Llama-3-Refueled
696
+ parameters:
697
+ normalize: false
698
+ int8_mask: true
699
+ dtype: bfloat16
700
+ ```
701
+
702
+ ### Eggs-and-Bread-Misc2
703
+
704
+ ```yaml
705
+ models:
706
+ - model: Casual-Autopsy/Eggs-and-Bread-Misc2-pt.1
707
+ - model: Casual-Autopsy/Eggs-and-Bread-Misc2-pt.2
708
+ merge_method: slerp
709
+ base_model: Casual-Autopsy/Eggs-and-Bread-Misc2-pt.1
710
+ parameters:
711
+ t:
712
+ - filter: self_attn
713
+ value: [0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5]
714
+ - filter: mlp
715
+ value: [0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5]
716
+ - value: 0.5
717
+ dtype: bfloat16
718
+ ```
719
+
720
+ ### Scrambled-Eggs-On-Toast-2
721
+
722
+ ```yaml
723
+ models:
724
+ - model: Casual-Autopsy/Eggs-and-Bread-Misc1
725
+ - model: Casual-Autopsy/Eggs-and-Bread-Misc2
726
+ merge_method: slerp
727
+ base_model: Casual-Autopsy/Eggs-and-Bread-Misc1
728
+ parameters:
729
+ t:
730
+ - value: [0.1, 0.15, 0.2, 0.4, 0.6, 0.4, 0.2, 0.15, 0.1]
731
+ dtype: bfloat16
732
+ ```
733
+
734
+ ### Scrambled-Eggs-On-Toast-3
735
+
736
+ ```yaml
737
+ models:
738
+ - model: Casual-Autopsy/Scrambled-Eggs-On-Toast-2
739
+ - model: Casual-Autopsy/Eggs-and-Bread-FFT
740
+ merge_method: slerp
741
+ base_model: Casual-Autopsy/Scrambled-Eggs-On-Toast-2
742
+ parameters:
743
+ t:
744
+ - value: [0.7, 0.5, 0.3, 0.25, 0.2, 0.25, 0.3, 0.5, 0.7]
745
+ dtype: bfloat16
746
+ ```
747
+
748
+ ### L3-Deluxe-Scrambled-Eggs-On-Toast-8B
749
+
750
+ ```yaml
751
+ models:
752
+ - model: Casual-Autopsy/L3-Scrambled-Eggs-On-Toast-8B
753
+ - model: Casual-Autopsy/Scrambled-Eggs-On-Toast-3
754
+ merge_method: slerp
755
+ base_model: Casual-Autopsy/L3-Scrambled-Eggs-On-Toast-8B
756
+ parameters:
757
+ t:
758
+ - value: [0.2, 0.25, 0.3, 0.4, 0.3, 0.25, 0.2, 0.25, 0.3, 0.4, 0.3, 0.25, 0.2]
759
+ dtype: bfloat16
760
+ ```