Letemoin commited on
Commit
c0b393f
1 Parent(s): e5d54f9

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +42 -0
README.md ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - jsfs11/Llama-3-RPMerge-8B-SLERP
4
+ - ehristoforu/llama-3-12b-instruct
5
+ library_name: transformers
6
+ tags:
7
+ - mergekit
8
+ - merge
9
+
10
+ ---
11
+ # merge
12
+
13
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
14
+
15
+ ## Merge Details
16
+ ### Merge Method
17
+
18
+ This model was merged using the passthrough merge method.
19
+
20
+ ### Models Merged
21
+
22
+ The following models were included in the merge:
23
+ * [jsfs11/Llama-3-RPMerge-8B-SLERP](https://huggingface.co/jsfs11/Llama-3-RPMerge-8B-SLERP)
24
+ * [ehristoforu/llama-3-12b-instruct](https://huggingface.co/ehristoforu/llama-3-12b-instruct)
25
+
26
+ ### Configuration
27
+
28
+ The following YAML configuration was used to produce this model:
29
+
30
+ ```yaml
31
+
32
+ slices:
33
+ - sources:
34
+ - model: ehristoforu/llama-3-12b-instruct
35
+ layer_range: [0, 49]
36
+ - sources:
37
+ - model: jsfs11/Llama-3-RPMerge-8B-SLERP
38
+ layer_range: [12, 32]
39
+ merge_method: passthrough
40
+ dtype: bfloat16
41
+
42
+ ```