README.md

--- 
license: apache-2.0 
library_name: transformers 
tags: 
- mergekit 
- merge 
base_model: 
- InferenceIllusionist/Excalibur-7b 
- flammenai/flammen26-mistral-7B 
model-index: 
- name: flammen27-mistral-7B 
 results: 
 - task: 
 type: text-generation 
 name: Text Generation 
 dataset: 
 name: AI2 Reasoning Challenge (25-Shot) 
 type: ai2_arc 
 config: ARC-Challenge 
 split: test 
 args: 
 num_few_shot: 25 
 metrics: 
 - type: acc_norm 
 value: 69.8 
 name: normalized accuracy 
 source: 
 url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=flammenai/flammen27-mistral-7B 
 name: Open LLM Leaderboard 
 - task: 
 type: text-generation 
 name: Text Generation 
 dataset: 
 name: HellaSwag (10-Shot) 
 type: hellaswag 
 split: validation 
 args: 
 num_few_shot: 10 
 metrics: 
 - type: acc_norm 
 value: 87.39 
 name: normalized accuracy 
 source: 
 url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=flammenai/flammen27-mistral-7B 
 name: Open LLM Leaderboard 
 - task: 
 type: text-generation 
 name: Text Generation 
 dataset: 
 name: MMLU (5-Shot) 
 type: cais/mmlu 
 config: all 
 split: test 
 args: 
 num_few_shot: 5 
 metrics: 
 - type: acc 
 value: 65.01 
 name: accuracy 
 source: 
 url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=flammenai/flammen27-mistral-7B 
 name: Open LLM Leaderboard 
 - task: 
 type: text-generation 
 name: Text Generation 
 dataset: 
 name: TruthfulQA (0-shot) 
 type: truthful_qa 
 config: multiple_choice 
 split: validation 
 args: 
 num_few_shot: 0 
 metrics: 
 - type: mc2 
 value: 68.87 
 source: 
 url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=flammenai/flammen27-mistral-7B 
 name: Open LLM Leaderboard 
 - task: 
 type: text-generation 
 name: Text Generation 
 dataset: 
 name: Winogrande (5-shot) 
 type: winogrande 
 config: winogrande_xl 
 split: validation 
 args: 
 num_few_shot: 5 
 metrics: 
 - type: acc 
 value: 81.69 
 name: accuracy 
 source: 
 url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=flammenai/flammen27-mistral-7B 
 name: Open LLM Leaderboard 
 - task: 
 type: text-generation 
 name: Text Generation 
 dataset: 
 name: GSM8k (5-shot) 
 type: gsm8k 
 config: main 
 split: test 
 args: 
 num_few_shot: 5 
 metrics: 
 - type: acc 
 value: 67.48 
 name: accuracy 
 source: 
 url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=flammenai/flammen27-mistral-7B 
 name: Open LLM Leaderboard 
--- 
![image/png](https://huggingface.co/nbeerbower/flammen13X-mistral-7B/resolve/main/flammen13x.png) 
 
# flammen27-mistral-7B 
 
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). 
 
## Merge Details 
### Merge Method 
 
This model was merged using the SLERP merge method. 
 
### Models Merged 
 
The following models were included in the merge: 
* [InferenceIllusionist/Excalibur-7b](https://huggingface.co/InferenceIllusionist/Excalibur-7b) 
* [flammenai/flammen26-mistral-7B](https://huggingface.co/flammenai/flammen26-mistral-7B) 
 
### Configuration 
 
The following YAML configuration was used to produce this model: 
 
```yaml 
models: 
 - model: InferenceIllusionist/Excalibur-7b 
 layer_range: [0, 32] 
 - model: flammenai/flammen26-mistral-7B 
 layer_range: [0, 32] 
merge_method: slerp 
base_model: flammenai/flammen26-mistral-7B 
parameters: 
 t: 
 - filter: self_attn 
 value: [0, 0.5, 0.3, 0.7, 1] 
 - filter: mlp 
 value: [1, 0.5, 0.7, 0.3, 0] 
 - value: 0.5 
dtype: bfloat16 
 
``` 

# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_flammenai__flammen27-mistral-7B)

| Metric |Value|
|---------------------------------|----:|
|Avg. |73.37|
|AI2 Reasoning Challenge (25-Shot)|69.80|
|HellaSwag (10-Shot) |87.39|
|MMLU (5-Shot) |65.01|
|TruthfulQA (0-shot) |68.87|
|Winogrande (5-shot) |81.69|
|GSM8k (5-shot) |67.48|