base_model: | |
- udkai/Turdus | |
- flemmingmiguel/MBX-7B | |
tags: | |
- merge | |
- mergekit | |
- lazymergekit | |
- udkai/Turdus | |
- flemmingmiguel/MBX-7B | |
# Beast-Soul-new | |
Beast-Soul-new is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing): | |
* [udkai/Turdus](https://huggingface.co/udkai/Turdus) | |
* [flemmingmiguel/MBX-7B](https://huggingface.co/flemmingmiguel/MBX-7B) | |
## 🧩 Configuration | |
```yaml | |
slices: | |
- sources: | |
- model: udkai/Turdus | |
layer_range: [0,32] | |
- model: flemmingmiguel/MBX-7B | |
layer_range: [0,32] | |
merge_method: slerp | |
base_model: udkai/Turdus | |
parameters: | |
t: | |
- filter: self_attn | |
value: [0,0.5,0.3,0.7,1] | |
- fliter: mlp | |
value: [1,0.5,0.7,0.3,0] | |
- value: 0.5 | |
dtype: bfloat16 | |
``` | |
## 💻 Usage | |
```python | |
!pip install -qU transformers accelerate | |
from transformers import AutoTokenizer | |
import transformers | |
import torch | |
model = "llmahmad/Beast-Soul-new" | |
messages = [{"role": "user", "content": "What is a large language model?"}] | |
tokenizer = AutoTokenizer.from_pretrained(model) | |
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) | |
pipeline = transformers.pipeline( | |
"text-generation", | |
model=model, | |
torch_dtype=torch.float16, | |
device_map="auto", | |
) | |
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95) | |
print(outputs[0]["generated_text"]) | |
``` |