Nice!
Thanks Joseph! I tried to do this merge myself but I got pretty terrible output from the resulting model. Did you test this out before uploading? I wonder what went wrong when I tried to do it...
You're welcome, Madison. 😁 What do you mean by terrible output? Its the base model, so its not trained for conversation?
Perhaps alplindale hadn't added the tokenizer from Mistral-7B-Instruct-v0.2 yet? 🤔
I just checked the checksums for alpindale/Mistral-7B-v0.2-hf's tokenizer.model with mistralai/Mistral-7B-Instruct-v0.2's tokenizer.model. And, they matched, so your issue could have been that you need to copy the tokenizer.model from mistralai/Mistral-7B-Instruct-v0.2's repo to make the model work. It's great to hear from you by the way. 😁
You're welcome, Madison. 😁 What do you mean by terrible output? Its the base model, so its not trained for conversation?
I mean like I tried a handful of basic text completions like "Hello, my name is" and the resulting paragraphs all had nonsense repeated words such as "I am a software developer with a passion for programming and programming" (I made this example up but that was the exact sort of issue I kept seeing). I dunno why. Did you do any sort of tests like that before uploading? Just curious. Maybe I'll download this myself and see. Hopefully it works!
I'm getting coherent text from a Q4_K_M GGUF quant of it. Using Faraday, and the default prompt format, which I believe is just a variation of alpaca/vicuna. I think it's safe; it's not rambling on for infinity on repeated responses. And, it has an intelligent understanding of the conversation we were having - even going so far as to roleplay (RP) on its own - without any prompt whatsoever, albeit I was roleplaying before it started (but still). 😁
Ah man, I think I'm just gonna have to re-do those two models I posted. I used the mergekit recipe from WestLake-10.7B but that's not the same as what you did/what was described in the twitter post you linked. I'll re-do them when I can.
At least you caught the mistakes, and are proactively going to fix them. That's what's important - that, and the fact that you are willing to admit your mistakes and learn from them. I can't wait to download your models and try them once they are uploaded again with the fixes applied. 😁
@Joseph717171 my repos of Starling-LM-10.7B-beta and Mistral-10.7B-Instruct-v0.2 have both been updated with the new, correct merges :)
Way to go, Madison!!! 🎉😁