Transformers
mpt
Composer
MosaicML
llm-foundry

Why not llama.cpp ?

#5
by lazyiitian - opened

Why can't this run on llama.cpp when it can run on others that use llama.cpp?
What's missing?

llama.cpp only supports Llama models. This is an MPT model - different model type, different code required to load it.

It's hoped that some time in the future - maybe in a month or so - llama.cpp will support more model types than just Llama, including MPT and Falcon. But for now, it doesn't.

But you can use other clients for this, like koboldcpp, LoLLMS-UI, ctransformers, etc.

Sign up or log in to comment