cekal

/

mpt-7b-gpt4

like 1

hey @cekal ,

since this is a lora model.
could you share a colab notebook for 8 bit inference of the mpt-7b-gpt4 model?
that could work in free tier colab?
if not, would be great to have a reference notebook for inference nonetheless.