seba/llama-3.2-1B-instruct-coreml-ane

Example commmand to run:

python -m src.coreml_example --model-path-mf ./Llama-3.2-1B-4bits-MF.mlmodelc --model-path-head ./Llama-3.2-1B-HEAD-6Bits.mlmodelc/ --model-path-emb ./Llama-3.2-1B-EMB-16Bits.mlmodelc --prompt "Write a poem joke about Harry Potter" --max-tokens 400 --temp 1.2

You need to have set HF_TOKEN or provide on execution