Question about fine-tuning recipe?
#3
by
phanhoang
- opened
Hi, thanks for your great model!
I have a question about fine-tuning process.
What are the values of the two parameters, lora_rank and lora_alpha, when fine-tuning this model?
And did you adjust the freeze_vision_tower = false parameter during fine-tuning?
phanhoang
changed discussion status to
closed
phanhoang
changed discussion status to
open
freeze_vision_tower = false is courageous. Needs to trial & errors.
Large datasets likes 2-3 millions, then you should open vision, better quality.
LoRA rank wise, 128 should be a good one, it improves reasoning.
Cheers,
Nguyên
@thusinh1969 thank you for your response!