metadata
license: gemma
datasets:
- RLHF4MATH/SFT_510K
language:
- en
pipeline_tag: text-generation
Gemma-7B-it-SFT3epoch
This model was fine-tuned from google/gemma-1.1-7b-it with parameters:
- learning_rate: 5e-06
- global batch size: 64
- optimizer: paged_adamw_32bit
- lr_scheduler_type: cosine
- num_train_epochs: 3.0
- warmup_steps: 50
- sequence_len: 4096
- sample_packing: true
- pad_to_sequence_len: true
Citation
TBD