weqweasdas's picture
Update README.md
1b1ecca verified
|
raw
history blame
501 Bytes
metadata
license: gemma
datasets:
  - RLHF4MATH/SFT_510K
language:
  - en
pipeline_tag: text-generation

Gemma-7B-it-SFT3epoch

This model was fine-tuned from google/gemma-1.1-7b-it with parameters:

  • learning_rate: 5e-06
  • global batch size: 64
  • optimizer: paged_adamw_32bit
  • lr_scheduler_type: cosine
  • num_train_epochs: 3.0
  • warmup_steps: 50
  • sequence_len: 4096
  • sample_packing: true
  • pad_to_sequence_len: true

Citation

TBD