File size: 501 Bytes
0da66c5
1b1ecca
 
 
 
 
 
0da66c5
 
1b1ecca
0da66c5
1b1ecca
0da66c5
1b1ecca
 
 
 
 
 
 
 
 
0da66c5
 
 
 
1b1ecca
 
 
 
 
0da66c5
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
---
license: gemma
datasets:
- RLHF4MATH/SFT_510K
language:
- en
pipeline_tag: text-generation
---

# Gemma-7B-it-SFT3epoch

This model was fine-tuned from [google/gemma-1.1-7b-it](https://huggingface.co/google/gemma-1.1-7b-it) with parameters:

- learning_rate: 5e-06
- global batch size: 64
- optimizer: paged_adamw_32bit 
- lr_scheduler_type: cosine 
- num_train_epochs: 3.0
- warmup_steps: 50
- sequence_len: 4096
- sample_packing: true
- pad_to_sequence_len: true




  
## Citation
```
TBD
```