flan-t5-small-lit-simplif

This model is a fine-tuned version of google/flan-t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.8832
• Rouge1: 57.5616
• Rouge2: 43.0588
• Rougel: 54.6246
• Rougelsum: 54.8382
• Gen Len: 18.4914

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.3943	1.0	698	1.0480	57.3763	43.1329	54.1155	54.4964	18.6571
1.1857	2.0	1396	0.9521	57.315	43.2483	54.4032	54.7664	18.6771
1.0406	3.0	2094	0.9075	57.6951	43.4451	54.8174	55.0469	18.5343
0.9861	4.0	2792	0.8873	57.8533	43.409	54.7583	55.0156	18.5971
0.9592	5.0	3490	0.8832	57.5616	43.0588	54.6246	54.8382	18.4914

Framework versions

• Transformers 4.33.0
• Pytorch 2.0.0+cu117
• Datasets 2.14.4
• Tokenizers 0.13.3