lnxdx commited on
Commit
83680d4
1 Parent(s): 3fb5390

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -21
README.md CHANGED
@@ -75,20 +75,6 @@ As you can see, my model performs better in maximum case :D
75
 
76
  ## Training procedure
77
 
78
- #### Model hyperparameters
79
- ```python
80
- model = Wav2Vec2ForCTC.from_pretrained(
81
- model_name_or_path if not last_checkpoint else last_checkpoint,
82
- # hp-mehrdad: Hyperparams of 'm3hrdadfi/wav2vec2-large-xlsr-persian-v3'
83
- attention_dropout = 0.05316,
84
- hidden_dropout = 0.01941,
85
- feat_proj_dropout = 0.01249,
86
- mask_time_prob = 0.04529,
87
- layerdrop = 0.01377,
88
- ctc_loss_reduction = 'mean',
89
- ctc_zero_infinity = True,
90
- )
91
- ```
92
  #### Training hyperparameters
93
 
94
  The following hyperparameters were used during training:
@@ -133,7 +119,7 @@ The following hyperparameters were used during training:
133
  Several models with differet hyperparameters were trained. The following figures show the training process for three of them.
134
  ![wer](wandb-wer.png)
135
  ![loss](wandb-loss.png)
136
- '20_2000_1e-5_hp-mehrdad' is the current model and it's hyperparameter are:
137
  ```python
138
  model = Wav2Vec2ForCTC.from_pretrained(
139
  model_name_or_path if not last_checkpoint else last_checkpoint,
@@ -146,8 +132,6 @@ model = Wav2Vec2ForCTC.from_pretrained(
146
  ctc_loss_reduction = 'mean',
147
  ctc_zero_infinity = True,
148
  )
149
-
150
- learning_rate = 1e-5
151
  ```
152
  The hyperparameters of '19_2000_1e-5_hp-base' are:
153
  ```python
@@ -162,8 +146,6 @@ model = Wav2Vec2ForCTC.from_pretrained(
162
  ctc_loss_reduction = 'mean',
163
  ctc_zero_infinity = True,
164
  )
165
-
166
- learing_rate = 1e-5
167
  ```
168
 
169
  And the hyperparameters of '22_2000_1e-5_hp-masoud' are:
@@ -179,9 +161,8 @@ model = Wav2Vec2ForCTC.from_pretrained(
179
  ctc_loss_reduction = 'mean',
180
  ctc_zero_infinity = True,
181
  )
182
-
183
- learning_rate = 1e-5
184
  ```
 
185
  As you can see this model performs better with WER metric on validation(evaluation) set.
186
 
187
  #### Framework versions
 
75
 
76
  ## Training procedure
77
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
78
  #### Training hyperparameters
79
 
80
  The following hyperparameters were used during training:
 
119
  Several models with differet hyperparameters were trained. The following figures show the training process for three of them.
120
  ![wer](wandb-wer.png)
121
  ![loss](wandb-loss.png)
122
+ '20_2000_1e-5_hp-mehrdad' is the current model and it's hyperparameters are:
123
  ```python
124
  model = Wav2Vec2ForCTC.from_pretrained(
125
  model_name_or_path if not last_checkpoint else last_checkpoint,
 
132
  ctc_loss_reduction = 'mean',
133
  ctc_zero_infinity = True,
134
  )
 
 
135
  ```
136
  The hyperparameters of '19_2000_1e-5_hp-base' are:
137
  ```python
 
146
  ctc_loss_reduction = 'mean',
147
  ctc_zero_infinity = True,
148
  )
 
 
149
  ```
150
 
151
  And the hyperparameters of '22_2000_1e-5_hp-masoud' are:
 
161
  ctc_loss_reduction = 'mean',
162
  ctc_zero_infinity = True,
163
  )
 
 
164
  ```
165
+ Learning rate is 1e-5 for all three models.
166
  As you can see this model performs better with WER metric on validation(evaluation) set.
167
 
168
  #### Framework versions