lmejias/w2v2-baseline

This model is a fine-tuned version of Jzuluaga/wav2vec2-large-960h-lv60-self-en-atc-uwb-atcc-and-atcosim on the LiveATC recent data dataset. It achieves the following results on the evaluation set:

• Loss: 4.3462
• Wer: 104.4118

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• training_steps: 3000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
5.5429	7.1429	50	6.3329	102.3396
3.6092	14.2857	100	3.6703	103.8770
2.9412	21.4286	150	3.2563	107.3529
2.7277	28.5714	200	3.1662	105.3476
2.462	35.7143	250	3.1332	104.8797
2.0927	42.8571	300	3.1455	105.1471
1.7015	50.0	350	3.3032	107.5535
1.3212	57.1429	400	3.3018	106.7513
1.1036	64.2857	450	3.4413	106.6176
0.8124	71.4286	500	3.4884	104.9465
0.674	78.5714	550	3.5686	106.3503
0.565	85.7143	600	3.5688	106.4171
0.488	92.8571	650	3.6488	106.2834
0.4201	100.0	700	3.6693	106.5508
0.3683	107.1429	750	3.6873	107.4198
0.313	114.2857	800	3.6209	104.5455
0.3457	121.4286	850	3.6447	105.4144
0.323	128.5714	900	3.6727	102.8743
0.2825	135.7143	950	3.6922	109.8262
0.2748	142.8571	1000	3.8867	102.5401
0.2109	150.0	1050	3.8843	104.8128
0.255	157.1429	1100	3.8725	104.4118
0.2079	164.2857	1150	3.9449	103.3422
0.1968	171.4286	1200	3.8255	104.7460
0.1641	178.5714	1250	3.8195	105.0802
0.1488	185.7143	1300	3.8621	103.7433
0.1487	192.8571	1350	3.7639	104.9465
0.1445	200.0	1400	3.7507	106.6845
0.1194	207.1429	1450	3.8920	104.4786
0.0906	214.2857	1500	3.9927	106.0829
0.093	221.4286	1550	3.9899	107.0856
0.1008	228.5714	1600	3.9051	104.4118
0.0911	235.7143	1650	3.9279	103.4759
0.0859	242.8571	1700	3.9341	104.1444
0.0876	250.0	1750	4.0044	105.2807
0.0768	257.1429	1800	3.9745	103.7433
0.0747	264.2857	1850	4.0634	105.7487
0.0672	271.4286	1900	4.1271	106.1497
0.0654	278.5714	1950	4.0620	106.3503
0.0559	285.7143	2000	4.1474	106.9519
0.0335	292.8571	2050	4.0337	104.4118
0.0661	300.0	2100	4.2306	102.8743
0.0517	307.1429	2150	4.1846	104.2781
0.0328	314.2857	2200	4.1616	105.2139
0.0488	321.4286	2250	4.1904	106.4171
0.0307	328.5714	2300	4.2941	104.9465
0.0304	335.7143	2350	4.2107	103.6096
0.0472	342.8571	2400	4.2483	105.1471
0.0181	350.0	2450	4.2711	104.3449
0.015	357.1429	2500	4.2480	104.2781
0.0122	364.2857	2550	4.3409	104.4118
0.008	371.4286	2600	4.3071	104.8797
0.0159	378.5714	2650	4.2340	104.6791
0.0297	385.7143	2700	4.3997	104.2781
0.0065	392.8571	2750	4.3840	104.0107
0.0154	400.0	2800	4.3097	103.6096
0.0065	407.1429	2850	4.3581	104.0775
0.0035	414.2857	2900	4.3293	104.8128
0.0201	421.4286	2950	4.3565	103.9439
0.0095	428.5714	3000	4.3462	104.4118

Framework versions

• Transformers 4.51.3
• Pytorch 2.9.1+cu128
• Datasets 4.4.1
• Tokenizers 0.21.4