For the 3 directions (ende, zhen, heen) covered in WMT2023, the model is pre-trained on top of XLMR-L using synthetic data generated by DCSQE.