Automated training dataset collection system design for machine learning application in optical networks: an example of quality of transmission estimation

Applications of machine learning (ML) models in optical communications and networks have been extensively investigated. For an optical wavelength-division-multiplexing (WDM) system, the quality of transmission (QoT) estimation generally depends on many parameters including the number and arrangement of WDM channels; launch power of each channel; number and distribution of fiber spans; attenuation, dispersion, and nonlinearity parameters and length of each fiber span; noise figure; gain and gain tilt of erbium-doped fiber amplifiers; transceiver noise; digital signal processing (DSP) performance; and so on. In recent years, ML-based QoT estimation schemes have gained significant attention. However, nearly all relevant works are conducted through simulations because it is difficult to obtain sufficient and high-quality datasets for training ML models. In this paper, we demonstrate completely automated generation and collection of an ultra-large-scale experimental training dataset for ML-model-based QoT estimation by automation of transceivers and optical link parameters, as well as data transfer and DSP. Implementation details and key codes of automation are presented. Artificial neural network models with one and two hidden layers are trained by the collected dataset, and brief QoT estimation results are evaluated and discussed to verify the performance and stability of the established automated system.