Memory-controlled deep LSTM neural network post-equalizer used in high-speed PAM VLC system

Linear and nonlinear impairments severely limit the transmission performance of high-speed visible light communication systems. Neural network-based equalizers have been applied to optical communication systems, which enables significantly improved system performance, such as transmission data rate and distance. In this paper, a memory-controlled deep long short-term memory (LSTM) neural network post-equalizer is proposed to mitigate both linear and nonlinear impairments in pulse amplitude modulation (PAM) based visible light communication (VLC) systems. Both 1.15-Gbps PAM4 and 0.9Gbps PAM8 VLC systems are successfully demonstrated, based on a single red-LED with bit error ratio (BER) below the hard decision forward error correction (HD-FEC) limit of 3.8 x 10 ⁻³ . Compared with the traditional finite impulse response (FIR) based equalizer, the Q factor performance is improved by 1.2dB and the transmission distance is increased by one-third in the same experimental hardware setups. Compared with traditional nonlinear hybrid Volterra equalizers, the significant complexity and system performance advantages of using a LSTM-based equalizer is demonstrated. To the best of our knowledge, this is the first demonstration of using deep LSTM in VLC systems.