Comprehensive inspection trains (CITs) run over the whole railway network in China and collect a huge amount of data. Occasionally, due to equipment faults or environmental interference, data anomalies appeared which affected the subsequent data processing and analysis. Visual inspection of these anomalies is time-consuming, tedious and subjective. A more automatic way is desired. In this study, this task is considered as an image classification problem to leverage the power of deep learning, which has become a powerful tool in almost every research area in recent years. The multichannel data collected from CITs are normalized, windowed and encoded into images to build a training data set. Each sample is marked with a label that represents the condition in which it was collected. A convolutional neural network (CNN) is then formulated, in which the filters are set to only slide the horizontal direction to imitate humans' judgment process with the assumption that translation invariance only exists in this direction. Different from the conventional approaches that can only detect data anomalies of a single channel, this deep learning-based approach enables to further integrate multichannel information to find out the cause of data anomalies. It is shown that the approach can achieve satisfactory accuracy in either binary and multi-class classification and thus release the practitioners from the tedious task.