Rail bolted joints (RBJs) are vital components served as either fastening rails or connecting adjacent rail sections in rail systems. In the era of high-speed rail (HSR), bolted joint fasteners are the most common types to keep rail tracks in position; although continuously welded rails (CWR) are mainly used in HSR lines for safety and ride comfort concern under high-speed operation, insulated rail joints (IRJ) are still widely utilized in low-speed zones (e.g., railway stations). Loosening of RBJs has potential risk to failure of circuit insulation, track displacement and misalignment, and subsequently even derailment. The monitoring of loosening of RBJs is therefore highly desired to maintain safe operation. The electromechanical impedance (EMI) is a promising ultrasonic structural health monitoring technique for structural deficiency or damage diagnosis. This paper introduces a novel EMI method for real-time monitoring of loosening of RBJs by employing a network of piezoelectric lead zirconate titanate (PZT) transducers pre-implemented on the host structure. The embedded PZT network and the host structure form a coupled system, and the coupled EMI which is highly sensitive to localized structural changes is measured to indicate the loosening of bolted joints. Convolutional neural networks (CNN) are applied to automatically extract key features from the joint EMI data and quantify the preloading forces on the bolted joints. While some damage metrics cannot well reflect the state of the RBJ, the proposed model can effectively and precisely predict the looseness of the bolted joint and outperform the model that utilizes only the data from one sensor.