Deep Learning for Scoliosis Assessment

    Student thesis: PhD


    Scoliosis is a medical condition in which the spine has a sideways deformity larger than 10 degrees. It occurs in 2-4% of teenagers. 3D ultrasound imaging shows great promise for scoliosis diagnosis thanks to its low-cost, radiation-free, and real-time characteristics. In clinical scoliosis diagnosis, experts need to segment all the spinal features for scoliosis assessment, which is a tedious and time-consuming job. Benefiting from the powerful feature extraction capability, deep learning-based algorithms have been widely used in computer vision recently. These algorithms make it possible to extract spinal bone features from images automatically. In this thesis, we performed an in-depth study on deep-learning-based spine segmentation via ultrasound imaging, and proposed robust and effective learning algorithms to facilitate ultrasound spine segmentation.

    Different from natural images, ultrasound images have low quality and tend to contain scan noise. General deep learning-based algorithms cannot capture bone features robustly. Considering that the pixel values in the noisy regions are changing frequently, we proposed total variance loss to reduce the sensitivity of the network to the image regions with high-frequency energy, which improves the robustness of the segmentation model to scan noise.

    Next, we analyze different spinal bone features in the ultrasound images, which contain strong structural relationships among each other.We revisit the selfattention mechanism in representation learning and propose to introduce the structural knowledge into the key representation in self-attention. By this means, the network explores contextual and structural information in the learned spine features, improving the segmentation accuracy as a result.

    Besides improving the robustness of the segmentation model by considering the loss function, we propose a weakly-supervised framework to remove scan noise from unpaired samples. A dual adversarial learning strategy is used to handle the problem of data imbalance. Moreover, we present a novel multi-task framework to achieve high inference efficiency. To establish the interaction between the noise removal and spine segmentation tasks, we propose a selective feature-sharing strategy to transfer only beneficial features.

    Finally, owing to the lack of information, such as the bone index and spine vertebrae landmark coordinates, a semantic segmentation algorithm might be difficult to measure the scoliosis degree automatically. To achieve a fully automatic measurement of scoliosis, we proposed a landmarks localization algorithm to estimate the spine vertebrae coordinates directly. As we usually have strong prior knowledge of the distribution of landmarks in medical images, we proposed a novel and effective landmark localization approach with landmark distribution prior. It leverages normalizing flows for the underlying landmark distribution estimation and utilizes the learned distribution prior to penalizing the landmark localization.

    The spinal segmentation frameworks proposed in this thesis are evaluated through comparisons with other state-of-the-art methods on our in-house spinal ultrasound dataset. On the other hand, owing to the lack of landmarks annotation in our dataset, we evaluate our proposed landmark localization on three publicly available datasets of X-ray images. Experimental results show that our proposed algorithms achieve encouraging performance. Ablation studies demonstrate that the proposed designs can facilitate the learning of spinal segmentation and landmark localization.
    Date of Award14 Jun 2023
    Original languageEnglish
    Awarding Institution
    • The Hong Kong Polytechnic University
    SupervisorHung Fat Frank Leung (Chief supervisor), Steve S.H. Ling (Co-supervisor) & Yongping Zheng (Co-supervisor)

    Cite this