Gross outlier removal and fault data recovery for SHM data of dynamic responses by an annihilating filter-based Hankel-structured robust PCA method

Si Yi Chen, You Wu Wang, Yi Qing Ni

Research output: Journal article publicationJournal articleAcademic researchpeer-review

8 Citations (Scopus)


In daily monitoring of structures instrumented with long-term structural health monitoring (SHM) systems, the acquired data is often corrupted with gross outliers due to hardware imperfection and/or electromagnetic interference. These unexpected spikes in data are not unusual and their existence may greatly influence the results of structural health evaluation and lead to false alarms. Hence, there is a high demand for executing data cleaning and data recovery, especially in harsh monitoring environment. In this paper, we propose a robust gross outlier removal method, termed Hankel-structured robust principal component analysis (HRPCA), to remove gross outliers in the monitoring data of structural dynamic responses. Different from the deep-learning-based approaches that possess only outlier identification or anomaly classification ability, HRPCA is a rapid and integrated methodology for data cleaning, which enables outlier detection, outlier identification, and recovery of fault data. It capitalizes on the fundamental duality between the sparsity of the signal and the rank of the structured matrix. Using annihilating filter-based fundamental duality, structural responses could be modeled as lying in a low-dimensional subspace with additional Hankel structure; thus, the gross outliers could be represented as a sparse component. Then the outlier removal issue turns into a matrix factorization problem, which could be successfully solved by robust principal component analysis (RPCA). To validate the denoising capability of HRPCA, a laboratory experiment is first conducted on a five-story building model where the reference clean signal is aware. Then real-world monitoring data with varying degrees of outliers (e.g., single outlier, multiple outliers, and periodic outliers) collected from a cable-stayed bridge and a high-rise structure is used to further illustrate the efficiency of the proposed approach.

Original languageEnglish
Article numbere3144
JournalStructural Control and Health Monitoring
Issue number12
Publication statusPublished - Dec 2022


  • data cleaning
  • Hankel-structured robust principal component analysis (HRPCA)
  • removal of gross outliers
  • structural health monitoring (SHM)
  • structured low-rank representation

ASJC Scopus subject areas

  • Civil and Structural Engineering
  • Building and Construction
  • Mechanics of Materials


Dive into the research topics of 'Gross outlier removal and fault data recovery for SHM data of dynamic responses by an annihilating filter-based Hankel-structured robust PCA method'. Together they form a unique fingerprint.

Cite this