An AR-Assisted Deep Learning-Based Approach for Automatic Inspection of Aviation Connectors

Shufei Li, Pai Zheng, Lianyu Zheng

Research output: Journal article publicationJournal articleAcademic researchpeer-review

1 Citation (Scopus)

Abstract

The mismatched pins inspection of the complex aviation connector is a critical process to ensure the correct wiring harness assembly, of which the existing manual operation is error-prone and time-consuming. Aiming to fill this gap, this article proposes an augmented reality (AR)-assisted deep learning-based approach to tackle three major challenges in the aviation connector inspection, including the small pins detection, multipins sequencing, and mismatched pins visualization. First, the proposed spatial-attention pyramid network approach extracts the image features in multilayers and searches for their spatial relationships among the images. Second, based on the cluster-generation sequencing algorithm, these detected pins are clustered into annuluses of expected layers and numbered according to their polar angles. Finally, the AR glass as the inspection visualization platform, highlights the mismatched pins in the augmented interface to warn the operators automatically. Compared with the other existing methodologies, the experimental result shows that the proposed approach can achieve better performance accuracy and support the operator's inspection process efficiently.

Original languageEnglish
Article number9112336
Pages (from-to)1721-1731
Number of pages11
JournalIEEE Transactions on Industrial Informatics
Volume17
Issue number3
DOIs
Publication statusPublished - Mar 2021

Keywords

  • Augmented reality (AR)
  • aviation connector
  • deep learning
  • industrial inspection
  • spatial-attention pyramid network

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Information Systems
  • Computer Science Applications
  • Electrical and Electronic Engineering

Cite this