TY - JOUR
T1 - An AR-Assisted Deep Learning-Based Approach for Automatic Inspection of Aviation Connectors
AU - Li, Shufei
AU - Zheng, Pai
AU - Zheng, Lianyu
N1 - Funding Information:
Manuscript received December 10, 2019; revised April 24, 2020 and May 22, 2020; accepted June 4, 2020. Date of publication June 9, 2020; date of current version November 20, 2020. This research was supported in part by the Civil Airplane Technology Development Program under Grant MJ-2017-G-70, in part by the Beijing Key Laboratory of Digital Design and Manufacturing Project, and in part by the Departmental General Research Fund (G-UAHH) from the Hong Kong Polytechnic University, Hong Kong. Paper no. TII-19-5279. (Corresponding author: Lianyu Zheng.) Shufei Li is with the Department of Industrial and Manufacturing Systems Engineering, Beihang University, Beijing 100191, China (e-mail: [email protected]).
Publisher Copyright:
© 2005-2012 IEEE.
Copyright:
Copyright 2020 Elsevier B.V., All rights reserved.
PY - 2021/3
Y1 - 2021/3
N2 - The mismatched pins inspection of the complex aviation connector is a critical process to ensure the correct wiring harness assembly, of which the existing manual operation is error-prone and time-consuming. Aiming to fill this gap, this article proposes an augmented reality (AR)-assisted deep learning-based approach to tackle three major challenges in the aviation connector inspection, including the small pins detection, multipins sequencing, and mismatched pins visualization. First, the proposed spatial-attention pyramid network approach extracts the image features in multilayers and searches for their spatial relationships among the images. Second, based on the cluster-generation sequencing algorithm, these detected pins are clustered into annuluses of expected layers and numbered according to their polar angles. Finally, the AR glass as the inspection visualization platform, highlights the mismatched pins in the augmented interface to warn the operators automatically. Compared with the other existing methodologies, the experimental result shows that the proposed approach can achieve better performance accuracy and support the operator's inspection process efficiently.
AB - The mismatched pins inspection of the complex aviation connector is a critical process to ensure the correct wiring harness assembly, of which the existing manual operation is error-prone and time-consuming. Aiming to fill this gap, this article proposes an augmented reality (AR)-assisted deep learning-based approach to tackle three major challenges in the aviation connector inspection, including the small pins detection, multipins sequencing, and mismatched pins visualization. First, the proposed spatial-attention pyramid network approach extracts the image features in multilayers and searches for their spatial relationships among the images. Second, based on the cluster-generation sequencing algorithm, these detected pins are clustered into annuluses of expected layers and numbered according to their polar angles. Finally, the AR glass as the inspection visualization platform, highlights the mismatched pins in the augmented interface to warn the operators automatically. Compared with the other existing methodologies, the experimental result shows that the proposed approach can achieve better performance accuracy and support the operator's inspection process efficiently.
KW - Augmented reality (AR)
KW - aviation connector
KW - deep learning
KW - industrial inspection
KW - spatial-attention pyramid network
UR - http://www.scopus.com/inward/record.url?scp=85097707890&partnerID=8YFLogxK
U2 - 10.1109/TII.2020.3000870
DO - 10.1109/TII.2020.3000870
M3 - Journal article
AN - SCOPUS:85097707890
SN - 1551-3203
VL - 17
SP - 1721
EP - 1731
JO - IEEE Transactions on Industrial Informatics
JF - IEEE Transactions on Industrial Informatics
IS - 3
M1 - 9112336
ER -