Human-object integrated assembly intention recognition for context-aware human-robot collaborative assembly

Yaqian Zhang, Kai Ding (Corresponding Author), Jizhuang Hui, Jingxiang Lv, Xueliang Zhou, Pai Zheng

Research output: Journal article publicationJournal articleAcademic researchpeer-review

Abstract

Human-robot collaborative (HRC) assembly combines the advantages of robot's operation consistency with human's cognitive ability and adaptivity, which provides an efficient and flexible way for complex assembly tasks. In the process of HRC assembly, the robot needs to understand the operator's intention accurately to assist the collaborative assembly tasks. At present, operator intention recognition considering context information such as assembly objects in a complex environment remains challenging. In this paper, we propose a human-object integrated approach for context-aware assembly intention recognition in the HRC, which integrates the recognition of assembly actions and assembly parts to improve the accuracy of the operator's intention recognition. Specifically, considering the real-time requirements of HRC assembly, spatial-temporal graph convolutional networks (ST-GCN) model based on skeleton features is utilized to recognize the assembly action to reduce unnecessary redundant information. Considering the disorder and occlusion of assembly parts, an improved YOLOX model is proposed to improve the focusing capability of network structure on the assembly parts that are difficult to recognize. Afterwards, taking decelerator assembly tasks as an example, a rule-based reasoning method that contains the recognition information of assembly actions and assembly parts is designed to recognize the current assembly intention. Finally, the feasibility and effectiveness of the proposed approach for recognizing human intentions are verified. The integration of assembly action recognition and assembly part recognition can facilitate the accurate operator's intention recognition in the complex and flexible HRC assembly environment.

Original languageEnglish
Article number101792
Number of pages12
JournalAdvanced Engineering Informatics
Volume54
DOIs
Publication statusPublished - Oct 2022

Keywords

  • Human intention recognition
  • Human-robot collaborative assembly
  • Improved YOLOX
  • Part recognition
  • ST-GCN

ASJC Scopus subject areas

  • Information Systems
  • Building and Construction
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Human-object integrated assembly intention recognition for context-aware human-robot collaborative assembly'. Together they form a unique fingerprint.

Cite this