Enhancing human-guided robotic assembly: AR-assisted DT for skill-based and low-code programming

Yue Yin, Pai Zheng, Chengxi Li, Ke Wan

Research output: Journal article publicationJournal articleAcademic researchpeer-review


Efficient and natural programming strategies play a crucial role in enabling human-guided robotic assembly to adapt quickly to dynamic tasks. The combination of Augmented Reality (AR) and Digital Twins (DT) has shown promising potential in enhancing the intuitiveness of human–robot interaction while leveraging digital representations of human intelligence to empower robots in manufacturing tasks. However, traditional programming methods lack intuitive interaction and rely heavily on simulation environments or pre-set CAD models, leading to high costs for both initial setup and sim-to-real deployment. On the other hand, existing AR-based robot control methods have primarily focused on the basic movements of robots, overlooking higher-level skills necessary for complex tasks. To address these limitations, this study introduces a four-layer system architecture that integrates AR-assisted DT into skill-based robotic assembly scenarios. Additionally, a skill-based and low-code programming system for human-guided robotic assembly is designed and implemented, which incorporates natural human guidance and robot autonomous intelligence to generate adaptive and feasible action plans. The feasibility and efficiency of the proposed system are verified by two case studies and a quantitative experiment comparing to traditional programming methods. The results demonstrate the usability of our AR-assisted DT approach in improving programming efficiency, intuitiveness, and safety for human-guided robotic assembly while reducing cognitive load.

Original languageEnglish
Pages (from-to)676-689
Number of pages14
JournalJournal of Manufacturing Systems
Publication statusPublished - Jun 2024


  • Augmented reality
  • Digital twin
  • Human–robot interaction
  • Robot programming
  • Robotic assembly

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Software
  • Hardware and Architecture
  • Industrial and Manufacturing Engineering


Dive into the research topics of 'Enhancing human-guided robotic assembly: AR-assisted DT for skill-based and low-code programming'. Together they form a unique fingerprint.

Cite this