A verification-oriented and part-focused assembly monitoring system based on multi-layered digital twin

In line with the human centricity characteristics of the emerging Industry 5.0 paradigm, manual assembly has long been an indispensable element of the small-batch and customized products. Nevertheless, the flexibility of manual operations is prone to assembly errors caused by human uncertainties. Since the newly assembled part directly determines the quality of products, the part-focused verification method has been widely adopted for manual assembly assistance. However, a part is partial to the whole assembly and occluded by other parts, which makes verification difficult. By leveraging the digital twin (DT) technology, one can construct a physical twin (PT) scene to represent the on-site assembly and a DT scene to represent the assembly's theoretical status in real time. Only if the part is assembled correctly, the newly assembled part can be mapped to the corresponding DT part model. Otherwise, the difference between the PT and the DT part can indicate a potential assembly error. Based on this idea, this paper proposes a verification-oriented and part-focused monitoring system based on multi-layered digital twin for manual assembly assistance. A multi-layered DT framework is constructed to build the geometric and logical connection between the physical and digital space. With the synchronization of DT, the newly assembled part and its corresponding template are generated using a point cloud reservation algorithm, and a two-direction matching algorithm is adapted to evaluate whether the newly assembled part is in the expected position for verification. The proposed monitoring system only requires one shot of the assembly scene in movable view for verification, which meets the flexible requirement for manual assistance in human-centric assembly. The experimental result shows that the proposed monitoring system can detect potential part errors and track assembly process information after manual assembly work, which is significant for human-centric intelligent manufacturing system development.