TY - JOUR
T1 - Smart training
T2 - Mask R-CNN oriented approach
AU - Su, Mu Chun
AU - Chen, Jieh Haur
AU - Trisandini Azzizi, Vidya
AU - Chang, Hsiang Ling
AU - Wei, Hsi Hsien
N1 - Funding Information:
This paper was partly supported by the Ministry of Science and Technology (MOST), Taiwan, for promoting academic excellent of universities under grant numbers MOST 109-2221-E-008-059-MY3 and MOST 110-2634-F-008-005.
Publisher Copyright:
© 2021 The Author(s)
PY - 2021/12/15
Y1 - 2021/12/15
N2 - This paper is aimed at the usage of an augmented reality assisted system set up on the smart-glasses for training activities. Literature review leads us to a comparison among related technologies, yielding that Mask Regions with Convolutional Neural Network (R-CNN) oriented approach fits the study needs. The proposed method including (1) pointing gesture capture, (2) finger-pointing analysis, and (3) virtual tool positioning and rotation angle are developed. Results show that the recognition of object detection is 95.5%, the Kappa value of recognition of gesture detection is 0.93, and the average time for detecting pointing gesture is 0.26 seconds. Furthermore, even under different lighting, such as indoor and outdoor, the pointing analysis accuracy is up to 79%. The error between the analysis angle and the actual angle is only 1.32 degrees. The results proved that the system is well suited to present the effect of augmented reality, making it applicable for real world usage.
AB - This paper is aimed at the usage of an augmented reality assisted system set up on the smart-glasses for training activities. Literature review leads us to a comparison among related technologies, yielding that Mask Regions with Convolutional Neural Network (R-CNN) oriented approach fits the study needs. The proposed method including (1) pointing gesture capture, (2) finger-pointing analysis, and (3) virtual tool positioning and rotation angle are developed. Results show that the recognition of object detection is 95.5%, the Kappa value of recognition of gesture detection is 0.93, and the average time for detecting pointing gesture is 0.26 seconds. Furthermore, even under different lighting, such as indoor and outdoor, the pointing analysis accuracy is up to 79%. The error between the analysis angle and the actual angle is only 1.32 degrees. The results proved that the system is well suited to present the effect of augmented reality, making it applicable for real world usage.
KW - Augmented reality
KW - Finger-pointing analysis
KW - Hand gesture recognition
KW - Mask Regions with Convolutional Neural Network (R-CNN)
KW - Smart training
UR - http://www.scopus.com/inward/record.url?scp=85111293710&partnerID=8YFLogxK
U2 - 10.1016/j.eswa.2021.115595
DO - 10.1016/j.eswa.2021.115595
M3 - Journal article
AN - SCOPUS:85111293710
SN - 0957-4174
VL - 185
JO - Expert Systems with Applications
JF - Expert Systems with Applications
M1 - 115595
ER -