TY - GEN
T1 - Object-Augmented Skeleton-Based Action Recognition
AU - Li, Zhengyu
AU - Guo, Heng
AU - Chau, Lap Pui
AU - Tan, Cheen Hau
AU - Ma, Xiaoxi
AU - Lin, Dan
AU - Yap, Kim Hui
N1 - Publisher Copyright:
© 2023 IEEE.
PY - 2023/6
Y1 - 2023/6
N2 - Human Skeleton-based Action Recognition (SAR) methods have made great advances in different applications in recent years. However, most existing SAR models mainly focus on the joint/limb pose estimation. They ignore the skeleton-object interaction, thereby resulting in underutilization of information available. For instance, drinking and eating actions may exhibit similar skeleton movements but different object interactions. Hence relying on joint/limb pose estimation alone may lead to incorrect action predictions; but leveraging skeleton-object interaction will help to discriminate such similar actions. In view of this, we propose a new effective method called Object-Augmented Skeleton-based Action Recognition (OA-SAR) to integrate skeleton-object interactions into action recognition. In specific, OA-SAR extracts the object and joint/limb heatmaps, and then integrates this information for subsequent action recognition. We evaluate the OA-SAR method on two benchmarks, NTU-RGB+D-60, and Drive&Act. Experimental results show that OA-SAR can achieve strong performance on both action datasets.
AB - Human Skeleton-based Action Recognition (SAR) methods have made great advances in different applications in recent years. However, most existing SAR models mainly focus on the joint/limb pose estimation. They ignore the skeleton-object interaction, thereby resulting in underutilization of information available. For instance, drinking and eating actions may exhibit similar skeleton movements but different object interactions. Hence relying on joint/limb pose estimation alone may lead to incorrect action predictions; but leveraging skeleton-object interaction will help to discriminate such similar actions. In view of this, we propose a new effective method called Object-Augmented Skeleton-based Action Recognition (OA-SAR) to integrate skeleton-object interactions into action recognition. In specific, OA-SAR extracts the object and joint/limb heatmaps, and then integrates this information for subsequent action recognition. We evaluate the OA-SAR method on two benchmarks, NTU-RGB+D-60, and Drive&Act. Experimental results show that OA-SAR can achieve strong performance on both action datasets.
KW - body keypoint heatmaps
KW - human action recognition
KW - skeleton-based action recognition
UR - http://www.scopus.com/inward/record.url?scp=85166373272&partnerID=8YFLogxK
U2 - 10.1109/AICAS57966.2023.10168565
DO - 10.1109/AICAS57966.2023.10168565
M3 - Conference article published in proceeding or book
AN - SCOPUS:85166373272
T3 - AICAS 2023 - IEEE International Conference on Artificial Intelligence Circuits and Systems, Proceeding
BT - AICAS 2023 - IEEE International Conference on Artificial Intelligence Circuits and Systems, Proceeding
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 5th IEEE International Conference on Artificial Intelligence Circuits and Systems, AICAS 2023
Y2 - 11 June 2023 through 13 June 2023
ER -