TY - JOUR
T1 - Transductive Zero-Shot Action Recognition via Visually Connected Graph Convolutional Networks
AU - Xu, Yangyang
AU - Han, Chu
AU - Qin, Jing
AU - Xu, Xuemiao
AU - Han, Guoqiang
AU - He, Shengfeng
N1 - Funding Information:
Manuscript received October 14, 2019; revised March 7, 2020 and June 24, 2020; accepted August 8, 2020. Date of publication August 21, 2020; date of current version August 4, 2021. This work was supported in part by NSFC under Grant 61702194, Grant 61972162, Grant 61772206, Grant U1611461, and Grant 61472145; in part by the Guangdong Research and Development Key Project of China under Grant 2018B010107003, in part by the Guangdong High-level Personnel of Special Support Program under Grant 2016TQ03X319, in part by the Guangdong Natural Science Foundation under Grant 2017A030311027, in part by the Guangzhou Key Project in Industrial Technology under Grant 201802010027 and Grant 201802010036, in part by the Key-Area Research and Development Program of Guangdong Province, China, under Grant 2020B010165004, Grant 2020B010166003, and Grant 2018B010107003; in part by the Guangdong High-Level Personnel Program under Grant 2016TQ03X319, in part by the Hong Kong Polytechnic University under Project YBZE, and in part by the CCF-Tencent Open Research Fund (CCF-Tencent RAGR20190112). (Yangyang Xu and Chu Han contributed equally to this work.) (Corresponding authors: Xuemiao Xu; Shengfeng He.) Yangyang Xu, Guoqiang Han, and Shengfeng He are with the School of Computer Science and Engineering, South China University of Technology, Guangzhou 510006, China (e-mail: [email protected]; [email protected]; [email protected]).
Publisher Copyright:
© 2012 IEEE.
PY - 2021/8
Y1 - 2021/8
N2 - With the explosive growth of action categories, zero-shot action recognition aims to extend a well-trained model to novel/unseen classes. To bridge the large knowledge gap between seen and unseen classes, in this brief, we visually associate unseen actions with seen categories in a visually connected graph, and the knowledge is then transferred from the visual features space to semantic space via the grouped attention graph convolutional networks (GAGCNs). In particular, we extract visual features for all the actions, and a visually connected graph is built to attach seen actions to visually similar unseen categories. Moreover, the proposed grouped attention mechanism exploits the hierarchical knowledge in the graph so that the GAGCN enables propagating the visual-semantic connections from seen actions to unseen ones. We extensively evaluate the proposed method on three data sets: HMDB51, UCF101, and NTU RGB + D. Experimental results show that the GAGCN outperforms state-of-the-art methods.
AB - With the explosive growth of action categories, zero-shot action recognition aims to extend a well-trained model to novel/unseen classes. To bridge the large knowledge gap between seen and unseen classes, in this brief, we visually associate unseen actions with seen categories in a visually connected graph, and the knowledge is then transferred from the visual features space to semantic space via the grouped attention graph convolutional networks (GAGCNs). In particular, we extract visual features for all the actions, and a visually connected graph is built to attach seen actions to visually similar unseen categories. Moreover, the proposed grouped attention mechanism exploits the hierarchical knowledge in the graph so that the GAGCN enables propagating the visual-semantic connections from seen actions to unseen ones. We extensively evaluate the proposed method on three data sets: HMDB51, UCF101, and NTU RGB + D. Experimental results show that the GAGCN outperforms state-of-the-art methods.
KW - Action recognition
KW - graph convolutional network (GCN)
KW - zero-shot learning (ZSL)
UR - http://www.scopus.com/inward/record.url?scp=85091290711&partnerID=8YFLogxK
U2 - 10.1109/TNNLS.2020.3015848
DO - 10.1109/TNNLS.2020.3015848
M3 - Journal article
C2 - 32822308
AN - SCOPUS:85091290711
SN - 2162-237X
VL - 32
SP - 3761
EP - 3769
JO - IEEE Transactions on Neural Networks and Learning Systems
JF - IEEE Transactions on Neural Networks and Learning Systems
IS - 8
M1 - 9173643
ER -