TY - JOUR
T1 - A Self-Training Approach for Point-Supervised Object Detection and Counting in Crowds
AU - Wang, Yi
AU - Hou, Junhui
AU - Hou, Xinyu
AU - Chau, Lap Pui
N1 - Funding Information:
Manuscript received July 25, 2020; revised December 22, 2020 and January 25, 2021; accepted January 25, 2021. Date of publication February 4, 2021; date of current version February 12, 2021. This work was supported in part by the Hong Kong Research Grants Council under Grant CityU 11219019 and Grant CityU 11202320. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Tao Mei. (Corresponding author: Lap-Pui Chau.) Yi Wang, Xinyu Hou, and Lap-Pui Chau are with the School of Electrical and Electronics Engineering, Nanyang Technological University, Singapore 639798 (e-mail: [email protected]; [email protected]; [email protected]).
Publisher Copyright:
© 1992-2012 IEEE.
PY - 2021/2
Y1 - 2021/2
N2 - In this article, we propose a novel self-training approach named Crowd-SDNet that enables a typical object detector trained only with point-level annotations (i.e., objects are labeled with points) to estimate both the center points and sizes of crowded objects. Specifically, during training, we utilize the available point annotations to supervise the estimation of the center points of objects directly. Based on a locally-uniform distribution assumption, we initialize pseudo object sizes from the point-level supervisory information, which are then leveraged to guide the regression of object sizes via a crowdedness-aware loss. Meanwhile, we propose a confidence and order-aware refinement scheme to continuously refine the initial pseudo object sizes such that the ability of the detector is increasingly boosted to detect and count objects in crowds simultaneously. Moreover, to address extremely crowded scenes, we propose an effective decoding method to improve the detector's representation ability. Experimental results on the WiderFace benchmark show that our approach significantly outperforms state-of-the-art point-supervised methods under both detection and counting tasks, i.e., our method improves the average precision by more than 10% and reduces the counting error by 31.2%. Besides, our method obtains the best results on the crowd counting and localization datasets (i.e., ShanghaiTech and NWPU-Crowd) and vehicle counting datasets (i.e., CARPK and PUCPR+) compared with state-of-the-art counting-by-detection methods. The code will be publicly available at https://github.com/WangyiNTU/Point-supervised-crowd-detection.
AB - In this article, we propose a novel self-training approach named Crowd-SDNet that enables a typical object detector trained only with point-level annotations (i.e., objects are labeled with points) to estimate both the center points and sizes of crowded objects. Specifically, during training, we utilize the available point annotations to supervise the estimation of the center points of objects directly. Based on a locally-uniform distribution assumption, we initialize pseudo object sizes from the point-level supervisory information, which are then leveraged to guide the regression of object sizes via a crowdedness-aware loss. Meanwhile, we propose a confidence and order-aware refinement scheme to continuously refine the initial pseudo object sizes such that the ability of the detector is increasingly boosted to detect and count objects in crowds simultaneously. Moreover, to address extremely crowded scenes, we propose an effective decoding method to improve the detector's representation ability. Experimental results on the WiderFace benchmark show that our approach significantly outperforms state-of-the-art point-supervised methods under both detection and counting tasks, i.e., our method improves the average precision by more than 10% and reduces the counting error by 31.2%. Besides, our method obtains the best results on the crowd counting and localization datasets (i.e., ShanghaiTech and NWPU-Crowd) and vehicle counting datasets (i.e., CARPK and PUCPR+) compared with state-of-the-art counting-by-detection methods. The code will be publicly available at https://github.com/WangyiNTU/Point-supervised-crowd-detection.
KW - Convolutional neural network (CNN)
KW - crowd counting
KW - object detection
KW - self-training
KW - weak supervision
UR - http://www.scopus.com/inward/record.url?scp=85101492704&partnerID=8YFLogxK
U2 - 10.1109/TIP.2021.3055632
DO - 10.1109/TIP.2021.3055632
M3 - Journal article
C2 - 33539297
AN - SCOPUS:85101492704
SN - 1057-7149
VL - 30
SP - 2876
EP - 2887
JO - IEEE Transactions on Image Processing
JF - IEEE Transactions on Image Processing
M1 - 9347744
ER -