TY - JOUR
T1 - Big data-driven pedestrian analytics
T2 - Unsupervised clustering and relational query based on tencent street view photographs
AU - Xue, Fan
AU - Li, Xiao
AU - Lu, Weisheng
AU - Webster, Christopher J.
AU - Chen, Zhe
AU - Lin, Lvwen
N1 - Funding Information:
The work presented in this paper was financially supported by The University of Hong Kong (Grant Nos. 102009741, 202011159253).
Publisher Copyright:
© 2021 by the authors. Licensee MDPI, Basel, Switzerland.
PY - 2021/8
Y1 - 2021/8
N2 - Recent technological advancements in geomatics and mobile sensing have led to various urban big data, such as Tencent street view (TSV) photographs; yet, the urban objects in the big dataset have hitherto been inadequately exploited. This paper aims to propose a pedestrian analytics approach named vectors of uncountable and countable objects for clustering and analysis (VUCCA) for processing 530,000 TSV photographs of Hong Kong Island. First, VUCCA transductively adopts two pre-trained deep models to TSV photographs for extracting pedestrians and surrounding pixels into generalizable semantic vectors of features, including uncountable objects such as vegetation, sky, paved pedestrian path, and guardrail and countable objects such as cars, trucks, pedestrians, city animals, and traffic lights. Then, the extracted pedestrians are semantically clustered using the vectors, e.g., for understanding where they usually stand. Third, pedestrians are semantically indexed using relations and activities (e.g., walking behind a guardrail, road-crossing, carrying a backpack, or walking a pet) for queries of unstructured photographic instances or natural language clauses. The experiment results showed that the pedestrians detected in the TSV photographs were successfully clustered into meaningful groups and indexed by the semantic vectors. The presented VUCCA can enrich eye-level urban features into computational semantic vectors for pedestrians to enable smart city research in urban geography, urban planning, real estate, transportation, conservation, and other disciplines.
AB - Recent technological advancements in geomatics and mobile sensing have led to various urban big data, such as Tencent street view (TSV) photographs; yet, the urban objects in the big dataset have hitherto been inadequately exploited. This paper aims to propose a pedestrian analytics approach named vectors of uncountable and countable objects for clustering and analysis (VUCCA) for processing 530,000 TSV photographs of Hong Kong Island. First, VUCCA transductively adopts two pre-trained deep models to TSV photographs for extracting pedestrians and surrounding pixels into generalizable semantic vectors of features, including uncountable objects such as vegetation, sky, paved pedestrian path, and guardrail and countable objects such as cars, trucks, pedestrians, city animals, and traffic lights. Then, the extracted pedestrians are semantically clustered using the vectors, e.g., for understanding where they usually stand. Third, pedestrians are semantically indexed using relations and activities (e.g., walking behind a guardrail, road-crossing, carrying a backpack, or walking a pet) for queries of unstructured photographic instances or natural language clauses. The experiment results showed that the pedestrians detected in the TSV photographs were successfully clustered into meaningful groups and indexed by the semantic vectors. The presented VUCCA can enrich eye-level urban features into computational semantic vectors for pedestrians to enable smart city research in urban geography, urban planning, real estate, transportation, conservation, and other disciplines.
KW - Big data
KW - Deep learning
KW - Hong Kong Island
KW - Object detection
KW - Pedestrian activity
KW - Semantic segmentation
KW - Streetscape
KW - Tencent street view (TSV)
KW - Urban informatics
UR - http://www.scopus.com/inward/record.url?scp=85114315089&partnerID=8YFLogxK
U2 - 10.3390/ijgi10080561
DO - 10.3390/ijgi10080561
M3 - Journal article
AN - SCOPUS:85114315089
VL - 10
JO - ISPRS International Journal of Geo-Information
JF - ISPRS International Journal of Geo-Information
SN - 2220-9964
IS - 8
M1 - 561
ER -