TY - JOUR
T1 - A framework to simplify pre-processing location-based social media big data for sustainable urban planning and management
AU - Abdul-Rahman, Mohammed
AU - Chan, Edwin H.W.
AU - Wong, Man Sing
AU - Irekponor, Victor E.
AU - Abdul-Rahman, Maryam O.
N1 - Funding Information:
The authors are grateful to the editors and anonymous reviewers for their insightful comments and suggestions. The work described in this paper was supported by a Ph.D. studentship from the Research Institute for Sustainable Development (RISUD) of the Hong Kong Polytechnic University, with the Department of Building and Real Estate, Hong Kong Polytechnic University, and AI Africa Lab providing other resources.
Funding Information:
The authors are grateful to the editors and anonymous reviewers for their insightful comments and suggestions. The work described in this paper was supported by a Ph.D. studentship from the Research Institute for Sustainable Development (RISUD) of the Hong Kong Polytechnic University, with the Department of Building and Real Estate, Hong Kong Polytechnic University, and AI Africa Lab providing other resources.
Publisher Copyright:
© 2020 Elsevier Ltd
PY - 2021/2
Y1 - 2021/2
N2 - Over the last decade, 90% of Big Data has been generated by people living in urban areas. With the advent of Internet of Things (IoT) and the increased use of the internet, Social Media has become an integral part of people's daily lives. Millions of unstructured data are being sent to the cloud every second, providing opinions practically on any discourse. This makes microblogs such as Twitter, Instagram, WeChat, and Facebook smart instruments for urban planners to harvest ‘big data’ on socioeconomics, urban dynamics, transportation, land uses, resilience, etc. This study proposed a framework for social media big data mining and data analytics using Twitter. It demonstrated the functionalities of the framework on a case study using Natural Language Processing and Machine Learning techniques like Latent Dirichlet Allocation and VADER Sentiment Analysis to mine, clean, process, and validate the data. The validated results from the case study showed high accuracy that Social Media Big Data can be used to study the spatiotemporal dynamism of community challenges.
AB - Over the last decade, 90% of Big Data has been generated by people living in urban areas. With the advent of Internet of Things (IoT) and the increased use of the internet, Social Media has become an integral part of people's daily lives. Millions of unstructured data are being sent to the cloud every second, providing opinions practically on any discourse. This makes microblogs such as Twitter, Instagram, WeChat, and Facebook smart instruments for urban planners to harvest ‘big data’ on socioeconomics, urban dynamics, transportation, land uses, resilience, etc. This study proposed a framework for social media big data mining and data analytics using Twitter. It demonstrated the functionalities of the framework on a case study using Natural Language Processing and Machine Learning techniques like Latent Dirichlet Allocation and VADER Sentiment Analysis to mine, clean, process, and validate the data. The validated results from the case study showed high accuracy that Social Media Big Data can be used to study the spatiotemporal dynamism of community challenges.
KW - Big data
KW - Community resilience
KW - Internet of things
KW - Natural language processing
KW - Social media
KW - Urban planning and management
UR - http://www.scopus.com/inward/record.url?scp=85095569425&partnerID=8YFLogxK
U2 - 10.1016/j.cities.2020.102986
DO - 10.1016/j.cities.2020.102986
M3 - Journal article
AN - SCOPUS:85095569425
SN - 0264-2751
VL - 109
JO - Cities
JF - Cities
M1 - 102986
ER -
