A framework to simplify pre-processing location-based social media big data for sustainable urban planning and management

Mohammed Abdul-Rahman, Edwin H.W. Chan, Man Sing Wong, Victor E. Irekponor, Maryam O. Abdul-Rahman

Research output: Journal article publicationJournal articleAcademic researchpeer-review

7 Citations (Scopus)


Over the last decade, 90% of Big Data has been generated by people living in urban areas. With the advent of Internet of Things (IoT) and the increased use of the internet, Social Media has become an integral part of people's daily lives. Millions of unstructured data are being sent to the cloud every second, providing opinions practically on any discourse. This makes microblogs such as Twitter, Instagram, WeChat, and Facebook smart instruments for urban planners to harvest ‘big data’ on socioeconomics, urban dynamics, transportation, land uses, resilience, etc. This study proposed a framework for social media big data mining and data analytics using Twitter. It demonstrated the functionalities of the framework on a case study using Natural Language Processing and Machine Learning techniques like Latent Dirichlet Allocation and VADER Sentiment Analysis to mine, clean, process, and validate the data. The validated results from the case study showed high accuracy that Social Media Big Data can be used to study the spatiotemporal dynamism of community challenges.

Original languageEnglish
Article number102986
Publication statusPublished - Feb 2021


  • Big data
  • Community resilience
  • Internet of things
  • Natural language processing
  • Social media
  • Urban planning and management

ASJC Scopus subject areas

  • Development
  • Sociology and Political Science
  • Urban Studies
  • Tourism, Leisure and Hospitality Management

Cite this