No-reference video quality assessment metric using spatiotemporal features through LSTM

Ngai Wing Kwong, Sik Ho Tsang, Yui Lam Chan, Daniel Pak Kong Lun, Tsz Kwan Lee

Research output: Chapter in book / Conference proceedingConference article published in proceeding or bookAcademic researchpeer-review

Abstract

Nowadays, a precise video quality assessment (VQA) model is essential to maintain the quality of service (QoS). However, most existing VQA metrics are designed for specific purposes and ignore the spatiotemporal features of nature video. This paper proposes a novel general-purpose no-reference (NR) VQA metric adopting Long Short-Term Memory (LSTM) modules with the masking layer and pre-padding strategy, namely VQA-LSTM, to solve the above issues. First, we divide the distorted video into frames and extract some significant but also universal spatial and temporal features that could effectively reflect the quality of frames. Second, the data preprocessing stage and pre-padding strategy are used to process data to ease the training for our VQA-LSTM. Finally, a three-layer LSTM model incorporated with masking layer is designed to learn the sequence of spatial features as spatiotemporal features and learn the sequence of temporal features as the gradient of temporal features to evaluate the quality of videos. Two widely used VQA database, MCL-V and LIVE, are tested to prove the robustness of our VQA-LSTM, and the experimental results show that our VQA-LSTM has a better correlation with human perception than some state-of-the-art approaches.

Original languageEnglish
Title of host publicationInternational Workshop on Advanced Imaging Technology, IWAIT 2021
EditorsMasayuki Nakajima, Jae-Gon Kim, Wen-Nung Lie, Qian Kemao
PublisherSPIE
ISBN (Electronic)9781510643642
DOIs
Publication statusPublished - Mar 2021
Event2021 International Workshop on Advanced Imaging Technology, IWAIT 2021 - Kagoshima, Virtual, Japan
Duration: 5 Jan 20216 Jan 2021

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
Volume11766
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X

Conference

Conference2021 International Workshop on Advanced Imaging Technology, IWAIT 2021
CountryJapan
CityKagoshima, Virtual
Period5/01/216/01/21

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering

Cite this