Redundancy Avoidance for Big Data in Data Centers: A Conventional Neural Network Approach

Chenhan Xu, Kun Wang, Yanfei Sun, Song Guo, Albert Zomaya

Research output: Journal article publicationJournal articleAcademic researchpeer-review

14 Citations (Scopus)


As the innovative data collection technologies are applying to every aspect of our society, the data volume is skyrocketing. Such phenomenon poses tremendous challenges to data centers with respect to enabling storage. In this paper, a hybrid-stream big data analytics model is proposed to perform multimedia big data analysis. This model contains four procedures, i.e., data pre-processing, data classification, data recognition and data load reduction. Specifically, an innovative multi-dimensional Convolution Neural Network (CNN) is proposed to assess the importance of each video frame. Thus, those unimportant frames can be dropped by a reliable decision-making algorithm. In order to ensure video quality, minimal correlation and minimal redundancy (MCMR) are combined to optimize the decision-making algorithm. Simulation results show that the amount of processed video is significantly reduced, and the quality of video is preserved due to the addition of MCMR. The simulation also proves that the proposed model performs steadily and is robust enough to scale up to accommodate the big data crush in data centers.

Original languageEnglish
Article number8371209
Pages (from-to)104-114
Number of pages11
JournalIEEE Transactions on Network Science and Engineering
Issue number1
Publication statusPublished - 1 Jan 2020


  • Data centers
  • big data
  • convolution neural network
  • multimedia
  • redundancy avoidance
  • storage

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Computer Science Applications
  • Computer Networks and Communications

Cite this