Light Field Compression With Disparity-Guided Sparse Coding Based on Structural Key Views

Jie Chen, Junhui Hou, Lap Pui Chau

Research output: Journal article publicationJournal articleAcademic researchpeer-review

54 Citations (Scopus)

Abstract

Recent imaging technologies are rapidly evolving for sampling richer and more immersive representations of the 3D world. One of the emerging technologies is light field (LF) cameras based on micro-lens arrays. To record the directional information of the light rays, a much larger storage space and transmission bandwidth are required by an LF image as compared with a conventional 2D image of similar spatial dimension. Hence, the compression of LF data becomes a vital part of its application. In this paper, we propose an LF codec with disparity guided Sparse Coding over a learned perspective-shifted LF dictionary based on selected Structural Key Views (SC-SKV). The sparse coding is based on a limited number of optimally selected SKVs; yet the entire LF can be recovered from the coding coefficients. By keeping the approximation identical between encoder and decoder, only the residuals of the non-key views, disparity map, and the SKVs need to be compressed into the bit stream. An optimized SKV selection method is proposed such that most LF spatial information can be preserved. To achieve optimum dictionary efficiency, the LF is divided into several coding regions, over which the reconstruction works individually. Experiments and comparisons have been carried out over benchmark LF data set, which show that the proposed SC-SKV codec produces convincing compression results in terms of both rate-distortion performance and visual quality compared with Joint Exploration Model: with 37.9% BD-rate reduction and 1.17-dB BD-PSNR improvement achieved on average, especially with up to 6-dB improvement for low bit rate scenarios.

Original languageEnglish
Article number8030107
Pages (from-to)314-324
Number of pages11
JournalIEEE Transactions on Image Processing
Volume27
Issue number1
DOIs
Publication statusPublished - Jan 2018
Externally publishedYes

Keywords

  • disparity
  • Light field
  • perspective shifting
  • sparse coding
  • structural key view

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design

Cite this