TY - JOUR
T1 - Light Field Image Compression Based on Bi-Level View Compensation with Rate-Distortion Optimization
AU - Hou, Junhui
AU - Chen, Jie
AU - Chau, Lap Pui
N1 - Funding Information:
Manuscript received February 15, 2017; revised August 13, 2017, November 11, 2017, and December 25, 2017; accepted January 29, 2018. Date of publication February 6, 2018; date of current version February 5, 2019. This work was supported by the CityU Start-up Grant for New Faculty under Grant 7200537/CS. This paper was recommended by Associate Editor Y. Wang. (Corresponding author: Lap-Pui Chau.) J. Hou is with the Department of Computer Science, City University of Hong Kong, Hong Kong (e-mail: [email protected]).
Publisher Copyright:
© 1991-2012 IEEE.
PY - 2019/2
Y1 - 2019/2
N2 - Compared with conventional color images, light field images (LFIs) contain richer scene information, which allows a wide range of interesting applications. However, such additional information is obtained at the cost of generating substantially more data, which poses challenges to both data storage and transmission. In this paper, we propose a new hybrid framework for effective compression of LFIs. The proposed framework takes the particular characteristics of LFIs into account so that the inter-and intra-view correlations of LFIs can be more efficiently exploited to produce better compression performance. Specifically, the proposed scheme partitions sub-Aperture images (SAIs) of an LFI into two groups, namely, key SAIs and non-key SAIs. Bi-level view compensation is proposed to exploit the inter-view correlation: first, based on the group of selected key SAIs, learning-based angular super-resolution is performed to compensate non-key SAIs in pixel-wise, during which heterogeneous inter-view correlation between the non-key SAIs is efficiently removed; second, the two groups of SAIs are respectively reorganized as pseudo-sequences, and block-wise motion compensation is carried out with a standard video encoder, during which the homogeneous inter-view correlation is subsequently exploited. The video encoder also helps to remove the intra-view correlation of the SAIs and finally generates the encoded bitstream. Moreover, the bits allocated to each group are optimally determined via model-based rate distortion optimization. Extensive experimental evaluations and comparisons demonstrate the advantage of the proposed framework over existing methods in terms of rate-distortion performance.
AB - Compared with conventional color images, light field images (LFIs) contain richer scene information, which allows a wide range of interesting applications. However, such additional information is obtained at the cost of generating substantially more data, which poses challenges to both data storage and transmission. In this paper, we propose a new hybrid framework for effective compression of LFIs. The proposed framework takes the particular characteristics of LFIs into account so that the inter-and intra-view correlations of LFIs can be more efficiently exploited to produce better compression performance. Specifically, the proposed scheme partitions sub-Aperture images (SAIs) of an LFI into two groups, namely, key SAIs and non-key SAIs. Bi-level view compensation is proposed to exploit the inter-view correlation: first, based on the group of selected key SAIs, learning-based angular super-resolution is performed to compensate non-key SAIs in pixel-wise, during which heterogeneous inter-view correlation between the non-key SAIs is efficiently removed; second, the two groups of SAIs are respectively reorganized as pseudo-sequences, and block-wise motion compensation is carried out with a standard video encoder, during which the homogeneous inter-view correlation is subsequently exploited. The video encoder also helps to remove the intra-view correlation of the SAIs and finally generates the encoded bitstream. Moreover, the bits allocated to each group are optimally determined via model-based rate distortion optimization. Extensive experimental evaluations and comparisons demonstrate the advantage of the proposed framework over existing methods in terms of rate-distortion performance.
KW - compression
KW - deep learning
KW - disparity
KW - Light field
KW - rate distortion optimization
KW - view compensation
UR - http://www.scopus.com/inward/record.url?scp=85041520169&partnerID=8YFLogxK
U2 - 10.1109/TCSVT.2018.2802943
DO - 10.1109/TCSVT.2018.2802943
M3 - Journal article
AN - SCOPUS:85041520169
SN - 1051-8215
VL - 29
SP - 517
EP - 530
JO - IEEE Transactions on Circuits and Systems for Video Technology
JF - IEEE Transactions on Circuits and Systems for Video Technology
IS - 2
M1 - 8283506
ER -