TY - JOUR
T1 - End-to-end varifocal multiview images coding framework from data acquisition end to vision application end
AU - Wu, Kejun
AU - Liu, Qiong
AU - Wang, Yi
AU - Yang, You
N1 - Funding Information:
Funding. National Key Research and Development Program of China (2020YFB2103501); National Natural Science Foundation of China (61991412); Major Project of Fundamental Research on Frontier Leading Technology of Jiangsu Province (BK20222006).
Publisher Copyright:
© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement.
PY - 2023/3/27
Y1 - 2023/3/27
N2 - The emerging data, varifocal multiview (VFMV) has an exciting prospect in immersive multimedia. However, the distinctive data redundancy of VFMV derived from dense arrangements and blurriness differences among views causes difficulty in data compression. In this paper, we propose an end-to-end coding scheme for VFMV images, which provides a new paradigm for VFMV compression from data acquisition (source) end to vision application end. VFMV acquisition is first conducted in three ways at the source end, including conventional imaging, plenoptic refocusing, and 3D creation. The acquired VFMV has irregular focusing distributions due to varying focal planes, which decreases the similarity among adjacent views. To improve the similarity and the consequent coding efficiency, we rearrange the irregular focusing distributions in descending order and accordingly reorder the horizontal views. Then, the reordered VFMV images are scanned and concatenated as video sequences. We propose 4-directional prediction (4DP) to compress the reordered VFMV video sequences. Four most similar adjacent views from the left, upper left, upper and upper right directions serve as reference frames to improve the prediction efficiency. Finally, the compressed VFMV is transmitted and decoded at the application end, benefiting potential vision applications. Extensive experiments demonstrate that the proposed coding scheme is superior to the comparison scheme in objective quality, subjective quality and computational complexity. Experiments on new view synthesis show that VFMV can achieve extended depth of field than conventional multiview at the application end. Validation experiments show the effectiveness of view reordering, the advantage over typical MV-HEVC, and the flexibility on other data types, respectively.
AB - The emerging data, varifocal multiview (VFMV) has an exciting prospect in immersive multimedia. However, the distinctive data redundancy of VFMV derived from dense arrangements and blurriness differences among views causes difficulty in data compression. In this paper, we propose an end-to-end coding scheme for VFMV images, which provides a new paradigm for VFMV compression from data acquisition (source) end to vision application end. VFMV acquisition is first conducted in three ways at the source end, including conventional imaging, plenoptic refocusing, and 3D creation. The acquired VFMV has irregular focusing distributions due to varying focal planes, which decreases the similarity among adjacent views. To improve the similarity and the consequent coding efficiency, we rearrange the irregular focusing distributions in descending order and accordingly reorder the horizontal views. Then, the reordered VFMV images are scanned and concatenated as video sequences. We propose 4-directional prediction (4DP) to compress the reordered VFMV video sequences. Four most similar adjacent views from the left, upper left, upper and upper right directions serve as reference frames to improve the prediction efficiency. Finally, the compressed VFMV is transmitted and decoded at the application end, benefiting potential vision applications. Extensive experiments demonstrate that the proposed coding scheme is superior to the comparison scheme in objective quality, subjective quality and computational complexity. Experiments on new view synthesis show that VFMV can achieve extended depth of field than conventional multiview at the application end. Validation experiments show the effectiveness of view reordering, the advantage over typical MV-HEVC, and the flexibility on other data types, respectively.
UR - http://www.scopus.com/inward/record.url?scp=85153475175&partnerID=8YFLogxK
U2 - 10.1364/OE.482141
DO - 10.1364/OE.482141
M3 - Journal article
C2 - 37155796
AN - SCOPUS:85153475175
SN - 1094-4087
VL - 31
SP - 11659
EP - 11679
JO - Optics Express
JF - Optics Express
IS - 7
M1 - 528440
ER -