TY - JOUR
T1 - Bitstream-Corrupted Video Recovery: A Novel Benchmark Dataset and Method
AU - Liu, Tianyi
AU - Wu, Kejun
AU - Wang, Yi
AU - Liu, Wenyang
AU - Yap, Kim Hui
AU - Chau, Lap Pui
N1 - Publisher Copyright:
© 2023 Neural information processing systems foundation. All rights reserved.
PY - 2023/12
Y1 - 2023/12
N2 - The past decade has witnessed great strides in video recovery by specialist technologies, like video inpainting, completion, and error concealment. However, they typically simulate the missing content by manual-designed error masks, thus failing to fill in the realistic video loss in video communication (e.g., telepresence, live streaming, and internet video) and multimedia forensics. To address this, we introduce the bitstream-corrupted video (BSCV) benchmark, the first benchmark dataset with more than 28, 000 video clips, which can be used for bitstream-corrupted video recovery in the real world. The BSCV is a collection of 1) a proposed three-parameter corruption model for video bitstream, 2) a large-scale dataset containing rich error patterns, multiple corruption levels, and flexible dataset branches, and 3) a new video recovery framework that serves as a benchmark. We evaluate state-of-the-art video inpainting methods on the BSCV dataset, demonstrating existing approaches' limitations and our framework's advantages in solving the bitstream-corrupted video recovery problem. The benchmark and dataset are released at https://github.com/LIUTIGHE/BSCV-Dataset.
AB - The past decade has witnessed great strides in video recovery by specialist technologies, like video inpainting, completion, and error concealment. However, they typically simulate the missing content by manual-designed error masks, thus failing to fill in the realistic video loss in video communication (e.g., telepresence, live streaming, and internet video) and multimedia forensics. To address this, we introduce the bitstream-corrupted video (BSCV) benchmark, the first benchmark dataset with more than 28, 000 video clips, which can be used for bitstream-corrupted video recovery in the real world. The BSCV is a collection of 1) a proposed three-parameter corruption model for video bitstream, 2) a large-scale dataset containing rich error patterns, multiple corruption levels, and flexible dataset branches, and 3) a new video recovery framework that serves as a benchmark. We evaluate state-of-the-art video inpainting methods on the BSCV dataset, demonstrating existing approaches' limitations and our framework's advantages in solving the bitstream-corrupted video recovery problem. The benchmark and dataset are released at https://github.com/LIUTIGHE/BSCV-Dataset.
UR - http://www.scopus.com/inward/record.url?scp=85191191844&partnerID=8YFLogxK
UR - https://proceedings.neurips.cc/paper_files/paper/2023/file/d7928f6dfb0c30d6a6917587dacbe4bc-Supplemental-Datasets_and_Benchmarks.pdf
U2 - 10.48550/arXiv.2309.13890
DO - 10.48550/arXiv.2309.13890
M3 - Conference article
AN - SCOPUS:85191191844
SN - 1049-5258
VL - 36
SP - 68420
EP - 68433
JO - Advances in Neural Information Processing Systems
JF - Advances in Neural Information Processing Systems
M1 - 198465
T2 - 37th Conference on Neural Information Processing Systems, NeurIPS 2023
Y2 - 10 December 2023 through 16 December 2023
ER -