Evaluating Image Super-Resolution Performance on Mobile Devices: An Online Benchmark

Deep learning-based image super-resolution (SR) has shown its strong capability in recovering high-resolution image details from low-resolution inputs. With the ubiquitous use of AI-accelerators on mobile devices (e.g., smartphones), increasing attention has been received to develop mobile-friendly SR models. Because of the complicated and tedious routines to deploy SR models on mobile devices, researchers have to use indirect indices, such as FLOPs, number of parameters, and activations, to evaluate and compare the efficiency of SR models. However, these indices cannot faithfully reflect the real performance of SR models on mobile devices. To mitigate this gap, we develop an online benchmark to automatically evaluate the performance of SR models on mobile devices. With a simple model definition file as input, e.g., PyTorch or ONNX file, our benchmark can generate the on-device evaluation indices and relevant statistics, including latency, memory, and energy consumption within 15 min, freeing the researchers from labor-intensive SR model deployment works. We further comprehensively study current SR models on mobile devices equipped with typical AI accelerators, such as Qualcomm, MediaTek, Hisilicon, and Samsung. Our benchmark provides a common platform for researchers to easily evaluate and compare the practical performance of their SR models on mobile devices. More details can be found at https://github.com/xindongzhang/MobileSR-Benchmark.