Chi Zhang1, Xiang Zhang2, Chenxu Jiang1, Lei Yu1,*
1. School of Electronic Information, Wuhan University, Wuhan 430079, China
2. Computer Graphics Lab of ETH Zurich, Switzerland
The perceptible visual blurring and loss of information between frames often result from extended exposure times inherent in frame-based cameras, ultimately diminishing video quality. To address this problem, this paper introduces a unified self-supervised framework of Event-based Video Deblurring and Interpolation (EVDI++), where the high temporal resolution of events is leveraged to alleviate motion blur and facilitate intermediate frame prediction. Specifically, the Learnable Double Integral (LDI) network is designed to estimate the mapping relation between reference frames and sharp latent images. Then, we refine the coarse results and optimize overall training efficiency by introducing an exposure-transferred reconstruction module, enabling images to be converted with varying exposure intervals. We devise an adaptive parameter-free fusion strategy to obtain the final results, utilizing the confidence embedded in the LDI outputs of concurrent events. A self-supervised learning framework is proposed to enable network training with real-world blurry videos and events by exploring the mutual constraints among blurry frames, latent images, and event streams. We further construct a dataset with it real-world blurry images and events using a DAVIS346 camera, demonstrating the generalizability of the proposed EVDI++ in real-world scenarios. Extensive experiments show that our method compares favorably against state-of-the-art approaches and achieves remarkable performance on synthetic and real-world datasets.