Smoothing and refinement of motion vector fields for video frame rate conversion applications

Abstract

The main goal of Video Enhancement (VE) applications is to improve the visual quality of the video by modifying the input frame sequence. The enhancement operations could be performed in the spatial domain (each frame of the sequence is modified by only using the information that is obtained from the same frame) and/or in the temporal domain (each frame of the sequence is processed by using the information that is obtained frame itself and its neighbors) depending on the type of the desired application. In order to achieve a higher visual quality level without producing any artifacts, it is very critical to correctly extract some key features of the frame sequence and carefully analyze them. Especially in the case of temporal domain processing, the motion information in the video content is possibly the most important feature. However, extracting the motion information of the 3D real world from the 2D frame sequence is an ill-posed problem. Unfortunately it is not possible to estimate true motion by using classical error minimization techniques. Therefore, it is inevitable to perform smoothing and refinement operations on the Motion Vector Fields (MVF) obtained via classical ME algorithms, if one wants to obtain an acceptable level of quality from the enhancement application. In this M.S. Thesis, a number of practical state-of-the-art ME methods are considered and some MVF post-processing (PP) operations (smoothing and refinement) are jointly investigated in detail. The effect of each algorithm on the quality of a Frame Rate Conversion (FRC) application is observed. Moreover, two contributions have been made in the area of MVF post-processing; the first contribution focuses on improving the performance of an existing strategy and the second contribution targets to decrease the computational load of another MVF post-processing strategy. It has been observed that the performance of the implemented FRC application is quite promising and fulfills the expectations of most of the today’s consumer electronics applications.