- Research Article
- Open Access
A Robust Subpixel Motion Estimation Algorithm Using HOS in the Parametric Domain
© E. M. Ismaili Aalaoui et al. 2009
- Received: 28 April 2008
- Accepted: 24 October 2008
- Published: 8 February 2009
Motion estimation techniques are widely used in todays video processing systems. The most frequently used techniques are the optical flow method and phase correlation method. The vast majority of these algorithms consider noise-free data. Thus, in the case of the image sequences are severely corrupted by additive Gaussian (perhaps non-Gaussian) noises of unknown covariance, the classical techniques will fail to work because they will also estimate the noise spatial correlation. In this paper, we have studied this topic from a viewpoint different from the above to explore the fundamental limits in image motion estimation. Our scheme is based on subpixel motion estimation algorithm using bispectrum in the parametric domain. The motion vector of a moving object is estimated by solving linear equations involving third-order hologram and the matrix containing Dirac delta function. Simulation results are presented and compared to the optical flow and phase correlation algorithms; this approach provides more reliable displacement estimates particularly for complex noisy image sequences. In our simulation, we used the database freely available on the web.
- Optical Flow
- Motion Vector
- Motion Estimation
- Parametric Domain
- Phase Correlation
The importance of image sequence processing is constantly growing with the ever increasing use of television and video systems in consumer, commercial, medical, and scientific applications. Image sequences can be acquired by film-based motion picture cameras or electronic video cameras. In either case, there are several factors related to imaging sensor limitations that contribute to the graininess (noise) of resulting images. Electronic sensor noise and film grain are among these factors . In many cases, graininess may result in visually disturbing degradation of the image quality, or it may mask important image information. Even if the noise may not be perceived at full-speed video due to the temporal masking effect of the eye, it often leads to unacceptable single-frame hardcopies and to poor-quality freeze-frames that adversely affect the performance of subsequent image analysis .
The motion estimation process must be able to track objects within a noisy source. In a noisy source, objects appear to change from frame to frame because of the noise, not necessarily as the result of object motion . Tracking objects within a noisy environment is difficult, especially if the image frames are severely corrupted by additive Gaussian noises of unknown covariance; second-order statistics methods do not work well.
Higher-order statistics (HOS) in general and the bispectrum (order 3) in particular have recently been widely used as an important tool for signal processing. The classical methods based on the power spectrum are now being effectively superseded by the bispectral ones due to some definite disadvantages of the former. These include the inability to identify systems fed by non-Gaussian noise (NGN) inputs and nonminimum phase (NMP) systems and identification of system nonlinearity . In these cases, the autocorrelation-based methods offer no answer. Out of all these, the identifiability of NMP systems has received the maximum attention from researchers.
HOS-based methods have been proposed to estimate motion between image frames [5–9]. In, the motion estimation is based on the bispectrum method for sub-pixel resolution of noisy image sequences. In , the displacement vector is obtained by maximizing a third-order statistics criterion. In , the global motion parameters are obtained by a new region recursive algorithm. In , several algorithms are developed based on a parametric cumulant method, a cumulant-matching method, and a mean kurtosis error criterion. The latter is an extension of the quadratic pixel-recursive method by Netravali and Robbins . In , it is shown that such statistical parameters are insensitive to additive Gaussian noises. In particular, bispectrum parameters are insensitive to any symmetrically distributed noise and also exhibit the capability of better characterizing NGN and identifying NMP linear systems as well as nonlinear systems. Therefore, transformation to a higher-order domain reduces the effect of noise significantly. In this correspondence, a novel algorithm for the detection of motion vectors in video sequences is proposed. The algorithm uses bispectrum model-based subpixel motion estimation in the parametric domain for noisy image sequences to obtain a measure of content similarity for temporally adjacent frames and responds very well to scene motion vectors. The algorithm is insensitive to the presence of symmetrically distributed noise.
The outline of this paper is as follows. First, the problem formulation is introduced in Section 2. In Section 3, we first present briefly the definitions and properties of the bispecrum and cross-bispectrum. Next, we describe the motion estimation in the parametric domain. High-accuracy subpixel motion estimation is discussed in Section 4. Section 5 presents an evaluation of the computational complexity of our algorithm. The results of the experimental evaluation of the proposed method are shown in Section 6 and compared to existing methods while Section 7 concludes the paper.
where denotes spatial image position of a point; and are observed image intensities at instants and respectively; and are noise-free frames; and are assumed to be spatially and temporally stationary, zero-mean image Gaussian (or non-Gaussian) noise sequences with unknown covariance; and is the displacement vector of the object during the time interval .
The goal is to estimate from and .
3.1. Definitions and Properties
In this subsection, some HOS functions are defined and their properties are described in order to provide the necessary tools to understand the motion estimation methodology.
where denotes the expectation operation; and are two shifted versions of the .
Also, can be non-Gaussian if it is independent and identically distributed (i.i.d.) and nonskewed (e.g., symmetrically distributed).
where denotes the 4D Fourier transform operation; and are the frequency coordinates for the 2D Fourier transform.
where indicates the complex conjugate.
These symmetry properties reduce the computational burden while calculating the bispectrum.
3.2. Parametric Model-Based Motion Estimation
where denotes the 4D inverse Fourier transform operation.
The least-squares solution is obtained and its maximum is determined. The image motion estimate is then .
Subpixel performance is a critical element of the proposed algorithm. With reference to our previously published work [16, 17], we are introducing a number of important new features, which improve the accuracy of the motion estimates.
where denotes the real part of complex array .
that is, the maximum peak of the phase correlation surface and its two neighboring values on either side, vertically and horizontally.
The fractional part of the vertical component can be obtained in a similar way using (24) instead of (23).
Finally the horizontal and vertical components of the subpixel accurate motion estimate are obtained by computing the location of the maxima of each of the above fitted quadratics.
In , it is shown that half-pixel accuracy motion vectors lead to a very significant improvement when compared to one pixel accuracy, whereas a higher precision results in negligible changes. Therefore, a half-pixel accuracy was chosen in our simulations.
The majority of the computational cost of the proposed bispectrum is due to the fast Fourier transform (FFT). Therefore, the fundamental computation required for bispectral estimates is given by (7), the triple product of the three individual Fourier transformations, while this computation is straightforward, limitations on computer time and statistical variance impose severe limitations on implementation of the definition of the bispectrum . On the other hand, we take advantage of the symmetrical properties of the bispectrum to reduce the computational complexity and memory requirements of calculating third-order statistics. It can now be calculated in any one sector and mapped onto the others .
The phase correlation is estimated by multiplying each coefficient by its complex conjugate, but each component of the bispectrum is estimated by a triple product of Fourier coefficients as demonstrated in (7). Thus, the number of operations required to compute the bispectrum is significantly increased relative to the phase correlation. There are independent components of the bispectrum while there are only independent components of the phase correlation for an image .
Our experiments have aimed at evaluating the performance of the proposed approach and comparing it with that of the optical flow and phase correlation techniques. For the optical flow method we used the implementation obtained from Bruhn method . In our simulation we used the database freely available on the web at http://vision.middlebury.edu/flow/. We contribute three types of data to test different aspects of all techniques: real sequences of independent motion; realistic synthetic sequences; and high frame-rate video. These sequences have been chosen for their difficult motion and their different characteristics. Although the original sequences are in color, only the luminance component is used to estimate the motion vectors.
The comparison between three methods for the computation time.
Average PSNR of motion compensated images for the three motion estimation techniques (unit: dB) for Tempete sequence.
where is the measured PSNR for frame and is the total number of frames. In Table 2, we observe that the decreases with larger apparent disparity between the global motion of the background and the local motion of the foreground. For each value of , we see that the is higher for the proposed scheme than the other methods.
In this paper, subpixel motion estimation algorithm using bispectrum in the parametric domain was presented. We have presented a collection of datasets for the evaluation of our method, available on the web at http://vision.middlebury.edu/flow/. In the case of the data is severely corrupted by additive Gaussian noises of unknown covariance, our method suppresses the effects of noise and simplifies the identification of the dominant peak on the correlation surface, unlike other techniques. At high noise levels SNR around 10 dB the optical flow and phase correlation techniques fail, yet even under these extreme conditions, the parametric bispectrum provides improvement in performance over the other algorithms. Overall, our scheme produces smoother displacement vector field with a more accurate measure of object motion in different SNR scenarios.
- Benmoussat N, Faouzi Belbachir M, Benamar B: Motion estimation and compensation from noisy image sequences: a new filtering scheme. Image and Vision Computing 2007,25(5):686-694. 10.1016/j.imavis.2006.05.010View ArticleGoogle Scholar
- Brailean JC, Kleihorst RP, Efstratiadis S, Katsaggelos AK, Lagendijk RL: Noise reduction filters for dynamic image sequences: a review. Proceedings of the IEEE 1995,83(9):1272-1292. 10.1109/5.406412View ArticleGoogle Scholar
- Armitano RM, Schafer RW, Kitson FL, Bhaskaran V: Robust block-matching motion-estimation technique for noisy sources. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '97), April 1997, Munich, Germany 4: 2685-2688.Google Scholar
- Bhattacharya S, Ray NC, Sinha S: 2-D signal modelling and reconstruction using third-order cumulants. Signal Processing 1997,62(1):61-72. 10.1016/S0165-1684(97)00115-1View ArticleMATHGoogle Scholar
- Ismaili Aalaoui EM, Ibn-Elhaj E: Estimation of subpixel motion using bispectrum. Research Letters in Signal Processing 2008, 2008:-5.Google Scholar
- Anderson JMM, Giannakis GB: Image motion estimation algorithms using cumulants. IEEE Transactions on Image Processing 1995,4(3):346-357. 10.1109/83.366482View ArticleGoogle Scholar
- Kleihorst RP, Lagendijk RL, Biemond J: Noise reduction of severely corrupted image sequences. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '93), April 1993, Minneapolis, Minn, USA 5: 293-296.Google Scholar
- Ibn-Elhaj E, Aboutajdine D, Pateux S, Morin L: HOS-based method of global motion estimation for noisy image sequences. Electronics Letters 1999,35(16):1320-1322. 10.1049/el:19990913View ArticleGoogle Scholar
- Sayrol E, Gasull A, Fonollosa JR: Motion estimation using higher order statistics. IEEE Transactions Image Processing 1996,5(6):1077-1084. 10.1109/83.503924View ArticleGoogle Scholar
- Netravali AN, Robbins JD: Motion-compensated television coding—part I. Bell System Technical Journal 1979,58(3):629-668.View ArticleGoogle Scholar
- Murino V, Ottonello C, Pagnan S: Noisy texture classification: a higher-order statistics approach. Pattern Recognition 1998,31(4):383-393. 10.1016/S0031-3203(97)00055-1View ArticleGoogle Scholar
- Sadler BM, Giannakis GB: Shift- and rotation-invariant object reconstruction using the bispectrum. Journal of the Optical Society of America A 1992,9(1):57-69. 10.1364/JOSAA.9.000057View ArticleGoogle Scholar
- Raghuveer MR, Nikias CL: Bispectrum estimation: a parametric approach. IEEE Transactions on Acoustics, Speech and Signal Processing 1985,33(5):1213-1230. 10.1109/TASSP.1985.1164679View ArticleGoogle Scholar
- Giannakis GB: On the identifiability of non-Gaussian ARMA models using cumulants. IEEE Transactions on Automatic Control 1990,35(1):18-26. 10.1109/9.45139View ArticleMathSciNetMATHGoogle Scholar
- Mendel JM: Tutorial on higher-order statistics (spectra) in signal processing and system theory: theoretical results and some applications. Proceedings of the IEEE 1991,79(3):278-305. 10.1109/5.75086View ArticleGoogle Scholar
- Ismaili Aalaoui EM, Ibn-Elhaj E: Estimation of motion fields from noisy image sequences: using generalized cross-correlation methods. Proceedings of the IEEE International Conference on Signal Processing and Communications (ICSPC '07), November 2007, Dubai, UAEGoogle Scholar
- Ismaili Aalaoui EM, Ibn Elhaj E: Estimation of displacement vector field from noisy data using maximum likelihood estimator. Proceedings of the 14th IEEE International Conference on Electronics, Circuits, and Systems (ICECS '07), December 2007, Marrakech, Morocco 1380-1383.Google Scholar
- Madec G: Half pixel accuracy in block matching. Proceedings on the Picture Coding Symposium (PCS '90), March 1990, Cambridge, Mass, USAGoogle Scholar
- Lii KS, Helland KN: Cross-bispectrum computation and variance estimation. ACM Transactions on Mathematical Software 1981,7(3):284-294. 10.1145/355958.355961View ArticleMathSciNetMATHGoogle Scholar
- Le Caillec J-M, Garello R: Comparison of statistical indices using third order statistics for nonlinearity detection. Signal Processing 2004,84(3):499-525. 10.1016/j.sigpro.2003.11.013View ArticleMATHGoogle Scholar
- Means RW, Wallach B, Busby D: Bispectrum signal processing on HNC's SIMD numerical array processor (SNAP). Proceedings of the ACM/IEEE Conference on Supercomputing (SC '93), November 1993, Portland, Ore, USA 535-537.View ArticleGoogle Scholar
- Bruhn A, Weickert J, Schnörr C: Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods. International Journal of Computer Vision 2005,61(3):211-231.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.