 Research
 Open access
 Published:
Phase congruency based on derivatives of circular symmetric Gaussian function: an efficient feature map for image quality assessment
EURASIP Journal on Image and Video Processing volume 2023, Article number: 11 (2023)
Abstract
Image quality assessment (IQA) has become a hot issue in the area of image processing, which aims to evaluate image quality automatically by a metric being consistent with subjective evaluation. The first stage of conventional IQA model design is the qualityaware feature selection. Taking advantages of early visual feature, Phase congruency (PC) operates in frequency domain to measure local structures such as edges, corners, lines, etc., by computing the local amplitudes and local energies in multiple scales. Conventional local PC features are calculated with logGaborbased filtrations in several orientations, and usually combined with other features for IQA model design. Generally, a directional filter is sensitive to the changes on specific direction, and insensitive to other directions. This leads to multidirectional calculation and much time consumption in practical applications. Recently, researchers suggested that spatially circular symmetric filters, such as gradient magnitude (GM) and Laplacian of Gaussian (LoG), are highly efficient quality features and hence have been widely used in various IQA model designs. With the oddsymmetric and evensymmetric properties of GM and LoG operators, the two features are a suitable pair for PC compositions and can be computed uniformly by a Gaussian function with one scale factor. In this regard, we propose to combine GM and LoG signals to construct a new PC model with nondirectional property. With ability to catch different types of distortions, the proposed PC feature can be promoted to a fullreference IQA model simply with average pooling or standard deviation pooling, and shows stateoftheart performance compared with existing methods. Furthermore, our proposed PC algorithm can take the place of conventional PC component in wellknown FSIM metric, which achieves improved performance and spends less in computation cost. This study suggests that the proposed circular symmetric PC feature is a highly efficient quality feature and can be exclusively used in IQA model designs.
1 Introduction
With the rapid growth of the technologies in digital communications and multimedia applications, more and more image data are produced for human observations. Human visual system (HVS) is the ultimate observer to judge the image quality. In order to improve efficiency, it is necessary to evaluate image quality automatically with a critical metric for different systems. Image quality assessment (IQA) model is developed to aim at estimating the objective quality of images as closely to subjective judgements as possible. Among different IQA algorithms, fullreference (FR) IQA works when the original reference image is completely provided, noreference (NR) is employed when the pristine reference image is not available, and reducedreference (RR) works at the situation where partial information of reference image is provided. Up to now, FRIQA has been extensively applied for various cases, such as image reconstruction, network transmission, image coding and compression, etc. In the premise of simultaneous presence of reference and distorted images, FRIQA metric can be applied in optimization of image processing systems [1,2,3,4], including the help of training deep neural networks for various vision tasks [2,3,4].
Conventional FRIQA metrics such as mean squared error (MSE) and the peak signaltonoise ratio (PSNR), which compute the image quality index on the intensity domain, evaluate the distortion degree by an arithmetic difference between reference and distorted images. The structural similarity (SSIM) index [5] can capture structural similarity information based on the assumption that the HVS is sensitive to local structure of visual signals. Based on that, the multiscale SSIM (MSSSIM) metric [6] compute the contrast and structural similarity at five scales altogether. Another variant of SSIM is the informationweighted SSIM (IWSSIM) metric [7], in which different types of local regions are considered to make different contributions to the quality of an image. Riesz transforms based feature similarity (RFSIM) [8] and spectral residualbased similarity (SRSIM) [9] are also improvements based on SSIM. The information fidelity criterion (IFC) [10] was proposed using the information theory, and was upgraded to more efficient metric named visual information fidelity (VIF) [11]. Based on property of HVS understanding the image in lowlevel vision [12, 13], the feature similarity (FSIM) index [14] measures the local structure by the value of phase congruency (PC) and image gradient. Image gradient has also been extensively applied to evaluate image distortion which yielded the gradient similarity (GSM) algorithm [15] and the gradient magnitude similarity deviation (GMSD) [16]. Another method to measure local structure of visual signal is to employ the Laplacian of Gaussian (LoG) filter, which proves to be approximate to the decorrelating mechanism of the retinal ganglion receptive field in HVS [17, 18]. Nonshift edge based ratio (NSER) [19] makes use of image edges produced by LoG filters, which are qualityaware in representing structural distortions. More related researches prove that LoG is highly efficient in FR [20, 21], RR [22,23,24], and blind IQA [25] model design. In these related studies, LoG shows ability to retain structural distortions in all directions because of its circularly symmetric property. Especially, the joint distribution of GM and LoG in [25] has proven efficient in IQA feature representation and IQA model design, and the relationship between GM and LoG was explored for the first time. Since the nondirectional filters have proven qualityaware, GM and LoG generated from Gaussian function on the same scale are more universal in theoretical calculation and subsequent optimization in IQA related applications. Comprehensive surveys and detailed comparisons of modern IQA metrics are discussed in many literature [26,27,28,29].
Aside from conventional IQA methods, convolutional neural networks (CNN) have been applied to IQA issues in recent researches [30,31,32]. Although existing CNNs have reached good performance in predicting image quality, studies on IQA models without training is still meaningful in practical applications. On the other hand, qualityaware feature maps have also been employed as similarity maps [33] and qualityaware loss [34] which helps to predict the discrepancy map. Therefore, qualityaware feature design is still a valuable issue in related fields.
In general, an FRIQA model is usually consisted of three methodical steps: feature extraction from the reference image and distortion image, pointwise quality measure between the features of the reference and distortion images, and pooling the local quality measures over the image [5]. The quality score is attained based on the pooling result [5]. Meanwhile, the image quality feature is acquired by handcrafted design or machine learning tech [35]. The quality measure is normally carried out by a distance metric [16] or a learnt network [25]. The pooling strategy is mostly either of mean or deviation computation over the local quality measures [5, 16]. Image feature reflects a specific aspect of image information by which meaningful image signal is represented and synthesized, and hence an image processing algorithm can be designed to realize a specified processing purpose. There are numerous image features proposed for various image processing tasks, such as image gradient, LoG signal, Gaborlike function, etc. However, conventional image features were proposed in accordance with natural images. They are efficient in representing natural image structures and have been used for usual image processing tasks, such as image denoising, superresolution, image restoration, etc. In IQA model design, image feature extraction is not only for natural (reference) images, but also for distorted images. For example, in lowlevel vision, a natural image consists of a plenty of directional features. In this type of image structures, pixel values are consistent along with its direction, so that Gaborlike functions are highly efficient to represent image lowlevel structures as independent components [36]. However, in distortion images, image structures are distorted to varied ones and IQA model measures the variation to assess image quality. In this case, distorted image structures may not be well represented by the directional features such as Gaborlike functions since the image values may be changed a lot along with its direction. Alternatively, circular symmetric filters, e.g., gradient magnitude, LoG signal, are more efficient in IQA design since circular symmetric filters do not have a preferred direction and hence easily sense the distortion information of the image. Indeed, there are a lot of successful IQA models that have been proposed based on the circular symmetric filters [16,17,18,19,20,21,22,23,24,25], as mentioned in the previous paragraph.
As one of the most important components in visual signal processing, phase information carries more structural information than the spectral amplitude does in an image [37], where lowlevel features such as edges and corners show consistence in phase according to Fourier translation. Based on physiological and psychophysical evidences, the PC theory provides a simple but biologically plausible model of how mammalian visual systems detect and identify features in an image [38,39,40]. The experiment based on odd and even symmetry of visual receptive fields [38] explains that it is an efficient means for the visual system to locate the edges by the sum of the squared output of odd and evensymmetric filters that always peak at points of phase congruence. As the result, points of high PC value represent highly informative features. As a dimensionless index, conventional PC algorithm was defined by Morrone et al. in 1986 [38] and was developed by Kovesi [41, 42] based on a local energy model, which assumed that features are more evidently perceived at points where the Fourier components are maximally in phase. Many scholars have made use of PC features in relevant fields, where the computation is operated in frequency domain after filtration with multiorientations [41, 42]. Multiscale PC has been applied in edge visual saliency detection [43], and the feature map can reflect fundamental structures and textures. Combined with Complex Wavelet Transform (CWT), the concept of PC is also efficient in image representation [44] and redundancy removal. The phasebased algorithms usually employ 2D Discrete Fourier Transform (DFT), Gabor filters [45], or logGabor filters [46] to calculate phase information. In studies on image quality evaluation, FSIM algorithm [14] combined PC with gradient magnitude, which is computed as the secondary feature to encode contrast information [47]. Combined GM and local binary pattern (LBP) in PC domain at multiple scales to design the NRIQA method with training a support vector regression model. A recent proposed metric based on symmetry phase congruency (SPCM) [48] also combined PC with GM in similarity computation.
In calculation, conventional PCbased metrics for IQA models use the evensymmetric and oddsymmetric components of Gaborlike functions with multiorientations, for example, the fourorientation Gaborbased PC on four scales employed in FSIM index [14]. In lowlevel vision, natural image consists of directional features, so that Gaborbased PC is highly efficient to represent lowlevel structures. However, structures are changed on arbitrary directions in distortion images. When pixel value varies along with the initial structural direction, distortion structure would be neglected by Gaborbased PC, since a single Gaborbased operator is only highly sensitive to changes orthogonal to the edge. Consequently, more orientations are needed in calculation, and thus computational complexity is increased. On the contrary, circular symmetric filters treat changes on all directions equally, which are more efficient in IQA design. This motivates us to design a new term of PC computing method where only circular symmetric filters are used. As is well known, gradient magnitude and LoG filter are the firstorder and secondorder derivatives of Gaussian function, respectively. GM and LoG are qualityaware features as mentioned above, and can be uniformly generated from circular symmetric Gaussian function. Obviously, gradient filter is oddsymmetric, and LoG filter is evensymmetric. Therefore, GM and LoG maps can represent the oddsymmetric and evensymmetric components of an image separately.
In this study, we utilize GM and LoG maps for PC computing to obtain a nondirectional PC operator. The proposed PC feature can be promoted to an FRIQA model by simply utilizing a similarity calculation and an average or standard deviation pooling strategy, and the model proves to be stateoftheart compared with the competitors. Furthermore, we replaced the PC algorithm in wellknown FSIM with our PC computation method to test the accuracy of the proposed method in representing phase information. The experimental results revealed that the proposed PC feature map can correctly take the place of conventional PC algorithm, and the calculation method is faster than the original one.
The rest of this paper is organized as follows. The proposed phase congruency method and a new FRIQA metric are introduced in Sect. 2. Section 3 describes the experimental setups. In Sect. 4, results and comparisons on three benchmark databases are presented. Section 5 concludes the paper.
2 Methods
2.1 Phase congruency
The phase congruency (PC) that is a dimensionless quantity was first proposed as a frequencybased algorithm [38] instead of spatially processing on images. The basic concept of PC algorithm is that the Fourier components are maximal in phase where the local structure is perceived in an image. According to the extensively used PC algorithm developed by Kovesi in [41], one consider a onedimensional signal \(f(x)\), and denote the evensymmetric filter and the oddsymmetric filter by \({M}_{n}^{e}\) and \({M}_{n}^{o}\) separately on scale n and define a vector to represent the responses of the signal \(f(x)\) after filtered by \({M}_{n}^{e}\) and \({M}_{n}^{o}\) on scale n as follows:
where \({e}_{n}\left(x\right)\) and \({o}_{n}\left(x\right)\) are the output of \({M}_{n}^{e}\) and \({M}_{n}^{o}\) filtering at position x. The local amplitude on scale n is defined as:
The local energy function can be written as:
where
The PC of onedimensional signal is defined as:
where \(\varepsilon\) is a small positive constant to prevent the denominator from being zero.
Different from conventional methods, in this study we apply the GM and LoG filters, which are the firstorder and the secondorder derivatives of Gaussian function, instead of the logGaborbased directional filters \({M}_{n}^{o}\) and \({M}_{n}^{e}\). As shown in Fig. 1, the normalized firstorder and secondorder derivatives of 1D Gaussian function are oddsymmetric and evensymmetric, respectively.
For 2D signals, the image gradient magnitude defined as the root mean square of image directional gradients along two orthogonal directions is still the firstorder derivative of 2D Gaussian filter. We denote the Gaussian function by \(G\), then the gradient filter on horizontal direction and vertical direction are defined as:
where the variables \(x\) and \(y\) denote the coordinate of the input image, parameter \(\sigma\) denotes the scale factor of the Gaussian function. We denote the image by \({\varvec{I}}\), and convolve the image with the two directional derivative filters to produce the horizontal and vertical gradient images \({{\varvec{d}}}_{n,x}\) and \({{\varvec{d}}}_{n,y}\) on scale \(n\), thus the GM of an image is computed as:
As the secondorder derivative of 2D Gaussian function, the LoG filter is defined as:
where the variables \(x\) and \(y\) denote the coordinate of the input image, parameter \(\sigma\) denotes the scale factor of the Gaussian function. Thus, the LoG map on scale \(n\) can be computed as:
In order to remove the contrast variation in the image of a large scale, we use divisive normalization [20, 25] as:
where \({c}_{0}\) is a positive constant to ensure the stability of calculation, and \({G}_{n}\left(x,y\right)\) represents a largescale Gaussian filter employed for each scale n.
Therefore, the 2D local amplitude and local energy on scale \(n\) can be written as:
where
Thus the PC can be computed by:
The value of PC ranges from 0 to 1. According to the definition, PC value equals 0 means no significance feature here, whereas the value 1 means the most important feature existed. Therefore, the PC map constructed from the oddsymmetric and the evensymmetric components can reflect the structural information of an image.
An analysis of the GM and LoG responses for different types of edge signals and distorted edges is shown in Fig. 2. A pristine 1D edge signal, a Gaussian blurred version, a Gaussian noise corrupted signal, and a DCT compressed signal are demonstrated in the first column of Fig. 2a–d, respectively. The next two columns are the corresponding GM and LoG responses on two different scales. The last column shows the phase congruency curve computed by GM and LoG responses. The result validated that the proposed PC feature gives the highest value at the edge position for both original ideal edge and the corresponding distorted version, no matter which type of edge is to be processed. Based on this property, the proposed PC feature is suggested to have the ability to reflect the structural information of reference and distorted images.
In order to explain why nondirectional PC feature is better than directional Gaborbased PC in representing changed structures, we demonstrate a comprehensible comparison of edge extraction from structural changes in Fig. 3. The first row shows the proposed PC map along with Gaborbased PC maps using one and four orientations in computation separately. The horizontal and vertical edges are not completely consistent with each other in a 1orient horizontal Gaborbased PC feature map. Although multiorientation neutralizes the difference a lot, some artifacts still exist. In the second row, the input image has small changes on both directions. Obviously, the feature map generated by 1orient Gaborbased PC gives different responses on horizontal and vertical edges, although the two changes share the same shape and value. When the number of orientations increases in Gaborbased PC, the difference between directions is reduced. Nevertheless, the feature map generated by 4orient Gabor PC is not as smooth as the proposed PC map. The third row is the same as the second row except the images are rotated with 45 degrees. 1orient Gabor PC responses the same on the two directions, but cannot clearly represent the image structure. The feature map of 4orient Gabor PC seems to be better in extracting edges, while artifacts are still difficult to eliminate. However, the proposed nondirectional PC operator does not suffer from this problem. Moreover, the proposed PC keeps steady on arbitrary directions and would not be affected by image rotation, which is more in line with the function of the HVS.
For further comparison and better illustration on how PC describe the image structure, Fig. 4 shows the proposed PC feature map on two reference images and corresponding distorted images, compared with GM, LoG, and conventional Gaborbased PC feature proposed by Kovesi [41]. Note that the proposed PC is computed on small scales. In the reference image, the proposed PC map reflects the significance of local structures even if the local contrast is low, thus PC is able to capture more details of structural information than GM and LoG maps. For distorted images, the new PC map still shows more distorted structures than GM and LoG maps, regardless of distortion types. Especially, the distorted edges in JPEG image can be clearly sensed by the proposed PC map, which are hard to directly emerge by GM and LoG maps according to Fig. 4. It is obvious that the Gaborbased PC operator cannot describe the distortion as clearly as the proposed PC map. For example, in the JPEG image, the proposed PC map displays the blocky contours clearly, but Gaborbased PC leads to ineluctable artifacts beyond edges. In the blur image, Gaborbased PC gives high response to the background, where human observers do not notice. Therefore, the proposed PC can figure out distortion structures more completely and clearly. This comparison proclaims that the PC map constructed by GM and LoG is an efficient feature map which contains enough structural distortion information to distinguish the faint features in distorted images, thus can be helpful to improve the prediction accuracy for IQA.
2.2 FRIQA model based on the proposed PC algorithm
Since the PC value represents the significance of edges, the quality map that measures local similarity can use PC between signal \({f}_{1}(x)\) and \({f}_{2}(x)\), as defined by Eq. (19):
where \({c}_{1}\) is a positive constant to prevent division by zero and increase the stability, the subscript \(g\) means the calculation is done on grayscale images or luma channel of color images. This is a commonly applied measure to define the similarity of two positive real numbers [5], and the result of each image pixel ranges within (0, 1]. Higher result means higher similarity between distorted and reference images.
The calculation of quality map can be directly applied on grayscale images. As for color images, we transform RGB signals to YIQ color space by a formula in [49]:
The similarity between chromatic channels is generated from:
where \({I}_{1}\), \({I}_{2}\), \({Q}_{1}\), \({Q}_{2}\) are chromatic channels of the reference and distorted images, \({c}_{2}\) and \({c}_{3}\) are constants that balance the data. Then the quality map for color images is defined as follows:
where \(\lambda\) is a constant to regulate the influence level of chromatic channels, the subscript \(c\) means the calculation is for color images.
In order to yield the overall score of an image, the pixelbased similarity map should be converted to a scalar score with a proper pooling strategy. Weighted pooling methods are widely discussed and many researches on pooling strategy have been done for image and video quality assessment [7, 50,51,52,53]. Average pooling is employed based on the hypothesis that each part of the image contributes the same importance in overall quality, which is the most commonly used method for pooling process. We compute the quality map with average pooling method as:
where N represents the number of pixels in the image, \(i\in \{g,c\}\) denotes whether the input images are grayscale or color images.
For further comparison, we utilize a standard deviation pooling strategy that considers different local structures with different degradations. It has been proven to be efficient for gradient similaritybased IQA method in [16], thus we compute the standard deviation of the similarity map of PC as follows:
The average pooling result gives higher score to better quality image since it measures the average similarity between reference and distorted images, whereas the standard deviation pooling gives higher score to lower quality image with larger distortion, on account of the ability to measure difference between distorted and reference images.
We made statistics on LIVE database [54] and found a nonlinear relationship of 1/3rd power law between the predicted scores and subjective scores. This nonlinearity also exists in CSIQ [55] and TID2013 [56] databases. Hence, in order to obtain a balanced relation between predicted quality scores and subjective scores, we use a nonlinear transformation to calculate the score as follows:
where \(i\in \{g,c\}\) denotes whether the input images are grayscale or color images. The modification makes the output of our model more reasonable and practical owing to the uniform distribution. Note that the transformation does not change the rank order of the estimated scores of distorted images, hence it has no influence in the evaluation of monotonicity. Nevertheless, we will explore the nonlinear relationship between objective score and subjective score in our further investigation.
2.3 Replace the PC computation in FSIM
FSIM metric [14] is a wellknown FRIQA model that has achieved outstanding quality evaluation performance and has been widely applied for various applications [57,58,59]. FSIM separates the feature similarity measurement between signal \({f}_{1}(x)\) and \({f}_{2}(x)\) into two components, each for PC or GM. The feature \({S}_{L}\left(x\right)\) combined PC with GM is defined as:
where \({S}_{G}(x)\) is the similarity measure of image gradient. We replace the PC computation with our proposed PC metric, and compute the objective score in the way as the FSIM algorithm does:
where
We compared this result with the original FSIM performance to test the validity and accuracy of our computation to express the structural features as a phase congruency expression.
3 Experimental setup
We test the proposed FRIQA model on three benchmark databases: LIVE [54], CSIQ [55], and TID2013 [56]. LIVE database contains 29 reference images and 779 distorted images generated with 5 distortions types: JPEG compression, JPEG2000 compression, white noise, Gaussian blur and simulated fast fading. CSIQ database consists of 30 reference images and 866 distorted images generated with 6 different distortions types: JPEG compression, JPEG2000 compression, additive white noise, additive pink Gaussian noise, Gaussian blur, and global contrast decrements. The Difference Mean Opinion Score (DMOS) values are provided in LIVE and CSIQ databases as the subjective score for distorted images, which is a positive score representing the degree of distortion from human evaluation. The TID2013 database is the largest of the three databases which contains 3000 distorted images created from 25 reference images with 24 types of distortions at 5 levels. The mean opinion score (MOS) are provided as subjective score of human evaluation, which gives higher value to higher subjective image quality. The most commonly applied methodology in evaluation of IQA models is the Spearman rankorder correlation coefficient (SROCC). It takes consideration of prediction monotonicity, which is a typical aspect of IQA performance [60].
The SROCC between predicted score and reference subjective score is defined as:
where \(X\) and \(S\) are the vectors of the predicted results and subjective scores of the test images, and \({d}_{i}\) is the difference between the rank of an objective score in \(X\) and the rank of its corresponding subjective score in \(S\).
The Pearson Linear Correlation Coefficient (PLCC) metric, which measures the prediction accuracy, should be applied after a nonlinear regression. A logistic function with an added linear term [26] is employed as follows:
where \({\beta }_{i}, i=1, 2, \dots , 5,\) are parameters to be fitted in the regression function. \({X}_{r}\) denotes the IQA scores after nonlinear regression. The PLCC is defined as:
where \({\overline{X} }_{r}^{T}\) and \(\overline{S }\) denote the vectors of scores with mean value removed.
The root mean square error (RMSE), which evaluates the prediction consistency of the IQA performance, is computed as:
In the experimental setup, the constant \({c}_{0}\) in divisive normalization by Eqs. (12) and (13) is selected as 120. According to the relationship between adjacent scales, the PC calculation is operated on two scales that larger filtering window is twice the width of the smaller one. We set the standard deviation of the original multiscale Gaussian functions as 0.3 and 0.6, respectively. The constant \(\varepsilon\) in PC calculation by Eq. (18) is selected as 25, and the constant \({c}_{1}\) in similarity map calculation by Eq. (19) is selected as \(3\times {10}^{5}\), which show the best property in the experimental performance of the proposed model on grayscale images. \({c}_{2}\) and \({c}_{3}\) in Eq. (21) and Eq. (22) are set as 200, and \(\lambda\) in Eq. (23) is selected as 0.03 according to [14]. When replacing the PC algorithm in FSIM, we select the scale factor of Gaussian functions as 2 and 4, since the image gradient in FSIM is calculated by Prewitt operator with a smallscale window, thus the phase computation needs to catch structural information with a larger range. According to the chosen scale of Gaussian function, the constant \({c}_{0}\) is adjusted to 60, \(\varepsilon\) is selected as 5.5, and \({c}_{1}\) is selected as 0.03.
4 Results and discussion
4.1 Experimental results on different databases
In order to validate the performance of the proposed metric, we investigate the model scores for images from the three benchmark databases, and compute the SROCC, PLCC, and RMSE between the model scores and subjective opinion scores provided by the databases as the performance criteria. The performances of proposed metric and competitors are shown in Table 1, where \({q}_{m,g}\) and \({q}_{\mathrm{sd},g}\) are the results on grayscale images, \({q}_{m,c}\) and \({q}_{\mathrm{sd},c}\) are results on color images. The source codes of the competitors in “Grayscale” group are publicly available and we have verified that they worked correctly according to the results in the original papers. Since the databases and calculation accuracies provided in the original literatures are not completely consistent with one another, we ran all the conventional algorithms on the three major databases with the same software and hardware systems in the whole process of our experiment to ensure the authenticity and fairness. On each database, the top three metric performances are presented in boldface. The “Color” group shows experimental results on color images, and the best metric for each database is shown in bold face. In addition, for a more comprehensive comparison, “DeepLearning” group shows results of two deeplearning metrics [61, 62], where the data are provided by their original papers.
In Table 1, the result of standard deviation pooling method is much more efficient than the average pooling method in grayscale group. For grayscale images, the proposed model with standard deviation pooling ranks 4th on LIVE database, 2nd on CSIQ, 1st on TID2013 database, and 1st on average across the three databases. Particularly, the proposed method performs significantly better than other metrics on TID2013 database. According to the experimental results, the proposed model shows stability and efficiency on a large range of different distortion types, since TID2013 is one of the most extensively used database which contains the most types of distortion and varies of image content. Besides, \({q}_{\mathrm{sd},g}\) performs better than DISTSGray [1, 61] on grayscale images, and \({q}_{\mathrm{sd},c}\) performs better than DISTSColor model in terms of SROCC. Although DeepSim [62] shows better performance on LIVE database, the proposed model still achieves stable performances on CSIQ and TID2013, and no training process is required. The result of \({q}_{\mathrm{sd},c}\) on color images is not higher than FSIMc, however, we checked the parameter settings and found that adjusting Gaussian scale factor can improve the accuracy of color image quality prediction, especially on TID2013 where chromatic distortions lead to a wide range of color change. According to the relationship between adjacent scales, the PC calculation is operated on two scales that larger filtering window is twice the width of the smaller one. The relationship between SROCC and the smallscale factor of Gaussian function is shown in Fig. 5. Although we selected small scales for PC in our proposed IQA model, the scale factor is an adjustable parameter for the proposed PC operator. That is, the scale factor can be selected according to the specific situation where PC feature is applied.
In Table 2, we tested the effectiveness of the proposed PC feature in representing phase congruency information in FSIM metric where PC is applied as a dimensionless measure for the significance of a local structure. The data of \({\mathrm{S}}_{\mathrm{FSIM}}\) and \({\mathrm{S}}_{\mathrm{FSIM}}\)c in Table 2 represent the results that we replace the PC algorithm in FSIM and FSIMc metrics with our proposed PC metric, compared with the original FSIM and FSIMc results in terms of SROCC.
It is shown that the performances of \({\mathrm{S}}_{\mathrm{FSIM}}\) and \({\mathrm{S}}_{\mathrm{FSIMc}}\) are very close to FSIM and FSIMc methods, and the result has been slightly improved on CSIQ and TID2013 databases. In fact, the proposed PC method reduces the computational complexity and shows better performance in average than the original PC algorithm. Such results validate that our PC calculation method is effective for phase information representation for IQA tasks compared with traditional PC method. The computational PC model constructed from GM and LoG maps works stably as a phasebased mechanism without calculating the phase information directly from the multiscale frequency domain.
Figure 6 shows the scatter plots of predicted quality scores versus subjective scores of the proposed model, compared with other metrics on LIVE database. The horizontal axis denotes the objective scores computed by different IQA metrics, while the vertical axis denotes the DMOS values. In this figure, we can see the monotonicity and consistency of the proposed and comparison IQA metrics more intuitively, since the scatter plots reflects the relationship between objective and subjective evaluations by pairs of coordinates.
4.2 Experimental results on individual distortion types
For further comparison of the performance between the proposed model and the competitors, we present the performance of proposed model and the competition metrics on each individual distortion type in terms of SROCC in Table 3. For each distortion type, the top three algorithms are presented in boldface. The last row counts the number of times that each algorithm reaches the top three across all distortion types.
According to the table, the proposed model with standard deviation pooling works stably and robustly on most distortion types across the three databases, and finally reaches the highest hit number compared with all the competitors. Particularly, it performs better on the distortion types where structural changes occurs rather than contrast and intensity changes, since the PC operator measures how salient the edge is.
4.3 Comparison of running time
Since the computational efficiency plays an important role in practical applications, it is necessary to improve the operation speed and efficiency of IQA metrics. In Table 4, we present the running time of the proposed method and 9 competing FRIQA models on each 512 × 512 image in average. Particularly, the running time of \({\mathrm{S}}_{FSIM}\) represents the result that we replace the PC algorithm in FSIM metric. We ran our algorithm and competitive conventional metrics using MATLAB R2019a on a personal desktop computer with Intel Core i56400 CPU @2.7 GHz and 8G RAM. The source codes of the competitors were provided by their authors.
According to Table 4, PSNR, GMSD and SSIM are the three fastest metrics owing to the low computational complexities, while the proposed \({q}_{\mathrm{sd},g}\) model ranks the 4th, since phase computation needs multiscale calculation. The proposed PCbased model runs 5.38 times faster than FSIM, 10.62 times faster than IWSSIM, and 22.71 times faster than VIF. In particular, when replacing the PC computation in FSIM with the proposed PC algorithm, the code runs 3.73 times faster than the original FSIM model, which validated that the proposed PC based on derivatives of Gaussian function is more efficient and less complex in computation than traditional PC method.
As a supplement for running time comparison, Table 5 shows the comparison of the running time of our proposed PC metric and conventional PC on 512 × 512 image where only the PC feature maps are calculated. Notice that the PC calculation here is done without downsampling, which is different from the IQA metric calculation in Table 4. Obviously, the proposed PC metric runs more than 2 times faster than Gaborbased PC. More importantly, we only calculate on two different scales when replacing the PC algorithm in FSIM, since the twoscale feature shows better performance. The experimental results show that the proposed PC algorithm saves a lot of time compared with Gaborbased PC metric. This is mostly because of the simplicity of the computation on symmetric Gaussian function and the nondirectional features generated by Gaussian derivatives, which shows less complexity compared with the Gaborbased features. In the experiment, both algorithms are implemented by MATLAB code only. Gaborbased PC uses Fast Fourier Transform (FFT) to separate odd and even components, where at least six parameters are necessary to control the scale and orientation selections. Meanwhile, a threshold is needed to penalize low PC values in order to reduce artifacts. On the contrary, the proposed circular symmetric PC is calculated in spatial domain with one scale factor for each scale, without consideration of direction selection and artifacts issues.
5 Conclusion
In this paper, we proposed a novel algorithm of phase congruency map computation to represent qualityaware structural information of an image, and then proposed an FRIQA model based on the quality feature. Instead of traditional multiscale logGabor filters with multiorientations, we utilized image gradient magnitude and Laplacian of Gaussian filters, which are the firstorder and the secondorder derivatives of Gaussian function, to generate the oddsymmetric and evensymmetric components of an image when computing the dimensionless phase congruency index. This calculation with Gaussianbased filters is much simpler in computation and more concise than traditional PC algorithm with logGabor filters. We have also validated that this phase congruency map contains enough structural information and can extract faint features such as the edges, lines, corners, and other local structures from both reference and distorted images, which makes it available to measure the degree of distortions.
The experimental results have indicated that the proposed method performs consistently and stably on different distortion types across three benchmark databases, while it is less computationally complex (faster to compute) compared with other outstanding metrics. Especially, the experiment on FSIM metric where we replaced the original PC algorithm with the proposed PC feature map shows that the firstorder and secondorder derivatives of Gaussian function can be constructed as an efficient PC alternative. Meanwhile, with performing similarly in prediction results but much faster in running time compared with conventional PC algorithms, the proposed PC shows to be a stateoftheart feature map for IQA model design.
Although the proposed model works slightly better than GMSD, the PC feature based on circular symmetric Gaussian derivatives actually reflect the characteristics of image information where image components in different frequencies show similar responses in phase. Therefore, PC feature detects structural information at all kinds of phase angles, whereas image gradient mostly focuses on step features with a phase angle of 0 or 180 degrees. Despite that the gradient map in the proposed feature resembles the gradient magnitude in GMSD and FSIM, Gaussian derivatives are strictly circular symmetric filters, which are different from Prewitt or Sobel operator. Because of the nondirectional properties and the ability to reflect image information in both odd and even phases, the proposed PC feature is expected to play an important role in image enhancement applications based on IQA features.
In conclusion, this paper proposes an efficient PC feature map based on derivatives of nondirectional Gaussian function. This symmetric operator proves to be qualityaware and works stably in the proposed FR model with reduction in running time compared with conventional PC metrics. Therefore, the proposed feature map would play an important role in the image qualityrelated applications in future researches.
Availability of data and materials
The MATLAB code of the proposed FRIQA model and the example on replacing PC in FSIM are publicly available at: https://gr.xjtu.edu.cn/web/xqmou/ccm.
Abbreviations
 FR:

Full reference
 GM:

Gradient magnitude
 IQA:

Image quality assessment
 LoG:

Laplacian of Gaussian
 MSE:

Mean squared error
 PC:

Phase congruency
 PLCC:

Pearson linear correlation coefficient
 PSNR:

Peak signaltonoise ratio
 SROCC:

Spearman rankorder correlation coefficient
References
K. Ding, K. Ma, S. Wang, E.P. Simoncelli, Comparison of fullreference image quality models for optimization of image processing systems. Int J Comput Vision 129(4), 1258–1281 (2021)
S. He, Q. Li, Y. Liu, et al. Semantic segmentation of remote sensing images with selfsupervised semanticaware inpainting. IEEE Geoscience and Remote Sensing Letters, 19, 2022.
A.B. Szczotka, D.I. Shakir, M.J. Clarkson et al., Zeroshot superresolution with a physicallymotivated downsampling kernel for endomicroscopy. IEEE Trans Med Imaging 40(7), 1863–1874 (2021)
H. Zhao, X. Qiao, Y. Ma et al., Transformerbased selfsupervised monocular depth and visual odometry. IEEE Sens J 23(2), 1436–1446 (2023)
Z. Wang, A.C. Bovik, H.R. Sheikh, E.P. Simoncelli, Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4), 600–612 (2004)
Z. Wang, E. P. Simoncelli, A. C. Bovik, Multiscale structural similarity for image quality assessment. Proc. IEEE 37th Conf. Rec. Asilomar Conf. Signals, Syst. Comput., 2, 2003, pp. 1398–1402.
Z. Wang, Q. Li, Information content weighting for perceptual image quality assessment. IEEE Trans Image Process 20(5), 1185–1198 (2011)
L. Zhang, L. Zhang, X. Mou, RFSIM: a feature based image quality assessment metric using Riesz transforms. Proc. IEEE Int. Conf. on Image Process, 2010, Hong Kong.
L. Zhang, H. Li, SRSIM: A fast and high performance IQA index based on spectral residual. IEEE Int. Conf. on Image Process, pp. 1473–1476, 2013.
H.R. Sheikh, A.C. Bovik, G. de Veciana, An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Trans Image Process 14(12), 2117–2128 (2005)
H.R. Sheikh, A.C. Bovik, Image information and visual quality. IEEE Trans Image Process 15(2), 430–444 (2006)
D. Marr, E. Hildreth, Theory of edge detection. Proc R Soc Lond B 207(1167), 187–217 (1980)
M.C. Morrone, D.C. Burr, Feature detection in human vision: a phasedependent energy model. Proc R Soc Lond B 235(1280), 221–245 (1988)
L. Zhang, L. Zhang, X. Mou, D. Zhang, FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process 20(8), 2378–2386 (2011)
A. Liu, W. Lin, M. Narwaria, Image quality assessment based on gradient similarity. IEEE Trans Image Process 21(4), 1500–1512 (2012)
W. Xue, L. Zhang, X. Mou et al., Gradient magnitude similarity deviation: a highly efficient perceptual image quality index. IEEE Trans Image Process 23(2), 684–695 (2014)
E.P. Simoncelli, B.A. Olshausen, Natural image statistics and neural representation. Annu Rev Neurosci 24, 1193–1216 (2001)
L.J. Croner, E. Kaplan, Receptive fields of P and M Ganglion cells across the primate retina. Vision Res 35(1), 7–24 (1995)
M. Zhang, X. Mou, L. Zhang, Nonshift edge based ratio (NSER): an image quality assessment metric based on early vision features. IEEE Signal Process Lett 18(5), 315–318 (2011)
W. Xue, X. Mou, Image quality assessment with mean squared error in a log based perceptual response domain. Signal and Information Processing (ChinaSIP), 2014 IEEE China Summit & International Conference on IEEE, pp. 315–319, 2014.
X. Mou, W. Xue, C. Chen, L. Zhang, LoG acts as a good feature in the task of image quality assessment. Proc. IS&T/SPIE Electronic Imaging, vol. 9023, California, USA, 2014.
X. Mou, W. Xue, L. Zhang, Reduced reference image quality assessment via subimage similarity based redundancy measurement. Proc. IS&T/SPIE Electronic Imaging, vol. 8291, California, USA, 2012.
Y. Chen, W. Xue, X. Mou, Reducedreference image quality assessment based on statistics of edge patterns. Proc. IS&T/SPIE Electronic Imaging, vol. 8299, California, USA, 2012.
C. Chen, X. Mou, A reducedreference image quality assessment model based on jointdistribution of neighboring LOG signals. Proc IS&T Electronic Imaging 18, 1–8 (2016)
W. Xue, X. Mou, L. Zhang, A.C. Bovik, Blind image quality assessment using joint statistics of gradient magnitude and Laplacian features. IEEE Trans Image Process 23(11), 4850–4862 (2014)
H.R. Sheikh, M.F. Sabir, A.C. Bovik, A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans Image Process 15(11), 3440–3451 (2006)
L. Zhang, L. Zhang, X. Mou, and D. Zhang, A comprehensive evaluation of full reference image quality assessment algorithms. Proc. 19th IEEE ICIP, 2012, pp. 1477–1480.
S. Athar, Z. Wang, A comprehensive performance evaluation of image quality assessment algorithms. IEEE Access 7, 140030–140070 (2019)
V. Domonkos. A comprehensive evaluation of fullreference image quality assessment algorithms on KADID10k, 2019. [Online]. Available: http://arxiv.org/abs/1907.02096
J. Kim, H. Zeng, D. Ghadiyaram et al., Deep convolutional neural models for picturequality prediction. IEEE Signal Process Mag 34(6), 130–141 (2017)
S. Bosse, D. Maniry, K. Muller et al., Deep neural networks for noreference and fullreference image quality assessment. IEEE Trans Image Process 27(1), 206–219 (2018)
J. Kim, S. Lee, Deep Learning of Human Visual Sensitivity in Image Quality Assessment Framework. 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1969–1977, 2017.
D. Pan, P. Shi, M. Hou, et al. Blind Predicting Similar Quality Map for Image Quality Assessment. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6373–6382, 2018.
K. Lin, and G. Wang, HallucinatedIQA: NoReference Image Quality Assessment via Adversarial Learning. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 732–741, 2018.
L. Tang, K. Sun, J. Bi et al., Feature comparison and analysis for new challenging research fields of image quality assessment. Digital Signal Proc 91, 3–10 (2019)
A.J. Bell, T.J. Sejnowski, The “independent components” of natural scenes are edge filters. Vision Res 37(23), 3327–3338 (1997)
A.V. Oppenheim, J.S. Lim, The importance of phase in signals. Proc IEEE 69(5), 529–541 (1981)
M.C. Morrone, J. Ross, D.C. Burr, R. Owens, Mach bands are phase dependent. Nature 324(6049), 250–253 (1986)
M.C. Morrone, R.A. Owens, Feature detection from local energy. Pattern Recognit Lett 6(5), 303–313 (1987)
L. Henriksson, A. Hyvärinen, S. Vanni, Representation of crossfrequency spatial phase relationships in human visual cortex. J Neuroscience 29(45), 14342–14351 (2009)
P. Kovesi, Image features from phase congruency. Videre J Comp Vis Res 1(3), 1–26 (1999)
P. Kovesi, Phase congruency: a lowlevel image invariant. Psychol Res 64(2), 136–148 (2000)
W. Gao, S. Kwong, Y. Zhou, et al. Multiscale phase congruency analysis for image edge visual saliency detection. International Conference on Machine Learning and Cybernetics (ICMLC), 2016.
T. Arathi, L. Parameswaran, Image representation method based on complex wavelet transform and phase congruency, with automatic threshold selection. Int J Math Comput Simul 15, 79–83 (2021)
D. Gabor, Theory of communication. J Inst Elec Eng 93(III), 429–457 (1946)
D.J. Field, Relations between the statistics of natural images and the response properties of cortical cells. J Opt Soc Am A 4(12), 2379–2394 (1987)
X. Miao, H. Chu, H. Liu, Y. Yang, X. Li, Quality assessment of images with multiple distortions based on phase congruency and gradient magnitude. Signal Process Image Commun. 79, 54–62 (2019)
F. Zhang, B. Zhang, R. Zhang, and X. Zhang, SPCM: image quality assessment based on symmetry phase congruency. Appl Soft Comput J, 87, 2020.
C. Yang, S.H. Kwok, Efficient gamut clipping for color image processing using LHS and YIQ. Opt Eng 42(3), 701–711 (2003)
C.F. Li, A.C. Bovik, Contentpartitioned structural similarity index for image quality assessment. Signal Process Image Commun. 25(7), 517–526 (2010)
Z. Wang and X. Shang, Spatial pooling strategies for perceptual image quality assessment. IEEE Int. Conf. Image Process. 2006, pp. 2945–2948.
A.K. Moorthy, A.C. Bovik, Visual importance pooling for image quality assessment. IEEE J Special Topics Signal Process 3, 193–201 (2009)
J. Park, K. Seshadrinathan, S. Lee, A.C. Bovik, VQpooling: video quality pooling adaptive to perceptual distortion severity. IEEE Trans Image Process 22(2), 610–620 (2013)
H. R. Sheikh, Z. Wang, L. Cormack, and A. C. Bovik. (2005) Live Image Quality Assessment Database Release 2. [Online]. Available: http://live.ece.utexas.edu/research/quality
E.C. Larson, D.M. Chandler, Most apparent distortion: fullreference image quality assessment and the role of strategy. J Electron Imaging 19(1), 011006 (2010)
N. Ponomarenko, O. Ieremeiev, V. Lukin, K. Egiazarian, L. Jin, J. Astola, B. Vozel, K. Chehdi, M. Carli, F. Battisti, and C.C. Jay Kuo, Color Image Database TID2013: Peculiarities and Preliminary Results. Proc. of 4th Euro. Workshop on Vis. Inf. Process., pp. 106–111, Paris, France, 2013.
S. Huang, F. Cheng, Y. Chiu, Efficient contrast enhancement using adaptive gamma correction with weighting distribution. IEEE Trans Image Process 22(3), 1032–1041 (2013)
H. Zhao, O. Gallo, I. Frosio et al., Loss functions for image restoration with neural networks. IEEE Trans Comput Imaging 3(1), 47–57 (2017)
J. Cai, S. Gu, L. Zhang, Learning a deep single image contrast enhancer from multiexposure images. IEEE Trans Image Process 27(4), 2049–2062 (2018)
Final Report from the Video Quality Experts Group on the Validation of Objective Models of Video Quality Assessment, Phase II VQEG, 2003. [Online]. Available: http://www.vqeg.org/.
K. Ding, K. Ma, S. Wang, and E. P. Simoncelli. Image quality assessment: Unifying structure and texture similarity. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(5), 2022.
F. Gao, Y. Wang, P. Li et al., DeepSim: deep similarity for image quality assessment. Neurocomputing 257, 104–114 (2017)
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China (NSFC) (No. 62071375), which is appreciated. The authors are grateful to the associate editor and anonymous reviewers for their useful feedback that helped to improve the quality of this paper.
Funding
This work is supported by the National Natural Science Foundation of China (NSFC, No. 62071375).
Author information
Authors and Affiliations
Contributions
Both authors have made contributions to this manuscript. CC: methodology, software coding, experimental analysis, original draft writing. XM: research plan designing, methodology, draft revising.
Corresponding author
Ethics declarations
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Chen, C., Mou, X. Phase congruency based on derivatives of circular symmetric Gaussian function: an efficient feature map for image quality assessment. J Image Video Proc. 2023, 11 (2023). https://doi.org/10.1186/s13640023006112
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13640023006112