A new evaluation function for face image enhancement in unconstrained environments using metaheuristic algorithms

Abstract

Image enhancement is an integral component of face recognition systems and other image processing tasks such as in medical and satellite imaging. Among a number of existing image enhancement methods, metaheuristic-based approaches have gained popularity owing to their highly effective performance rates. However, the need for improved evaluation functions is a major research concern in the study of metaheuristic-based image enhancement methods. Thus, in this paper, we present a new evaluation function for improving the performance of metaheuristic-based image enhancement methods. Essentially, we applied our new evaluation function in conjunction with metaheuristic-based optimization algorithms in order to select automatically the best enhanced face image based on a linear combination of different key quantitative measures. Furthermore, different from other existing evaluation functions, our evaluation function is finitely bounded to determine easily whether an image is either too dark or too bright. This makes it better suited to find optimal solutions (best enhanced images) during the search process. Our method was compared with existing metaheuristic-based methods and other state-of-the-art image enhancement techniques. Based on the qualitative and quantitative measures obtained, our approach is shown to enhance facial images in unconstrained environments significantly.

1 Introduction

Pre-processing in face recognition systems involves enhancing an input face image in order to improve its quality by making more facial features in the image visible. Pre-processing enhances the performance of face recognition techniques [1, 2]. Further, the pre-processing stage amends distorted images and acquires regions of interest in an image for onward feature extraction. One important pre-processing task is image enhancement, which is essential to improve the performance of face recognition systems. However, most face recognition systems do not often incorporate face image enhancers in their designs, and whenever they do incorporate them, these are often less effective methods prior to the recognition process [3, 4].There are many image enhancement methods used in face recognition systems today. These methods have their pros and cons. Some recent methods transform an input image in order to achieve a more detailed or less noisy output image [5, 6]. In this regard, an image enhancement technique (IET) is considered effective if it is self-adaptive, i.e., if it adjusts its parameters to improve its performance automatically over different images. An IET must enhance an input image without introducing overstretching, excessive brightness, or loss of important features [7]. They should be simple with low computational complexity [8, 9]. These qualities are desired in a typical IET.

However, in face recognition systems, the quality of face images in unconstrained environments may be notably degraded for many reasons such as lighting conditions, i.e., in dark or too bright environments. Also, various facial expressions, change of pose, occluded faces, and other facial conditions may change the appearance of face images by hiding important features in the face [10]. Some IETs have been developed for various image-processing tasks; however, researchers have done minimal work concerning the development of IETs for face recognition in unconstrained environments. Further, most IETs often produce less enhanced outputs or unnatural effects and over enhancement in some cases that negatively affect the performance of face recognition systems. For these reasons, experts need to develop better IETs to improve further the performance of face recognition systems.

In terms of existing IETs, the histogram equalization (HE) method is a popular and simple approach for image enhancement [11, 12]. It works by adjusting the image’s contrast either by increasing or decreasing the global contrast of the image, especially when the image is characterized by close contrast values [13]. It operates by spreading the intensities of image pixels based on the information from the entire image. This results in conditions where low occurring intensities are transformed and fused with neighboring high occurring intensities, thus leading to over enhancement [14]. Further, mean shift issues may arise in such a situation, which maintains the image’s brightness and limits the performance of HE. The bi-histogram equalization (BHE) method was developed to address this problem [15], and it displayed better performance while maintaining the quality of the original image. However, the BHE is limited when the image pixel distribution is not symmetrical. Other methods such as the gamma correction and logarithm transformations have been used with lower computational complexities. However, they cannot manage complex illumination differences. Furthermore, other extensions of HE have been proposed such as the block-based histogram equalization, oriented local histogram equalization, and adaptive histogram equalization (AHE) methods. Nevertheless, these methods typically underperform in face recognition tasks, particularly in complex illumination conditions. This poor performance occurs because these methods rescind the entire distribution, which may contain important image characteristics [10].

Singh and Kapoor [16] presented an exposure-based sub-image histogram equalization method for contrast enhancement in low exposure grayscale images. Their method obtains thresholds, which are computed in order to split the original image into sub-images of various intensity levels. To control the enhancement rate, the histogram is clipped utilizing a threshold value as an average number of gray-level occurrences. Their method performed better than other conventional histogram equalization approaches. However, their approach does not adjust the level of enhancement, thereby resulting in darker or brighter enhanced images. Zhuang and Guan [17] used the mean and variance-based sub-image histogram equalization method to increase the contrast of the input image with brightness while retaining important features. However, because some IETs produce over-enhanced images and artifacts, Hussain et al. [8] proposed a dark image enhancement approach where local transformation of the image pixels is considered. The experiments in [8] showed that their method improved satisfactorily the quality of images. However, artifacts were present in the images. Reddy et al. [12] presented the dynamic clipped histogram equalization for enhancing low contrast images. Their approach selects a clipped level at the occupied bins and conducts histogram equalization on the clipped histogram to produce the output image. Reddy’s method uses three variants of the occupied bin space to enhance the low-contrasted dark, bright, and gray images. Shi et al [2] presented a dual channel prior-based method for nighttime low illumination image enhancement using a single image that is based on two existing image priors i.e., bright and dark priors. They used the bright channel prior to obtain the initial transmission estimate and used the dark prior as a complementary channel to adjust any wrong transmission estimate produced by the bright channel prior.

Linear contrast stretching (LCS) is an IET that uses linear transformation to increase the dynamic range of gray levels present in an original image [18]. LCS improves the contrast grade of an image; however, the LCS’s threshold value must be manually configured. If a wrong threshold value is used, the quality of the enhanced image will be low. Since no universal standard exists for image quality assessment, it becomes difficult to improve on an image by simply stretching its histogram or utilizing simple gray-level transformations [19, 20].

Thus, to solve these general issues, particularly when computers need to decide autonomously how good an enhanced image is, researchers have recently proposed methods based on evolutionary computation and metaheuristic optimization algorithms [21]. Metaheuristic algorithms are generally used to finding solutions involving non-linear optimization tasks. Munteanu and Rosa [5] pioneered the application of metaheuristic algorithms for image enhancement. They used an evaluation function (EF) to select automatically the most appropriate enhanced image without requiring human intervention. Thus, they proposed a novel EF and an evolutionary algorithm to globally search for the best enhanced image from among a solution space of candidate-enhanced images.

An EF plays a vital role in metaheuristic-based image enhancement methods. It automatically selects the best enhanced image by assessing the image’s quality without human involvement. Martens and Meesters [22] proposed an EF by using the statistical variable of the enhanced image. However, it could only be applied to a small set of test images. In 2015, Ye et al. [13] enhanced low contrast images by using a combination of the cuckoo search optimization (CSO) and the particle swarm optimization (PSO) algorithms. They derived contrast enhancement by the global transformation of the input intensities, while using an incomplete beta function as the transformation function. Three factors, namely the entropy value, threshold, and the probability density of the image, were used in [13] to measure the quality of the enhanced image. They evaluated and compared their approach to other IETs and showed improved performances. However, their model does not select always the most enhanced image, which necessitates the need for better methods.

Following in this paper, we present a metaheuristic-based IET for enhancing face images in an automatic and more effective manner than previous approaches. The primary contributions of our research are as follows:

• 1. We present a new EF as a component within the IET for face images.

• 2. We present a defined scaling mechanism for the EF in order to ensure that extreme values of an enhanced image depict either completely dark or completely white images.

• 3. Our proposed IET exhibits stability in selecting the most enhanced image based on both qualitative and quantitative measures.

• 4. Extensive quantitative and qualitative experiments were carried out to compare different existing standard EFs.

• 5. A comprehensive evaluation of the different standard metaheuristic-based algorithms was carried out.

The remainder of the paper is organized as follows: Section 2 discusses the proposed IET, where the related functions used are highlighted. Section 3 describes the data samples and the performance evaluation used. Section 4 discusses the experiments and simulation results compared with various state-of-the-art algorithms. Section 5 concludes the research work giving highlights of the merits of our proposed algorithm and scope for future research work.

2 Proposed method

2.1 Image enhancement technique

An IET uses an effective transformation function to efficiently map the intensity values of an original input image in order to produce an enhanced output image [23]. In metaheuristic-based methods, this process requires that an EF selects automatically the optimal enhancement parameters of a transformation function in order to appropriately enhance an image [24]. This section describes the transformation function used in our research, our proposed EF, and the metaheuristic algorithm that we considered.

2.2 Transformation function

Generally, the process of enhancing images in the spatial domain requires the use of a transformation function, which assigns new intensity values to each image pixel of the original image in order to produce an enhanced image. Local enhancement approaches apply transformation functions based on the gray-level distribution in the neighborhood of every pixel in a given input image. In our research, we used the transformation function proposed by Munteanu and Rosa in [5]. Munteanu’s approach applies a transformation function, T, to each pixel at a location (i, j) using the gray-level intensity of the pixel within the input image, f(i, j), and converts these intensities to another value, g(i, j), i.e., the gray-level intensity of the output image. The horizontal and vertical size of the image is denoted as H_size and V_size, respectively. Hence, T is defined according to [5] as:

$$ g\left(i,j\right)=T\left(f\left(i,j\right)\right)=k\left(\frac{M}{\sigma \left(i,j\right)+b}\right)\cdot \left[f\left(i,j\right)-c\cdot m\left(i,j\right)\right]+m{\left(i,j\right)}^a $$

(1)

where m(i, j) and σ(i, j) represent the mean and standard deviation of the gray scale image obtained for the pixels in the neighborhood centered at (i, j). The global mean, M, of the original image is computed as $M={\sum}_{i=0}^{H_{\mathrm{size}-1}}{\sum}_{j=0}^{V_{\mathrm{size}-1}}f\left(i,j\right)$. The parameters a, b, c, and k in Eq. (1) have the following effects: Parameter a introduces a brightening bias in the output image based on the last term m(i, j) in Eq. (1). It enables further control over the amount of smoothening effect required in the output image. Parameter b ensures that a zero-standard deviation value in the local neighborhood pixels does not have a huge whitening effect on the final output image. Thus, by its introduction, the denominator component in Eq. (1) typically remains nonzero. Parameter c allows only a fraction of the mean, m(i,j), to be subtracted from the original pixels of the input image. The parameter c controls the degree of darkening introduced in the output image. Parameter k is introduced to create a fair balance between pixels existing in the mid-range boundaries of the gray scale. Essentially, these pixels are prevented from being either too dark or too white during the enhancement process. The following parameter values were noted to be highly effective based on an extensive empirical parameter-tuning exercise conducted in our work: 2 ≤ a ≤ 2.5, 0.3 ≤ b ≤ 0.5;  0 ≤ c ≤ 3, and 3 ≤ k ≤ 4. These values formed the limits of the constraints used in the optimization process of our method.

2.3 Evaluation function

An EF is used to assess the quality of an enhanced image, g(i, j) without the need for visual assessment by a human operator. It is used to determine the optimal parameter values of the transformation function, T, which produces the best enhanced image. The development of our EF was motivated following three ideas, which we describe as follows: Firstly, we began by identifying particular key metrics in the literature that suitably describe how well an image is enhanced or not. We considered as many metrics as possible, which differentiates our model from existing models in the literature. Secondly, we introduced the concept of normalizing the metrics in our model in order to provide a bound for our method. This innovation made it possible to define a linear function that differs from existing functions in the literature. Interestingly, by this innovation, our method defines specific boundaries for images in very black and very white regions. These clear boundaries further enable an optimization algorithm to find better solutions in a well-defined range. Thirdly, we investigated three different optimization methods in order to use the best method, which led to the choice of the cuckoo search optimization algorithm. We present evidence of its performance in Section 4.

Thus, in developing our EF, and similar to [25], we quantified the following qualities of a well-enhanced image as follows: A well enhanced image should have a higher number of edge pixels than the original image. Furthermore, an enhanced image is expected to have a higher measure of information than the original image. This measure of information can be quantified using an entropic metric such as the histogram of the image. This approach is similar to the information measure used in the histogram equalization technique. Similarly, more pixels belonging to the foreground objects in an enhanced image should be better revealed than in the original image. An enhanced image should contain less alien artifacts than the original image. Based on the above qualities of an enhanced image, we propose a new EF that comprises of different performance metrics. These metrics, which are used to measure the qualities mentioned above, include the number of edge pixels, number of foreground pixels, entropic measure, and the peak signal-to-noise ratio (PSNR).

We describe the process of our EF as follows: First, the number of edge pixels, N_g, in the enhanced image is computed. To achieve this, a Sobel threshold, T_f, is automatically computed from the original image, f(i, j), using the Sobel edge detector. This threshold, T_f, is then used in the Sobel edge detector to obtain the edge intensities, E_g(i, j), of the enhanced image. In addition to being invariant, T_f was considered in our EF for computing E_g(i, j) in order to ensure a fair comparison between the original image and the different instances of the enhanced image. Thus, the number of edge pixels, N_g, in the enhanced image is obtained as:

$$ {N}_g=\sum \limits_{i=1}^H\sum \limits_{j=1}^V{E}_g\left(i,j\right) $$

(2)

Secondly, the number of pixels, ϕ_g, belonging to the foreground objects in g(i, j) is computed. To achieve this, the variance ϑ_g(i, j), of g(i, j), and the variance ϑ_f(i, j), of f(i, j), are computed within a neighborhood (window) having n × n pixels. A threshold value, η_f, is automatically computed for ϑ_f(i, j) using Otsu’s threshold algorithm. A representation, D_g(i, j), revealing pixels belonging to the foreground objects in the enhanced image is obtained as:

$$ {D}_g\left(i,j\right)=\left\{\begin{array}{c}1\kern1em \mathrm{if}\ {\vartheta}_g\left(i,j\right)\ge {\eta}_f\\ {}\\ {}0\kern1em \mathrm{if}\ \mathrm{otherwise}\kern0.75em \end{array}\right.\kern0.5em \mathrm{for}\ i=1,2,\dots, H;j=1,2,\dots, V. $$

(3)

Thus, ϕ_g, is obtained as

$$ {\phi}_g=\sum \limits_{i=1}^H\sum \limits_{j=1}^V{D}_g\left(i,j\right) $$

(4)

Thirdly, an entropic measure, β_g, of g(i, j) is computed as

$$ {\beta}_g=\left\{\begin{array}{c}-\sum \limits_m{\Omega}_m\log \left({\Omega}_m\right)\kern0.75em \mathrm{for}\ {\Omega}_m\ne 0\\ {}\\ {}0\kern5.25em \mathrm{for}\ {\Omega}_m\ne 0\end{array}\right. $$

(5)

where Ω_m is the frequency of pixels having gray levels in the histogram bin, m = 1, … , 256. The PSNR, ρ_g, of g(i, j) is obtained as

$$ {\rho}_g=10{\log}_{10}\left[\frac{{\left(L-1\right)}^2}{\mathrm{MSE}}\right] $$

(6)

where L is the maximum pixel intensity value in g(i, j) and MSE is given as

$$ \mathrm{MSE}=\frac{1}{H\times V}\sum \limits_{i=1}^H\sum \limits_{j=1}^V{\left|f\left(i,j\right)-g\Big(i,j\Big)\right|}^2 $$

(7)

Based on the parameters computed in Eqs. (2)–(7), a new EF, Ε, is proposed as

$$ E=1-\exp \left(-\frac{\rho_g}{100}\right)+\frac{N_g+{\phi}_g}{H\times V}+\frac{\beta_g}{8} $$

(8)

where Ε is a linear combination of the normalized values of the different metrics described in Eqs. (2)–(7). By normalizing each metric in Eq. (8), we confine each parameter to values between 0 and 1. Thus, based on this linear combination, our EF is described by a defined scale bounded between a minimum value of 0 and maximum value of 4. A minimum value of 0 represents an entirely black enhanced image, while a maximum value of 4 represents an entirely white enhanced image.

2.4 Metaheuristic algorithm

Metaheuristic optimization algorithms generally find solutions for highly non-linear optimization tasks [26]. They follow an iterative process that directs a subordinate heuristic by automatically fusing various ideas to explore a solution search space. Learning strategies are utilized to arrange information in order to find efficient optimal solutions. There are different types of metaheuristic algorithms known to be effective, such as the particle swarm optimization (PSO), genetic algorithm (GA), and the cuckoo search optimization (CSO) algorithms. In our research work, the CSO was used following an extensive comparison of the different metaheuristic algorithms as presented in the result section (see Section 4.2). The CSO algorithm was developed in [27], and it is among the most recent metaheuristic algorithms used for global optimization, with a look-alike process of the brood parasitic behavior of certain cuckoo species. The CSO was considered for its simplicity, fast convergence, and for its useful capability proven over series of experiments carried out in [28]. These attributes are essential requirements, which we considered worthwhile in order to enhance the contrast of images. The CSO algorithm based on Levy flight was applied in our work for searching new solutions based on the model given as:

$$ {a_i}^{\left(c+1\right)}={a_i}^{(c)}+\alpha \otimes \mathrm{Levy}\left(\lambda \right) $$

(9)

where a_i^(c + 1) depicts the latest solutions for a cuckoo, i, using flight function, with λ representing the Levy walk parameter, α represents the step size associated to the scale of the problem of interest, and ⊗ product represents entry wise multiplication. The Levy flight produces a random walk while the arbitrary step length is derived from a Levy distribution as Levy ∼u = t^−λ, (1 < λ ≤ 3). Generally, the process of the CSO is described as follows: every egg present in a nest denotes a solution and a cuckoo egg depicts a new solution. The purpose is to utilize the latest and supposedly better solutions, i.e., cuckoo to swap the less efficient solutions in the nests [27]. The adoption of the CSO in our work considers the situation for just an egg, as we are interested in only one solution. Table 1 describes the algorithmic process in the CSO. In the next section, we describe the entire flow process involved in our proposed image enhancement method based on an integration of the transformation function, our proposed EF, and the CSO algorithm.

Table 1 Steps involved in the Proposed Face Image Enhancement Technique

2.5 Summary of the proposed image enhancement algorithm

In this section, we provide a summary of our proposed image enhancement method as stipulated in Table 1. Typically, the inputs to our method are the original image to be enhanced and the range of values describing the lower and upper constraints of each parameter. The process converts the original image to its corresponding gray scale image. This gray scale image is passed to the CSO algorithm where the entire optimization process selects the best enhanced image. Essentially, we describe the process involved in a single iteration of the CSO algorithm. However, the iteration typically continues until a stopping criterion is met or there is no further change in the fitness value.

Firstly, the original gray scale image is subjected to the transformation function in Eq. (1) to obtain a supposed enhanced image. The transformation function is applied based on a set of initial parameter values selected by the CSO algorithm in a random manner. These parameter values are used in the transformation function in order to obtain a supposed enhanced image. This enhanced image is then passed to the EF were a specific value is computed for the enhanced image as described in Section 2.3. This EF value is one of several possible values computed for different enhanced images belonging to a population of possible enhanced images. Thus, each enhanced image in the population is subjected to the transformation function using different sets of parameter values and then passed to the EF to obtain a fitness value. The best (or largest) EF value from among a population of different enhanced images is selected per generation (or iteration). The iteration proceeds until no better value is obtained. Thus, the enhanced image that produces the highest EF value after several number of iterations (or generation) is outputted as the best enhanced image.

3 Performance evaluation and data samples used

In our work, we enhanced face images in unconstrained environments. Thus, we used face images obtained from three different standard benchmark face datasets, i.e., the AR face database [29], Yale face dataset (YF), and the Olivetti research laboratory face dataset (ORL). Face images affected by different lighting conditions, different facial expressions, and different pose variations were selected from each face database. The lighting conditions used were from the right, left, and both sides. Further, for the different facial expressions, face images with smile, anger, and scream were selected.

We describe the protocol used to select the representative images considered in the evaluation of our method and other state-of-the art approaches as follows: Essentially, six different images were selected based on our protocol. Firstly, since our research focused on facial images and their constraints, we used six different types of representative facial conditions in unconstrained environments, namely smile, anger, scream, right light illumination, left light illumination, and both side illumination. Secondly, we categorized all images in each dataset into these six different facial conditions. Thirdly, one representative image from each category was randomly selected, thus accounting for the six different images presented in Section 4 per dataset.

The effectiveness of an IET can be measured qualitatively by visualizing the enhanced output image. However, it is also required to describe quantitatively the degree of enhancement of an image. We describe the following metrics used to assess quantitatively the measure of enhancement of an image. The metrics considered are number of edges, number of pixels in the foreground, entropic measure, PSNR, and absolute mean brightness error (AMBE), which are defined in the following sub-sections.

3.1 Number of edges

The number of edges produced by an IET must provide a more substantial number of edges on the enhanced image as compared to the original input image. A higher number of edges on the enhanced image are desired as compared to the input image. The number of edges N_g can be obtained as stated in Eq. (2).

3.2 Number of pixels in the foreground

An effective IET must be able to reveal more pixels that belong to the foreground object in the enhanced image as compared to the original image. Hence, a higher value of the number of pixels in the foreground is desired to quantify the effectiveness of an IET. The number of pixels in the foreground ϕ_g can be obtained as stated in Eq. (4).

3.3 Entropic measure

An entropic measure is regarded as the process of quantifying the details of information in the image. The larger the entropic measure value, the more detailed an enhanced image will be. Also, the entropic value of an image is independent of a different image because comparison is done on the same image before and after the processing [30]. The entropic measure of the enhanced image β_g can be obtained as stated in Eq. (5).

3.4 PSNR

An IET must not only have the ability to improve the images but also control the level at which artifacts is introduced into the enhanced image, i.e., the level of noise should not be increased during the enhancement process. The PSNR ρ_g is used to evaluate the increase in quality between the original and the enhanced image [31]. The PSNR value can be obtained as stated in Eq. (6).

3.5 AMBE

The AMBE, ξ, is generally used to measure the rate at which the mean brightness is preserved, which can be represented mathematically as in Eq. (10). It shows the change in mean brightness value between the original and the enhanced image. Furthermore, the mean brightness of the original and enhanced image can be calculated as shown in Eqs. (11) and (12), respectively. Thus, a lower AMBE value is desired, while a zero AMBE value is considered the ideal result.

$$ \xi =\mid \delta \left(f\left(i,j\right)\right)-\delta \left(g\left(i,j\right)\right)\mid, $$

(10)

$$ \delta \left(f\left(i,j\right)\right)=\frac{1}{HV}\sum \limits_i\sum \limits_j\delta \left(f\left(i,j\right)\right), $$

(11)

$$ \delta \left(g\left(i,j\right)\right)=\frac{1}{HV}\sum \limits_i\sum \limits_j\delta \left(g\left(i,j\right)\right) $$

(12)

where δ(f(i, j)) depicts the mean brightness of the original image and δ(g(i, j)) represents the mean brightness of the enhanced image.

4 Simulation results and discussion

In this section, we discuss the effects of various experiments carried out in our research work. Firstly, an experiment to determine the choice of appropriate CSO parameters was conducted. Then, simulations using different metaheuristic algorithms were conducted to assess the respective performances of each algorithm based on the fitness value and time of convergence. Furthermore, we carried out experiments to compare our function with other EFs using the CSO algorithm in order to verify the effectiveness of our proposed EF. Finally, to confirm the efficacy of our image enhancement method, quantitative and qualitative comparisons were conducted based on standard performance metrics across different standard benchmark datasets.

4.1 Choice of CSO parameter

To effectively select appropriate values of the parameter, Pa, for the CSO algorithm, an experiment with different values ranging between Pa = 0.1 to 1.0 was conducted. Results were plotted based on the number of iterations and the fitness value. It is seen in Fig. 1 below that Pa = 0.2 converged the earliest. Hence, in our research, we selected Pa = 0.2.

Fig. 1

Choice of CSO parameter

4.2 Evaluation of different metaheuristic algorithms

To confirm the selection of the metaheuristic algorithm used in our research, an evaluation of different metaheuristic algorithms was carried out. The algorithms considered include the CSO, PSO, and genetic algorithm (GA) methods. The number of iterations for each metaheuristic algorithm was set at 100, which we plotted against the fitness value for each metaheuristic algorithm. In order to avoid bias, we selected three different images from the AR dataset and evaluated the performance of the different algorithms as shown in Fig. 2.

Fig. 2

Performance and the convergence analysis of the different metaheuristic algorithms on different images. a Image 1. b Image 2. c Image 3

In Fig. 2, it is seen that across the different images labeled (a), (b), and (c), the CSO algorithm outperformed other metaheuristic algorithms. For image 1, the fitness function value obtained by the CSO is 1.534; followed closely by the PSO at a fitness value of 1.532, and lastly the GA with a fitness value of 1.4509. For image 2, the fitness function of the CSO reached a value of 1.31, which is followed by the PSO value of 1.304 and GA obtained a value of 1.302. Similarly, evaluating the different algorithms on image 3, the CSO algorithm achieved the highest fitness function value of 1.417, while being followed by the PSO algorithm with a fitness function value of 1.406, and lastly the GA with a fitness function value of 1.404. The convergence analysis of the different algorithms was also observed with the CSO algorithm converging the earliest across all images at the 20th, 63rd, and 95th iteration for images 1–3, respectively. Furthermore, as shown in Table 2, we compared the different metaheuristic algorithms based on our proposed EF using all the performance evaluation metrics such as the number of pixels in the foreground, number of edges, PSNR, entropic measure, AMBE, and fitness value.

Table 2 Comparison of the different metaheuristic algorithm with our proposed EF based on all the performance evaluation metrics

The various performance evaluation metrics are defined in Section 3. From Table 2, the CSO algorithm in conjunction with our EF provided exciting and useful results reported as follows for image 1; the number of pixels in the foreground value of 6026 was attained for the CSO outperforming PSO and GA with values of 4267 and 5806, respectively. Similarly, in image 3, the highest number of pixels in the foreground value of 4567 was attained using the CSO with our proposed algorithm, outperforming the values of 4318 and 4291 generated by PSO and GA, respectively. Furthermore, the values generated for the number of edges on the different images by the CSO algorithm with the proposed EF outperformed all other techniques. This implies that the CSO algorithm with the proposed EF is able to improve on the original images, thereby revealing more information in the image.

To confirm our selection of the CSO metaheuristic algorithm, we qualitatively analyzed the enhanced images produced by the different metaheuristic algorithms with the proposed EF as shown in Figs. 3, 4, and 5. We inserted red bounding boxes in Figs. 3, 4, 5, 6, 7, and 8 to emphasize particular regions of interest. For example, observe the bounding box in the original image of Fig. 3 and notice that the beards of the subject are significantly revealed in the subsequent enhanced images (see Fig. 3b, c, and d). We applied our method based on different metaheuristic algorithms and showed qualitatively (see Figs. 3, 4, and 5) that there is not much to differentiate between these optimization algorithms by qualitative analysis. However, we showed slight advantages of the CSO algorithm over other methods by the quantitative results presented in Table 2. Furthermore, consider Fig. 4 and observe that the four spots within the bounding box in the original image are unobvious; however, they are clearly enhanced and made visible in the enhanced images (see Fig. 4b, c, and d). Similarly, we show bounding boxes on other images to emphasize interesting features that have been clearly enhanced over the respective original images. In essence, the CSO algorithm provided better quantitative performance as compared to the PSO and GA algorithms.

Fig. 3

Qualitative comparison of the different metaheuristic algorithm with our proposed evaluation function on Image 1. a Original. b CSO + proposed evaluation method. c PSO + proposed evaluation method. d GA + proposed evaluation method

Fig. 4

Qualitative comparison of the different metaheuristic algorithm with our proposed evaluation function on Image 2. a Original. b CSO + proposed evaluation method. c PSO + proposed evaluation method. d GA + proposed evaluation method

Fig. 5

Qualitative comparison of the different metaheuristic algorithm with our proposed evaluation function on Image 3. a Original. b CSO + proposed evaluation method. c PSO + proposed evaluation method. d GA + proposed evaluation method

Fig. 6

Qualitative comparison of the different evaluation function with the CSO algorithm on Image 1. a Original. b Munteanu + CSO. c Ye + CSO. d Proposed + CSO

Fig. 7

Qualitative comparison of the different evaluation function with the CSO algorithm on Image 2. a Original. b Munteanu + CSO. c Ye + CSO. d Proposed + CSO

Fig. 8

Qualitative comparison of the different evaluation function with the CSO algorithm on Image 3. a Original. b Munteanu + CSO. c Ye + CSO. d Proposed + CSO

4.3 Comparison of different EFs

In this section, we confirm the performance of our proposed EF based on the CSO algorithm. Hence, we compared the different EFs used in Munteanu [5] and Ye [13] using the CSO algorithm based on three different selected face images that vary among individuals as shown in Table 3.

Table 3 Quantitative comparison for different EFs

Considering Table 3, the different EFs were compared based on all the performance evaluation metrics using the CSO algorithm. The results obtained confirm the effectiveness of our proposed EF by displaying decent results reported as follows: across images 1, 2, and 3, our proposed EF produced the highest foreground value of 2104, 1116, and 4827, respectively as compared to the other EFs. Similarly, values generated for the number of edges by our proposed algorithms outperformed other EF techniques with higher values of 2304, 2057, and 3368 for images 1, 2, and 3, respectively. Furthermore, the fitness value generated by the different algorithms was analyzed, and it showed that our proposed algorithm produced the highest fitness value for all the images.

To confirm the effectiveness of our proposed EF, a comparison of the different EFs was conducted by analyzing qualitatively the enhanced images as shown in Figs. 6, 7, and 8. Following these figures, it is evident that our proposed EF outperforms the other algorithms.

4.4 Comparison of different image enhancement methods

The experiments in this section were designed to confirm the performance of our proposed algorithm by comparing our method with other state-of-the-art image enhancement methods. The methods considered are the linear contrast stretching (LCS), histogram equalization (HE), image intensity adjustment (IIA), Munteanu with particle swarm optimization (M_PSO), Munteanu with genetic algorithm (M_GA), brightness preserving dynamic fuzzy histogram equalization (BPDFHE), contrast limited adaptive histogram equalization (CLAHE), and low-light image enhancement (LLIE). The results obtained are presented in Tables 4, 5, 6, 7, 8, and 9. We conducted these experiments using different standard benchmark face datasets by selecting six different face images with different real-world conditions. The chosen face images selected from each face dataset were labeled images 1–6, and they represent multiple faces with lightning conditions, pose variations, and facial expressions such as smile, anger, scream, left light, right light, and both light on, respectively.

Table 4 Number of pixels in the foreground value comparison between results obtained using different enhancement methods on standard benchmark face datasets

Table 5 Number of edge value comparison between results obtained using different enhancement methods on standard benchmark face datasets

Table 6 Peak signal-to-noise ratio value comparison between results obtained using different enhancement methods on three different standard benchmark face datasets

Table 7 Entropic measure value comparison between results obtained using different enhancement methods on standard benchmark face datasets

Table 8 AMBE value comparison between results obtained using different enhancement methods on three different standard benchmark face datasets

Table 9 Fitness value comparison between results obtained using different enhancement methods on standard benchmark face datasets

From Table 4, the performance of the different image enhancement methods was analyzed based on the number of pixels in the foreground, which represents the amount of information introduced to the enhanced image. Hence, the goal is to obtain a higher number of pixels in the foreground. Our proposed algorithm produced the highest number of pixels in the foreground value across all images within the different face datasets. For image 1 from the AR face dataset, the number of pixels in the foreground value was 3294 achieved by our method, followed by the M_PSO with a value of 2565, then M_GA with a value of 2249, and LCS with a value of 451. Our enhancement method produced for images 2–6 the following number of pixels in the foreground values: 2565, 5424, 5956, 6865, and 3947, respectively, which surpasses the values presented by other enhancement methods. For the YF face dataset, our method produced a significant number of pixel values in the foreground across all face images. Similarly, for the ORL face dataset, our method produced the largest number of pixels in the foreground based on the different selected face images. Using this dataset, the values produced for images 1–6 include 10,483, 3873, 7618, 6499, 6248, 5946, and 4705, respectively.

The number of edges produced by the different enhancement methods was compared across the different standard face datasets as shown in Table 5. Our method demonstrated a better performance based on the performance metric used across all images in the different datasets. We desire a higher value of the number of edges because it demonstrates that more information has been added to the enhanced image. M_GA follows with the largest number of edge values across all images, while LCS and BPDFHE produced the least number of edge values.

Tables 6, 7, and 8 display the PSNR value, entropic measure value, and the AMBE value, respectively. The PSNR value obtained with our method produced the lowest value across all face images within the different face datasets. This implies that a lower PSNR value produces a more enhanced image. The entropic value by our proposed technique produced a higher value for most face images as compared to the other methods. This shows that more information has been added to the enhanced image to make the face image unique. Further, the AMBE values of each method, representing the mean brightness value, were compared. Our method preserved the absolute mean brightness error value significantly. Furthermore, the fitness values of each method were computed and presented in Table 9. The fitness value is an important metric that determines the effectiveness of each technique, and a higher fitness value is desired. Our method when compared to the other IETs produced the highest fitness value across most face images used from the different face datasets. Our method is followed by the M_PSO method across all images. The CLAHE and BPDFHE methods produced a satisfactory performance while the HE method produced the least fitness value across all images.

To confirm further the performance of our proposed algorithm, we compared qualitatively the enhanced images produced by the different image enhancement methods as shown in Figs. 9, 10, and 11 with each figure representing different lightning conditions, pose variation, and facial expressions such as smile, anger, scream, left light on, right light on, and both light on, respectively.

Fig. 9

Qualitative comparison of the different image enhancement algorithms on the AR face dataset where Figs. 1, 2, 3, 4, 5, and 6 represent images of different subjects respectively. a–j denote the methods labeled as a original, b LCC, c HE, d IIA, e M_PSO, f M_GA, g BDPFHE, h CLAHE, i LLIE, and j proposed

Fig. 10

Qualitative comparison of the different image enhancement algorithms on the Yale face dataset where Figs. 1, 2, 3, 4, 5, and 6 represent images of different subjects respectively. and a–j denote the methods labeled as a original, b LCC, c HE, d IIA, e M_PSO, f M_GA, g BDPFHE, h CLAHE, i LLIE, and j proposed

Fig. 11

Qualitative comparison of the different image enhancement algorithms on the ORL face dataset where Figs. 1, 2, 3, 4, 5, and 6 represent images of different subjects respectively. a–j denote the methods labeled as a original, b LCC, c HE, d IIA, e M_PSO, f M_GA, g BDPFHE, h CLAHE, i LLIE, and j proposed

Figures 9, 10, and 11 display the effectiveness of our proposed IETs. The enhanced images produced by our algorithm show a much positive difference in quality than the original and other enhanced images by other methods. The images affected by lighting conditions, i.e., images 4, 5, and 6 representing right light, left light and both light on, respectively were effectively enhanced as compared to other enhancement techniques. Furthermore, our method produced decent enhanced images considering images with different pose and expressions. Generally, unlike the images produced by other methods, our method produced more enhanced face images. This better performance was achieved because our method considers different essential metrics in its design. Metrics such as the number of edges and the number of the pixel value in the foreground will undoubtedly add more features to the image, thus, producing a more enhanced image.

4.5 Processing time per image

In this section, we provide results concerning the processing time of each algorithm per image considered in our research. We conducted the experiments in this section using a PC built on an Intel Core i5 processor with an Intel HD Graphics 4400 and 8GB RAM. We tested each algorithm using three different pixel sizes in order to assess their respective average processing times. The results obtained are presented in Table 10. Our algorithm experienced the longest average processing time per image. The processing time increased as the image pixel size increased. Our method experienced a longer processing delay because it computes iteratively both in the transformation and in the evaluation blocks for each potentially enhanced image in a population of several solutions. This iteration in the optimization process further prolonged the processing time of our method. This indicates that a trade-off thus exists between achieving better-enhanced images at the expense of speed. Though our method experienced a longer delay than other alternatives; however, this delay was at a far better enhancement performance than the other alternatives. Practical application areas where our method can be applied are as follows: in face enhancement and editing software in smartphones, face enhancement for forensic purposes, and face enhancement for cosmetic and dermatological purposes. In these and other similar application areas, experts are notably interested more in better-enhanced images than in the speed of enhancement. Nevertheless, we have observed that metaheuristic-based methods tend to provide improved performance than other methods at the expense of longer processing times.

Table 10 Processing time per facial image comparison using the different image enhancement methods on different image pixel size

5 Conclusion

Image enhancement is an essential pre-processing stage in typical face recognition systems. Hence, an efficient IET is required in order to improve face recognition performance. In this paper, we have presented a new EF for face image enhancement in unconstrained environments using metaheuristic algorithms. Our EF is used in conjunction with the cuckoo search optimization (CSO) algorithm to determine the best enhanced image, similar to the visual role played by a human evaluator. We achieved improved performance by introducing a scaled mechanism in our EF that prevents the enhanced image from assuming extreme dark or bright images. We have shown that our proposed EF outperforms other standard EFs. In addition, extensive quantitative and qualitative comparisons with other metaheuristic and state-of-the-art image enhancement methods were conducted in order to demonstrate the effectiveness of our method. We evaluated our method using different face images from different standard benchmark face datasets that represent different real-life scenarios. The performance metrics considered in our research demonstrated the superior performance of our method over other methods. For future works, we will look at the possibility of exploring other metaheuristic algorithms. In addition, approaches to reduce the processing time per image using the proposed method will be investigated. We note that our work can be extended to other application areas of image processing in order to improve their respective performance rates.