Image offset density distribution model and recognition of hand knuckle

2.1 Horizontal density clustering and Markov assumptions for discrete observations

Among them, the marker amount constitutes a hidden variable at the observation grid Z. To learn the distribution model of the observations by using observations, the relationship between observations, label classes, and offset measures on grid Z must be established. On V and Z\V, respectively, the position in the observed set V has a definite marker class 1 on the image. However, the labeling category Z\V on the unobserved position set is uncertain. Under the assumption of the continuity of the distribution model, the position where the marker category is indefinite should be understood as not observed, and the 0 marker cannot directly determine the corresponding observation result. The label category indicates whether the observation position belongs to the offset set under the level c. However, when estimating the overall offset measure using the observation data, it is necessary to further specify the mark relationship between the elements of the sets V and Z\V to integrate the mark relationships on the entire grid point Z. In connection with the data-extraction process in the previous article [30], the dependency relationship between grid observations can be established on Z using the mixture graph structure as a basis for subsequent inference learning using observation data. The relationship between marker categories and grid positions is established through a directed graph structure. At the same time, pairwise Markov random fields are used to establish a dependency relationship between the discrete grid points on the imaging domain, i.e., the distribution of p(⋅) on Z.

Thus, the hidden Markov model with observation markers is constructed on the grid point Z, as shown in Fig. 1. Among them, the hidden variable is the mark type, and the correlation factor is the local dependency on the offset set. The observations extracted based on the density estimate have neighborhood structures similar to those observed in the original grayscale image. Therefore, the corresponding semantic p(⋅) on the grid point Z not only forms a meaning on the image as a whole but also has a dependency in the local area and is a local Markov hypothesis on the corresponding undirected graph model:

$$ p\left\{{x}_i\in \mathrm{V}|{x}_{Z\backslash i}\right\}=p\left\{{x}_i\in \mathrm{V}|{x}_{\Gamma (i)}\right\}. $$

(5)

That is to say, observations that depend on the overall distribution are separated from the whole in the form of local associations.

According to the above analysis, on the one hand, the offset measure on the random hyperparameter field f reflects the characteristics of the observation mark classification and the density distribution agglomeration under the local relation. On the other hand, considering that when the offset set level parameter is higher, the Euler indicative number of the offset set is larger, and it shows that the local coverage of the offset set at the high level in the planar domain is more complete and shows more of a clustering trend. Therefore, the learning problem for p(⋅) can be transformed to a random clustering learning problem. This section uses the infinite Dirichlet process mixture model of the clustering model to construct the probability density p(⋅) in Eq. 2. The Gibbs sampling method is used to iteratively study the density structure of the hierarchical probability form under the assumption of the Markov neighborhood.

2.2 Nonparametric distributions and infinite Dirichlet processes

where θ is a hyperparameter, which is not limited to a limited form of distribution to improve the learning effectiveness and image-recognition rate.

The random process DP(α, H) is defined by the central parameter α and the base measure H.

where N is the number of samples and k_s may have a degeneration value of K.

By sampling the above distribution, an effective clustering parameter update can be obtained, and the update learning of parameters in each mixed Gaussian component can be realized.

2.3 Infinite Dirichlet process mixed model based on collapsed Gibbs sampling

According to the N observations \( x={\left\{{x}_i\right\}}_{i=1}^N \) of the Dirichlet process mixed model, the hidden variable label z_i, the total number of clusters, and the corresponding parameter \( {\left\{{\theta}_k\right\}}_{k=1}^K \) are inferred. The exact posterior distribution p(π, θ| x) contains the distributions corresponding to all possible category labeling spaces, and it uses a collapsed Gibbs sampling algorithm to implement iterative learning of an infinite clustering mixture model. First, all observed variables are sampled with their corresponding hidden variables z_i, then the posterior edge π of the polynomial corresponding to the current label class distribution and all clustering hyperparameters \( {\left\{{\theta}_k\right\}}_{k=1}^K \) is calculated.

3.1 Mid-level data distribution training based on Gaussian process classification

Due to the complexity of the distribution patterns in the middle-level data, it is difficult to obtain a mid-level migration density distribution model with relatively obvious features and a certain resolution. According to the nonparametric density kernel estimation result, the image gray position data corresponding to the offset parameter in the selected interval segment are taken as an observation of the random offset image bilateral offset set. The endpoints of the offset parameter interval are c₂₁ = 0.50 and c₂₂ = 0.85, and the middle-layer data are further divided into a multilayer structure corresponding to the three types of labels, according to the level of the corresponding density level, as shown in Fig. 2. Figure 2 a–c respectively correspond to observations in the intervals 0.70–0.85, 0.60–0.75, and 0.50–0.65. With the decrease of the offset parameter, the distribution pattern in Fig. 2a has certain regression characteristics. Figure 2b shows a clustering trend, and Fig. 2c shows the spread features. Comparing the transitions between the three graphs, it can be seen that the mid-level data distribution does not obviously have the clustering patterns or trends implicit in the high-level data distribution. Instead, it reflects the characteristics of random fields, i.e., the overall distribution of middle-level data, has the transition characteristics from clustering to irregular diffusion. In the above judgment of the distribution characteristics of the mid-level data, a random distribution modeling method can be used to learn the typical distribution states of the three parameter segments in the “clustering-diffusion” classification mode and to obtain \( {{\tilde{\mu}}_{{\mathbb{G}}_2\mid \mathrm{\mathbb{P}}}}^{\hbox{'}} \) in Eq. 1, the estimated results on the plane domain.

where the location i tag \( {y}_i^c \) has {0, 1} value, and the f vector form is \( f=f\left({f}_1^1,\dots, {f}_n^1,{f}_1^2,\dots, {f}_n^2,{f}_1^3,\dots, {f}_n^3\right) \). It has a prior form \( f\mid X\sim \mathcal{N}\left(0,K\right) \), where K is the corresponding covariance function and n is the amount of training data. Assuming the category information is not related, K has the form of a diagonal matrix, K =  diag {k₁, k₂,  … , k_c}, where k_c represents the trust relationship between each type of tag data. Therefore, the learning of the middle-level migration measure is transformed to the learning of the random quantity f.

3.2 Posterior calculations on Gaussian fields with multiple binary classifications

The maximum posterior estimate of the implicit function f is defined as \( \widehat{f}=\arg {\max}_fp\left(f|X,y\right) \), \( A=-\mathrm{\nabla \nabla}\log p\left(f=\widehat{f}|X,y\right). \)

where Π is a Gibbs distribution π corresponding to a cn × n scale column block matrix.

where E = (K + D⁻¹)⁻¹ = D^1/2(I + D^1/2KD^1/2)⁻¹D^1/2.

3.3 Structure of positive definite kernel function of random information in the middle level

and the design parameter array is p = [1, 0.5  ,  0.25; 0.5  ,  1  , 0.5; 0.25  ,  0.5  , 1].

In the above equation, the sub-diagonal array k₁ corresponds to the position of the image observation data at the density estimation level of 75 to 85% in the mid-level dataset of the image. After testing, it was found that although the aggregation level of this category dataset is weaker than the aforementioned high-level data model, it still has a certain clustering trend. Therefore, the associated credits between this category of data can be designed as an exponential clustering pattern, and the closest clustering component of the observation data can be found. The final correlation result of the clustering trend between x and x^' is determined using an isotropic exponential function. The design of the parameter array p_ij further confirms the labeling of the best high-level clustering component to which the two observations belong, i.e., when the data in the two images belong to the same clustering component in a high-level model, they have a higher degree of trust.

Figure 3 shows an example of the covariance matrix in the learning process of a multi-class model of middle-finger data in the distal phalanx and middle-finger images. The covariance scale is 3n × 3n, and n is the data volume of the layer density observation set in the training image. It can be seen that there is a certain difference in the degree of trust between the different diagonal block arrays for class positions. Only the sub-array k₃, in the form of a distance has a stronger associative relationship with respect to the positions of gray particles in the same class. Considering that the degree of data association carried by the sub-array k₁ in the form of clustering is the smallest, it indicates that the overall diffusion trend of the middle-level data is strong and the clustering trend is relatively weak.

3.4 Model prediction process

4.1 Binary classification and image offset information fusion based on Gaussian process

Depending on whether it belongs to the hand joint image, the test image is given the y mark {−1, 1} at the corresponding observed data point in the two-layer model likelihood space. In this way, the aforementioned fusion process can be presented as Binary Gaussian process learning. The learning result is the probability distribution of the marker y = 1 on the discriminant field for the joint target and non-joint targets. Different from middle-level information modeling, in the learning process of the classification information for the marker information y in this section, the sample domain is a training dataset generated from the two types of model likelihood values corresponding to the test image set. The learning domain is a normalized feature plane.

Since the value of label y is 0 in the range, the specific form of conditional field f ∣ X can be given by using the Gaussian process a priori \( f\mid X\sim \mathcal{N}\left(0,K\right) \), where K is the binary covariance function on field f. It can be seen that the main content of classification learning is the posterior update of f and the prediction of p(f_∗| f, X, y, x_∗) at the test position x_∗.

where \( \widehat{f}=\arg {\max}_fp\left(f|X,y\right) \) and \( A=-\mathrm{\nabla \nabla}\log p\left(f=\widehat{f}|X,y\right) \).

Equations 51 and 53 yield the posterior format \( q\left(f|X,y\right)=N\left(\widehat{f},{\left({K}^{-1}+W\right)}^{-1}\right) \) of \( \widehat{f} \).

Equation 56 obviously produces a diagonal band matrix. The inverse matrix can be quickly calculated by means of Cholesky decomposition. The inversion format in Eq. 57 is more stable than the directly solved inverse matrix of A.

The predicted edge distribution with tag category 1 is \( {\overline{\pi}}_{\ast } \).

4.2 Knuckle target recognition algorithm based on offset feature distribution

Based on the learned image layered offset fusion GP model, the model likelihood of the fusion image of the test image is used as the image feature. In the range of the test image domain, according to this feature combined with the maximum between-class variance method, self-adaptive threshold recognition is performed for the far finger and middle finger in the image. The concrete manifestation is that sub-image extraction is performed after the template data are calculated from the test image data. The nonparametric density kernel estimation calculations and the evolution of the level set of interest regions are performed on the obtained sub-images to obtain high- and mid-level data for density kernel estimation. High-level data are used to calculate the high-level data model likelihood through the high-level data model (DPMM). After the middle-level data are transformed by the data discrete group, the mid-level data model is used to calculate the mid-level data model likelihood. We then combine the two to calculate the two-level data fusion GP model likelihood.

The likelihood value of the fusion GP model is calculated at each template position of the test image, and this likelihood value is used as the information matrix corresponding to the image feature. Due to the limitations of the integrity of the model, detection points with higher likelihood values may appear at non-joint target locations. The largest cluster-like variance method is used to eliminate the position with the highest GP-likelihood in the case of threshold adaptation. Then, nonlinear evolution is performed on the feature information matrix with the high likelihood value removed, the features are enhanced, and the joint target is detected again using the adaptive threshold detection method. Because the detection position of the high threshold at non-joint positions is mostly unstable, it is difficult to recover it by the neighborhood information after rejection by the threshold. At the same time, a high threshold value at the joint location is more stable, so it can be recovered by the evolution of the neighborhood information.

5.1 DPMM model learning process

Examples of iterative learning monitoring of high-level density data DPMM in far- and middle-knuckle gray scale images are shown in Figs. 4 and 5. Using the above collapsed Gibbs sampling method, the Dirichlet process model for high-level data distribution of knuckle images is iteratively learned. Iterative initialization uses the k-means algorithm to classify the results and records the first 300 steps of likelihood monitoring values. The a priori parameters in the normal-inverse-Wishart distribution are taken as κ = 0.1, ν = 4; the a priori hyper parameters in the mixed Gamma distribution are taken as a = 0.1, b = 0.1; and the parameters of the Dirichlet process are initialized as α = 10. To improve the sampling accuracy of matrix parameters, the Cholesky decomposition of the covariance matrix obtained by iterative updating is performed. Sample moment statistics are made on its eigenvalues and feature directions, and matrix sampling is used to recover valid matrix samples.

From the results, the convergence speed of DPMM is faster and the smoothness of the likelihood curve is greater. On the one hand, because the number of clusters is flexible, the model has further improved the identification of the structure within the training dataset. The process of testing the number of linked random clusters can further clarify the sampling results. In the initial phase of the iterative process, the number of clusters suddenly increases by several times the convergence value. As shown in Figs. 4 and 5, different from the parameter optimization in the traditional finite mixture model, this stage corresponds to the sampling algorithm performing a random search in a wide range of clustering models, so that the model can quickly determine a more stable clustering mode. On the other hand, the Dirichlet distribution uses an a priori structure, so that the update process of the DPMM internal parameters can be more effectively controlled under higher-level conditional distributions, manifesting that the convergence curve has higher smoothness in the stable region.

5.2 Offset measurement data learning results

Examples of the DPMM model learning results on the training image library are shown in Fig. 6. Under the condition of K = 3 clustering initialization, the flexible random clustering modeling for image high-offset density position distribution is realized. A high-level distribution likelihood model of knuckle images with a more complex internal structure is obtained. The far phalanx goal learning results shown in the figure clearly show that the clustering self-adaptive process has similar results to data density clustering. The model scale has a strong ability to adapt to the training set. The distribution of clustering represented by the model likelihood results is not only consistent with the observed characteristics on the whole. The characteristic orientation of the internal clustering component also reflects the features of the knuckles under the grip of the hand. It shows that this algorithm has better modeling ability for the high-level distribution of far-knuckle images.

Using the aforementioned middle-level distribution learning and prediction algorithm, the multi-classification model of middle-level data for each image is learned in 51 positive images of distal phalanx images and 51 positive sample images of middle phalanx images, respectively, as shown in Fig. 7. In Fig. 7, from left to right, there are first-, second-, and third-class hidden flags.

From the prediction results in Fig. 7, it can be seen that the three-category tag learning results of layer data in the finger image can more clearly show the design goals of the model. The marker results of the learning prediction also conform to the hypothesis of the distribution of layer data in the knuckle image.

where the scale parameter is κ = 0.007. In the distal phalanx and middle-finger image test libraries (51 positive and negative samples), the high- and middle-level data-extraction process, feature likelihood calculation, and fusion model learning are completed. Based on the high-level data DPMM model learning results, the high-level data model likelihoods are obtained. We combine the mid-level three-classification model to calculate the likelihood value of the corresponding observation data and normalize the two similarity values as a labeled test set for the supervised learning of the two-class Gaussian process. Multi-offset feature likelihood distributions, covariance functions, and fusion model (GP) learning results are shown in Fig. 8, in which the left graphs of each map are far-knuckle results, and the right graphs are the middle-finger results.

According to Fig. 8b, in the normalized feature plane, the first characteristic direction of the positive sample fusion distribution follows the characteristic line (0, 0)-(100, 100) direction in the feature plane. The second feature direction is nearly perpendicular to the feature line; the first feature direction of the negative sample fusion distribution is close to the vertical direction of the feature line. The angle relationship between the feature direction and the feature line indicates that the two types of offset features that are fused constitute a certain degree of discrimination between positive and negative samples, and the fusion results show a stronger forecast of this differentiation. Comparing the left and right graphs shown in Fig. 8b, the high-end model of far-knuckle images has better discrimination between positive and negative samples than the middle-level model, while the fusion prediction of middle-finger images shows the opposite result. The main reason for the difference between the above models is the obvious differences in the random structure of the distal phalanx and middle phalanx images, which are embodied in the differences in distribution patterns at different levels of displacement.

5.3 Recognition for various algorithms

Under the fixed threshold condition, the recognition ability of the high-level data DPMM model, the middle-level data implicit marking GP model, and the DPMM+ GP model combining the two are briefly analyzed. We artificially produced four finger-knuckle image databases with library capacity of 330, 1344, 1896, and 1400 images. These knuckles are taken by industrial cameras and come from people of different genders, ages and sizes. In these image libraries, positive and negative samples each make up half. In these image libraries, two are far-finger image libraries and two are middle-finger image libraries. To test the adaptability of the recognition algorithm to the fuzzy objects, the joint features corresponding to all the knuckle images in the four image libraries are artificially selected to be weaker than those in the training library. That is, testing on a feature-rich joint image can achieve higher recognition capabilities. Three models are used in the four image libraries for detection to compare the optimal recognition ability of each algorithm for each image library; through the test analysis, the threshold of the highest recognition capability of the above three models in each image library is taken as the best recognition threshold of each algorithm in the library and plotted to a receiver operating characteristic curve (ROC). At the same time, through actual measurement, it is found that the difference between the best recognition thresholds of the same algorithm in different image databases is small. Therefore, the following four ROC curves can be compared and analyzed as a whole, as shown in Fig. 9.

Considering that the area under the curve (AUC) on the ROC is a measure of the recognition ability of the identifier, it can be clearly seen from Fig. 9 that in the far-knuckle test library, the comprehensive model recognition ability of the middle-level data model and the two-tier data is not as good as that of high-level image features. In the middle-finger test library, the recognition curves of the two high-level data models are low, and the corresponding AUC is less than 0.5. The recognition ability of the high-level data model in the far-finger library is obviously higher, while the middle-tier data model in the middle-finger library has stronger recognition ability. The above shows that the existing data model has great differences in the ability to identify different types of knuckle objects. It also potentially indicates that there is a certain difference between deep model categories in the distribution of far- and middle-finger image data.

In Fig. 9b, the AUC values of the high-level data model corresponding to the two ROCs are 0.5134 and 0.2332. It can be seen that the recognition effect of high-level models in the middle-finger image database (2) is not obvious, and wrong classifications even appear. Further tests show that the high-level model with high likelihood is the middle-finger area, not the finger joint area. This phenomenon occurs because the intermediate region has relatively small local information entropy due to the smooth grayscale distribution. The high-level data volume is larger and denser than the usual joint image data, which undermines the model’s assumptions on data distribution, and therefore, it has poor recognition capability. At the same time, according to Fig. 9, it can be seen that the ROC corresponding to the two-layer data fusion model is located between the high- and middle-level models. It shows that the recognition based on the fusion model has the effect of comprehensively judging the two features. In the case that the high- and middle-level models differ greatly in their ability to identify models, they can provide effective comprehensive evaluation, which is more prominent in Fig. 9b. The minimum area under the curve for the fusion model is (a) 0.4512 on the left of the figure, and the maximum is (b) 0.7880 on the right of the figure. The results show that the fixed threshold identification method has stable and correct classification ability under the condition of existing limited data and test set in the environment where the light intensity is relatively stable and the imaging angle does not change much.

To further improve the recognition ability of the fusion model, combined with the learned DPMM+GP model, adaptive threshold joint detection is performed on a hand test image containing a finger joint. The size of the test image is controlled to include the size of about 1000 templates, as shown in Fig. 10. Among them, black indicates that there is no finger joint at the position, the gray portion is the artificially marked finger joint region, and the white position is the joint point position result recognized by the adaptive threshold segmentation. When the detected joint position falls within the manually determined gray area, the joint identification is correct. Being away from the gray area indicates that the detected position has a large deviation from the true position. With the aid of diffusion evolution, the recognition result is closer to the real target area, which obviously improves the accuracy of finger joint target recognition. It is worth noting that the detection results and the marked areas shown in Fig. 10 are all at the pixel level. Therefore, the detection error of the above algorithm in the actual image is also at the pixel level. For actual detection tasks, joint detection can be initially implemented.