The influence of photo elements on EEG signal recognition

Abstract

With the increasing demand for confidentiality of information, traditional identification tools are no longer sufficient. As a new biological signal, EEG signal has the advantage of being difficult to forge and easy to carry and has become a hot topic in the identification research. In the study of EEG signals, the EEG signals triggered by vision have the advantage of stable characteristics. Therefore, many researchers use the photos of the subjects themselves as stimuli to achieve identification research, but the EEG signals caused by vision are easily stimulated. Object interference, such as photo size, resolution, and color. The main purpose of this paper is to study the effect of these disturbances on the recognition effect. The results show that when the number of electrodes is increased to 9, the effect of increasing the electrode to improve the classification accuracy is not significant; the recognition effect of black-and-white photos is lower than that of color, the size of the picture is 800 × 600 or 1024 × 768. When the resolution is 72, the classification effect is the optimal.

1 Introduction

In order to protect their own property, people began to research on identity early on, from early passwords to later fingerprints, faces, genes, etc. With the development of technology, the tools used for identity recognition are also gradually developing. The existing identification can easily forget (passwords) and easily be forged (fingerprints, faces), and the cost is too expensive (genetic testing). In some special circumstances, such as for the higher confidentiality departments, the existing identification tools are not suitable, so they need to find new identification tools.

The development of biometric technology has provided a new way to provide a reliable identification tool. Any physiological or behavioral characteristic of a person can be used as a biometric for identification in principle as long as it satisfies the following conditions: (1) universality, everyone has; (2) uniqueness, each person is different; (3) stability, relatively stable over a period of time; and (4) collectability, which can be conveniently measured quantitatively. Of course, it is not always feasible to satisfy the above conditions. The actual system should also consider the following: (1) performance, i.e., accuracy, speed, robustness, and resources needed to meet the requirements; (2) acceptability, people: the degree of acceptance of such biometrics; and (3) deceptiveness, whether it is possible to deceive the system through subjective fraud. At present, more mature biometric technologies include fingerprints, irises, faces, hand shapes, voiceprints and signatures, but there are still some problems as follows: (1) there are always some people whose biometrics are not significant and are excluded, such as fingerprints. Unclear or injured may not pass the fingerprint test. It may be difficult for iris recognition when red eye, and the sound may change when there is a cold; (2) the safety factor is not high, for example, face recognition is powerless for twins and photo imitation; voiceprint and signature recognition is easy to be imitated; fingerprint and hand type identification are easy to be stolen; (3) it is difficult to perform living body detection, and the fingerprints, faces, and hands of dead or isolated bodies are still effective. Therefore, research on the theory and methods of new biometric technologies is a challenge for biometric research institutes.

EEG signal is the external manifestation of brain thinking activity. Since EEG signals have been recorded, people use a variety of methods to use EEG signals to reveal brain activity [1,2,3,4,5]. In these studies, EEG signals induced by image stimuli have been widely used in many studies due to stable features. For example, Mamoru N et al. [6] use visual stimuli to make objective evaluation of educational videos. Visual stimuli are induced by pictures displayed on the liquid crystal display in the shielded room. The results showed a negative correlation between the EEG signal and pupil size in the 9–11 Hz frequency band; Yan X D [7] et al. used image stimulation to study the “image advantage effect.” Through the discussion of the two types of self-demolition, the feature extraction of the image-stimulated EEG signals is used to improve the accuracy of the test case’s brain in test civil and criminal cases. Liu F J et al. [8] used picture-evoked EEG signals to study the different responses of teenagers and older adults to voice, pictures, and Chinese characters. The results showed that the ERP components of the elderly and young people are different.

As a tool for identification, EEG has the following characteristics: First, it is universal, and each living person has an EEG signal. Secondly, because each person’s brain characteristics, thinking mode, memory, etc. are different, there are differences between people. The characteristic EEG signal is unique. Thirdly, the EEG signal has stability. The EEG signal can be relatively stable for a certain period of time. Fourth, the EEG signal is easy to collect. Currently, there are commercial acquisition devices. After years of research and development, today’s EEG signal acquisition has been simplified from the original steps of grounding, setting reference, coating conductive paste, and mounting electrode caps, and simplifying it to only one earphone-like device. Corresponding EEG signal.

EEG signal is a spontaneous physiological signal. With the development of acquisition techniques, the memorylessness, difficulty of counterfeiting, and convenience of EEG signals have become new research directions for many identity researchers [9,10,11,12]. People’s sensitivity to their own photos is higher than that of other people’s photos, which can use this difference to induce stable EEG signals with their own characteristics to achieve identification.

Identity recognition using photo-induced EEG signals has achieved good results [13,14,15]. From the point of view of the experimental model, these results are based on a small part of the same photo recognition research. However, in the specific use process, whether the attributes such as the size, color, and resolution of the photograph will have influence on the final recognition effect, which is rarely studied. This paper uses photos as stimuli to induce subjects’ brain electrical signals. Wavelet decomposition is used as a feature extraction tool to extract the features of the subject’s EEG signals, and Fisher distance is used as a feature screening method for feature screening. SVM is used as a classification algorithm for identification. In the process of choosing photos, choose different size, color, and resolution photos to experiment. The results show that the attributes of photos have an impact on the accuracy of recognition. When the best size, color, and resolution are selected, the recognition rate can reach 97.8%, which is much higher than that of similar identity recognition research. In summary, the main contribution of this paper is that, by the influence of the size, resolution, and color of the subject’s photos on the classification and recognition results, the best photo attributes are finally obtained.

2 Experimental paradigm

The purpose of the experiment in this paper is to investigate the effect of photos on identification results under different attributes. The photo attributes are set to size, resolution, and black-and-white color, and the size is set to 400px × 300px, 640px × 480px, 800px × 600px, 1024px × 768px, and 1280px × 960px, and the resolution is set to 50 dpi (pixels/inches), 72 dpi (pixels/inch), 100 dpi (pixels/inch), 200 dpi (pixels/inch), and 300 dpi (pixels/inch).

The subjects performed experiments in the laboratory. To ensure the quality of the experimental data, the laboratory used a silent mode. In front of the subject is a 32-in. widescreen LCD monitor with a resolution of 1920 × 1080. During the experiment, the background of the screen is black. The subject hears a “drip” to indicate the start of the experiment. The first is a 500-ms black screen time. Then a random photo is selected from the photo bank on the background color, and the screen is blanked again after 1.2 s. Then the subject’s photo is displayed for 1.2 s. The experiment completes a trial. A trial takes a total of 3.4 s (experimental mode is shown in Fig. 1(1)). Each trial completes 100 trials for a total of 340 s. Each subject completed a photo attribute experiment over a period of time. According to the photo attributes, the size, resolution, and black and white of the photo library used in this paper are 26 groups. Therefore, each subject needs to complete at least 26 experiments. The whole experiment takes 50 days.

Fig. 1

Image stimulation experiment process

The 20 subjects in this study are from college students, including 10 males and 10 females (mean=20.45, variance=1.05). Before the experiment, the subjects knew the contents of the experiment. The subjects did not drink alcohol before the experiment, and they did not take any irritating drugs. Subjects were guaranteed adequate sleep (8 h or more) before the experiment. The experiments in this paper are all filed at the Research Office.

The design of EEG acquisition mode, in the analysis of EEG signals, generally consider the following three aspects: First, how to design the experimental process to “inducate” the subject to “transmit” a reasonable signal; Second, the setting of various parameters; Third, the number and position of the electrodes, the more electrodes, the more complex the algorithm, the effect may be better; the fewer the electrodes, the simpler the algorithm, the worse the effect may be.

This paper uses neruoscan’s 40-lead brain electrical signal acquisition device to collect brain electrical signals at a sampling frequency of 1000 Hz. The left mastoid is used as a reference electrode, and the 50-Hz notch is used. The data are stored for 8 min each time, including 100 trials and preparations before and after the experiment. Before the analysis, the experimental data was pre-processed with the offline processing software (scan 4.3), which is provided with the acquisition equipment, including the removal of the electro-oculogram (vertical electro-oculogram), filtering (1–100 Hz band-pass filtering), and photo display time (the first 20–80%) segmentation, the final segmentation results are stored in the form of a sample, each sample is cutoff for one sampling period (1 s), and each subject’s pretreated sample set behaves as 30 × 200 (electrode number of *samples).

As shown in Fig. 1(2), the electrodes are distributed on the international 10–20 standard, in which the markers in each electrode respectively represent the 10–20 standard nomenclature and the serial numbers used in this paper.

3 Method

3.1 Wavelet entropy

The EEG signal is a weak electric signal, so the EEG signal characteristics are easily interfered by the outside world. In order to highlight the essential features of EEG signals, it is a common type to convert EEG signals from time domain signals to other domain signals; means, in which the frequency domain feature is a commonly used method, the wavelet transform can transform the time domain feature into the time-frequency domain and integrate the time and frequency features to achieve the filtering effect. Wavelet entropy better reflects the order of this time-frequency domain feature, so this paper chooses wavelet entropy as the feature transformation method.

Wavelet analysis is used to perform multi-scale decomposition of EEG signals. If the wavelet coefficient vector on scale i is denoted as W_i = (w_i1, w_i2, …, w_in), in which w_in refers to wavelet coefficient parameters, n is the decomposition length, so each scale has a coefficient vector corresponding to it. For EEG signals, the more similar coefficient vectors at different scales, the greater the entropy of the signal, the greater the complexity of the signal, the more similar each signal component, and the less obvious the signal characteristics.

Here, the norm of the space vector Rⁿ is used to measure the degree of closeness between the vectors, that is:

$$ {\left|{w}_s\right|}_2={\left[\sum \limits_{i=1}^n\left|{w}_{si}\right.\right]}^2\left|{}^{1/2}\right. $$

(1)

In which, |w_s| is the norm sequence.

According to the definition of wavelet energy, the closeness of each scale is measured by a norm sequence of wavelet coefficient vectors. Normalize the energy sequence E₁, E₂, …, E_m. The signal structure and complexity are analyzed by the distribution of normalized energy sequences. The process is described as follows:

1) Assuming that the signal is decomposed on the M scale, let the wavelet coefficient vector on scale i be defined as: W_i = (w_i1, w_i2, …, w_in), then the energy at scale i is defined as:

$$ {E}_i={\left|{w}_i\right|}^2=\sum \limits_{j=1}^n{\left|{w}_{ij}\right|}^2 $$

(2)

3.2 Fisher distance

This paper introduces the Fisher distance to calculate the feature distance between different subjects who see themselves and look at others. The Fisher distance calculation method is as follows:

$$ {F}_{i,j}=\frac{{\left({\mu}_i-{\mu}_j\right)}^2}{\sigma_i^2-{\sigma}_j^2} $$

(3)

In which, F_i,j refers to the Fisher distance matrix between subjects’ normal state and fatigue state EEG signals, and μ and σ are the mean value and variance.

3.3 SVM classifier

Supportive Vector Machine (SVM) is a supervised machine learning method that can learn the characteristics of different types of known samples and predict unknown samples. It is essentially a two-class algorithm. For an input sample of n-dimensional space, it finds an optimal categorical hyperplane so that the two types of samples can obtain the best classification effect under this hyperplane.

In order to ensure the robustness of the classification results, this paper adopts a 30-fold cross-validation method to test and divide all the samples into 30 equal samples. The 300 self photos and the 300 photos of others were equally divided into 10 parts. Each time, 8 of them were selected as learning sets and 2 were used as test sets. Randomly selected 10 times, the average result was finally calculated.

This paper selects SVM as a classifier and RBF as a kernel function. The RBF kernel function can be described by the following formula:

$$ K\left(x,{x}_i\right)=\exp \left(-{\gamma}^{\ast }{\left\Vert x-{x}_i\right\Vert}^2\right) $$

(4)

In which, γ is the width of the kernel function.

The penalty factor C and the kernel function σ in the classification calculation are determined using a grid search method.

4 Results and discussion

For each subject, the sample of brain electrical signals collected from the photo collection of each attribute included 100 samples (positive samples) of self-photographs and 100 samples (negative samples) that were not self-photographs. A total of 20 subjects were added together to obtain 2000 samples of one’s own and 2000 samples of watching others, for a total of 26 groups. Using the above method for feature extraction, we can get the following results (the wavelet entropy used in this paper is based on the original value in addition to e⁵ after taking the absolute value).

4.1 Electrode selection

The smaller the number of electrodes, the smaller the amount of data involved in the algorithm operation and the smaller the time complexity of the algorithm. Can 30-lead EEG signals be identified using only a few electrodes? Because there is no hair in the forehead area, it is easy to collect and practice. Can we use the two leads of brain signals in the forehead area to achieve identification?

Fisher’s distance calculates the difference of two characteristics of different subjects in terms of mean and variance. Therefore, using the Fisher distance as a criterion, the feature distances between the subjects who see themselves and the photos of others can be calculated (Fisher distance is a combination of 26 groups of samples), and electrode screening is used as a basis. The results are shown in Fig. 2. Figure 2(2) is the increase of the Fisher distance with the increase of the number of electrodes (first sorting the Fisher distance in descending order). Then, with the increase of the electrode, the Fisher distance is superposed. Figure 2(1) is the accuracy of the classification with the increase of the number of electrodes and the accuracy of the two electrodes with only the forehead (FP1 and FP2).

Fig. 2

The effect of the number of electrodes on the accuracy. (1) is the effect of the number of electrodes on the accuracy. In (1), the x-axis is the number of electrodes, y is the Fisher distance, and 1.2 x represents the number of electrodes, where 0 is used only for the two electrodes on the forehead, y indicates the classification accuracy, and (2) shows the effect of the number of electrodes on the Fisher distance

As can be seen from Fig. 2, with the increase in the number of electrodes, Fisher’s distance increasing trend gradually decreased, indicating that under different electrodes, subjects see themselves and see others’ different characteristics of the difference. Therefore, the contribution to the classification is also different. The classification accuracy can also be seen that when the number of electrodes increases from 8 to 9, the accuracy rate increases by 0.01, from 9 to 10 only increases by 0.0008, and then the accuracy increases as the number of electrodes increases. The rate has barely increased. Moreover, as the number of electrodes increases, there is also a phenomenon in which the accuracy rate is lowered (for example, 18). Therefore, it can be concluded that when the number of electrodes is selected at 9, the effect of 30 electrodes can be achieved.

If only the two electrodes in the forehead region are used, the classification accuracy is 81%, which is only 8% less than when selecting 9 electrodes. Therefore, for cases where the classification requirement speed is not strict, it is feasible to use the forehead EEG signal.

4.2 Influence of black and white photos

Black and white pictures and color pictures are two common sources of stimulus for visual stimuli. Figure 3 is the entropy result of the subject using black-and-white photos and color photos as stimulating sources. The blue part is the entropy value of 100 samples to see oneself and others, and the green part is a sample of 25 color patterns (2500). Look at yourself and 2500 to see others).

Fig. 3

Comparison of entropy values of different stimuli. Blue is a black-and-white photo source, green is a photo source, the x-axis is the entropy value of 9 pictures, and y is the entropy value of 9 pictures

As can be seen from Fig. 3, the entropy value of the black-and-white photos is lower than that of the color stimulus sources as shown in Table 1 (where 1, 2, and 3 represent the electrode labels, which correspond to the 9 electrodes selected in result 1). Color (self) indicates the color photo’s own mode, other indicates the other mode, and B-W represents the black and white photo. The value in Table 1 is the average of the sample entropy values of the mode. From the results in Table 1, it can be seen that the average entropy of color photo-stimulus source is greater than that of black-and-white photos, and the average of 9 electrodes can be known. The color photo can be seen when the entropy is 5.96, and the picture of others is 6.16; 3.14 for myself and 3.78 for looking at other people’s photos. Therefore, from the average point of view, the entropy value of looking at yourself and looking at other people’s photos is smaller, and looking at the color photo is larger than looking at the entropy value of the black and white photo.

Table 1 Contrast between the entropy of looking at yourself and looking at others

For the above result, because the subject knew that it was an identity recognition experiment, his photos would appear during the experiment, so the expectation was not obvious. And because they are more familiar with their own photos is much higher than that of other photos, the volatility (the amount of information in the signal) that appears in the expression of entropy is not obvious. But when looking at other people’s photos, because of the background photos selected from the sample database, randomness may cause each photo to be different. The brain needs more information to process these photos that are not “familiar.” Therefore, the volatility of the signal (the amount of information contained in the signal) is large, causing the entropy value to be less than the average of the photos of others.

The black and white of photographs are more colorful than color photographs, and people have more attention to color photographs. Color photographs also highlight the subtle features of people. Therefore, the reaction of the human brain is also more intense, causing the entropy of the color photo to be larger than the entropy of black and white.

In contrast to the recognition rate, Fig. 4 shows the subject’s recognition rate in black and white photo stimulus and color stimuli. As can be seen from the figure, the recognition rate is 71.56% for black and white stimuli and 90.13% for color photos. Color stimuli (superimposing all 25 patterns of color stimuli) are significantly higher than black and white photo stimuli.

Fig. 4

Black and white and color photo recognition rate (where B-W is the black and white stimulus and color is the color stimulus)

4.3 Classification accuracy

From result 2, it can be seen that the recognition rate of black and white photographs is lower than that of color, but the properties of color photographs are varied, and the factors that affect the resolution include the resolution and size. Figure 5 shows the effect of different resolutions and sizes on entropy characteristics. As can be seen from the figure, when the subjects looked at their own photos, at the same resolution, the photos were 800px × 600px, and the maximum entropy was 1024px × 768px. When the resolution is 72 dpi, the photo entropy of these two sizes is maximum. Then, as the resolution increases, the entropy value gradually decreases. At a resolution of 300 dpi, the size of the photo no longer affects the size of the entropy, but when looking at other people’s photos (Fig. 5(2)), at different sizes and at the resolution, the difference in entropy is not significant.

Fig. 5

The effect of resolution and size on entropy

From the above results, the following conclusions can be drawn:

• 1. In order to highlight the characteristics of your own photos, the small size of the photos is suitable for 800 × 600 or 1024 × 768.

• 2. The resolution of 72 is more appropriate.

• 3. The main reason for the above results may be that with the popularity of smart phones, photos at this resolution and size are more popular. Especially in the case of various self-portraits and online self-photographs, such photographs are the most numerous, and therefore, the subject is affected and the subject is most familiar with such photographs.

Figure 5 shows the effect of resolution and size on entropy (where 1 is the effect of looking at your own photo. 2 is the effect on looking at other people’s photos, where the x-axis is the resolution (1, 2, 3, 4, and 5 corresponds to 50 dpi, 72 dpi, 100 dpi, 200 dpi, and 300 dpi), and y is the resolution entropy. The five bars at each resolution in the figure correspond to five different photo sizes of 400 px × 300 px, 640 px × 480 px, 800 px × 600 px, 1024 px × 768 px, 1280 px × 960 px), respectively.

Table 2 shows the recognition rate under different resolutions and sizes. From the results in Table 2, it can be seen that as the size of the picture gradually increases, the recognition rate generally exists from small to large and then to small. Among them, the resolution ratio is 640 × 480, 800 × 600, and 1024 at 5 resolutions. * 768 may reach the maximum. When the resolutions are 50, 72, and 300, it reaches the maximum at 640 × 480, and the resolution reaches 100 at 800 and 600. When it reaches 200, it reaches the maximum at 1024 × 768. The results in Table 2 show that when the resolution is 100 and the size is 800 × 600, the resolution is optimal.

Table 2 Recognition rates with different resolutions and sizes

EEG signals are used as identification tools. From the perspective of the signal source, it is mainly the use of sports imagination [16, 17], picture stimulation (visual evoked potentials [18, 19] and photo stimulation [13, 15]), and resting state [20, 21]. Different signal sources and different analysis methods have different classification accuracy. The results from Table 3 show that the recognition rate of photos using stimuli is higher than that of other stimuli, but the identities of photographs are not optimal due to the fact that they do not take into account the attributes of photographs [13, 14]. This article uses wavelet entropy as features, and the highest recognition rate is 97.8%.

Table 3 Comparison of existing representative identification results

5 Conclusion

EEG signals as an identification and recognition tool have advantages over other biological information. Biometric systems based on EEG signals can achieve certain accuracy and speed. Collecting EEG signals will not cause any harm to the human body. It is easy to accept, and because the EEG signal is derived from the brain’s thinking activities, the way and the way of thinking between people are different. On the one hand, it is difficult to “decipher” the individual’s thinking. On the other hand, even if it is deciphered, it is difficult to forge the same. Using the EEG signal, it is impossible to “decipher” through subjective fraud, with the continuous enrichment of signal analysis tools and research results, to achieve “validation is not this person,” using the classification method to achieve, and through feature extraction and classification method design; basic can achieve verification purposes.

The use of EEG signals as an identification tool is an important branch of the current identity research. Because of the stability of the EEG signal characteristics induced by the vision, the picture is often used as a tool for inducing EEG signals. Among the visually evoked EEG signals, there are many research findings on the identification of EEG signals based on the differences in characteristics of EEG signals seen by the subjects themselves and their photos. However, this study does not have a good analysis of the effect of photo properties as a stimulus on the accuracy of classification. In this paper, the brain electrical signals of 20 subjects were used as a tool to analyze the influence of the size of the photo, black and white photos, and resolution attributes on the accuracy rate. The results show that the recognition rate results under different attributes are different.