- Open Access
Fatigue driving detection based on electrooculography: a review
EURASIP Journal on Image and Video Processing volume 2021, Article number: 33 (2021)
To accurately identify fatigued driving, establishing a monitoring system is one of the important guarantees of improving traffic safety and reducing traffic accidents. Among many research methods, electrooculogram signal (EOG) has unique advantages. This paper presents a systematic literature review of these technologies and summarizes a basic framework of fatigue driving monitoring system based on EOGs. Then we summarize the advantages and disadvantages of existing technologies. In addition, 80 primary references published during the last decade were identified. The multi-feature fusion technique based on EOGs performs better than other traditional methods due to its low cost, low power consumption and low intrusion, while its application is still limited which needs more efforts to obtain good and generalizable results. And then, an overview of the literature on technology is given, revealing a premier and unbiased survey of the existing empirical research of classification techniques that have been applied to fatigue driving analysis. Finally, this paper adds value to the current literature by investigating the application of EOG signals in fatigued driving and the design of related systems, future guidelines have been provided to practitioners and researchers to grasp the major contributions and challenges in the state-of-the-art research.
According to the global status report on road safety conducted by the World Health Organization (WHO) in 2015, 1.25 million people die from traffic accident every year worldwide and millions of more people suffer serious injuries in traffic accidents. It is estimated that the main cause of death of young people is road traffic injures, especially those who aged 15–29 years . During the driving process, the driver’s mental state seriously affects the driving behavior, which is a major hidden danger of traffic safety driving . In Europe, a study of 19 countries showed that 17 percent of drivers felt drowsy when driving, while 7% of those who fell asleep were involved in an accident .
In other kinds of illegal driving situations, such as overloading, speeding, overcrowding, drunk driving, we can take effective measures to supervise and reduce the occurrence of accidents. However, there are no quantitative and scientific measurement standards of fatigue driving. Different from other illegal driving behaviors, fatigued driving reflects a complex operation behavior, and it is difficult to be judged and identified. How to correctly and scientifically define driving fatigue and implement more accurate monitoring in real time is the main research focus for researchers.
The European Transport Safety Council (ETSC) defines the fatigue as “concerns the inability or disinclination to continue an activity, generally because the activity has been going on for too long” . There are four different kinds of fatigue according to the ETSC: local physical fatigue, general physical fatigue, central nervous fatigue and mental fatigue. Sleep in humans is considered as a passive process, whereas sleep actually is an active process . Sleep in humans is generally divided into two types: rapid eye moment (REM) sleep and non-rapid eye movement (NREM) sleep . Although researchers have made significant progress for indicating fatigue status, there are still very few findings that can be put into practice because of the environmental complexity and under-developed mechanisms of fatigue in the transport field .
Operator fatigue analysis contains diverse research questions. A reliable fatigue detection device in the future is not only about the development of a fatigue detection algorithm and hardware in mechanical engineering, but also about the understanding of the underlying mechanism of fatigue in a transport scenario. There are many existing fatigue monitoring technologies and classification methods, which can be divided into five categories according to the detection parameters and application technologies: (1) subjective measures: these methods are applied to measure the fatigue degree of the driver through drivers' subjective self-assessment; (2) driver’s biological measures: these methods are applied to detect fatigue by detecting changes of drivers’ biological signals, including electrocardiogram (ECG) , electroencephalogram (EEG) [8, 9], EOG  and other fatigue detection methods; (3) driver physical measures: these methods are applied to detect fatigue degree by human physiological response [11,12,13,14]; (4) driving performance measures: these methods are applied to judge the fatigue degree of drivers by driving performance ; (5) hybrid measures: these methods are often applied to integrate the two or more detection methods mentioned above to improve the monitoring accuracy [16,17,18] to deal with larger data and have better performance.
The existing methods of driving fatigue monitoring have some limitations. Although the subjective evaluation method is widely used, it has inherent defects which include the expectation bias and it can interfere with normal work, not suitable for continuous real-time monitoring of mental fatigue. Moreover, it is unrealistic to always ask drivers to report their state . The vision-based monitoring method is greatly affected by lighting. In the case of sunlight reflection and glasses reflection, the performance may decrease by 30% . In the current vision-based monitoring methods, the work for frontal faces is good, yet extreme head posture will lead to errors in monitoring results . Monitoring driver behavior based on vehicle operation information requires modification of vehicle structure, which is unrealistic and unwise in reality . The development of vehicle automation also has a certain restrictive effect on the technology of obtaining fatigue state by vehicle movement . In the monitoring method based on physiological signals, EOG is a potential fatigue monitoring technology due to its relatively low cost, low power consumption, corresponding speed, and it does not block the driver’s vision. In addition, EOG also has extensive eye movement tracking capability.
There is no doubt that the previous reviews on drivers’ fatigue monitoring collectively covered a wide range of fatigue systems, but those reviews vary in scope [22, 23]. Comparatively, the main contributions of this review on this subject is as below:
The importance of mental fatigue and fatigue driving is discussed and emphasized.
The limitations of fatigue driving monitoring methods in the current transportation field are discussed and analyzed to simplify the current research on non-invasive fatigue monitoring.
This review adds value to the current literature by studying the application of EOG signals in fatigued driving and the design of related systems.
In the rest of this review, the research results of the latest fatigue driving monitoring technology based on EOG are discussed in the second section. Next, the feature extraction and correlation extraction methods of EOG signals are discussed in the third section. And then, the feature classification method and the fatigue driving system based on EOG are discussed in the fourth section. Finally, several future interests with a discussion regarding the methodology of conducting operator fatigue research are provided.
In the past few years, there have been a number of fatigue driving monitoring techniques based on a variety of features. In the traditional methods, various signals generated by the driver are regarded as the effective characteristics of fatigue monitoring. Different from other bioelectrical signal, as its great potential of practicality and economy, the system based on EOGs has been developing rapidly in the field of fatigue driving condition monitoring in recent years.
A theoretical framework of fatigue driving system based on EOGs using the current technology and equipment is summarized. The structure of the proposed framework is illustrated in Fig. 1. The recognition technology based on biological signal belongs to one of the objective detection methods which is mainly applied to judge whether the driver is in the state of driving fatigue by analyzing the change rule of biological signal of the driver's body. The physiological status can be detected by supervising driver’s related physiological parameters, such as eyes activity, facial expressions, head nodding, body sagging posture, physiological electrical signals, etc.
As one of the physiological and electrical signals of the human body, EOG contains abundant information on features of eyelid movement. Different types of eye movement features can be extracted from EOGs, including REM , slow eye movement (SEM) , and eyelid movement , etc. To solve the problem that the electrode placement of the traditional EOG, Zhang et al. proposed a novel electrode placement on forehead to extract horizontal electrooculogram (HEO) and vertical electrooculogram (VEO) from forehead EOG . In addition to being used alone to monitor mental fatigue, EOGs are often combined with other signals to form new quantitative tools for mental fatigue [28, 29]. Ahn et al. used multimodal EEG/ECG/EOG and functional near infra-red spectroscopy (fNIRS) data to explore drivers’ mental fatigue states . Picot et al. used both brain and visual activity to detect drowsiness which enabled the false alarm rate to be reduced to 5% . In the field of machine learning, Jiao et al.  proposed a method, some new features were extracted based on wavelet singularity analysis and statistics to detect SEMs. Classifiers such as SVM, the discriminative graph regularized Extreme Learning Machine (GELM), and KNN were compared with a 2 s HEO signal which could finally be recognized as the category of SEMs or non-SEMs. The combination of the wavelet energy features and the new features based on wavelet singularity analysis and statistics improved the detection performance. The man–machine response mode (MRM) based on EOG and other eye movement characteristics can relieve long-term driving fatigue .
With the many studies already done on fatigue driving monitoring based on EOGs and the importance of these findings and their contribution to traffic safety, these kinds of work are the most essential to be explored as they laid the foundation for future technology.
Extraction of EOG features
Driving fatigue detection is a nonlinear problem. Physiological signals such as EEG and EOG are not fixed, and EOG is considered as a potential and effective method of monitoring fatigue status. Different physiological signals have different characteristics, such as the baseline drift of EOG and the non-repeatability and high signal-to-noise ratio (SNR) , the repeatability and good adaptability to baseline drift of ECG. In recent years, feature extraction methods for different physiological signals have also made considerable headway.
Baseline drift of EOG
Implementing an EOG-based interface raises several problems. Baseline drift is a major concern, since it decreases the tracking accuracy. Although drift changes the baseline slowly and it can be ignored over short periods, it can’t be ignored over longer periods, such as fatigue driving monitoring which can give an inaccurate absolute eye angle.
There are several strategies to overcome the baseline drift. The first strategy tries to lower the impact of the drift using a wavelet transform  and multiple EOGs . This strategy is effective in extending the period of ignoring the drift not for a long period. Accoupled EOGs are an easy solution to the drift problem which can be used to estimate the absolute eye angle . However, once an estimation error occurs, the error will persist all the time. Different from the previous strategy, frequent offset calibration can offset the drift . However, the current methods require explicit effort which can interrupt the main function. Finally, tuning the interaction is the other solution. The drift can be ignored when the interaction is based on the short-time eye movements instead of absolute eye angle when it needs an additional eye gesture.
As indicated previously, a single solution strategy is often unable to completely solve the drift problem. Thus, they are often combined in practical systems to deal with the drift issue. EOGs are useful for estimating eye gaze but it can add artifacts to EEG [38, 39]. Many researchers tried to remove EOG components from other signals [40, 41]. Zahan proposed a method based on the independent component analysis and multivariate empirical mode decomposition to remove EOG artifacts from EEGs . Cheng et al. used a combination of singular spectrum analysis and second-order blind identification method to remove diverse artifacts . However, EOGs and other bioelectrical signals bidirectionally contaminate each other . Unlike EEG, the nonlinearity of EOG is small, but the drift problem will eventually reduce the accuracy of the system when researchers regard it as a linear problem. Manabe et al. proposed a gaze estimation technique based on the nonlinearity between the EOG and eye angle which provided a practical solution to the drift problem .
Preprocessing of EOGs based on wavelet transform
Existing techniques for processing signals include Fourier transformation (FFT) , short-time Fourier transformation (STFT) , discrete wavelet transformation (WT) , wavelet packet transformation (WPT) , and entropy . Wavelet transform is more suitable than FFT for unstable signal processing, such as EEG and EOG, and Shannon entropy is more suitable for compression than classification problems. In fatigue driving EOG signal processing, there are a lot of researches based on wavelet transform.
Ho et al. used the wavelet methodology for EOG state classification which emphasized wavelet pattern recognition methods . The EOGs were processed by WT and then produced fuzzy signals for a neural network. High precision in time localization in the high frequency band can be achieved at the expense of reduced frequency resolution in WT and it can be scaled to match most of the high and low frequency signals. Therefore, it can achieve the optimal resolution with the least number of base functions. Calculating the percentage of eye closure over time (PERCLOS) is not robust in practical applications due to the complexity of eye detection methods . WT is sensitive to singularity and can produce a better result than the derivative method. While highly dynamical alterations are better reflected by EOGs than by integral measures such as PERCLOS for not containing any assessment of eye and eyelid movements. Few methods can detect overlong eye lid closures (more than 3 s) . New features were extracted based on wavelet singularity analysis and statistics to detect SEMs which can improve the classification results. For traditional EOGs, WT can be used to obtain many features which can be classified with higher classification accuracy [53, 54].
Preprocessing of EOGs based on wavelet packet
Using the WPT can construct features correlate with alertness and drowsiness. Different from WT, the WPT not only decomposes the approximation coefficients and the detail coefficients, but also can deal with stationary, nonstationary, or transitory characteristics of different signals .
The WPT was introduced by Coifman et al.  by generalizing the link between multiresolution approximations and wavelets. WPT has more subsequent decomposition levels than WT. Zhang et al.  utilized the relative energies of the WPT subspaces with Shannon entropy as a measure for feature suitability for detecting drowsiness which called HWPT. Wang et al.  proposed the optimal wavelet-packet feature-extraction method which called OWP. This method produced a better result. Khushaba et al. proposed a fuzzy mutual-information (MI)-based WPT feature-extraction method for classifying the driver drowsiness state . This method estimated the required MI using the fuzzy memberships which achieved a classification accuracy of 95–97% on an average across all subjects. Vigilance states are intrinsic mental states that involve temporal evolution rather than a time point . EOGs are easier to implement and ultimately more feasible than EEG for large-scale implementations for its higher signal-to-noise ratio. Zheng et al. used WPT to extract features from the forehead EOGs and applied two temporal dependency models, continuous conditional neural field (CCNF) and continuous conditional random field (CCRF) . Ding et al.  extracted different features including WPT coefficient and WT coefficient for EOG signals and the experimental results indicated that feature-level fusion (FLF) strategies achieved better classification accuracies than decision-level fusion (DLF) strategies.
Other methods of EOGs preprocessing
Different from the approach methods mentioned above, traditional linear spatial filters include principal component analysis (PCA), independent component analysis (ICA) , common spatial patterns for BCI applications  or beamforming methods for source localization . Chambon et al. used the linear spatial filters to exploit the array of sensors which increased the SNR . The last layer fed the features to a softmax classifier. As signal-gathering tools have improved, specialized manufacturers have produced devices that can quickly extract features. Kim et al. used the teager energy operator (TEO) by differential operator circuit with high-pass filter and low-pass configurations to extract features from multiple physiological signals . Li et al. used a wearable eye tracker to acquire a number of eye-movement features without signal preprocessing process . Advanced driving simulator can easily obtain a large number of signal features required by fatigue driving monitoring algorithm in laboratory environment .
With the many studies already done on extracting features from EOGs and the importance of these findings and their contribution to work, great progress has been made in fatigue driving feature extraction technology based on EOGs. Meanwhile, these methods also promote the research and development of fatigue driving system based on EOG signals.
Processing of EOGs and classifier models
Theoretical frameworks are essential contributions in grasping insight about the mechanisms influencing fatigue driving monitoring, and in the development of improving transportation safety . A theoretical framework for fatigue driving detection systems is summarized to more specifically study the relevant issues in this field. Accordingly, practitioners will be better directed in the future development of intervention strategies designed to improve safety outcomes.
The driver, vehicle and environment model (DVE model) are based on the concept of the ‘‘joint’’ cognitive system, where the dynamic interactions of the DVE model are represented in a complex way. This model can be seen a as closed loop system whose impact on the driver is observed in a driving situation.
Framework design of systems based on EOGs
In the DVE system, there are lots of actions of drivers to control the vehicle based on current driving environment. Fatigue driving monitoring technology has been greatly developed in recent years based on various body information of the driver during the driving process. As a potential approach of bioelectrical signal, EOGs plays an important role in fatigue driving condition monitoring system. A certain number of basic components affecting variability in driver’s driving monitoring systems based on EOGs are identified and listed as follows:
Chen et al.  improved the traditional feature selection methods of EEG, ECG and EOG. Gao et al.  proposed a method of fatigue detection by eye tracking glasses to evaluate driving fatigue detection algorithms. This method provides a new option for wireless and convenient detection of fatigue monitoring. Damousis et al.  proposed a fuzzy expert system for the driving fatigue detection. A system could predict fatigue accurately with a fuzzy combination of eyelid activity parameters. Barua et al. proposed a method of considering the effect of individual differences, that accuracy increased 10% .
Considering the various parameters associated with mental fatigue abound, there are few revealed on how mental fatigue influences drivers’ ability to detect hazardous situations. Unlike electrophysiological signal monitoring, eye tracking can be unaffected by electromagnetic, temperature and vibration. The accuracy of physiological signals has attracted the attention of many researchers, but the visual examination of continuous physiological signals is still a difficult and challenged task even for trained neuroscientists.
Although all of these research papers provide a significant contribution to furthering our work of the fatigue monitoring, there is still a lack of broad overviews based on relevant theoretical frameworks .
Signal analysis and modality
Eye movement characteristics, such as saccades, fixations, and blinks detected by EOG signals, have already been used for fatigue driving monitoring . The work detecting the three eye-movement characteristics demonstrates the promise of eye-based activity recognition (EAR) . There are a large number of parameters obtained by the preprocessing of EOGs, researchers usually need to select appropriate characteristic parameters when using the model to monitor fatigued driving. Table 1 describes some of the features of eye movements commonly used in fatigue drive monitoring after preprocessing of electrical eye signals. Other eye movement characteristics such as pupil dilation, microsaccades, vestibulo-ocular reflex, or smooth pursuit movements have few applications for its hard-to-measure with EOG .
This section presents a categorization of EOG modality regarding the various processing models. Table 2 shows the performance and accuracy of the fatigue driving detection model based on EOG as a comparison method of performance.
In fact, fatigue driving monitoring technology is the artificial construction of some algorithms, through the computer ability to automatically process the monitored data, extraction and classification, and automatic classification. Then the function of high precision fatigue monitoring can be realized [74,75,76,77]. Combining multiple non-invasive fatigue methods would provide good reliability . Just like the frame design mentioned in Fig. 1 the fatigue driving detection system based on EEGs that integrates other features can realize the modernized and convenient function in the future.
In this literature review, a range of peer-reviewed survey from journals or conferences is completed. The steps involved in conducting this survey include the identification of resources and selection of papers, along with data extraction and synthesis. Recognizing the relevant keywords plays an important role in identifying the proper studies. An important and impartial survey, by searching for relevant keywords and filtering them, shows only articles written in English over the past decade. To select relevant studies, we excluded duplicates and ambiguous research questions, and assessed the accuracy and specificity of the remaining papers.
From the selected studies, we categorized the classification techniques used for driving detection relating to the EOGs as follows:
Artificial neural network (ANN)
Support vector machines (SVM)
Fuzzy and neuro fuzzy based (NF)
Inductive rule based (IR)
Spectral bruit analysis (SBA)
Bayesian learners (BL)
Integration and differentiation based (ID)
Extremely learning machine (ELM)
Relevant vector machine (RVM)
Principal component analysis (PCA)
Independent component analysis (ICA)
As shown in Fig. 2 presenting the amount of researcher attention that each type of technique received during the last decade, an apparent publication peak is shown around the years 2019 and 2020. Overall, SVM, ANN, CL and NF are the four most frequently used ones; they together were adopted by 62% of the selected studies, as illustrated in Fig. 3. Figure 2 shows the distribution of research interest in each publication year. As can be seen, the activity of publications in this field is growing.
Compared to other fatigue driving monitoring techniques based on EOGs, SVM, ANN, CL and NF seem to have received dominant attention in many years. In addition to these four classification methods, traditional methods such as ICA, PCA and IR have also been developed in recent years. In addition, the new methods, such as RVM and ELM, are also developing at an accelerated pace in recent years, showing good classification performance and practicality.
One of the main goals of this review is to pave the way for future researchers and practitioners by gaining insight into fatigue driving monitoring methods. To validate the stability of the developed machine learning model and verify its effectiveness, the researchers used performance measures to describe the reliability required for the evaluation. In terms of validation methods, cross validation is a common method. For example, reference 19 used the leave-one subject-out verification method, and the cross-validation accuracy of reference 10 was 80.74%. In addition to cross-validation, other performance measures are currently being used by researchers. An overview of the proportion of studies using each performance metric is illustrated in Fig. 4. According to the figure, it is found that Accuracy and correlation coefficient are popular performance metrics. 78.13% of the studies used Accuracy as a metric, while 12.5% used correlation coefficients. The most popular performance metrics was Accuracy, followed by correlation coefficient, root mean square error (RMSE), cross-validation, Recall, specificity, sensitivity, and the area under the receiver operating characteristic (ROC). Figure 5 shows the performance of the various classification algorithms in terms of Accuracy. According to the figure, in terms of Accuracy, ANN performed the best, followed by RVM, SVM and ELM, and NF and BL performed the worst.
The corresponding extraction algorithms and classification methods are different based on different signal types. In classification algorithms such as SVM, as a non-parametric method, the use of nonlinear discriminant function can overcome the limitation of parameter statistics . SVM has outstanding capabilities of redundant feature processing, high-dimensional and small sample data processing. The ANN has been applauded for its excellent anti-noise ability, learning ability and adaptability. Although the current monitoring accuracy of SVM and ANN is high, the learning speed will decline when processing a large number of samples. Compared with other classifiers, ELM can greatly improve the learning speed . ELM is also relatively robust to over fitting, outliers and noise .
Although the current research on fatigue driving is beginning to pay off, the performance of these methods is always different for different scenarios and different database tasks. Subsequent researchers will improve on previous studies to overcome their inherent shortcomings and form new research techniques for application.
Nowadays, with the rapid development of the transportation industry, traffic safety accidents related to the fatigue driving state occur frequently. In recent years, the research on relevant methods is also booming. However, different research methods, directions and algorithms vary in the accuracy of fatigue driving state detection. This paper summarizes and sorts out the current methods of fatigue driving state analysis and detection based on EOG signals, and also summarizes relevant feature classification methods. Fair and unbiased comments are given in this paper. In addition, this review does not include any work in progress, unpublished or non-peer reviewed publications. Finally, the strengths and weaknesses of fatigue driving detection techniques based on EOGs were extracted directly from the selected studies. This paper is a systematic literature review, 80 primary studies published during the last decade were identified. Although we cannot cover all relevant studies in this field, we believe it will be of some benefit to anyone who is interested in this field of research. The main findings obtained from the selected primary studies are:
The classification techniques used for driving detection relating to the EOGs included Artificial Neural Network (ANN), Support Vector Machines (SVM), Fuzzy & Neuro Fuzzy based (NF), Clustering (CL), Inductive Rule Based (IR), Spectral Bruit Analysis (SBA), Bayesian Learners (BL), Integration and Differentiation based (ID), Extremely Learning Machine (ELM), Relevant Vector Machine (RVM), Principal Component Analysis (PCA) and Independent Component Analysis (ICA). Among them, SVM, ANN, CL and NF are used most frequently.
Accuracy and correlation coefficient are the most commonly used for performance measurement in the primary studies. The overall estimation accuracy of most techniques is close to the acceptable level.
ANN performed the best, followed by RVM, SVM and ELM, and NF and BL performed the worst in terms of accuracy.
The general trend is as follows: It is feasible to study driving fatigue by simulating test with driving simulator. On the basis of single feature monitoring, multi-feature fusion technology status monitoring will become the mainstream of research. Future technologies need to be improved from the following aspects:
Improve the real-time performance of the monitoring method. Traffic accidents happen in a very short time, which requires that the detection of driving fatigue can be rapid, accurate and timely, and can give early warning to drivers. However, the existing driving fatigue detection methods are poor in real time so far.
Eliminate or significantly reduce the intrusion of the detection system. The method based on EOGs has a high accuracy, because it can monitor the physiological state from the human body in real time, but it may affect the driver's operation. Low contact is a major challenge for future hardware facilities.
Improve the cost performance of relevant technologies. Even though EOG is cheaper than other methods, the equipment required by existing conditions is still complex and expensive, which makes it difficult to popularize. The post-processing data of multi-feature fusion technology is huge and the algorithm is complex, so simplifying the algorithm and controlling the cost is a great challenge in the future.
Establish a set of accurate and recognized fatigue classification criteria. The current fatigue detection technology has no mature reference to accurately classify the fatigue detection level.
The development of relevant technologies should consider different driving environments and road conditions, and develop relevant fatigue detection technologies suitable for different professional drivers.
Ideally, reliable fatigue detection equipment in the future can not only accurately identify drivers’ fatigue, then warn them, but also notify the third party to intervene when necessary, which will greatly improve the safety of transportation.
Availability of data and materials
This paper is a literature review. All data generated or analyzed during this study are included in this published article. The table and picture information in this paper are all collected from the references. The data in Figs. 2 and 3, is collected from the uploaded excel, and the other data is collected from the references.
The World Health Organization
The European Transport Safety Council
Rapid eye moment
Non-rapid eye movement
Slow eye movement
Discriminative graph regularized extreme learning machine
Short-time Fourier transformation
Wavelet packet transformation
Percentage of eye closure over time
Continuous conditional neural field
Continuous conditional random field
Principal component analysis
Independent component analysis
Teager energy operator
Eye-based activity recognition
Artificial neural network
Support vector machines
Fuzzy and neuro fuzzy based
Inductive rule based
Spectral bruit analysis
Integration and differentiation based
Extremely learning machine
Relevant vector machine
Root mean square error
The area under the receiver operating characteristic
WHO | Global status report on road safety 2015. 2015. Retrieved from http://www.who.int/violenceinjuryprevention/roadsafetystatus/2015/en/
F.W. Wang, Q. Xu, R.R. Fu, Study on the effect of man-machine response mode to relieve driving fatigue based on EEG and EOG. Sensors (2019). https://doi.org/10.3390/s19224883
G. Marta, A. Roberto, L. Raquel, Å. Torbjörn, C. Fabio, H. Jim, L. Damien, W.T. McNicholas, P. Markku, T.S. Joaquín, P. Philippe, G. Ludger, Sleepiness at the wheel across Europe: a survey of 19 countries. J. Sleep Res. 24(3), 242–253 (2015). https://doi.org/10.1111/jsr.12267
H. D. Croo, M. Bandmann, G. M. Mackay, K. Rumar, and P. Vollenhoven, The role of driver fatigue in commercial road transport crashes. (Eur. Transp. Safety Council, Brussels, Belgium, Tech. Rep. 2001), p. 47
A. Swetapadma, Novel approach for sleep disorder monitoring using a finite-state machine for localities lacking specialist physicians. IET Sci. Meas. Technol. 11(8), 1099–1103 (2017). https://doi.org/10.1049/iet-smt.2017.0240
X.Y. Hu, G. Lodewijks, Detecting fatigue in car drivers and aircraft pilots by using non-invasive measures: the value of differentiation of sleepiness and mental fatigue. J. Safety Res. 72, 173–187 (2020). https://doi.org/10.1016/j.jsr.2019.12.015
R.N. Khushaba, S. Kodagoda, S. Lal, G. Dissanayake, Uncorrelated fuzzy neighborhood preserving analysis based feature projection for driver drowsiness recognition. Fuzzy Sets Syst. 221, 90–111 (2013). https://doi.org/10.1016/j.fss.2012.12.003
W.Z. Kong, W.C. Lin, B. Fabio, S.Q. Hu, B. Gianluca, Investigating driver fatigue versus alertness using the granger causality network. Sensors 15(8), 19181–19198 (2015). https://doi.org/10.3390/s150819181
Z.K. Gao, X.M. Wang, Y.X. Yang, C.X. Mu, Q. Cai, W.D. Dang, S.Y. Zuo, EEG-based spatio-temporal convolutional neural network for driver fatigue evaluation. IEEE Trans. Neural Netw. Learning Syst. 30(9), 2755–2763 (2019). https://doi.org/10.1109/TNNLS.2018.2886414
S.Y. Hu, G.T. Zheng, Driver drowsiness detection with eyelid related parameters by Support Vector Machine. Expert Syst. Appl. 36(4), 7651–7658 (2009). https://doi.org/10.1016/j.eswa.2008.09.030
X. Fan, B. C. Yin, Y. F. Sun, Yawning detection for monitoring driver fatigue. Machine Learning and Cybernetics, 2007 International Conference on. IEEE, 2: 664–668. (2007). https://doi.org/10.1109/ICMLC.2007.4370228
A. Mittal, K. Kumar, S. Dhamija and M. Kaur. Head movement-based driver drowsiness detection: a review of state-of-art techniques. Engineering and Technology (ICETECH), 2016 IEEE International Conference on. IEEE, 903–908. (2016). https://doi.org/10.1109/ICETECH.2016.7569378
A. Eskandarian, A. Mortazavi, Evaluation of a smart algorithm for commercial vehicle driver drowsiness detection. Intelligent Vehicles Symposium, 2007 IEEE: IEEE, 553–559. (2007). https://doi.org/10.1109/IVS.2007.4290173
V. Vijayan, K.P. Pushpalatha, A comparative analysis of RootSIFT and SIFT methods for drowsy features extraction. Procedia Comput. Sci. 171, 171 (2020). https://doi.org/10.1016/j.procs.2020.04.046
V.J. Kartsch, S. Benatti, P.D. Schiavone, D. Rossi, L. Benini, A sensor fusion approach for drowsiness detection in wearable ultra-low-power systems. Inf. Fusion 43, 66–76 (2018). https://doi.org/10.1016/j.inffus.2017.11.005
F. Laurent, M. Valderrama, M. Besserve, M. Guillard, J.P. Lachaux, J. Martinerie, G. Florence, Multimodal information improves the rapid detection of mental fatigue. Biomed. Signal Process. Control 8(4), 400–408 (2013). https://doi.org/10.1016/j.bspc.2013.01.007
G. Borghini, G. Vecchiato, J. Toppi, L. Astolfi, A. Maglione, R. Isabella, C. Caltagirone, W. Kong, D. Wei, Z. Zhou, L. Polidori, S. Vitiello and F. Babiloni, Assessment of mental fatigue during car driving by using high resolution EEG activity and neurophysiologic indices. 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, 6442–6445. (2012). https://doi.org/10.1109/EMBC.2012.6347469
F.Y. Song, X.Y. Tan, X. Liu, S.C. Chen, Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients. Pattern Recogn. 47(9), 2825–2838 (2014). https://doi.org/10.1016/j.patcog.2014.03.024
K. Bylykbashi, E. Qafzezi, M. Ikeda, K. Matsuo, L. Barolli, Fuzzy-based Driver Monitoring System (FDMS): implementation of two intelligent FDMSs and a testbed for safe driving in VANETs. Futur. Gener. Comput. Syst. 105, 665–674 (2020). https://doi.org/10.1016/j.future.2019.12.030
Y. Wang, R. Huang, L. Guo, Eye gaze pattern analysis for fatigue detection based on GP-BCNN with ESM. Pattern Recogn. Lett. 123, 61–74 (2019). https://doi.org/10.1016/j.patrec.2019.03.013
J. Li, H. Li, W. Umer, H.W. Wang, X.J. Xing, S.K. Zhao, J. Hou, Identification and classification of construction equipment operators’ mental fatigue using wearable eye-tracking technology. Autom. Constr. (2020). https://doi.org/10.1016/j.autcon.2019.103000
Y. Dong, Z. Hu, K. Uchimura and N. Murayama, Driver inattention monitoring system for intelligent vehicles: a review. 2009 IEEE Intelligent Vehicles Symposium, Xi’an, China, pp. 875–880. (2009). https://doi.org/10.1109/IVS.2009.5164395
J. Horne, L. Reyner, Vehicle accidents related to sleep: a review. Occup. Environ. Med. 56(5), 289–294 (1999). https://doi.org/10.1136/oem.56.5.289
R. Schleicher, N. Galley, S. Briest, L. Galley, Blinks and saccades as indicators of fatigue in sleepiness warnings: looking tired? Ergonomics 51(7), 982–1010 (2008). https://doi.org/10.1080/00140130701817062
C. Filippo, P. Fabio, P. Federica, M. Elisa, An improved algorithm for the automatic detection and characterization of slow eye movements. Med. Eng. Phys. 36(7), 954–961 (2014). https://doi.org/10.1016/j.medengphy.2014.03.019
Y.S. Kim, H.J. Baek, J.S. Kim, H.B. Lee, J.M. Choi, K.S. Park, Helmet-based physiological signal monitoring system. Eur. J. Appl. Physiol. 105(3), 365–372 (2009). https://doi.org/10.1007/s00421-008-0912-6
Y. Zhang, X. Gao, J. Zhu, W. Zheng and B. Lu, A novel approach to driving fatigue detection using forehead EOG. 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER) Montpellier, 707–710. (2015). https://doi.org/10.1109/NER.2015.7146721
S.S.H. Nazari, A. Moradi, K. Rahmani, A systematic review of the effect of various interventions on reducing fatigue and sleepiness while driving. Chin. J. Traumatol. 20(5), 249–258 (2017). https://doi.org/10.1016/j.cjtee.2017.03.005
H. Wang, C. Wu, T. Li, Y. He, P. Chen, A. Bezerianos, Driving fatigue classification based on fusion entropy analysis combining EOG and EEG. IEEE Access 7, 61975–61986 (2019). https://doi.org/10.1109/ACCESS.2019.2915533
S. Ahn, T. Nguyen, H. Jang, J.G. Kim, S.C. Jun, Exploring neuro-physiological correlates of drivers’ mental fatigue caused by sleep deprivation using simultaneous EEG, ECG, and fNIRS data. Front Hum Neurosci 10, 219 (2016). https://doi.org/10.3389/fnhum.2016.00219
A. Picot, S. Charbonnier, A. Caplier, On-line detection of drowsiness using brain and visual information. IEEE Trans. Syst., Man, Cybern. Part A: Syst. Humans 42(3), 764–775 (2012). https://doi.org/10.1109/TSMCA.2011.2164242
Y. Y. Jiao, Yong Peng, B. L. Lu, X. P. Chen, S. G. Chen and C. H. Wang, Recognizing slow eye movement for driver fatigue detection with machine learning approach. International Joint Conference on Neural Networks. IEEE; 4035–4041. (2014). https://doi.org/10.1109/IJCNN.2014.6889615
A. Bulling, J.A. Ward, G. Hans, T. Gerhard, Eye movement analysis for activity recognition using electrooculography. IEEE Trans. Pattern Anal. Machine Intell. (2011). https://doi.org/10.1109/TPAMI.2010.86
A. Bulling, D. Roggen, G. Troester, What’s in the eyes for context-awareness? IEEE Pervasive Comput (2010). https://doi.org/10.1109/MPRV.2010.49
H. Manabe and M. Fukumoto, Full-time wearable headphone-type gaze detector. in Proc. Extended Abstracts Conf. Human Factors Comput. Syst. 1073–1078. (2006). https://doi.org/10.1145/1125451.1125655
N. Itakura, K. Sakamoto, A new method for calculating eye movement displacement from AC coupled electro-oculographic signals in head mounted eye-gaze input interfaces. Biomed. Signal Process. Control 5(2), 142–146 (2010). https://doi.org/10.1016/j.bspc.2009.12.002
T. Yagi, Y. Kuno, K. Koga and T. Mukai, Drifting and blinking compensation in electro-oculography (EOG) eye-gaze interface. in Proc. IEEE Conf. Syst., Man, Cybern. p. 3222–3226. (2006). https://doi.org/10.1109/ICSMC.2006.384613
M.C. Song, L.N. Li, J.T. Guo, T. Liu, S.Y. Li, Y.T. Wang, Q.U. Ain, J. Wang, A new method for muscular visual fatigue detection using electro-oculogram. Biomed. Signal Proc. Control (2020). https://doi.org/10.1016/j.bspc.2020.101865
J.W. Kelly, D.P. Siewiorek, A. Smailagic, J.L. Collinger, D.J. Weber, W. Wang, Fully automated reduction of ocular artifacts in high-dimensional neural data. IEEE Trans. Biomed. Eng. 58(3), 598–606 (2011). https://doi.org/10.1109/ICSMC.2006.384613
A.K. Maddirala, R.A. Shaik, Removal of EOG artifacts from single channel EEG signals using combined singular spectrum analysis and adaptive noise canceler. IEEE Sens. J. 16(23), 8279–8287 (2016). https://doi.org/10.1109/JSEN.2016.2560219
X. Li, C. Guan, H. Zhang, K.K. Ang, Discriminative ocular artifact correction for feature learning in EEG analysis. IEEE Trans. Biomed. Eng. 64(8), 1906–1913 (2017). https://doi.org/10.1109/TBME.2016.2628958
S. Zahan, Removing EOG artifacts from EEG signal using noise-assisted multivariate empirical mode decomposition. 2016 2nd International Conference on Electrical, Computer & Telecommunication Engineering (ICECTE), Rajshahi, 1–5. 8–10, (2016)
J. Cheng, L.C. Li, C. Li, Y. Liu, A. Liu, R.B. Qian, X. Chen, Remove diverse artifacts simultaneously from a single-channel EEG based on SSA and ICA. A Semi-Simulated Study. IEEE Access 7, 60276–60289 (2019). https://doi.org/10.1109/ACCESS.2019.2915564
A. Bulling, D. Roggen and G. Tröster, Wearable EOG goggles: seamless sensing and context-awareness in everyday environments. in Proc. Extended Abstracts Conf. Human Factors Comput. Syst. p. 3259–3264. (2009). https://doi.org/10.3233/AIS-2009-0020
M. Hiroyuki, F. Masaaki, Y. Tohru, Direct gaze estimation based on nonlinearity of EOG. IEEE Trans. Bio-medical Eng. (2015). https://doi.org/10.1109/TBME.2015.2394409
X. Gao, Y. Zhang, W. Zheng and B. Lu, Evaluating driving fatigue detection algorithms using eye tracking glasses. 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER), Montpellier, p. 767–770. 22–24 (2015)
B.T. Jap, S. Lal, P. Fischer, E. Bekiaris, Using EEG spectral components to assess algorithms for detecting fatigue. Expert Syst. Appl. 36(2), 2352–2359 (2009). https://doi.org/10.1016/j.eswa.2007.12.043
S. Iampetch, Y. Punsawad and Y. Wongsawat, EEG-based mental fatigue prediction for driving application. The 5th 2012 Biomedical Engineering International Conference, Ubon Ratchathani 2012; 1–5. 5–7, (2012)
X. Ding, Z. Lv, C. Zhang, X. Gao, B. Zhou, A robust online saccadic eye movement recognition method combining electrooculography and video. IEEE Access 5, 17997–18003 (2017). https://doi.org/10.1109/ACCESS.2017.2750701
C. Zhang, H. Wang, R. Fu, Automated detection of driver fatigue based on entropy and complexity measures. IEEE Trans. Intell. Transp. Syst. 15(1), 168–177 (2014). https://doi.org/10.1109/TITS.2013.2275192
C. K. Ho and M. Sasaki. Brain-wave bio potentials based mobile robot control: wavelet-neural network pattern recognition approach. 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236), Tucson 2001; 322–328. (2001). https://doi.org/10.1109/ICSMC.2001.969832
A. Chowdhury, R. Shankaran, M. Kavakli, M.M. Haque, Sensor applications and physiological features in drivers’ drowsiness detection: a review. IEEE Sens. J. 18(8), 3055–3067 (2018). https://doi.org/10.1109/JSEN.2018.2807245
X. Q. Huo, W. Zheng and B. Lu, Driving fatigue detection with fusion of EEG and forehead EOG. 2016 International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, 897–904. 24–29, (2016)
L. Du, W. Liu, W. Zheng and B. Lu, Detecting driving fatigue with multimodal deep learning. 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER), Shanghai 2017; 74–77. (2017). https://doi.org/10.1109/NER.2017.8008295
L. Deqiang, W. Pedrycz, N.J. Pizzi, Fuzzy wavelet packet based feature extraction method and its application to biomedical signal classification. IEEE Trans. Biomed. Eng. 52(6), 1132–1139 (2005). https://doi.org/10.1109/TBME.2005.848377
R. R. Coifman, Y. Meyer, S. Quake, and V. Wickerhauser, Wavelet analysis and Signal processing. in Wavelets and Their Applications. (Jones and Barlett, Sudbury, 1992), p. 153–178
C. Zhang, C.X. Zheng, X.L. Yu, Automatic recognition of cognitive and fatigue from physiological indices by using wavelet packet transform and kernel learning methods. Expert Syst. Appl. 36(3), 4664–4671 (2009). https://doi.org/10.1016/j.eswa.2008.06.022
G. Wang, Z. Wang, W. Chen, J. Zhuang, Classification of surface EMG signals using optimal wavelet packet method based on Davies-Bouldin criterion. Med. Biol. Eng. Compu. 44(10), 1741–2444 (2006). https://doi.org/10.1007/s11517-006-0100-y
R.N. Khushaba, S. Kodagoda, S. Lal, G. Dissanayake, Driver drowsiness classification using fuzzy wavelet-packet-based feature-extraction algorithm. IEEE Trans. Biomed. Eng. 58(1), 121–131 (2011). https://doi.org/10.1109/TBME.2010.2077291
W.L. Zheng, K. Gao, G. Li, W. Liu, C. Liu, J.Q. Liu, G.X. Wang, B.L. Lu, Vigilance estimation using a wearable EOG device in real driving environment. IEEE Trans. Intell. Transp. Syst. 21(1), 170–184 (2020). https://doi.org/10.1109/TITS.2018.2889962
L.C. Parra, C.D. Spence, A.D. Gerson, P. Sajda, Recipes for the linear analysis of EEG. Neuroimage 28(2), 326–341 (2005). https://doi.org/10.1016/j.neuroimage.2005.05.032
B. Blankertz, R. Tomioka, S. Lemm, M. Kawanabe, K.R. Müller, Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Process. Mag. 25(1), 41–56 (2008). https://doi.org/10.1109/MSP.2008.4408441
B.D. Van Veen, W. van Drongelen, M. Yuchtman, A. Suzuki, Localization of brain electrical activity via linearly constrained minimum variance spatial filtering. IEEE Trans. Biomed. Eng. 4(9), 867–880 (1997). https://doi.org/10.1109/10.623056
S. Chambon, M.N. Galtier, P.J. Arnal, G. Wainrib, A. Gramfort, A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series. IEEE Trans. Neural Syst. Rehabil. Eng. 26(4), 758–769 (2018). https://doi.org/10.1109/TNSRE.2018.2813138
S. Kim, K. Lee, J. Yeom, T. Lee, D. Kim, J.J. Kim, Wearable multi-biosignal analysis integrated interface with direct sleep-stage classification. IEEE Access 2020(8), 46131–46140 (2020). https://doi.org/10.1109/ACCESS.2020.2978391
J. Li, H. Li, H.W. Wang, W. Umer, H. Fu, X.J. Xing, Evaluating the impact of mental fatigue on construction equipment operators’ ability to detect hazards using wearable eye-tracking technology. Autom. Constr. (2019). https://doi.org/10.1016/j.autcon.2019.102835
J. Li, H. Li, H.W. Wang, W. Umer, H. Fu, X.J. Xing, Driver drowsiness detection using mixed-effect ordered logit model considering time cumulative effect. Anal. Methods Accid. Res. (2020). https://doi.org/10.1016/j.amar.2020.100114
Z.E.A. Elassad, H. Mousannif, H.A. Moatassime, A. Karkouch, The application of machine learning techniques for driving behavior analysis: a conceptual framework and a systematic literature review. Eng. Appl. Artif. Intell. 87, 103312 (2020). https://doi.org/10.1016/j.engappai.2019.103312
L.L. Chen, A. Zhang, X.G. Lou, Cross-subject driver status detection from physiological signals based on hybrid feature selection and transfer learning. Expert Syst. Appl. 137, 266–280 (2019). https://doi.org/10.1016/j.eswa.2019.02.005
X. Y. Gao, Y. F. Zhang, W. L. Zheng and B. L. Lu, Evaluating driving fatigue detection algorithms using eye tracking glasses. 7th Annual International IEEE EMBS Conference on Neural Engineering (NER), 22–24. (2015). https://doi.org/10.1109/NER.2015.7146736
I.G. Damousis, D. Tzovaras, Fuzzy fusion of eyelid activity indicators for hypovigilance-related accident prediction. IEEE Trans. Intell. Transp. Syst. 9(3), 491–500 (2008). https://doi.org/10.1109/TITS.2008.928241
S. Barua, M.U. Ahmed, C. Ahlström, S. Begum, Automatic driver sleepiness detection using EEG, EOG and contextual information. Expert Syst. Appl. 115, 121–135 (2018). https://doi.org/10.1016/j.eswa.2018.07.054
Y.C. Dong, Z.C. Hu, K. Uchimura, N. Murayama, Driver inattention monitoring system for intelligent vehicles: a review. IEEE Trans. Intell. Transp. Syst. 12(2), 596–614 (2011). https://doi.org/10.1109/TITS.2010.2092770
D. Suman, M. Malini and S. Anchuri, EOG based vigilance monitoring system. 2015 Annual IEEE India Conference (INDICON), New Delhi 2015; 1–6. (2015). https://doi.org/10.1109/INDICON.2015.7443210
C. Minho, S. Minseok, L.J. Seong, K.S. Woo, Fuzzy support vector machine-based personalizing method to address the inter-subject variance problem of physiological signals in a driver monitoring system. Artif. Intell. Med. (2020). https://doi.org/10.1016/j.artmed.2020.101843
B.S.M. Caio, J.D.C.M. Márcio, S.J.M. Marques, I.D. Li, Real-time classification for autonomous drowsiness detection using eye aspect ratio. Expert Syst. Appl. 15, 158 (2020). https://doi.org/10.1016/j.eswa.2020.113505
B. Wang, Y.D. Sun, T. Zhang, T. Sugi, X.Y. Wang, Bayesian classifier with multivariate distribution based on D-vine copula model for awake/drowsiness interpretation during power nap. Biomed. Signal Proc. Control (2020). https://doi.org/10.1016/j.bspc.2019.101686
G. Borghini, L. Astolfi, G. Vecchiato, D. Mattia, F. Babiloni, Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci. Biobehav. Rev. 44, 58–75 (2014). https://doi.org/10.1016/j.neubiorev.2012.10.003
D. Sommer,M. Golz, Evaluation of PERCLOS based current fatigue monitoring technologies. Conference proceedings: ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference; (2010). https://doi.org/10.1109/IEMBS.2010.5625960
J.C. Chen, H. Wang, C.C. Hua, Electroencephalography based fatigue detection using a novel feature fusion and extreme learning machine. Cogn. Syst. Res. 52, 715–728 (2020). https://doi.org/10.1016/j.cogsys.2018.08.018
The authors are grateful to professor Haitao Guo and associate professor Fuwang Wang for the guidance of this paper.
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Science Foundation of China (61661038).
No conflict of interest exits in the submission of this manuscript, and manuscript is approved by all authors for publication. I would like to declare on behalf of my co-authors that the work described was original research that has not been published previously, and not under consideration for publication elsewhere, in whole or in part. All the authors listed have approved the manuscript that is enclosed.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Tian, Y., Cao, J. Fatigue driving detection based on electrooculography: a review. J Image Video Proc. 2021, 33 (2021). https://doi.org/10.1186/s13640-021-00575-1
- Fatigue driving state monitoring
- Driving behavior monitoring
- Literature review