Fatigue driving is one of the main causes of traffic accidents. In recent years, considerable attention has been paid to fatigue detection systems, which is an important solution for preventing fatigue driving. In order to prevent and reduce fatigue driving, a driver fatigue detection system based on computer vision is proposed. In this system, an improved face detection method is used to detect the driver’s face from the image obtained by a charge coupled device (CCD) camera. Then, the feature points of the eyes and mouth are located by an ensemble of regression trees. Next, fatigue characteristic parameters are calculated by the improved percentage of eyelid closure over the pupil over time algorithm. Finally, the state of drivers is evaluated by using a fuzzy neural network. The system can effectively monitor and remind the state of drivers so as to significantly avoid or decrease the occurrence of traffic accidents. The experimental results show that the system is of wonderful real-time performance and accurate recognition rate, so it meets the requirements of practicality in driver fatigue detection greatly.

Keywords:

References

1. C. Zhao, Y. Zhang, X. Zhang and J. He , Recognition of driver’s fatigue expression using local multiresolution derivative pattern, J. Intell. Fuzzy Syst. 30(1), 547–560 (2016). Crossref, Web of Science, Google Scholar
2. M. H. Abdullah, K. J. Raman, A. Azman, S. Yogarayan, H. A. A. Elbendary, M. F. A. Abdullah and S. Z. Ibrahim , Driver fatigue detection, in Information Science and Applications (ICISA) (Singapore, 2016), pp. 269–278. Crossref, Google Scholar
3. C. Lin, R. Wu, S. Liang, W. Chao, Y. Chen and T. Jung , EEG-based drowsiness estimation for safety driving using independent component analysis, IEEE Trans. Circuits Syst. 52(12), 2726–2738 (2005). Crossref, Google Scholar
4. C. Lin, Y. Chen, T. Huang, T. Chiu, W. Li, S. Liang, H. Y. Hsieh, S. H. Hsu and J. R. Duann , Development of wireless brain computer interface with embedded multitask scheduling and its application on real-time driver’s drowsiness detection and warning, IEEE. Trans. Biomed. Eng. 55, 1582–1591 (2008). Crossref, Web of Science, Google Scholar
5. A. Sangtae, T. Nguyen, H. Jang, J. G. Kim and S. C. Jun , Exploring neuro-physiological correlates of drivers’ mental fatigue caused by sleep deprivation using simultaneous EEG, ECG, and fNIRS data, Front. Hum. Neurosci. 10, 219 (2016). Web of Science, Google Scholar
6. G. Borghini, L. Astolfi, G. Vecchiato, D. Mattia and F. Babiloni , Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness, Neurosci. Biobehav. Rev. 44, 58–75 (2014). Crossref, Web of Science, Google Scholar
7. L. Salvati, M. d’Amore, A. Fiorentino, A. Pellegrino, P. Sena and F. Villecco , On-road detection of driver fatigue and drowsiness during medium-distance journeys, Entropy 23(2), 135 (2021). Crossref, Web of Science, Google Scholar
8. A. Muhammad, B. Nasreen and D. Micheal , A hybrid approach to detect driver drowsiness utilizing physiological signals to improve system performance and wearability, Sensors 17(9), 1991 (2017). Crossref, Web of Science, Google Scholar
9. K. Chui, K. F. Tsang, H. Chi, B. W. K. Ling and C. Wu , An accurate ECG-based transportation safety drowsiness detection scheme, IEEE Trans. Ind. Electron. 12(4), 1438–1452 (2016). Google Scholar
10. A. D. McDonald, J. D. Lee, C. Schwarz and T. L. Brown , Steering in a random forest, Hum. Factors 56(5), 986–998 (2014). Crossref, Web of Science, Google Scholar
11. J. Wang, C. Lin and S. Chen , Applying fuzzy method to vision-based lane detection and departure warning system, Expert Syst. Appl. 37(1), 113–126 (2017). Crossref, Web of Science, Google Scholar
12. T. Abe, T. Nonomura, S. Komada, T. Asaoka, A. U. Sasai and Y. Inoue , Detecting deteriorated vigilance using percentage of eyelid closure time during behavioral maintenance of wakefulness tests, J. Psychophysiol. 82(3), 269–274 (2011). Crossref, Web of Science, Google Scholar
13. R. Gupta, K. Aman, N. Shiva and Y. Singh , An improved fatigue detection system based on behavioral characteristics of driver, in International Conference on Intelligent Transportation Engineering (ICITE) (Singapore, 2017), pp. 227–230. Crossref, Google Scholar
14. Z. Liu, Y. Peng and W. Hu , Driver fatigue detection based on deeply-learned facial expression representation, J. Vis. Commun. Image Represent. 71, 102723 (2020). Crossref, Web of Science, Google Scholar
15. F. Friedrichs and B. Yang , Camera-based drowsiness reference for driver state classification under real driving conditions, in Intelligent Vehicles Symposium (IVS) (IEEE, La Jolla, CA, USA, 2010), pp. 101–106. Crossref, Google Scholar
16. N. Azmi, A. S. M. M. Rahman, S. Shirmohammadi and A. E. Saddik , LBP-based driver fatigue monitoring system with the adoption of haptic warning scheme, in International Conference on Virtual Environments (ICVE) (IEEE, Ottawa, OA, Canada, 2011), pp. 1–4. Crossref, Google Scholar
17. Q. Zhao, J. Jiang, Z. Lei and J. Yi , Detection method of eyes opening and closing ratio for driver’s fatigue monitoring, IET Intell. Transp. Syst. 15(1), 31–42 (2021). Crossref, Web of Science, Google Scholar
18. T. Liu, Y. Yang, G. Huang, K. Yong and Z. Lin , Driver distraction detection using semi-supervised machine learning, IEEE Trans. Ind. Electron. 17(4), 1108–1120 (2015). Google Scholar
19. K. Dwivedi, K. Biswaranjan and A. Sethi , Drowsy driver detection using representation learning, in International Advance Computing Conference (IACC) (Gurgaon, India, 2014), pp. 995–999. Crossref, Google Scholar
20. M. Hajinoroozi, Z. Mao and Y. Huang , Prediction of driver’s drowsy and alert states from EEG signals with deep learning, in Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP) (Cancun, Mexico, 2015), pp. 493–496. Crossref, Google Scholar
21. Y. Wang and R. Qu , Research on driver fatigue state detection method based on deep learning, J. Phys.: Conf. Ser. 25 (2021) 644–647. Google Scholar
22. K. Yuen, S. Martin and M. M. Trivedi , Looking at faces in a vehicle: A deep CNN based approach and evaluation, in International Conference on Intelligent Transportation Systems (ITSC) (Rio de Janeiro, Brazil, 2016), pp. 649–654. Crossref, Google Scholar
23. Z. Zhao, N. Zhou, L. Zhang, H. Yan, Y. Xu and Z. Zhang , Driver fatigue detection based on convolutional neural networks using EM-CNN, Comput. Intell. Neurosci. 2020(1) (2020) 1–11. Google Scholar
24. L. F. Ibrahim, M. Abulkhair and A. D. AlShomrani , Using Haar classifiers to detect driver fatigue and provide alert, Multimed. Tools Appl. 71(3), 1857–1877 (2014). Crossref, Web of Science, Google Scholar
25. Y. Freund and R. E. Schapire , A decision-theoretic generalization of on-line learning and an application to boosting, J. Comput. Syst. Sci. 55(1), 119–139 (1997). Crossref, Web of Science, Google Scholar
26. V. Paul and J. J. Michael , Robust real-time face detection, Int. J. Comput. Vis. 57(2), 137–154 (2004). Crossref, Web of Science, Google Scholar
27. B. Mandal, Y. Li, G. Wang and L. Jie , Towards detection of bus driver fatigue based on robust visual analysis of eye state, IEEE Trans. Intell. Transp. Syst. 18(3), 545–557 (2017). Crossref, Web of Science, Google Scholar
28. V. Kazemi and J. Sullivan , One millisecond face alignment with an ensemble of regression trees, in Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, Columbus, OH, USA, 2014), pp. 1867–1874. Crossref, Google Scholar
29. F. S. C. Clement, A. Vashistha and E. R. Milind , Driver fatigue detection system, in Robotics and Biomimetics (RB) (IEEE, Pune, India, 2015), pp. 229–234. Crossref, Google Scholar
30. A. M. Tayyaba, J. Arfan and M. M. Anwar , Fully automated real time fatigue detection of drivers through Fuzzy Expert Systems, Appl. Soft Comput. 18, 25–38 (2014). Crossref, Web of Science, Google Scholar
31. X. Tang, P. Zhou and P. Wang , Real-time image-based driver fatigue detection and monitoring system for monitoring driver vigilance, in Proceedings of the 35th Chinese Control Conference (CCC) (IEEE, Chengdu, China, 2016), pp. 4188–4193. Crossref, Google Scholar