Studying Doze-Off in student using electroencephalography system

Science & Technology Development Journal – Engineering and Technology, 3(SI):xxx-xxx Open Access Full Text Article Research Article Faculty of Applied Science, Ho Chi Minh University of Technology, VNU-HCM, Vietnam Correspondence Le Q. Khai, Faculty of Applied Science, Ho Chi Minh University of Technology, VNU-HCM, Vietnam Email: quockhai@hcmut.edu.vn History  Received: 10-02-2020  Accepted: 24-11-2020  Published: xx-11-2020 DOI : Copyright © VNU-HCM Press. This is an open- access article distributed under the terms of the Creative Commons Attribution 4.0 International license. Studying DOZE-OFF in student using ELECTROENCEPHALOGRAPHY system Le Q. Khai*, Pham T. H. Trang, Nguyen T. Hieu, Huynh T. D. Thy, Huynh Q. Linh Use your smartphone to scan this QR code and download this article ABSTRACT Sleep deprivation of high school and university students is currently an actual concerning issue. Sleep deprivation is one of the leading causes of dozing-off during daytime. Usually, the state of drowsiness is very little concerned, but studies on drowsiness show the importance of investigat- ing the frequency of occurrence as well as the need to clarify the cause and propose limited mea- sures appeared dozing-off. Dozing-off is not only an undesirable state and disrupts daily activities, but also provides information on personal health status. In that case, early alertness for dozing-off event is very helpful in preventing unwanted consequences. The study has designed a process to record dozing-off event, then constructed and implemented the hypnogram processing program that evaluated quantitative changes in polysomnography signals at sleep onset, the transition time from wake stage to sleep stage. By analyzing the energy spectrum of the signal and using wavelet transform in combine with the support vector machine algorithm, the research allows a compre- hensive evaluation of the state of dozing-off. Determining the exact time of onset of sleep is very important in the study of drowsiness. Extracting the time of this event appears to help develop an application for early warning dozing-off. Besides, it allows making an initial assessment of the condition of the subject when the time of drowsiness begins suddenly. Six recordings from vol- unteered students were processed from a 45-minute vigilance test. All of the volunteers had no neuropathy and were well explained to the procedures used in this study. The results show that depending on kinds of different applications, signals such as EEG, EOG, or EMG are used individually or in combination to fetch suitable results. The result presented a successful method to distinguish dozing-off event with other stages. Key words: dozing-off, sleep onset, polysomnography, support vector machine INTRODUCTION1 OZING-off can be observed in many workplaces,2 public spots and especially in educational environ-3 ment. This phenomenon causes many unwanted con-4 sequences and becomes an actual concerning issue5 since it leads to serious occupational accidents and re-6 duces life quality. For students, dozing-off causes dis-7 traction from studying which reduces their produc-8 tivity.9 Dozing-off can be a symptom of many severe illness10 and disorders such as narcolepsy or sleep depriva-11 tion1. Researchers also found that people who fre-12 quently doze-off has the hazard ratio for getting stroke13 2.6 times higher than normal 2. Due to its harmful-14 ness, early alertness is necessary in order to prevent15 people from accidents and keep their work on track.16 Also, a full record of dozing-off events can be useful17 for further diagnostic and therapy.18 Reasons for daytime unwanted sleep can be listed as19 a side effect of medicine or fatigue due to overwork.20 Above all, lacking of sleep is the most mentioned rea-21 sonwhen talking about dozing-off. This is also the fre- 22 quent answer when asking students why they doze-off 23 during class or when they drive to school in themorn- 24 ing3. 25 Many researches were conducted to detect this unde- 26 sirable sleeping condition using infrared camera an- 27 alyzing movements of the head and pupils. This de- 28 vice is attached to cars as an alarm for drivers and 29 is now widely used thanks to its noninvasive func- 30 tion. However, there are some limitations in this 31 device. As a peripheral equipment, it is affected by 32 noise from outside environment which leads to er- 33 rors in detection4. In addition to this, sleeping con- 34 dition is controlled by the brain. Then, a detection 35 system based on brain signals will work better at dis- 36 tinguish the stages. In that case, 3 special charac- 37 ters usually used for classifying sleep stages are Elec- 38 troencephalogram (EEG), Electromyogram (EMG) 39 and EOG (Electrooculogram) signals which are ac- 40 quired from the Electroencephalography system. 41 Dozing-off event can be detected by determining the 42 transition time between wake and sleep stages (or 43 Cite this article : Khai LQ, TrangPTH,HieuNT, ThyHTD, LinhHQ.StudyingDOZE-OFF in studentusing ELECTROENCEPHALOGRAPHY system. Sci. Tech. Dev. J. – Engineering and Technology; 3(SI):xxx-xxx. 1 Un c rre cti on pr o f Science & Technology Development Journal – Engineering and Technology, 3(SI):xxx-xxx the sleep on-set condition) when a subject is doing44 some particular actions. To have a better result of the45 dozing-off records, a precise measurement for transi-46 tion time is necessary. This work can be best achieved47 by the convergence of a number of indices5. In this48 project 3 indices were data collected from 3 channels49 of EEG, EMG and EOG signals.50 When the transition period happens between wake51 and sleep stages, muscle tone presents a reduction in52 amplitude. Normally, in relaxation period, EEG dis-53 plays a predominant of alpha activity (8 to 13Hz).54 When sleep enters, the theta activity (4 to 7 Hz) re-55 places and usually accounts for 50% of epoch dura-56 tion. For the EOG performance, during wake stage,57 the channels in both eyes show eye blinking and rapid58 eye movements. As the patient become extremely59 drowsy, slow eye movement appears, and sleep may60 come right after. That is why the presence of slow eye61 movement is considered to be a proof for sleep on-62 set. When combining with other characters, the fea-63 ture increases the confidence in recognizing the sleep64 on-set condition5.65 However, when the body relaxes, muscle tone also66 scales down as well as in the transition time between67 REM andNREM sleeps. Because of that reason, EMG68 signal itself cannot be used alone as an indicator for69 sleep on-set scoring. Similarly for EEG signal, the pre-70 dominant replacement may take a long period, some-71 times up to a couple of minutes for the phenomenon72 to be seen clearly. For those alarming applications,73 using EEG channel for detection dozing-off will not74 be sufficient due to its latency. Slow eye movement-75 likes other discriminators – is not always precisely re-76 lated to sleep on-set, however, its appearance is a reli-77 able sign that sleep on-set has happened. In that case,78 analyzing all three 3 channels to define the transition79 event is a need.80 MATERIALS ANDMETHODS81 This research applies theDeclaration ofHelsinki prin-82 ciples in human studies.83 Experiment apparatus84 All the experimental process was conducted in85 Biomedical Laboratory 204B4 at Ho Chi Minh Uni-86 versity of Technology. The recording room was de-87 signed based on characteristics of Faraday cage in or-88 der to reduce the effect of 50Hz electrical noise. The89 data was recorded using NicoletOne V32 device (Na-90 tus Neurology Incorporated, WI, USA) providing 3291 channels with the ability to amplify the original signal92 and filter noise at 50Hz in order to gain high quality93 signal. The device is specially designed for sleep anal- 94 ysis and is widely used in many neuroscience labora- 95 tories around the world. 96 Signal gained from NicoletOne device is extracted 97 into digital signal and processed using Matlab soft- 98 ware (The Mathworks, Natick, MA, USA). 99 Technique 100 Data acquisition 101 Six recordings were collected from volunteered stu- 102 dents age ranged from 20 to 25 years old without hav- 103 ing neuropathy. Since dozing-off in students happen 104 mostly due to lacking of sleep, all subjects participated 105 in the experiment were asked to reduce the amount of 106 sleep the night before and not to use any alcohol or 107 caffeine before recording. The experiment executed 108 after lunch time 30 to 60 minutes. Before recording, 109 all the objectives and recording process were carefully 110 instructed to subjects. 111 The data acquisition started with 10-minute period 112 calibration when subject seated, kept their eyes open 113 and relaxed for 5 minutes and thereafter sat with their 114 eyes close for the remaining 5 minutes. The recording 115 in this period was used as training data for later classi- 116 fying process. During the subsequent 35-minute vig- 117 ilance test, subject stayed on the same sitting position 118 in front of a screen and a video camera. The task re- 119 quired subjects to watch the display and press a time- 120 lapse button whenever they saw a signal (a white 3x1 121 centimeters arrow) appeared randomly on the screen. 122 The signal showed up 2 to 4 times per minute and was 123 visible during 4 seconds at each exposure. Any omis- 124 sion of the signal was noticed as a sign of dozing-off 125 event. Data recorded in this periodwas used as testing 126 data. 127 Data analysis 128 Each signal index required different parameter to ex- 129 amine. The signal processing procedure is shown in 130 Figure 1. 131 For EEG signal, the predominance of brain waves 132 was calculated based on the Ratio of Power Spectrum 133 (RPS) presented as following formula: 134 RPSx = Px SP (1) While X are theta and alpha. 135 In EMG signal, the reduction in muscle tone was ex- 136 amined by changes in mean square amplitude and 137 identified as: 138 ______ A2EMG = åNi=1A 2 i N (2) 2 Un co rr ct o pr o f Science & Technology Development Journal – Engineering and Technology, 3(SI):xxx-xxx While N: number of data points in each epoch139 Ai : amplitude of each data point (mV)140 EOG channel required a more complicated process to141 identify slow eye movement feature. By extraction142 method as executed in previous work6, epoch con-143 taining SEM was classified automatically under the144 support of Support Vector Machines algorithm 7.145 First of all, the efficiency of SEM recognition algo-146 rithmwas evaluated in order to ensure EOG signal an-147 alyzing result contribute to the determination of sleep148 on-set.149 After classifying using Support VectorMachines algo-150 rithm, epochs with or without SEM appearance were151 then confronted with the categorized result for each152 epoch conducted by observation in EEG Viewer soft-153 ware. Epochs were arranged in 4 groups: TP (True154 Positive) for epochs that were both calculated and155 observed with SEM; TN (True Negative) for epochs156 that were both calculated and observed without SEM157 (non-SEM or NSEM); FP (False Positive) for epochs158 observed as NSEM but got calculated result as SEM;159 and FN (False Negative) for epochs observed as SEM160 but calculated result was NSEM.161 Figure 1: The whole data analysis process was con- ducted orderly as the following. The effectiveness of SVM classification algorithm is162 evaluated based on the following parameters:163 Accuracy= TP+TN total number o f epochs 100 (3) Accuracy gave the general estimation for classifying 164 algorithm. 165 Sensitivity= TP TP+FN 100 (4) Sensitivity showed the ability to classify epochs with 166 SEM. 167 Speci f icity= TN TN+FP 100 (5) Specificity showed the ability to recognize epochs 168 without SEM. 169 Ideally, all three parameters needed to be high to 170 prove the effectiveness of the dozing-off event recog- 171 nition algorithm. There were two parameters that 172 needed to be noticed: sensitivity and specificity. High 173 sensitivity indicated the optimal effect of SEM recog- 174 nition algorithm. However, in a successive epoch 175 sequence with SEM, some epoch might be missed 176 and would not change its contribution in detecting 177 dozing-off event but few seconds delay of recognizing 178 the event. Specificity identified NSEM epoch. If the 179 parameter was high, it meant that the errors in iden- 180 tifying the time of sleep on-set due to false detection 181 of slow eye movement was minimized. 182 RESULTS ANDDISCUSSION 183 Epoch classification results withNSEMand SEMwere 184 listed in Table 1 based on the mentioned parameters: 185 accuracy, sensitivity, specificity. 186 Table 1: Estimation of SEM classifying algorithm in EOG channel. Sample number Accuracy (%) Sensitivity (%) Specificity (%) 1 93.43 66.67 94.61 2 96.74 66.67 98.84 3 93.70 76.47 94.86 4 96.43 54.55 98.14 5 82.09 62.27 89.19 6 91.82 18.75 97.55 According to the results table, the reason of the per- 187 centage of sensitivity quite low compared to the per- 188 centage of accuracy and specificity is that SEM does 189 not occur alone but rather in a series of repetitive cy- 190 cles. If the algorithm did not identify all SEMs cor- 191 rectly but correctly identified the first SEMs, this study 192 could identify the time of drowsiness. So as a result, 193 we did not add additional training steps to increase 194 the percentage of sensitivity. 195 3 Un co rre cti on pr oo f Science & Technology Development Journal – Engineering and Technology, 3(SI):xxx-xxx From the analysis of samples 2 and 3, it is not possible196 to use only the EEG signal to determine the state of197 sleep due to abnormalities occurring in brain waves.198 Besides, it is not correct to consider chin EMG alone,199 because this factor has a variable amplitude even in200 the waking period (sample 3). In the EOG signal,201 SEM appears in both phases: drowsiness andN1 stage202 of sleep. Because of these inadequacies, it is necessary203 to combine many factors to get more accurate and ap-204 propriate results in the case of wanting to determine205 the time of the subject’s drowsiness events to moni-206 toring or treatment.207 In addition, the time of drowsiness will be detected208 more slowly due to continuously changing brain wave209 signals, whichmakes the state of drowsiness and alert-210 ness interwoven in experiments of samples 1 and 4. It211 also means that with doze-off alert devices, it is more212 effective to analyze EMG and EOG signals before in-213 vestigating the EEG signal when the object is work-214 ing continuously and having risk of drowsiness. This215 is very logical because when you fall into a state of216 muscle tone loss and eye is no longer under control,217 the accident may occur. Therefore, through the above218 results, depending on the purpose of the application,219 consideration can be given to selecting specific signals220 for analysis and identification.221 With the results of the study, it is possible to open a222 wide range of applications: an alert device for drowsi-223 ness or a warning of the imminent drowsiness trend.224 When early warning is required, priority can be given225 to EMG and EOG signals. However, when it is nec-226 essary to accurately and quantitatively identify trends227 and the time when drowsiness occurs, an EEG signal228 analysis can be used.229 The suspected dozing-off events were re-checked with230 the video recorded from camera to determine the real231 events. The events detected by algorithm were then232 demonstrated in the same time axis with real events233 to evaluate the accuracy of the method. There was234 a total of 11 dozing-off events happened in 6 record-235 ings and 9 events were recognized by algorithms with236 mean delay time was approximately 50 seconds. Max-237 imum delay time was 2 minutes 20 seconds caused by238 the steady changes in EEG signal (Figure 2).239 Reverse event was observed in a female subject. Dur-240 ing the time sleep occurred, the alpha wave took the241 predominance instead of theta wave (Figure 3). This242 was an obvious evidence that EEG signal alone could243 not use to efficiently detect the phenomenon.244 CONCLUSIONS245 As can be seen in the result, recording of brain waves246 only could not be used to determine the state of sleep247 due to an abnormality might occur in the subject’s 248 brainwaves. Also, it is not possible to consider only 249 the muscle tone of the chin because this factor has a 250 variable amplitude. In EOG signal, slow eye move- 251 ment, as mentioned, appeared in both stages: drowsi- 252 ness and the N1 stage of sleep. Because of the above 253 inadequacies, the needs of combining all the factors to 254 bringmore accurate and optimal results when record- 255 ing dozing-off events for the purpose of diagnosis and 256 treatment were huge. 257 Additionally, the phenomenon was more difficult to 258 be detected if the state of sleeping and waking mixed 259 together. When the subject sleepily did a particular 260 action during the experiment, it led to mixing signals 261 in brainwaves. This might force an exclusion of EEG 262 signal and considering EOG and EMG signals instead 263 for amore accurate notification in dozing-offwarning 264 device. This was also reasonable since once the mus- 265 cles lost its ability to stress and eyes was uncontrol- 266 lable, accidents increased their chances to happen. 267 To conclude, depending on the purpose of the appli- 268 cation, consideration of the specific signals may be 269 taken into account for analysis and identification. 270 ACKNOWLEDGMENT 271 This research is funded by Vietnam National Uni- 272 versity Ho Chi Minh City (VNU-HCM) under grant 273 number C2020-20-10. 274 LIST OF ABBREVIATION 275 EEG: Electroencephalogram. 276 EMG: electromyogram. 277 EOG: electrooculogram. 278 SEM: Slow Eye Movement. 279 REM: Rapid Eye Movement. 280 NREM: Non-Rapid Eye Movement. 281 NSEM: Non- Slow Eye Movement. 282 AUTHOR S’ CONTRIBUTIONS 283 All authors contributed equally to this work. All au- 284 thors have read and agreed to the published version of 285 the manuscript. 286 CONFLICT OF INTEREST 287 We declare that there is no conflict of whatsoever in- 288 volved in publishing this research. 289 REFERENCES 290 1. Chokroverty S. Overview of sleep & Sleep disorders. Indian J 291 Med Res. 2010;131(2):126–140. 292 2. Susman E. Daytime naps maybe a risk factor for a stroke. Neu- 293 rology Today. 2008;p. 13–15. Available from: https://doi.org/10. 294 1097/01.NT.0000319946.26513.47. 295 3. GreenbergMBG. New Study Finds One in Three Teens Are Driv- 296 ing While Drowsy. Liberty Mutual Insurance, Boston. 2016;. 297 4 Un o r ec tio n p r o f Science & Technology Development Journal – Engineering and Technology, 3(SI):xxx-xxx Figure 2: Data analyzing result in EOG, EEG and EMG signals respectively along with dozing-off events marked by algorithm and real events. At the time when sleep on-set occurred, there were the appearance of SEM and reduction in muscle tone. However, the predominance of alpha wave replacing theta wave came after quite a period of time leading to delay in dozing-off recognition. Figure 3: Data analyzing result in EOG, EEG and EMG signals respectively along with dozing-off events marked by algorithm and real events. During sleeping period, subject showed a reverse phenomenon comparing to normal: alpha wave accounted for most of the time while theta wave had less predominance. This event reduplicated when doing another attempt to the same subject with similar conditions. 4. Dinh TQ, QuangND. Hệ thống phát hiện tình trạng ngủ gật của298 lái xe. Tạp chí Khoa học Trường Đại học Cần Thơ, Số chuyên đề:299 Công nghệ Thông tin (2015). 2015;p. 160–167.300 5. Ogilvie RD, McDonagh DM, Stone SN, Wilkinson RT. Eye Move-301 ments and the Detection of Sleep Onset. Psychophysiology.302 1988;25(1):81–91. PMID: 3353488. Available from: https://doi.303 org/10.1111/j.1469-8986.1988.tb00963.x.304 6. Trang PTH., Hieu NT, Khai LQ, Linh HQ. Detection of Slow Eye305 Movement indozing-offeventusingmorphological andNeural306 Network Method. SEATUC Symposium 2019, Ha Noi, Vietnam.307 2019;.308 7. ShinD, Sakai H, UchiyamaY. Slow eyemovement detection can 309 prevent sleep-related accidents effectively in a simulated driv- 310 ing task. Journal of Sleep Research. 2010;20(3):416–424. PMID: 311 21070424. Available from: https://doi.org/10.1111/j.1365-2869. 312 2010.00891.x. 313 5 Un co rre cti on pr oo f