NSF CAREER Project – MINDWATCH
Acknowledgment: This material is based upon work supported by the National Science Foundation under Grant Numbers 1942585/2226123.Disclaimer: Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
- Project Overview
Smartwatch-like wearables have enabled seamless tracking of vital signs and physical activities, but still lack a significant feature: they are currently unable to provide any information about brain states or to modulate brain function for optimizing human health and performance. This project aims to make it possible for wearables to feature such capabilities. Being aware of brain states is not only extremely valuable in clinical studies but is also crucial to improving human performance in various everyday life activities. While recording neural signals directly from the scalp region is possible, it is impractical for use in everyday life. In order to fill this gap, the goal of this project is to pioneer a closed-loop brain-aware wearable architecture called MINDWATCH. This enables (1) decoding multidimensional brain states from noninvasive wearable devices and (2) applying corrective control. MINDWATCH will transform healthcare delivery (e.g., aging, autism, dementia) as well as human performance and productivity enhancement (e.g., online learning, smart workplaces). For instance, knowledge of mental health and cognitive engagement can enable detecting if a student is depressed or is not cognitively engaged/learning, which makes it possible to take corrective action early on.
This project seeks to overcome the barriers to achieving brain-aware wearables by pioneering a transformative system-theoretic computational toolset for noninvasive closed-loop wearable architectures that monitor and modulate brain function without needing neural recordings. The proposed framework will (1) infer discrete brain-related events in real-world settings, (2) decode multidimensional latent neurobehavioral states based on inferred brain activity, and (3) apply robust adaptive control to maintain the neurobehavioral states within desired ranges. The closed-loop framework will be rigorously validated using experiments on interactive human-technology environments and mental health.
– E-Books:
[1] McConnell, G.C., Santaniello, S., Gale, J.T., Faghih, R.T., Kemere, C., Han, M., Hilliard, J.D.,“Towards the Next Generation of Deep Brain Stimulation Therapies: Technological Advancements, Computational Methods, and New Targets“, Frontiers in Neuroscience, 2021.
Journal Articles:
[16] Wickramasuriya, D.S., Khazaei, S., Kiani, R. and Faghih, R.T., 2023. A Bayesian Filtering Approach for Tracking Sympathetic Arousal and Cortisol-Related Energy From Marked Point Process and Continuous-Valued Observations. IEEE Access, 11, pp.137204-137247.
[15] Alam, S., Amin, M.R. and Faghih, R.T., 2023. Sparse Multichannel Decomposition of Electrodermal Activity With Physiological Priors. IEEE Open Journal of Engineering in Medicine and Biology, 4, pp.234-250.
[14] Fekri Azgomi, H., F. Branco, L.R., Amin, M.R., Khazaei, S. and Faghih, R.T., 2023. Regulation of brain cognitive states through auditory, gustatory, and olfactory stimulation with wearable monitoring. Scientific reports, 13(1), p.12399.
[13] Amin, M.R. and Faghih R.T., “Physiological characterization of electrodermal activity enables scalable near real-time autonomic nervous system activation inference” PLOS Computational Biology, 2022.
[12] Branco LRF, Ehteshami A, Fekri Azgomi H, and Faghih R.T., “Closed-Loop Tracking and Regulation of Emotional Valence State From Facial Electromyogram Measurements“. Frontiers in computational neuroscience, 2022.
[11] Amin, M.R, Pednekar D, Fekri Azgomi H, Van Wietmarschen H, Aschbacher K, and Faghih, R.T., “Sparse System Identification of Leptin Dynamics in Women With Obesity“. Frontiers in endocrinology, 2022.
[10] Fekri Azgomi H, and Faghih R.T., “Enhancement of Closed-Loop Cognitive Stress Regulation Using Supervised Control Architectures” IEEE open journal of engineering in medicine and biology 2022. (Supplementary Info).
[9] Steel A. G, Parekh S, Fekri Azgomi H, Badri Ahmadi M, Craik A, Pati S, Francis J. T.Contreras-Vidal J.L., and Faghih R.T., “A Mixed Filtering Approach for Real-Time Seizure State Tracking Using Multi-Channel Electroencephalography Data” IEEE Transactions on Neural Systems and Rehabilitation Engineering 2021.(Supplementary Info).
[8] Fekri Azgomi H, Hahn J-O, and Faghih R.T., “Closed-Loop Fuzzy Energy Regulation in Patients With Hypercortisolism via Inhibitory and Excitatory Intermittent Actuation,” Frontiers in Neuroscience,2021. (Supplementary Info).
[7] Fekri Azgomi H, Cajgas I, and Faghih R.T., “Closed-Loop Cognitive Stress Regulation Using Fuzzy Control in Wearable-Machine Interface Architectures,” IEEE Access, 2021.
[6] Casile A, Faghih R.T., Brown E. N., “Robust point-process Granger causality analysis in presence of exogenous temporal modulations and trial-by-trial variability in spike trains,” PLoS computational biology, 2021.
[5] Wickramasuriya D.S., and Faghih R.T., “A Marked Point Process Filtering Approach for Tracking Sympathetic Arousal from Skin Conductance” IEEE Access, 2020.
[4] Amin, M.R. and Faghih, R.T., 2020. “Identification of Sympathetic Nervous System Activation from Skin Conductance: A Sparse Decomposition Approach with Physiological Priors“. IEEE Transactions on Biomedical Engineering, 2020. (Supplementary Info).
[3] Genty J. X., Amin M.R., Shaw N. D., Klerman E. B., and Faghih R.T., “Sparse Deconvolution of Pulsatile Growth Hormone Secretion in Children,” IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2021. (Supplementary Info).
[2] Pednekar D, Amin M.R., Fekri Azgomi H, Aschbacher K, J. Crofford L, and Faghih R.T., “Characterization of Cortisol Dysregulation in Fibromyalgia and Chronic Fatigue Syndromes: A State-Space Approach” IEEE Transactions on Biomedical Engineering, 2020. (supplementary info).
[1] Wickramasuriya, D.S., and Faghih R.T., “A Mixed Filter Algorithm for Sympathetic Arousal Tracking from Skin Conductance and Heart Rate Measurements in Pavlovian Fear Conditioning,” PLOS ONE, 2020.
– Conference Proceedings:
[10] Raju, V., Gibbison, B., Hajihossainlou, B., Klerman, E.B. and Faghih, R.T., 2023, July. Sparse Deconvolution and Causality Analysis of Inflammatory Markers During Cardiac Surgery. In 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (pp. 1-7). IEEE.
[9] Raju, V., Gibbison, B., Klerman, E.B. and Faghih, R.T., 2023, July. Characterizing Alterations in Cortisol Secretion During Cardiac Surgery. In 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (pp. 1-6). IEEE.
[8] Reddy, R., Khazaei, S. and Faghih, R.T., 2023, July. A Point-Process Approach for Tracking Valence using a Respiration Belt. In 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (pp. 1-7). IEEE.
[7] Reddy, R., Guo, Y., Raju, V. and Faghih, R.T., 2023, July. Characterization of Leptin Secretion in Premenopausal Obese Women Treated with Bromocriptine. In 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (pp. 1-6). IEEE.
[6] Khazaei, S., Amin, M.R. and Faghih, R.T., 2022, October. Decoding a Neurofeedback-Modulated Performance State in Presence of a Time-Varying Process Noise Variance. In 2022 56th Asilomar Conference on Signals, Systems, and Computers (pp. 990-996). IEEE.
[5] Wickramasuriya D.S., Amin M.R., and Faghih R.T., “A Wearable Exam Stress Dataset for Predicting Grades using Physiological Signals“. 2022 IEEE Healthcare Innovations and Point of Care Technologies (HI-POCT) 2022.
[4] Khazaei, S., Amin, M.R, and Faghih, R.T., “Decoding a Neurofeedback-Modulated Cognitive Arousal State to Investigate Performance Regulation by the Yerkes-Dodson Law.” The Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2021
[3] Amin, M.R, Tahir, M., and Faghih, R.T.,”A State-Space Investigation of Impact of Music on Cognitive Performance during a Working Memory Experiment.“The Annual International Conference of the IEEE Engineering in Medicine and Biology Society,2021.
[2] Yaghmour, A., Amin, M.R, and Faghih, R.T., “Decoding a Music-Modulated Cognitive Arousal State using Electrodermal Activity and Functional Near-infrared Spectroscopy Measurements.“The Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2021.
[1] Seet M. S., Amin M. R., Abbasi N. I., Hamano J., Bezerianos A., Faghih R.T., Thakor N. V., Dragomir A., “Olfactory-induced Positive Affect and Autonomous Response as a Function of Hedonic and Intensity Attributes of Fragrances,” The Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2020.
- Selected Media Attention
CML graduate students and Rose published a research article outlining the influence of everyday activities such as listening to music, drinking coffee and use of perfume in Scientific Reports. This work is a key demonstration of utilizing commonly available interventions to regulate cognitive states such as performance and arousal. The first author is Rose’s former Ph.D. student Hamid Fekri Azgomi who is now a postdoctoral scholar in neurological surgery at the University of California San Francisco School of Medicine. This work has received attention from several international media outlets around the world including: Yahoo News, EurekAlert!, The Independent, The Mirage, News Medical Life Science, SciTechDaily, Science Daily, and Medical Xpress. Read the Scientific Reports article
A new dataset, “A Wearable Exam Stress Dataset for Predicting Cognitive Performance in Real-World Settings,” comprising physiological signals measured using wearables to estimate stress was generated by CML. Research based on this dataset has been published in IEEE Healthcare Innovations in Point of Care Technology (HIPOCT) and can be accessed here.
CML research work titled “Enhancement of closed-loop cognitive stress regulation using supervised control architectures“ in the IEEE Open Journal of Engineering in Medicine and Biology. This work was covered by Medical Xpress in an article which reads: “Feeling overwhelmed, anxious and agitated are among the symptoms associated with high levels of cognitive stress…” Read the article on Medical Xpress.
CML research work on the estimation of emotional valence from facial electromyogram measurements in Frontiers in Neuroscience. This work was completed as part of a course project for the graduate course titled “State-Space Estimation with Physiological Applications.”
The work was also featured in a research brief by NYU Tandon School of Engineering. Read more in NYU News
CML paper on “Sparse system identification of leptin dynamics in women with obesity” has been described by Medical Xpress and labroots. The paper studied the differences in leptin hormone dynamics before and after treatment in obese women, and the causality relationship between leptin and the stress hormone, cortisol. Read in UH News.
CML Proposed algorithm in PLOS Computational Biology has received a vast media attention and has been outlined by Yahoo news , MSN, Gadgets and Wearables, Global Health News Wire, News Azi, The Mirage, News Medical Life Science, Medical Republic, and Medical Xpress.
Paper from CML outlines new Approach for Management of Cushing’s disease
New Smartwatch Technology Uses Sweat on Skin to Infer Brain Stress
ECE students Fekri Azgomi, Amin among 21 students presenting at national meeting
Faghih tabbed for 2020-21 Interstellar Initiative
Computational Medicine Lab’s Research Featured by IEEE Xplore
UH’s Faghih Named an MIT Technology Review 2020 Innovator Under 35
College honors 17 with yearly Faculty and Student Excellence Awards
Faghih featured in IEEE publication as ‘Woman to Watch’
Continuously Tracking Fear Response Could Improve Mental Health Treatment
Students provide videos to public modeling spread of COVID-19
Adapting Smartwatches to Improve Distance Learning and Health
Intellectual Merit
Publications: Journal papers: 17, Peer-reviewed conference papers: 7; Pending Patent: 1; Provisional Patents: 2
– Integration of research into a graduate course resulted in 5 of these peer-reviewed publications (2 conference & 3 journal).
– Created 22 educational videos and 13 open-source code repositories
– Released a publicly available toolbox for Bayesian filtering analysis (8 classes of filters) on GitHub
– Collected and shared Wearable Exam Stress and Regulation of Cognitive States Datasets on PhysioNet, a repository of freely-available medical research data.
– 2 PhD and 1 Masters’ students currently under supervision. 3 PhD students & 2 Masters’ students graduated, and 6 Undergraduate projects (Honors’ Thesis, REU, Summer Undergraduate Research Fellowship) were completed under PI’s supervision. 3 Capstone teams designed wearables for emotion monitoring.
– PI gave several laymen talks about this research to general audience & broader community
– PI participated in outreach to the next generation of engineers at the International Summer School on Bio X: Data Science and Engineering in Medicine and Biology
– PI was featured as a panelist at the 2023 IEEE EMBS Inaugural Women in Biomedical Engineering Forum
– Computational Medicine Lab Won First Place in the 2023 NYU Tandon Research Excellence Exhibit for the MINDWATCH Project
– PI was recognized as MIT Technology Review 2020 Innovator under 35 (FuturoProssimo has predicted that MINDWATCH research has a high potential for a Nobel prize)
Patents
– Faghih, R. T., E. N. Brown, and A. K. Styer. System and method for automated ovarian follicular monitoring, 2023. US Patent 11,622,744.
– Faghih, R.T., Wickramasuriya, D.S. and Amin, M.R., University of Houston System, 2022. Systems and methods for estimating a nervous system state based on measurement of a physiological condition. U.S. Patent Application 17/514,129.
Filed Provisional Patents
– Faghih, R. T., R. Reddy, and S. Khazaei. A system and method for estimating emotional valence based on measurements of respiration, 2023. US Patent App 63/514,825.
– Faghih, R. T. and S. Khazaei. Systems and methods for estimating interoceptive awareness state using eye-tracking measurements, 2023. US Patent App 63/584,814.
Software
Emotion Recognition from Heartbeat Dynamics
Educational and Outreach Activities
Girls Engineering the Future Outreach Event (2021) – (Video Link)
Educational VideosState-Space Modeling and control of power systems – Video by Thejas Rajan (Spring 2020)
CartPloe LQR Control Analysis – Video by Pengyu (Ben) Yuan (Spring 2020)
An Application of Optimal Control in EM – Video by Neil Jerome A. Egarguin (Summer 2020)
Matrix Algebra and Linear Equations – Video by Prasanna Kumar Reddy Gade (Fall 2018)
Video on Stability by Nitin Bhabsar (Fall 2018)
Gilbert Realization and Pole Zero Cancellation – Video by Divesh Pednekar
Reference Tracking and McMillan Degree – Video by Aruj Khandelwal
Web Outreach
Computational Medicine Lab’s Youtube Channel
K-12 and College Level Mentor on Engineer Girl, a website by the National Academy of Engineering.
- 71% of US workers reported feeling lonely or isolated in the wake of the COVID-19 pandemic
- 32% of US adults and 50% of young adults report symptoms of anxiety or depression
- 88% of US physicians want patients to monitor their health at home
- 35% of US employers use medical wearable technology to facilitate wellness programs