NSF CAREER: MINDWATCH

NSF CAREER Project – MINDWATCH

Acknowledgment: This material is based upon work supported by the National Science Foundation under Grant Numbers 1942585/2226123.

Disclaimer: Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

Project Overview

Smartwatch-like wearables have enabled seamless tracking of vital signs and physical activities, but still lack a significant feature: they are currently unable to provide any information about brain states or to modulate brain function for optimizing human health and performance. This project aims to make it possible for wearables to feature such capabilities. Being aware of brain states is not only extremely valuable in clinical studies but is also crucial to improving human performance in various everyday life activities. While recording neural signals directly from the scalp region is possible, it is impractical for use in everyday life. In order to fill this gap, the goal of this project is to pioneer a closed-loop brain-aware wearable architecture called MINDWATCH. This enables (1) decoding multidimensional brain states from noninvasive wearable devices and (2) applying corrective control. MINDWATCH will transform healthcare delivery (e.g., aging, autism, dementia) as well as human performance and productivity enhancement (e.g., online learning, smart workplaces). For instance, knowledge of mental health and cognitive engagement can enable detecting if a student is depressed or is not cognitively engaged/learning, which makes it possible to take corrective action early on. The research is integrated with educational activities with an emphasis on increasing the participation of students in science and engineering. These activities include hosting hands-on STEM K12 events, supervising undergraduate research interns and capstone senior design projects, creating educational videos, and interdisciplinary course development.

This project seeks to overcome the barriers to achieving brain-aware wearables by pioneering a transformative system-theoretic computational toolset for noninvasive closed-loop wearable architectures that monitor and modulate brain function without needing neural recordings. The proposed framework will (1) infer discrete brain-related events in real-world settings, (2) decode multidimensional latent neurobehavioral states based on inferred brain activity, and (3) apply robust adaptive control to maintain the neurobehavioral states within desired ranges. The closed-loop framework will be rigorously validated using experiments on interactive human-technology environments and mental health. The proposed research will provide foundational statistical signal processing and control-theoretic tools for the analysis of multimodal binary and continuous physiological observations that occur on multiple time-scales. While the initial focus is on two aspects of brain function, namely mental health and cognitive engagement, the proposed framework opens up the opportunity to investigate broader questions in computational neuroscience.

Effects of Music and Coffee on Cognitive States Demo

MINDWATCH Demonstration Video

  • MINDWATCH Publications

– E-Books:

[1] McConnell, G.C., Santaniello, S., Gale, J.T., Faghih, R.T., Kemere, C., Han, M., Hilliard, J.D.,“Towards the Next Generation of Deep Brain Stimulation Therapies: Technological Advancements, Computational Methods, and New Targets“, Frontiers in Neuroscience, 2021.

– Journal Articles:

[23] Reddy, R. and Faghih, R.T., “A Marked Point Process Approach for Continuous Valence Estimation using Respiration Activity.“ IEEE Access, vol. 13, pp. 4067-4080, 2025.

[22] Khazaei, S. and Faghih, R.T., “Decoding a Cognitive Performance State from Behavioral Data in the Presence of Auditory Stimuli.“ IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 32, pp. 4270-4283, 2024.

[21] Khazaei, S. and Faghih, R.T., “Eye tracking is more sensitive than skin conductance response in detecting mild environmental stimuli.“ PNAS nexus, 3(9), p.pgae370, 2024.

[20] Alam, S., Khazaei, S., and Faghih, R.T., “Unveiling productivity: The interplay of cognitive arousal and expressive typing in remote work.“ PLoS one, 19(5), p.e0300786, 2024.

[19] Amin, M.R., Alam, S., Khazaei, S., Azgomi, H.F., and Faghih, R.T., “Skin Conductance Response Artifact Reduction: Leveraging Accelerometer Noise Reference and Deep Breath Detection.“ IEEE Access, vol. 12, pp. 68208-68231, 2024.

[18] Khazaei, S., Amin, M.R., Tahir, M., and Faghih, R.T., “Bayesian Inference of Hidden Cognitive Performance and Arousal States in Presence of Music.“ IEEE Open Journal of Engineering in Medicine and Biology, vol. 5, pp. 627-636, 2024.

[17] Khazaei, S., Parshi, S., Alam, S., Amin, M.R., and Faghih, R.T., “A multimodal dataset for investigating working memory in presence of music: a pilot study.“ Frontiers in Neuroscience, 18, p.1406814, 2024.

[16] Wickramasuriya, D.S., Khazaei, S., Kiani, R. and Faghih, R.T., 2023. A Bayesian Filtering Approach for Tracking Sympathetic Arousal and Cortisol-Related Energy From Marked Point Process and Continuous-Valued Observations. IEEE Access, 11, pp.137204-137247.

[15] Alam, S., Amin, M.R. and Faghih, R.T., 2023. Sparse Multichannel Decomposition of Electrodermal Activity With Physiological Priors. IEEE Open Journal of Engineering in Medicine and Biology, 4, pp.234-250.

[14] Fekri Azgomi, H., F. Branco, L.R., Amin, M.R., Khazaei, S. and Faghih, R.T., 2023. Regulation of brain cognitive states through auditory, gustatory, and olfactory stimulation with wearable monitoring. Scientific reports, 13(1), p.12399.

[13] Amin, M.R. and Faghih R.T., “Physiological characterization of electrodermal activity enables scalable near real-time autonomic nervous system activation inference” PLOS Computational Biology, 2022.

[12] Branco LRF, Ehteshami A, Fekri Azgomi H, and Faghih R.T., “Closed-Loop Tracking and Regulation of Emotional Valence State From Facial Electromyogram Measurements“. Frontiers in computational neuroscience, 2022.

[11] Amin, M.R, Pednekar D, Fekri Azgomi H, Van Wietmarschen H, Aschbacher K, and Faghih, R.T., “Sparse System Identification of Leptin Dynamics in Women With Obesity“. Frontiers in endocrinology, 2022.

[10] Fekri Azgomi H, and Faghih R.T., “Enhancement of Closed-Loop Cognitive Stress Regulation Using Supervised Control Architectures” IEEE open journal of engineering in medicine and biology 2022. (Supplementary Info).

[9] Steel A. G, Parekh S, Fekri Azgomi H, Badri Ahmadi M, Craik A, Pati S, Francis J. T.Contreras-Vidal J.L., and Faghih R.T., “A Mixed Filtering Approach for Real-Time Seizure State Tracking Using Multi-Channel Electroencephalography Data” IEEE Transactions on Neural Systems and Rehabilitation Engineering 2021.(Supplementary Info).

[8] Fekri Azgomi H, Hahn J-O, and Faghih R.T., “Closed-Loop Fuzzy Energy Regulation in Patients With Hypercortisolism via Inhibitory and Excitatory Intermittent Actuation,” Frontiers in Neuroscience,2021. (Supplementary Info).

[7] Fekri Azgomi H, Cajgas I, and Faghih R.T., “Closed-Loop Cognitive Stress Regulation Using Fuzzy Control in Wearable-Machine Interface Architectures,” IEEE Access, 2021.

[6] Casile A, Faghih R.T., Brown E. N., “Robust point-process Granger causality analysis in presence of exogenous temporal modulations and trial-by-trial variability in spike trains,” PLoS computational biology, 2021.

[5] Wickramasuriya D.S., and Faghih R.T., “A Marked Point Process Filtering Approach for Tracking Sympathetic Arousal from Skin Conductance” IEEE Access, 2020.

[4] Amin, M.R. and Faghih, R.T., 2020. “Identification of Sympathetic Nervous System Activation from Skin Conductance: A Sparse Decomposition Approach with Physiological Priors“. IEEE Transactions on Biomedical Engineering, 2020. (Supplementary Info).

[3] Genty J. X., Amin M.R., Shaw N. D., Klerman E. B., and Faghih R.T., “Sparse Deconvolution of Pulsatile Growth Hormone Secretion in Children,” IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2021. (Supplementary Info).

[2] Pednekar D, Amin M.R., Fekri Azgomi H, Aschbacher K,  J. Crofford L, and Faghih R.T., “Characterization of Cortisol Dysregulation in Fibromyalgia and Chronic Fatigue Syndromes: A State-Space Approach” IEEE Transactions on Biomedical Engineering, 2020. (supplementary info).

[1] Wickramasuriya, D.S., and Faghih R.T., “A Mixed Filter Algorithm for Sympathetic Arousal Tracking from Skin Conductance and Heart Rate Measurements in Pavlovian Fear Conditioning,” PLOS ONE, 2020.

– Conference Proceedings:

[10] Raju, V., Gibbison, B., Hajihossainlou, B., Klerman, E.B. and Faghih, R.T., 2023, July. Sparse Deconvolution and Causality Analysis of Inflammatory Markers During Cardiac Surgery. In 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (pp. 1-7). IEEE.

[9] Raju, V., Gibbison, B., Klerman, E.B. and Faghih, R.T., 2023, July. Characterizing Alterations in Cortisol Secretion During Cardiac Surgery. In 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (pp. 1-6). IEEE.

[8] Reddy, R., Khazaei, S. and Faghih, R.T., 2023, July. A Point-Process Approach for Tracking Valence using a Respiration Belt. In 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (pp. 1-7). IEEE.

[7] Reddy, R., Guo, Y., Raju, V. and Faghih, R.T., 2023, July. Characterization of Leptin Secretion in Premenopausal Obese Women Treated with Bromocriptine. In 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (pp. 1-6). IEEE.

[6] Khazaei, S., Amin, M.R. and Faghih, R.T., 2022, October. Decoding a Neurofeedback-Modulated Performance State in Presence of a Time-Varying Process Noise Variance. In 2022 56th Asilomar Conference on Signals, Systems, and Computers (pp. 990-996). IEEE.

[5] Wickramasuriya D.S., Amin M.R., and Faghih R.T., “A Wearable Exam Stress Dataset for Predicting Grades using Physiological Signals“. 2022 IEEE Healthcare Innovations and Point of Care Technologies (HI-POCT) 2022.

[4] Khazaei, S., Amin, M.R, and Faghih, R.T., “Decoding a Neurofeedback-Modulated Cognitive Arousal State to Investigate Performance Regulation by the Yerkes-Dodson Law.” The Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2021

[3] Amin, M.R, Tahir, M., and Faghih, R.T.,”A State-Space Investigation of Impact of Music on Cognitive Performance during a Working Memory Experiment.“The Annual International Conference of the IEEE Engineering in Medicine and Biology Society,2021.

[2] Yaghmour, A., Amin, M.R, and Faghih, R.T., “Decoding a Music-Modulated Cognitive Arousal State using Electrodermal Activity and Functional Near-infrared Spectroscopy Measurements.“The Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2021.

[1] Seet M. S., Amin M. R., Abbasi N. I., Hamano J., Bezerianos A., Faghih R.T., Thakor N. V., Dragomir A., “Olfactory-induced Positive Affect and Autonomous Response as a Function of Hedonic and Intensity Attributes of Fragrances,” The Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2020.

  • Selected Media Attention

CML work on “Eye tracking is more sensitive than skin conductance response in detecting mild environmental stimuli” was published in PNAS Nexus. In this work, a framework for decoding one’s hidden interoceptive awareness state from eye tracking data was developed. 

Read more on Medical Xpress
Read more on qq.com
 Link to the article

CML graduate students and Rose published a research article outlining the influence of everyday activities such as listening to music, drinking coffee and use of perfume in Scientific Reports. This work is a key demonstration of utilizing commonly available interventions to regulate cognitive states such as performance and arousal.  The first author is Rose’s former Ph.D. student Hamid Fekri Azgomi who is now a postdoctoral scholar in neurological surgery at the University of California San Francisco School of Medicine. This work has received attention from  several international media outlets around the world including: Yahoo News, EurekAlert!, The Independent, The Mirage, News Medical Life Science, SciTechDaily, Science Daily, and Medical Xpress.   Read the Scientific Reports article

A new dataset, “A Wearable Exam Stress Dataset for Predicting Cognitive Performance in Real-World Settings,” comprising physiological signals measured using wearables to estimate stress was generated by CML. Research based on this dataset has been published in IEEE Healthcare Innovations in Point of Care Technology (HIPOCT) and can be accessed here.

Access dataset          Read more on ScienceDaily

CML research work titled “Enhancement of closed-loop cognitive stress regulation using supervised control architectures“ in the IEEE Open Journal of Engineering in Medicine and Biology. This work was covered by  Medical Xpress in an article which reads: “Feeling overwhelmed, anxious and agitated are among the symptoms associated with high levels of cognitive stress…” Read the article on Medical Xpress.

CML research work on the estimation of emotional valence from facial electromyogram measurements in Frontiers in Neuroscience. This work was completed as part of a course project for the graduate course titled “State-Space Estimation with Physiological Applications.” The work was also featured in a research brief by NYU Tandon School of Engineering.  Read more in NYU News

CML paper on obesity  has been described by Medical Xpress and labroots. The paper studied the differences in leptin hormone dynamics before and after bromocriptine treatment, and the causality relationship between leptin and the stress hormone, cortisol. Read in UH News.

CML Proposed algorithm in PLOS Computational Biology has received a vast media attention and has been outlined by Yahoo news , MSN, Gadgets and Wearables, Global Health News Wire, News Azi, The Mirage, News Medical Life Science, Medical Republic, and Medical Xpress.

Paper from CML outlines new Approach for Management of Cushing’s disease

New Smartwatch Technology Uses Sweat on Skin to Infer Brain Stress

ECE students Fekri Azgomi, Amin among 21 students presenting at national meeting

New research method from Faghih, Amin allows more reliable brain information inference using electrodermal activity

Faghih tabbed for 2020-21 Interstellar Initiative

Computational Medicine Lab’s Research Featured by IEEE Xplore

UH’s Faghih Named an MIT Technology Review 2020 Innovator Under 35

College honors 17 with yearly Faculty and Student Excellence Awards

Faghih featured in IEEE publication as ‘Woman to Watch’

Continuously Tracking Fear Response Could Improve Mental Health Treatment

Students provide videos to public modeling spread of COVID-19

Adapting Smartwatches to Improve Distance Learning and Health

  • Intellectual Merit

Publications: Journal papers: 23, Peer-reviewed conference papers: 8; Pending Patent: 3

  • Integration of  research into a graduate course resulted in 5 of these peer-reviewed publications (2 conference & 3 journal).
    • Created 22 educational videos and 13  open-source code repositories
  • Released a publicly available toolbox for Bayesian filtering analysis (8 classes of filters) on GitHub- Collected and shared Wearable Exam Stress and  Regulation of Cognitive States Datasets on PhysioNet, a repository of freely-available medical research data.
  • 4  PhD and 1 Masters’ students currently under supervision. 3 PhD students & 3 Masters’ students graduated, and 8 Undergraduate projects (Honors’ Thesis, REU, Summer Undergraduate Research Fellowship) were completed under PI’s supervision. 4 Capstone teams designed wearables for emotion monitoring.
  • PI gave several layman’s talks about this research to general audience and broader community.
  • MINDWATCH demo featured in INet-TV’s (South Korea) documentary The Human Lyricist The AI Composer.
  • PI participated in an educational activity for the next generation of engineers at the International Summer School on Bio-X: Data Science and Engineering in Medicine and Biology.
  •  PI received the 2024 IEEE EMBS Early Career Achievement Award.
  • Computational Medicine Lab Won First Place in the 2023 NYU Tandon Research Excellence Exhibit for the MINDWATCH Project.
  • PI was recognized as MIT Technology Review 2020 Innovator under 35.(FuturoProssimo has predicted that MINDWATCH research has a high potential for a Nobel prize).

Patents

  • Faghih, R.T. and Khazaei, S.. Systems and methods for estimating interoceptive awareness state using eye-tracking measurements, 2023. U. S. Provisional Patent App 63/584,814.

Software

Deconvolution

Estimation

Closed-Loop Control

Emotion Recognition from Heartbeat Dynamics

Real-Time Seizure Tracking

Educational Activities 

Engineering the Future Event  (2021) – (Video Link)

Educational Videos

State-Space Modeling and control of power systems – Video by Thejas Rajan (Spring 2020)

Feedback displacement control of Dielectric Elastomer Actuator (DEA) – video by Shengbin Wang (Spring 2020)

CartPloe LQR Control Analysis – Video by Pengyu (Ben) Yuan (Spring 2020)

An Application of Optimal Control in EM – Video by Neil Jerome A. Egarguin (Summer 2020)

Matrix Algebra and Linear Equations – Video by Prasanna Kumar Reddy Gade (Fall 2018)

Video on Stability by Nitin Bhabsar (Fall 2018)

Lyapunov’s Indirect Method and Linking Lyapunov Function to LQR –  Video by Harshitha Kethyreddy (Fall 2018)

Gilbert Realization and Pole Zero Cancellation – Video by Divesh Pednekar

Reference Tracking and McMillan Degree –  Video by Aruj Khandelwal

Web-Based Educational Activities for the Next Generation of Engineers

Computational Medicine Lab’s Youtube Channel

K-12 and College Level Mentoring, a website by the National Academy of Engineering.

Broader Impact

  • 71% of US workers reported feeling lonely or isolated in the wake of the  COVID-19 pandemic
  • 32% of US adults and 50% of young adults report symptoms of anxiety or depression
  • 88% of US physicians want patients to monitor their health at home
  • 35% of US employers use medical wearable technology to facilitate wellness programs