Application of Large-Scale Foundation Models and Multimodal Fusion for Stress Detection in Glider Pilots Under Real-World Flight Conditions

A. Wolszczak (Poznan Univ. of Techn., Poland), M. De Marsico (Sapienza Univ. of Rome, Italy)

Monitoring the cognitive and emotional workload of pilots under real-world flight operations remains a critical challenge due to severe environmental noise, dynamic artifacts, and pronounced inter-subject variance. This paper presents a comprehensive multimodal pipeline for processing neurophysiological (EEG) and behavioral (FACS) data collected from six pilots during actual flight training on an SZD-50 Puchacz glider. To address the absence of reliable in-flight self-assessments, ground truth labels were generated via automated facial expression analysis, synchronized with EEG streams using an inertial head-nod protocol. The study evaluates a wide spectrum of machine learning architectures: classical algorithms (XGBoost, SVM, Random Forest), deep convolutional models (EEGNet, TSception) under a Transfer Learning paradigm, and novel large-scale foundation models (LaBraM). The findings empirically demonstrate that generic convolutional models suffer severe performance degradation due to Domain Shift and Spatial Mismatch when transferring knowledge from simulator-based datasets (NASA MATB-II, MDPI). The highest predictive stability and performance (ROC AUC of 0.73) were achieved using the LaBraM foundation model. The paper concludes that generic, subject-independent BCI models are fundamentally insufficient for aviation, necessitating paradigm shifts toward subject-specific calibration and temporal context modeling.