Machine Learning-Based Evaluation of Cleft Lip and Palate Severity from Facial Symmetry and Speech Analysis

Cleft Lip and Palate (CLP) significantly affects facial esthetics and speech production, presenting challenges for objective severity assessment, particularly after surgical interventions. Current evaluations rely on subjective clinical judgment, limiting reproducibility. To address this issue, this study proposes a dual-modality machine learning based framework that integrates facial symmetry and speech-derived features for the objective classification of the severity of CLP. The research follows an observational, cross-sectional, and exploratory design with a quantitative approach, aiming to identify and characterize visual and auditory features extracted from facial videos of children with and without CLP, to explore potential severity patterns. A novel multimodal dataset was constructed from 99 Spanish-speaking children, of whom 39.4% were diagnosed with CLP condition. Severity labels (mild, moderate, severe) were assigned at the subject level by three raters: an experienced CLP surgeon and two trained non-clinical raters, using majority voting. Inter- and intra-rater reliability were assessed using established agreement metrics, demonstrating consistent severity labeling across raters and sessions. Facial features were extracted using a Convolutional Neural Network and reduced through Uniform Manifold Approximation and Projection to compute a Symmetry Index quantifying facial asymmetry. Speech representations were obtained from a pre-trainedWav2Vec2 model, and audiovisual features were combined for supervised severity classification. The proposed Symmetry Index effectively distinguished the CLP from the control participants (Bhattacharyya distance = 2.45, p < 0.01). The multimodal classifier achieved 76.5% accuracy, outperforming single-modality baselines. The proposed framework, strengthened by class-balancing strategies, demonstrates the feasibility of automated severity assessment and shows potential as a future clinical decision-support tool.