Computer vision detects covert voluntary facial movements in unresponsive brain injury patients

Background: Many brain injury patients who appear unresponsive retain subtle, purposeful motor behaviors, signaling capacity for recovery. We hypothesized that low-amplitude movements precede larger-amplitude voluntary movements detectable by clinicians after acute brain injury. To test this hypothesis, we developed a novel, as far as we are aware, computer vision-based tool (SeeMe) that detects and quantifies low-amplitude facial movements in response to auditory commands. Methods: We enrolled 16 healthy volunteers and 37 comatose acute brain injury patients (Glasgow Coma Scale ≤8) aged 18–85 with no prior neurological diagnoses. We measured facial movements to command assessed using SeeMe and compared them to clinicians’ exams. The primary outcome was the detection of facial movement in response to auditory commands. To assess comprehension, we tested whether movements were specific to command type (i.e., eye-opening to open your eyes and not stick out your tongue) with a machine learning-based classifier. Results: Here we show that SeeMe detects eye-opening in comatose patients 4.1 days earlier than clinicians. SeeMe also detects eye-opening in more comatose patients (30/36, 85.7%) than clinical examination (25/36, 71.4%). In patients without an obscuring endotracheal tube, SeeMe detects mouth movements in 16/17 (94.1%) patients. The amplitude and number of SeeMe-detected responses correlate with clinical outcome at discharge. Using our classifier, eye-opening is specific (81%) to the command open your eyes. Conclusion: Acute brain injury patients have low-amplitude movements before overt movements. Thus, many covertly conscious patients may have motor behavior currently undetected by clinicians.