Journal of Neuroscience and Neuropharmacology

Automatic Diagnosis of Schizophrenia using Multimodality Media and a Text Reading Task

Farkas A^{* *}Correspondence: Farkas A, Department of Neuropharmacology, Belgium, Email:

Author info »

Introduction

Schizophrenia is a crippling chronic mental illness that affects people all over the world. In this paper, an automatic schizophrenia detection algorithm based on schizophrenic patients' reading deficit is proposed. The automatic schizophrenia detection algorithm illustrates abnormal speech, head movement, and reading fluency during the reading task using speech and video modalities. An acoustic model of speech emotional flatness in schizophrenia is established in the speech modality to reflect the emotional expression flatness of schizophrenic speech from the standpoint of speech production and perception. The reading-fluency-related features are proposed in the video modality to convey the damaged degree of schizophrenic patients' reading fluency, and the head-movement-related features are proposed to illustrate the spontaneous head movement caused by repeated reading and unconscious movement. Schizophrenia is a disabling disease that is accompanied by speech or behavior disorders all over the world. Schizophrenia not only causes social functional defects in patients, but it also has a negative impact on their physical health. Patients benefit from early and accurate diagnosis and treatment of schizophrenia in the long run. Currently, schizophrenia is diagnosed using standardized scales created by authorities. The subjective personal experience of doctors is added to the diagnosis during the clinical diagnosis process. Biological indicators that are objective and effective would aid doctors in making auxiliary diagnoses.

Clinical studies have revealed that schizophrenic patients have reading dyslexia. Researchers discovered that schizophrenic patients have a dyslexia-related risk gene DCDC2 (Doublecortin Domain-Containing Protein 2) and a significantly reduced volume of brain regions related to reading. Schizophrenic patients perform significantly worse than normal controls on single-word reading tasks. They also demonstrate deficiencies in reading speed and comprehension ability in connected text reading tasks. Defects in reading tasks have been shown in studies to be one of the biological markers for diagnosing schizophrenia. Previous research has shown those schizophrenic patients' reading deficiencies can be seen in their abnormal speech, head movement, and reading fluency.

1) Schizophrenia-related abnormal speech: Clinical research shows that the brain nerve activation pattern involved in speech processing differs from normal controls in reading tasks in schizophrenic patients. The decrease in saccade amplitude during reading in schizophrenic patients is related to impaired speech processing ability. In the reading task, schizophrenic patients have a smaller emotional range than normal controls, which is reflected in speech as smaller variations in pitch and intensity.

2) Schizophrenia-related abnormal head movement: The reading process is accompanied by body movements. Body movement necessitates motor processing and speed discrimination.

Clinical studies have shown that in patients with schizophrenia, activation of the V5/MT (Visual Area 5/Middle Temporal) region decreases during motor processing and speed discrimination. The lower the activation level of the V5/MT region, the higher the speed discrimination threshold and the abnormal body movement. When reading the specified text in a fixed position, head movement is primarily body movement. According to studies, schizophrenic patients have significantly higher head movement rates than controls and produce unconscious and unnecessary head movement in visual tasks.

3) Motion energy analysis and head-mounted motion sensors were used to measure head movement in this study. MEA reflects the subject's head movement through grey changes between video frames, which quantifies the amount of movement but is oblivious to the movement trajectory. The headmounted motion sensors make direct contact with the subjects, adding to the psychological burden on schizophrenic patients.

4) Schizophrenia's abnormal reading fluency: One of the obvious characteristics of schizophrenic patients in reading tasks is a lack of reading fluency. Clinical studies have revealed that the left upper temporal cortex of schizophrenic patients fails to show normal blood flow reduction during the verbal fluency task, resulting in abnormal fluency. Furthermore, negative symptoms and attention disorder are factors influencing speech fluency in patients with schizophrenia. Experiments revealed that schizophrenic patients have deficiencies in the fluency of expressing verbs and nouns. It was also discovered that schizophrenic patients had fluency problems in various emotional expression tasks.

In this work, an automatic schizophrenia detection algorithm based on the abnormalities of schizophrenic patients in the reading task is proposed to automatically diagnose schizophrenia from speech and video modalities. The acoustic model of speech emotional flatness in schizophrenia is established in the speech modality to reflect the emotional expression defects in schizophrenia. The relevant features in the video modality are extracted from the aspects of head movement and reading fluency. RHR and UHD are extracted in the head-movement aspect to quantify head movement in the repeated reading dimension and unconscious movement dimension, respectively. The transfer learning method is used in the reading fluency aspect to mine the features in the proposed mouth movement time-domain map and three-dimensional spectrogram. In subsection Results of the Automatic Schizophrenia Detection Algorithm Based on Multimodality in the Reading Task, the results of the automatic schizophrenia detection algorithm based on multimodality in the reading task are illustrated. In the subsections Results of the Proposed Acoustic Model of Speech Emotional Flatness in Schizophrenia and Results of the Proposed Features in Video Modality, the results of the acoustic model of speech emotional flatness in schizophrenia and the proposed features in video modality are displayed and analyzed, respectively.

Based on the abnormal performance of schizophrenic patients in the reading task, a new automatic schizophrenia detection algorithm is proposed in this study. The features proposed by the speech and video modalities are incorporated into the automatic schizophrenia detection algorithm. An acoustic model of speech emotional flatness in schizophrenia is established in the speech modality from the standpoint of speech production and perception, which reflects the emotional flatness of schizophrenia. Headmovement-related features are proposed in the video modality based on the mapping from three-dimensional head movement to two-dimensional images, which reflect spontaneous head movement caused by repeated reading and unconscious movement. The RRFs are proposed from the mouth movement using transfer learning, which reflects the level of reading fluency of the subjects. Experiments on the proposed automatic schizophrenia detection algorithm and the state-of-the-art are carried out to test the classification performance. The performance evaluation indices of the experiments are Accuracy (Acc.), specificity (Spec.), Sensitivity (Sens.), and AUC.

The diagnostic accuracy of the proposed acoustic model of speech emotional flatness in schizophrenia, head-movement-related features, and reading-fluency-related features range from 94.38 to 96.50 %, 73.38 to 83.38 %, and 79.50 to 83.63 %, respectively, in experiments with 10- fold cross-validation. The proposed automatic schizophrenia detection algorithm combines features from the speech and video modalities, with classification accuracy ranging from 96.25 to 97.50 %.