The Construction and Validation of an Automatic Crisis Balance Analysis Model

Abstract

AbstractBackground: With the development of Internet, many people with suicide risk tend to express their thoughts on social media platforms. AI-based model can early identify social media users with suicide risk and analyze their cognitive and interpersonal characteristics. Then we can do early intervention to help them.Objective: To build an automatic crisis balance analysis model based on artificial intelligence which can perform automatic early suicide identification, suicide risk classification and analyze cognitive distortion and interpersonal relationship of users. Then to validate the predictive efficiency of model.Method: Firstly, based on the suicide knowledge graph, free annotation data set was generated and then Bert-based model was built. Secondly, the data set was refined by psychology students and experts to build fine-tuning model and Psychology+ model. The Psychology+ model was used as final suicide risk assessment model. We enriched and quantified the variables of cognitive and interpersonal characteristics and built the cognitive distortion and interpersonal relationship analysis model. Using F1 score, precision, recall and accuracy to evaluate the model performance and the consistence of model results with expert judgment and scales results to evaluate the model prediction ability.Results: For the suicide risk assessment model, the F1 score, precision, recall rate and accuracy rate of the model are 77.98%, 80.75%, 75.41% and 78.68% respectively. For the cognitive distortion and interpersonal relationship analysis model, the F1 score, accuracy and recall rate of the model are 77.26%, 78.22% and 76.33% respectively. Comparing the results with the results of the scale by chi square test, there was no significant difference in cognitive distortion(P = 0.521) and interpersonal relationship(P = 0.189) aspect.Conclusion: The model showed good performance and can be used as a guideline and evaluation tool for intervention.KeywordsSuicideArtificial intelligenceSocial mediaModel validation