Automated extraction of quality indicators for treatment of children with complex developmental disorders: A feasibility study using the example of attention-deficit/hyperactivity disorder.

Abstract

Quality assessment is challenging in children with developmental disorders. Previously, a set of quality indicators (QIs) was developed and analyzed in terms of feasibility of use with patients with attention-deficit/hyperactivity disorder (ADHD). QI assessment turned out to be possible but highly complex. Thus, we compared different technologies for automated extraction of data for assessment of QIs. Four automated extraction technologies (regular expressions, Apache Solr, Apache Mahout, Apache OpenNLP) were compared with respect to their properties regarding the complexity of implementing the QI, the complexity of implementing a check module, the reliability and quality of results, the complexity of preparation of interdisciplinary medical reports, and the complexity of deployment and installation. Twenty medical reports from different institutions were reviewed for compliance with three QIs by these technologies and compared with expert opinions. Among the four technologies, Apache Solr had the best overall performance. For manual extraction of the three QIs, at least 77 s were necessary per medical report, whereas the prototype evaluated and extracted the QIs automatically in 8 s on average. Unexpectedly, different assessments of the degree of compliance by the experts turned out to be one of the stumbling blocks. An in-depth evaluation compared results on a semantic level. It is possible to extract QIs by post-processing automated technologies. This approach can also be applied to other developmental disorders. However, a more uniform documentation throughout institutions involved will be necessary in order to implement this method in daily practice.