Artificial intelligence / machine-learning tool for post-market surveillance of in vitro diagnostic assays

Abstract

The study compares an artificial intelligence technology with traditional manual search of literature databases to assess the accuracy and efficiency of retrieving relevant articles for post-market surveillance of in vitro diagnostic and medical devices under the Medical Device Regulation and In Vitro Diagnostic Medical Device Regulation. Over a 3-year period, literature searches and technical assessment searches were performed manually or using the Huma.AI platform to retrieve relevant articles related to the safety and performance of selected in vitro diagnostic and medical devices. The manual search involved refined keyword searches, screening of titles/abstracts / full text, and extraction of relevant information. The Huma.AI search utilized advanced caching techniques and a natural language processing system to identify relevant reports. Searches were conducted on PubMed and PubMed Central. The number of identified relevant reports, precision rates, and time requirements for each approach were analyzed. The Huma.AI system outperformed the manual search in terms of the number of identified relevant articles in almost all cases. The average precision rates per year were significantly higher and more consistent with the Huma.AI search compared with the manual search. The Huma.AI system also took significantly less time to perform the searches and analyze the outputs than the manual search. The study demonstrated that the Huma.AI platform was more effective and efficient in identifying relevant articles compared with the manual approach.