Approaches for Using Machine Learning Algorithms with Large Label Sets for Rotorcraft Maintenance

Abstract

The US Army Aviation and Missile Research, Development, and Engineering Center (AMRDEC), in collaboration with the US Army Engineer Research and Development Center (ERDC), is using machine learning (ML) to transform the way rotorcraft maintenance logbook event data is scored for reporting purposes. Traditionally, human analysts manually inspect data fields completed by maintenance personnel and, using published guidelines in conjunction with their personal expertise, provide sets of labels or scores for each event. These labels are stored with the data and provide valuable insight into maintenance event histories. However, the inequity between the enormous volume of maintenance data generated daily and the ability of analysts to score the data results in only 10% of all data receiving scores; therefore, 90% of the recorded data does not contain this important value added feature. Classification algorithms for automating this scoring process trained on existing labeled data sets have been implemented with promising results. A particularly challenging element of this problem, however, involves the classification of the specific component on which maintenance was performed. Greater than 1200 unique labels exist that can be used to describe a rotorcraft component that is the subject of a maintenance action. Furthermore, the component labels are hierarchically structured, resulting in the occurrence of multiple levels of precision in identifying a component in the expert-labeled data used for training. Although computational efficiency of common classification algorithms has improved considerably, it is still quite challenging to harness these methods for problems that include large numbers of unique class labels. This paper describes several novel strategies for solving this problem, based on hierarchical ensemble models and strategic label set segmentation. Through these approaches, a best overall total component classification accuracy of 96% was achieved, in conjunction with a total per record accuracy for three different label categories of 93%. The approaches implemented to handle the large label set for classification of rotorcraft components, along with classification performance measures, are discussed.