Image feature extraction via local binary patterns for marbling score classification in beef cattle using tree-based algorithms

Abstract

The objective of this study was to investigate the potential of creating a pipeline to classify the marbling score obtained from ribeye area (REA) images using computer vision and machine learning methods. Our database consisted of images and measurements (N = 2,446) from the transversal cut between the 12th and 13th ribs of the Longissimus dorsi muscle from carcasses of a beef cattle population (Bos taurus). Each sample was previously labeled by the industry using a low, medium or high marbling score. The prediction accuracies of two tree-based Machine Learning (ML) algorithms (Decision Tree - DT and Random Forest - RF) were compared, as well as different measures for extracting features from the REA images, which were used as input for the ML algorithms. In order to extract features based on detectable color patterns and textures contained in smaller parts of the grayscale image, we proposed the application of the local binary pattern (LBP) method prior to the adoption of ML methods. Mean classification accuracies for the test set ranged from 45.78% to 91.25% for different test scenarios. The results were mostly impacted by the feature extraction metrics, ML methods, potential subjectivity during the classification process by the industry, and the number of classes evaluated together. The best prediction accuracy results were achieved after performing the cross-validation (20% in each balanced group, 5 folds, and 10 repetitions), considering solely the extreme groups (low and high marbling scores) and pre-selecting from each group the 400 most visually representative samples. The RF algorithm outperformed the DT for most scenarios. After increasing the number of images to 580 samples for the same two groups, the highest testing accuracies were reduced to 83.05% for RF and 75.58% for DT. Such a decrease in the classification accuracies may be associated with the addition of erroneously classified images, due to the subjective nature of the industry evaluation. In conclusion, our preliminary studies showed the LBP method as a powerful feature extraction strategy considering a scenario where the labels were well defined. Our results revealed high accuracies for the classification of marbling extremes, but there is an evident need to improve the understanding of the biological and visual aspects that led to a sharp drop in classification accuracy after the insertion of the intermediate groups of marbling. In addition, the authors highlight the importance of an accurate labeling process for achieving better classification accuracy when applying supervised classification methods.