Descriptor Learning Research Articles

Machine learning descriptors for crystal materials: applications in Ni-rich layered cathode and lithium anode materials for high-energy-density lithium batteries

Lithium batteries have revolutionized energy storage with their high energy density and long lifespan, but challenges such as energy density limitations, safety, and cost still need to be addressed. Crystalline materials, including Ni-rich cathodes and lithium anodes, play pivotal roles in the performance of high-energy-density lithium batteries. Understanding the micro-scale behavior and degradation mechanisms of these materials is crucial for improving macro-scale battery performance. Simulation methods, particularly machine learning (ML) techniques, have become indispensable tools in elucidating these intricate processes because of great efficiency and acceptable accuracy. ML methods depend on descriptors, which bridge the gap between crystal structures and input matrices of models. These descriptors encode essential atomic-level details in crystal structures, enabling predictions of material properties and behaviors relevant to lithium batteries. This paper reviews and discusses the diverse array of descriptors employed in the simulation of crystalline materials for lithium batteries with high energy density. Case studies highlight the effectiveness of different descriptors in simulating cathode behaviors such as Li/Ni disordering, screening of stable LiNi0.8Co0.1Mn0.1O2 (NMC811) configurations, and lithium deposition behaviors at the anode interface. The discussed descriptors can also be applied to other crystalline cathode, anode, and electrolyte materials in lithium batteries and advance the development of lithium batteries with superior energy density.

Assessing children’s health literacy using a curricular approach: Tasmanian primary school teacher perspectives

ABSTRACT Improving teachers’ understanding of how they can support children’s health literacy development relies on having appropriate measurement tools. A curricular approach to health literacy assessment is one possible means of overcoming the limitations of existing approaches. In a curricular approach, health literacy progress can be recorded through open-ended assessments, linked to curriculum learning descriptors and assessed against health literacy rubrics. Using semi-structured interviews and thematic analysis, this study explored generalist primary teachers’ opinions of the suitability, acceptability, and feasibility of the proposed approach. It was found that it may only be suitable in schools where generalist teachers are responsible for teaching health education and assessing health literacy. Teachers had positive views of the framework (acceptability) but believed that barriers (e.g. lack of time and confidence) could hinder feasibility. Suggestions to support the adoption of a curricular approach included receiving further professional development, and the clear allocation of teaching time and responsibility.

Cross-Modal Prediction of Spectral and Structural Descriptors via a Pretrained Model Enhanced with Chemical Insights.

Proposing and utilizing machine learning descriptors for chemical property prediction and material screening have become a cutting-edge field in artificial intelligence-enabled chemical research. However, a single descriptor typically captures only partial features of a chemical object, resulting in an information deficiency and limiting generalizability. Obtaining a comprehensive set of descriptors is essential but challenging, especially when accessing some microlevel structural and electronic features due to technological limitations. Herein, we exploit multimodal chemical descriptors to construct an encoder-decoder machine learning framework that enables the cross-modal prediction of spectral and structural descriptors. By pretraining the model to endow it with chemical insights, the multimodal data fusion is implemented in a descriptor-encoded hidden layer. The model's capabilities are validated in the system of CO/NO adsorption on Au/Ag surfaces, demonstrating successful reciprocal prediction of infrared spectra, Raman spectra, and internal coordinates. This work provides a proof-of-concept for the feasibility of cross-modal predictions between different chemical features and will significantly reduce the machine learning model's dependence on complete physicochemical parameters and improve its multitarget prediction capabilities.

ConKeD: multiview contrastive descriptor learning for keypoint-based retinal image registration.

Retinal image registration is of utmost importance due to its wide applications in medical practice. In this context, we propose ConKeD, a novel deep learning approach to learn descriptors for retinal image registration. In contrast to current registration methods, our approach employs a novel multi-positive multi-negative contrastive learning strategy that enables the utilization of additional information from the available training samples. This makes it possible to learn high-quality descriptors from limited training data. To train and evaluate ConKeD, we combine these descriptors with domain-specific keypoints, particularly blood vessel bifurcations and crossovers, that are detected using a deep neural network. Our experimental results demonstrate the benefits of the novel multi-positive multi-negative strategy, as it outperforms the widely used triplet loss technique (single-positive and single-negative) as well as the single-positive multi-negative alternative. Additionally, the combination of ConKeD with the domain-specific keypoints produces comparable results to the state-of-the-art methods for retinal image registration, while offering important advantages such as avoiding pre-processing, utilizing fewer training samples, and requiring fewer detected keypoints, among others. Therefore, ConKeD shows a promising potential towards facilitating the development and application of deep learning-based methods for retinal image registration.

RNNPose: 6-DoF Object Pose Estimation via Recurrent Correspondence Field Estimation and Pose Optimization.

6-DoF object pose estimation from a monocular image is a challenging problem, where a post-refinement procedure is generally needed for high-precision estimation. In this paper, we propose a framework, dubbed RNNPose, based on a recurrent neural network (RNN) for object pose refinement, which is robust to erroneous initial poses and occlusions. During the recurrent iterations, object pose refinement is formulated as a non-linear least squares problem based on the estimated correspondence field (between a rendered image and the observed image). The problem is then solved by a differentiable Levenberg-Marquardt (LM) algorithm enabling end-to-end training. The correspondence field estimation and pose refinement are conducted alternately in each iteration to improve the object poses. Furthermore, to improve the robustness against occlusion, we introduce a consistency-check mechanism based on the learned descriptors of the 3D model and observed 2D images, which downweights the unreliable correspondences during pose optimization. We evaluate RNNPose on several public datasets, including LINEMOD, Occlusion-LINEMOD, YCB-Video and TLESS. We demonstrate state-of-the-art performance and strong robustness against severe clutter and occlusion in the scenes. Extensive experiments validate the effectiveness of our proposed method. Besides, the extended system based on RNNPose successfully generalizes to multi-instance scenarios and achieves top-tier performance on the TLESS dataset.

Outcome Prediction Using Multi-Modal Information: Integrating Large Language Model-Extracted Clinical Information and Image Analysis.

Survival prediction post-cystectomy is essential for the follow-up care of bladder cancer patients. This study aimed to evaluate artificial intelligence (AI)-large language models (LLMs) for extracting clinical information and improving image analysis, with an initial application involving predicting five-year survival rates of patients after radical cystectomy for bladder cancer. Data were retrospectively collected from medical records and CT urograms (CTUs) of bladder cancer patients between 2001 and 2020. Of 781 patients, 163 underwent chemotherapy, had pre- and post-chemotherapy CTUs, underwent radical cystectomy, and had an available post-surgery five-year survival follow-up. Five AI-LLMs (Dolly-v2, Vicuna-13b, Llama-2.0-13b, GPT-3.5, and GPT-4.0) were used to extract clinical descriptors from each patient's medical records. As a reference standard, clinical descriptors were also extracted manually. Radiomics and deep learning descriptors were extracted from CTU images. The developed multi-modal predictive model, CRD, was based on the clinical (C), radiomics (R), and deep learning (D) descriptors. The LLM retrieval accuracy was assessed. The performances of the survival predictive models were evaluated using AUC and Kaplan-Meier analysis. For the 163 patients (mean age 64 ± 9 years; M:F 131:32), the LLMs achieved extraction accuracies of 74%~87% (Dolly), 76%~83% (Vicuna), 82%~93% (Llama), 85%~91% (GPT-3.5), and 94%~97% (GPT-4.0). For a test dataset of 64 patients, the CRD model achieved AUCs of 0.89 ± 0.04 (manually extracted information), 0.87 ± 0.05 (Dolly), 0.83 ± 0.06~0.84 ± 0.05 (Vicuna), 0.81 ± 0.06~0.86 ± 0.05 (Llama), 0.85 ± 0.05~0.88 ± 0.05 (GPT-3.5), and 0.87 ± 0.05~0.88 ± 0.05 (GPT-4.0). This study demonstrates the use of LLM model-extracted clinical information, in conjunction with imaging analysis, to improve the prediction of clinical outcomes, with bladder cancer as an initial example.

Leveraging Descriptor Learning and Functional Map-based Shape Matching for Automatic Landmark Acquisition.

Geometric morphometrics is widely employed across the biological sciences for the quantification of morphological traits. However, the scalability of these methods to large datasets is hampered by the requisite placement of landmarks, which can be laborious and time consuming if done manually. Additionally, the selected landmarks embody a particular hypothesis regarding the critical geometry pertinent to the biological inquiry at hand. Modifying this hypothesis lacks flexibility, necessitating the acquisition of an entirely new set of landmarks on the entire dataset to reflect any theoretical adjustments. In our research, we investigate the precision and accuracy of landmarks derived from the comprehensive set of functional correspondences acquired through the functional map framework of geometry processing. We use a deep functional map network to learn shape descriptors that effectively yield functional map-based and point-to-point correspondences between the specimens in our dataset. We then interrogate these maps to identify corresponding landmarks given manually placed landmarks from the entire dataset. We assess our method by automating the landmarking process on a dataset comprising mandibles from various rodent species, comparing its efficacy against MALPACA, a cutting-edge technique for automatic landmark placement. Compared to MALPACA, our model is notably faster and maintains competitive accuracy. The Root Mean Square Error (RMSE) analysis reveals that while MALPACA generally exhibits the lowest RMSE, our models perform comparably, especially with smaller training datasets, suggesting strong generalizability. Visual evaluations confirm the precision of our landmark placements, with deviations remaining within an acceptable range. These findings underscore the potential of unsupervised learning models in anatomical landmark placement, providing a viable and efficient alternative to traditional methods.

D3Former: Jointly learning repeatable dense detectors and feature-enhanced descriptors via saliency-guided transformer

Establishing accurate and representative matches is a crucial step in addressing the point cloud registration problem. A commonly employed approach involves detecting keypoints with salient geometric features and subsequently mapping these keypoints from one frame of the point cloud to another. However, methods within this category are hampered by the repeatability of the sampled keypoints. In this paper, we introduce a saliency-guided transformer, referred to as D3Former, which entails the joint learning of repeatable Dense Detectors and feature-enhanced Descriptors. The model comprises a Feature Enhancement Descriptor Learning (FEDL) module and a Repetitive Keypoints Detector Learning (RKDL) module. The FEDL module utilizes a region attention mechanism to enhance feature distinctiveness, while the RKDL module focuses on detecting repeatable keypoints to enhance matching capabilities. Extensive experimental results on challenging indoor and outdoor benchmarks demonstrate that our proposed method consistently outperforms state-of-the-art point cloud matching methods. Notably, tests on 3DLoMatch, even with a low overlap ratio, show that our method consistently outperforms recently published approaches such as RoReg and RoITr. For instance, with the number of extracted keypoints reduced to 250, the registration recall scores for RoReg, RoITr, and our method are 64.3%, 73.6%, and 76.5%, respectively.

SDGMNet: Statistic-Based Dynamic Gradient Modulation for Local Descriptor Learning

Rescaling the backpropagated gradient of contrastive loss has made significant progress in descriptor learning. However, current gradient modulation strategies have no regard for the varying distribution of global gradients, so they would suffer from changes in training phases or datasets. In this paper, we propose a dynamic gradient modulation, named SDGMNet, for contrastive local descriptor learning. The core of our method is formulating modulation functions with dynamically estimated statistical characteristics. Firstly, we introduce angle for distance measure after deep analysis on backpropagation of pair-wise loss. On this basis, auto-focus modulation is employed to moderate the impact of statistically uncommon individual pairs in stochastic gradient descent optimization; probabilistic margin cuts off the gradients of proportional triplets that have achieved enough optimization; power adjustment balances the total weights of negative pairs and positive pairs. Extensive experiments demonstrate that our novel descriptor surpasses previous state-of-the-art methods in several tasks including patch verification, retrieval, pose estimation, and 3D reconstruction.

Semantics-enhanced discriminative descriptor learning for LiDAR-based place recognition

LiDAR-based place recognition (LPR) aims to localize autonomous vehicles and mobile robots relative to pre-built maps or retrieve previously visited places. However, the complexity of real-world scenes and changes in viewpoint are significant challenges for place recognition. As high-level information, semantics makes it easier to distinguish geometrically similar scene situations. Unlike most existing methods that rely solely on a single type of information (geometric or semantic) to construct scene descriptors, we consider the complementary nature of the semantic and geometric information and propose a semantics-enhanced discriminative feature learning method for LPR. Specifically, we first develop a Multi-layer Fusion Feature Extraction Network (MFFEN) based on the transformer encoder to hierarchically fuse local geometric and semantic information and utilize contextual information for extracting discriminative local features. To obtain semantic information, we introduce the dynamic graph convolution network to extract local semantic features with local relations. In addition, to weaken the interference of redundancy and dynamic objects in the scene, we design a semantics-guided local attention network (SLAN) to focus on salient local features that are helpful for recognizing scenes, thereby enhancing the descriptive ability of the global descriptor. Extensive experiments on public datasets KITTI and KITTI-360 demonstrate that the proposed method performs better than recent LiDAR-based methods on the 3D place recognition task. For instance, it achieves the mean F1max score of 96.9% on the KITTI dataset, surpassing the strongest prior model by 2.7%.

Covariant Peak Constraint for Accurate Keypoint Detection and Keypoint-Specific Descriptor Learning

Covariant Peak Constraint for Accurate Keypoint Detection and Keypoint-Specific Descriptor Learning

Integrated Analysis of Machine Learning and Deep Learning in Silkworm Pupae (Bombyx mori) Species and Sex Identification

Hybrid pairing of the corresponding silkworm species is a pivotal link in sericulture, ensuring egg quality and directly influencing silk quantity and quality. Considering the potential of image recognition and the impact of varying pupal postures, this study used machine learning and deep learning for global modeling to identify pupae species and sex separately or simultaneously. The performance of traditional feature-based approaches, deep learning feature-based approaches, and their fusion approaches were compared. First, 3600 images of the back, abdomen, and side postures of 5 species of male and female pupae were captured. Next, six traditional descriptors, including the histogram of oriented gradients (HOG), and six deep learning descriptors, including ConvNeXt-S, were utilized to extract significant species and sex features. Finally, classification models were constructed using the multilayer perceptron (MLP), support vector machine, and random forest. The results indicate that the {HOG + ConvNeXt-S + MLP} model excelled, achieving 99.09% accuracy for separate species and sex recognition and 98.40% for simultaneous recognition, with precision–recall and receiver operating characteristic curves ranging from 0.984 to 1.0 and 0.996 to 1.0, respectively. In conclusion, it can capture subtle distinctions between pupal species and sexes and shows promise for extensive application in sericulture.

Improving the performance of learned descriptors in the matching of high spatial resolution aerial images by proposing a large-scale dataset of vertical images

Improving the performance of learned descriptors in the matching of high spatial resolution aerial images by proposing a large-scale dataset of vertical images

Learning General Descriptors for Image Matching With Regression Feedback

Recent advances on feature descriptors for image matching put more emphasis on encoding invariances ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g</i> . illumination invariance) to promote the descriptors’ discriminative power. However, according to the information entropy, more invariance implies greater certainty and less informativeness in a descriptor. Consequently, descriptors encoding too many invariances usually show poor generalization to unknown image changes, lacking enough informativeness to cover the large uncertainty in unseen scenes. This limits the application scenarios of learned descriptors. In this paper, we propose to alleviate this issue from the perspective of informativeness and we thus design hierarchical consistent constraint by introducing regression feedback in a self-supervised manner. Combined with the hardest-within-batch matching constraint, we form a novel dual supervision framework, to encourage the descriptor to learn an informative representation while maintaining a good discriminative power. Moreover, to fully mine the context information hidden in image and boost the informativeness in turn, we present AANet, a descriptor network that efficiently predicts dense description by the powerful Attentional Aggregation of multi-level features. Experiments across challenging feature matching on HPatches, RDNIM datasets, and visual localization tasks on Aachen Day-night dataset show that our method outperforms recent state-of-the-art descriptors while keeping encouraging efficiency. The application of visual 3D reconstruction on various scenarios also demonstrates the high generalization ability of our method.

Comparison of Energy-Based Machine Learning Descriptors for Gas Adsorption

Comparison of Energy-Based Machine Learning Descriptors for Gas Adsorption

Reliable Feature Matching for Spherical Images via Local Geometric Rectification and Learned Descriptor

Spherical images have the advantage of recording full scenes using only one camera exposure and have been becoming an important data source for 3D reconstruction. However, geometric distortions inevitably exist due to the spherical camera imaging model. Thus, this study proposes a reliable feature matching algorithm for spherical images via the combination of local geometric rectification and CNN (convolutional neural network) learned descriptor. First, image patches around keypoints are reprojected to their corresponding tangent planes based on a spherical camera imaging model, which uses scale and orientation data from the keypoints to achieve both rotation and scale invariance. Second, feature descriptors are then calculated from the rectified image patches by using a pre-trained separate detector and descriptor learning network, which improves the discriminability by exploiting the high representation learning ability of the CNN. Finally, after classical feature matching with the ratio test and cross check, refined matches are obtained based on an essential matrix-based epipolar geometry constraint for outlier removal. By using three real spherical images and an incremental structure from motion (SfM) engine, the proposed algorithm is verified and compared in terms of feature matching and image orientation. The experiment results demonstrate that the geometric distortions can be efficiently reduced from rectified image patches, and the increased ratio of the match numbers ranges from 26.8% to 73.9%. For SfM-based spherical image orientation, the proposed algorithm provides reliable feature matches to achieve complete reconstruction with comparative accuracy.

Accelerating the prediction of CO2 capture at low partial pressures in metal-organic frameworks using new machine learning descriptors

Metal-Organic frameworks (MOFs) have been considered for various gas storage and separation applications. Theoretically, there are an infinite number of MOFs that can be created; however, a finite amount of resources are available to evaluate each one. Computational methods can be adapted to expedite the process of evaluation. In the context of CO2 capture, this paper investigates the method of screening MOFs using machine learning trained on molecular simulation data. New descriptors are introduced to aid this process. Using all descriptors, it is shown that machine learning can predict the CO2 adsorption, with an R2 of above 0.9. The introduced Effective Point Charge (EPoCh) descriptors, which assign values to frameworks’ partial charges based on the expected CO2 uptake of an equivalent point charge in isolation, are shown to be the second most important group of descriptors, behind the Henry coefficient. Furthermore, the EPoCh descriptors are hundreds of thousands of times faster to obtain compared with the Henry coefficient, and they achieve similar results when identifying top candidates for CO2 capture using pseudo-classification predictions.

RDLNet: A Regularized Descriptor Learning Network.

Local image descriptor learning has been instrumental in various computer vision tasks. Recent innovations lie with similarity measurement of descriptor vectors with metric learning for randomly selected Siamese or triplet patches. Local image descriptor learning focuses more on hard samples since easy samples do not contribute much to optimization. However, few studies focus on hard samples of image patches from the perspective of loss functions and design appropriate learning algorithms to obtain a more compact descriptor representation. This article proposes a regularized descriptor learning network (RDLNet) that makes the network focus on the learning of hard samples and compact descriptor with triplet networks. A novel hard sample mining strategy is designed to select the hardest negative samples in mini-batch. Then batch margin loss concerned with hard samples is adopted to optimize the distance of extreme cases. Finally, for a more stable network and preventing network collapsing, orthogonal regularization is designed to constrain convolutional kernels and obtain rich deep features. RDLNet provides a compact discriminative low-dimensional representation and can be embedded in other pipelines easily. This article gives extensive experimental results for large benchmarks in multiple scenarios and generalization in matching applications with significant improvements.

WSDesc: Weakly Supervised 3D Local Descriptor Learning for Point Cloud Registration.

In this work, we present a novel method called WSDesc to learn 3D local descriptors in a weakly supervised manner for robust point cloud registration. Our work builds upon recent 3D CNN-based descriptor extractors, which leverage a voxel-based representation to parameterize local geometry of 3D points. Instead of using a predefined fixed-size local support in voxelization, we propose to learn the optimal support in a data-driven manner. To this end, we design a novel differentiable voxelization layer that can back-propagate the gradient to the support size optimization. To train the extracted descriptors, we propose a novel registration loss based on the deviation from rigidity of 3D transformations, and the loss is weakly supervised by the prior knowledge that the input point clouds have partial overlap, without requiring ground-truth alignment information. Through extensive experiments, we show that our learned descriptors yield superior performance on existing geometric registration benchmarks.

Revisiting Unsupervised Local Descriptor Learning

Constructing accurate training tuples is crucial for unsupervised local descriptor learning, yet challenging due to the absence of patch labels. The state-of-the-art approach constructs tuples with heuristic rules, which struggle to precisely depict real-world patch transformations, in spite of enabling fast model convergence. A possible solution to alleviate the problem is the clustering-based approach, which can capture realistic patch variations and learn more accurate class decision boundaries, but suffers from slow model convergence. This paper presents HybridDesc, an unsupervised approach that learns powerful local descriptor models with fast convergence speed by combining the rule-based and clustering-based approaches to construct training tuples. In addition, HybridDesc also contributes two concrete enhancing mechanisms: (1) a Differentiable Hyperparameter Search (DHS) strategy to find the optimal hyperparameter setting of the rule-based approach so as to provide accurate prior for the clustering-based approach, (2) an On-Demand Clustering (ODC) method to reduce the clustering overhead of the clustering-based approach without eroding its advantage. Extensive experimental results show that HybridDesc can efficiently learn local descriptors that surpass existing unsupervised local descriptors and even rival competitive supervised ones.