Similar Neighbors Research Articles

Spatial phylogenetic and phenotypic patterns reveal ontogenetic shifts in ecological processes of plant community assembly

The analysis of the spatial phylogenetic and phenotypic structure of plant communities can provide insight into the underlying processes and interactions governing their assembly, and how these may change during plant ontogeny. We used point pattern analysis to find out if saplings and adult plants are surrounded by phylogenetically and phenotypically more similar or more dissimilar neighbours than expected by chance, and whether these associations change from the sapling to the adult stage. To this end, we combined information on the phylogenetic structure and eight phenotypic traits of 15 woody plant species in two Mediterranean mixed forests of southeastern Spain. At the community level, we found that the sapling bank at both sites did not show phylogenetic or phenotypic spatial patterns, but adults showed phylogenetic clustering (i.e. heterospecific neighbours were more similar than expected). At the species level, we found frequently repulsive patterns in the sapling bank of less abundant species (i.e. heterospecific sapling or adult neighbours were more dissimilar than expected) in both, phylogenetic and phenotypic analyses. For the adult stage, we found phylogenetic attraction (i.e. more similar neighbours) in just one species and phenotypic clustering in four species. The processes driving the assembly of the communities of saplings and adults leave detectable signals in the spatial phylogenetic and phenotypic structure of our two forest communities. Our findings reinforce the existence of ontogenetic shifts in the mechanisms involved in plant community assembly. Facilitation between phylogenetically distant and phenotypically divergent species favours the recruitment of less abundant species. However, processes acting later in the ontogeny ameliorate the competition between close relatives and determine the spatial structure of adult plants. Nevertheless, the role of phenotype in shaping the interactions between adult plants was context‐ and trait‐dependent. The use of spatial point pattern analysis allowed a nuanced interpretation of the phylogenetic and phenotypic structures of plant communities.

An Efficient Hyperspectral Image Classification Method: Inter-Class Difference Correction and Spatial Spectral Redundancy Removal

Existing remote sensing images of ground objects are difficult to annotate, and building a hyperspectral dataset requires huge resources. To tackle these problems, this paper proposes a new method with low requirements for the scale of the dataset that involves correcting the inter-class differences of hyperspectral images and eliminating the redundant information of spatial–spectral features. Firstly, the algorithm introduces the spatial information of hyperspectral images into the classification task through the superpixel definition based on the entropy rate, which not only reduces the spatial redundancy information of hyperspectral images but also uses the similarity of spatial neighbors to increase the differences between classes and alleviate the differences within classes. Secondly, under the theoretical guidance of similar spectral fluctuation trends of similar objects, a feature extraction method based on fast Fourier transform is proposed to alleviate spectral feature redundancy and further eliminate the inter-class differences of the algorithm. Finally, to verify the improvement effect of the proposed idea on the traditional classification method, the idea was applied to the traditional SVM algorithm, and experiments were carried out based on the PaviaU and Indian Pines hyperspectral datasets. The simulation results show that the proposed (ERS–FFT–SVM) algorithm shows a significant improvement in classification accuracy when compared with the traditional classification algorithm and is able to perform the small sample classification of hyperspectral images.

A multiview graph collaborative filtering by incorporating homogeneous and heterogeneous signals

In the industrial e-commerce recommender systems, the sparsity of user–item interaction limits the improvement of the performance of collaborative filtering recommendation. Some studies have leveraged attribute co-occurrence or similar neighbors to enhance the semantic representation quality of users and items. Previous methods consider collaborative signals of homogeneous type nodes, such as <user,user>→user and <item,item>→item. By exploiting homogeneous and heterogeneous signals of attribute and neighbor views, we design a multiview graph collaborative filtering (MVGCF) network for recommendation. The MVGCF model utilizes both co-occurrence features of various attribute values and collaborative preference of various neighbors to learn the embedding representation of nodes. Experimental results show that the MVGCF is superior to the state-of-the-art models in AUC and logloss metrics by 1.41% and 3.12% for MovieLens 1M dataset, and by 2.35% and 2.31% for BookCrossing dataset. Aiming at the sparse problem with a small amount of interaction records, our findings is that attribute co-occurrence and neighbor collaboration can improve the accuracy and provide a good explanation for e-commerce recommender systems.

Multisource forest inventories: A model-based approach using k-NN to reconcile forest attributes statistics and map products

Forest map products are widely used and have taken benefit from progresses in the multisource forest inventory approaches, which are meant to improve the precision of forest inventory estimates at high spatial resolution. However, estimating errors of pixel-wise predictions remains difficult, and reconciling statistical outcomes with map products is still an open and important question.We address this problem using an original approach relying on a model-based inference framework and k-nearest neighbours (k-NN) models to produce pixel-wise estimations and related quality assessment. Our approach takes advantage of the resampling properties of a model-based estimator and combines it with geometrical convex-hull models to measure respectively the precision and accuracy of pixel predictions. A measure of pixel reliability was obtained by combining precision and accuracy.The study was carried out over a 7,694 km2 area dominated by structurally complex broadleaved forests in centre of France. The targeted forest attributes were growing stock volume, basal area and growing stock volume increment. A total of 819 national forest inventory plots were combined with auxiliary data extracted from a forest map, Landsat 8 images, and 3D point clouds from both airborne laser scanning and digital aerial photogrammetry. k-NN models were built independently for both 3D data sources. Both selected models included 5 auxiliary variables, and were generated using 5 neighbours, and most similar neighbours distance measure. The models showed relative root mean square error ranging from 35.7% (basal area, digital aerial photogrammetry) in calibration to 63.4% (growing stock volume increment, airborne laser scanning) in the validation set. At pixel level, we found that a minimum of 86.4% of the predictions were of high precision as their bootstrapped coefficient of variation fall below calibration’s relative root mean square error. The amount of extrapolation varied from 4.3% (digital aerial photogrammetry) to 6.3% (airborne laser scanning). A relationship was found between extrapolation and k-NN distance, opening new opportunities to correct extrapolation errors. At the population level, airborne laser scanning and digital aerial photogrammetry performed similarly, offering the possibility to use digital aerial photogrammetry for monitoring purposes.The proposed method provided consistent estimates of forest attributes and maps, and also provided spatially explicit information about pixel predictions in terms of precision, accuracy and reliability. The method therefore produced high resolution outputs, significant for either decision making or forest management purposes.

Siamese Graph-Based Dynamic Matching for Collaborative Filtering

Behaviorally similar neighbors in the interaction graph have been actively explored to facilitate the collaboration between users and items and address the interaction sparsity issue. We investigate homogenous neighbors between users or items to mine collaborative signals for embedding learning. In the case of multiple and complex composition of user interests and item attributes, traditional uniform embedding is insufficient to depict matching between a specific user-item pair. Therefore, we propose a siamese graph-based dynamic matching (SGDM) model for collaborative filtering. A target-aware dual attention module is introduced to update neighbors with their varying contributions to unveil users’ interests, which explicitly encodes the clue of the candidate matching target. For a target user-item pair, homogeneous neighbors participate in the interest propagation through graph convolution, which learns user/item embeddings respectively on the siamese homogeneous graphs. The dual dynamic aggregation in graph convolution endows a specific user-item pair with dynamic matching, which is expected to meet the needs of fine-grained filtering and promote the performance of collaborative filtering. Extensive experimental results confirm the collaboration between siamese homogeneous graphs of users and items. It further illustrates the effectiveness of the proposed SGDM in mining homogeneous collaborative signals for embedding learning and collaborative filtering.

Topological influence and locality in swap schelling games

Residential segregation is a wide-spread phenomenon that can be observed in almost every major city. In these urban areas residents with different racial or socioeconomic background tend to form homogeneous clusters. Schelling’s famous agent-based model for residential segregation explains how such clusters can form even if all agents are tolerant, i.e., if they agree to live in mixed neighborhoods. For segregation to occur, all it needs is a slight bias towards agents preferring similar neighbors. Very recently, Schelling’s model has been investigated from a game-theoretic point of view with selfish agents that strategically select their residential location. In these games, agents can improve on their current location by performing a location swap with another agent who is willing to swap. We significantly deepen these investigations by studying the influence of the underlying topology modeling the residential area on the existence of equilibria, the Price of Anarchy and on the dynamic properties of the resulting strategic multi-agent system. Moreover, as a new conceptual contribution, we also consider the influence of locality, i.e., if the location swaps are restricted to swaps of neighboring agents. We give improved almost tight bounds on the Price of Anarchy for arbitrary underlying graphs and we present (almost) tight bounds for regular graphs, paths and cycles. Moreover, we give almost tight bounds for grids, which are commonly used in empirical studies. For grids we also show that locality has a severe impact on the game dynamics.

Neighborhood-based inference and restricted Boltzmann machine for microbe and drug associations prediction.

BackgroundEfficient identification of microbe-drug associations is critical for drug development and solving problem of antimicrobial resistance. Traditional wet-lab method requires a lot of money and labor in identifying potential microbe-drug associations. With development of machine learning and publication of large amounts of biological data, computational methods become feasible.MethodsIn this article, we proposed a computational model of neighborhood-based inference (NI) and restricted Boltzmann machine (RBM) to predict potential microbe-drug association (NIRBMMDA) by using integrated microbe similarity, integrated drug similarity and known microbe-drug associations. First, NI was used to obtain a score matrix of potential microbe-drug associations by using different thresholds to find similar neighbors for drug or microbe. Second, RBM was employed to obtain another score matrix of potential microbe-drug associations based on contrastive divergence algorithm and sigmoid function. Because generalization ability of individual method is poor, we used an ensemble learning to integrate two score matrices for predicting potential microbe-drug associations more accurately. In particular, NI can fully utilize similar (neighbor) information of drug or microbe and RBM can learn potential probability distribution hid in known microbe-drug associations. Moreover, ensemble learning was used to integrate individual predictor for obtaining a stronger predictor.ResultsIn global leave-one-out cross validation (LOOCV), NIRBMMDA gained the area under the receiver operating characteristics curve (AUC) of 0.8666, 0.9413 and 0.9557 for datasets of DrugVirus, MDAD and aBiofilm, respectively. In local LOOCV, AUCs of 0.8512, 0.9204 and 0.9414 were obtained for NIRBMMDA based on datasets of DrugVirus, MDAD and aBiofilm, respectively. For five-fold cross validation, NIRBMMDA acquired AUC and standard deviation of 0.8569 ± −0.0027, 0.9248 ± −0.0014 and 0.9369 ± −0.0020 on the basis of datasets of DrugVirus, MDAD and aBiofilm, respectively. Moreover, case study for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) showed that 13 out of the top 20 predicted drugs were verified by searching literature. The other two case studies indicated that 17 and 17 out of the top 20 predicted microbes for the drug of ciprofloxacin and minocycline were confirmed by identifying published literature, respectively.

Data Mining and Video Target Detection-Based Analysis of Martial Arts Cultural Communication and Martial Arts Athletes’ Posture

With the continuous deepening of economic globalization and the continuous development of Internet big data, the spread of martial arts culture has attracted people’s attention. The advent of the Internet big data era has brought new opportunities and media for the dissemination of martial arts culture but also brought certain risks and challenges. This paper firstly uses the literature method, logical analysis method, and related theories of communication, with the help of the SWOT strategic analysis method, to analyze the internal advantages and disadvantages faced by martial arts culture communication itself, as well as the external opportunities in the context of the Internet big data era and challenges. In order to realize the target detection and tracking task of Martial athletes, this paper proposes a target detection and tracking algorithm for Martial athletes. First, the algorithm constructs and trains a fully CNN-based athlete target detection model, which can detect the first frame of athlete targets in martial arts competitions or martial arts training videos. Experiments show that the athlete positioning information output by the detection model and the video frame sequence are jointly input into the athlete target tracking module based on neighborhood similarity, so as to realize the target tracking of martial arts athletes.

A Graph-Based k-Nearest Neighbor (KNN) Approach for Predicting Phases in High-Entropy Alloys

Traditional techniques for detecting materials have been unable to coordinate with the advancement of material science today due to their low accuracy and high cost. Accordingly, machine learning (ML) improves prediction efficiency in material science and high-entropy alloys’ (HEAs’) phase prediction. Unlike traditional alloys, HEAs consist of at least five elements with equal or near-equal atomic sizes. In a previous approach, we presented an HEA interaction network based on its descriptors. In this study, the HEA phase is predicted using a graph-based k-nearest neighbor (KNN) approach. Each HEA compound has its phase, which includes five categories: FCC, BCC, HCP, Multiphase and Amorphous. A composition phase represents a state of matter with a certain energy level. Phase prediction is effective in determining its application. Each compound in the network has some neighbors, and the phase of a new compound can be predicted based on the phase of the most similar neighbors. The proposed approach is performed on the HEA network. The experimental results show that the accuracy of the proposed approach for predicting the phase of new alloys is 88.88%, which is higher than that of other ML methods.

Spatial responses of soil carbon stocks, total nitrogen, and microbial indices to post-wildfire in the Mediterranean red pine forest

Wildfire is a key ecological event that alters vegetation and soil quality attributes including biochemical attributes at spatial scale. This knowledge can provide insights into the development of better rehabilitation or restoration strategies that depend on the ecological dynamics of vegetation, fungi, and animals. The present study aimed to understand the causes and consequences of spatial variability of soil organic carbon, microbial biomass C concentrations, and soil quality indices as impacted by wildfire in a red pine forest. This study was conducted using kriging and inverse distance neighborhood similarity (IDW) interpolations methods. The carbon stocks were significantly (P = 0.002) higher in burned areas compared to those of unburned areas by 255% whereas microbial biomass carbon and microbial respiration were significantly (P < 0.0001 and P = 0.02) lower in burned areas by 66% and 90%, The Pearson's correlation analysis showed that carbon stocks were positively correlated with pH (0.61), total nitrogen (0.60) and ash quantity (0.41), but negatively correlated with microbial biomass carbon (−0.46) and nitrogen (−0.61), and microbial respiration (−0.48). The IDW interpolation method better-predicted pH, bulk density, and microbial biomass carbon and nitrogen compared to kriging interpolation, whereas the kriging interpolation method was better than IDW interpolation for the other studied soil properties. We concluded that pH, EC, SOC, C/N, MR, MBC/SOC, and MBC/MBN can be reliable indicators to monitor the effect of wildfire on forest soils. The wildfire event increased soil carbon stocks, TN, pH, and qCO2, but decreased MBC and MBN.

Time-series surface water gap filling based on spatiotemporal neighbourhood similarity

Optical satellite-derived surface water monitoring is challenging because of the spatial gaps in images caused by clouds, cloud shadows, voids, etc. Here, an efficient method for filling gaps in time-series surface water images is proposed, based on the spatiotemporal characteristics of water. This method utilises the accurately classified historical ternary (gap, water, non-water) or binary (water, non-water) water image time-series and the clear part of the ternary gap water image. Pixels with values of 0 and 1 in the same period water occurrence image are first used to correct the gap water image. The spatial neighbourhood similarity is then calculated as a quality control band for mosaicking the accurately classified historical water images. The final result is generated by replacing the gap pixels with a mosaic image. The proposed method was implemented on the Google Earth Engine, and 93 Landsat 8 top-of-atmosphere (TOA) images were used to verify its validity. Quantitative evaluations were adequate, with a mean accuracy, recall, and precision of 0.98, 0.90, and 0.85, respectively. The proposed method could improve the utilisation of optical remote sensing data and would be applicable to the production of large-area homogeneous surface water time-series and water resource monitoring.

A new overlapping community detection algorithm based on similarity of neighbors in complex networks

A new overlapping community detection algorithm based on similarity of neighbors in complex networks

Enhancing the scalability of distance-based link prediction algorithms in recommender systems through similarity selection.

Slope One algorithm and its descendants measure user-score distance and use the statistical score distance between users to predict unknown ratings, as opposed to the typical collaborative filtering algorithm that uses similarity for neighbor selection and prediction. Compared to collaborative filtering systems that select only similar neighbors, algorithms based on user-score distance typically include all possible related users in the process, which needs more computation time and requires more memory. To improve the scalability and accuracy of distance-based recommendation algorithm, we provide a user-item link prediction approach that combines user distance measurement with similarity-based user selection. The algorithm predicts unknown ratings based on the filtered users by calculating user similarity and removing related users with similarity below a threshold, which reduces 26 to 29 percent of neighbors and improves prediction error, ranking, and prediction accuracy overall.

TraTSA: A Transprecision Framework for Efficient Time Series Analysis

Time series analysis (TSA) comprises methods for extracting information in domains as diverse as medicine, seismology, speech recognition and economics. Matrix Profile (MP) is the state-of-the-art TSA technique, which provides the most similar neighbor to each subsequence of the time series. However, this computation requires a huge amount of floating-point (FP) operations, which are a major contributor (≈ 50%) to the energy consumption in modern computing platforms. In this sense, Transprecision Computing has recently emerged as a promising approach to improve energy efficiency and performance by using fewer bits in FP operations while providing accurate results.In this work, we present TraTSA, the first transprecision framework for efficient time series analysis based on MP. TraTSA allows the user to deploy a high-performance and energy-efficient computing solution with the exact precision required by the TSA application. To this end, we first propose implementations of TraTSA for both commodity CPU and FPGA platforms. Second, we propose an accuracy metric to compare the results with the double-precision MP. Third, we study MP’s accuracy when using a transprecision approach. Finally, our evaluation shows that, while obtaining results accurate enough, the FPGA transprecision MP (i) is 22.75× faster than a 72-core server, and (ii) the energy consumption is up to 3.3× lower than the double-precision executions.

CBIM: Community-based influence maximization in multilayer networks

Selecting seed nodes (most influential nodes) in networks has attracted attention due to seed nodes’ ability to influence and spread information. Seed nodes are essential to understanding the spreading and controlling of the information dynamics of the networks. Influence maximization (IM) is predominant in monolayer networks. After the advancements and widespread usage of social networks, applying influence maximization to multilayer networks is gaining popularity. Identifying influential nodes precisely in multilayer networks is a challenging and yet unexplored task. Based on studies, individuals in a community interact frequently and are more likely to influence each other. Motivated by this observation, this paper proposes community-based influence maximization (CBIM) model to find k seed nodes in multilayer networks. CBIM has two phases: In the first phase, CBIM uses the function FIC(M) to find the small communities from a multilayer network based on dice neighborhood similarity. It uses the function CSC(CSinit,θ) to merge smaller communities and generate larger communities to improve communities’ quality. In the second phase, CBIM computes Edge Weight Sum (EWS) for each node in a community and ranks the nodes based on EWS. CBIM uses the quota-based approach to select the seed node set from the communities based on the ranks. A comparative study of various influence maximization (IM) algorithms shows that the CBIM algorithm performs better than the state-of-the-art. The simulation studies have shown that CBIM can detect a set of most influential nodes on real-time datasets under various settings and environments.

Achieving Differential Privacy Publishing of Location-Based Statistical Data Using Grid Clustering

Statistical partitioning and publishing is commonly used in location-based big data services to address queries such as the number of points of interest, available vehicles, traffic flows, infected patients, etc., within a certain range. Adding noise perturbation to the location-based statistical data according to the differential privacy model can reduce various risks caused by location privacy leakage while keeping the statistical characteristics of the published data. The traditional statistical partitioning and publishing methods realize the decomposition and indexing of 2D space from top to bottom. However, they can easily cause the over-partitioning or under-partitioning phenomenon, and therefore need multiple times of data scan. This paper proposes a grid clustering and differential privacy protection method for location-based statistical big data publishing scenarios. We implement location-based big data statistics in units of equal-sized grids and perform density classification on uniformly distributed grids by discrete wavelet transform. A bottom-up grid clustering algorithm is designed to perform on the blank and the uniform grids of the same density level based on neighborhood similarity. The Laplacian noise is incorporated into the clustering results according to the differential privacy model to form the published statistics. Experimental comparison of the real-world datasets manifests that the grid clustering and differential privacy publishing method proposed in this paper is superior to other existing partition publishing methods in terms of range querying accuracy and algorithm operating efficiency.

Clustered vector textures

Repetitive vector patterns are common in a variety of applications but can be challenging and tedious to create. Existing automatic synthesis methods target relatively simple, unstructured patterns such as discrete elements and continuous Bézier curves. This paper proposes an algorithm for generating vector patterns with diverse shapes and structured local interactions via a sample-based representation. Our main idea is adding explicit clustering as part of neighborhood similarity and iterative sample optimization for more robust sample synthesis and pattern reconstruction. The results indicate that our method can outperform existing methods on synthesizing a variety of structured vector textures. Our project page is available at https://phtu-cs.github.io/cvt-sig22/.

DeepMNE: Deep Multi-Network Embedding for lncRNA-Disease Association Prediction.

Long non-coding RNA (lncRNA) participates in various biological processes, hence its mutations and disorders play an important role in the pathogenesis of multiple human diseases. Identifying disease-related lncRNAs is crucial for the diagnosis, prevention, and treatment of diseases. Although a large number of computational approaches have been developed, effectively integrating multi-omics data and accurately predicting potential lncRNA-disease associations remains a challenge, especially regarding new lncRNAs and new diseases. In this work, we propose a new method with deep multi-network embedding, called DeepMNE, to discover potential lncRNA-disease associations, especially for novel diseases and lncRNAs. DeepMNE extracts multi-omics data to describe diseases and lncRNAs, and proposes a network fusion method based on deep learning to integrate multi-source information. Moreover, DeepMNE complements the sparse association network and uses kernel neighborhood similarity to construct disease similarity and lncRNA similarity networks. Furthermore, a graph embedding method is adopted to predict potential associations. Experimental results demonstrate that compared to other state-of-the-art methods, DeepMNE has a higher predictive performance on new associations, new lncRNAs and new diseases. Besides, DeepMNE also elicits a considerable predictive performance on perturbed datasets. Additionally, the results of two different types of case studies indicate that DeepMNE can be used as an effective tool for disease-related lncRNA prediction. The code of DeepMNE is freely available at https://github.com/Mayingjun20179/ DeepMNE.

O-014 An expected benefit analysis of using an interpretable machine learning model for gonadotropin starting dose selection during ovarian stimulation

Abstract Study question What is the expected benefit of using a machine learning model for predicting the optimal starting dose of gonadotropin during ovarian stimulation? Summary answer Patients who had an optimal starting gonadotropin dose had improved outcomes and used significantly less total FSH compared to propensity matched patients who did not. What is known already The relationship between the starting dose of follicle-stimulating hormones (FSH) and ovarian response is complex. In general, too little starting FSH may lead to inadequate follicle recruitment, while too much may lead to excessive response. In completed cycles, there exists conflicting evidence of whether higher doses are beneficial or detrimental to the number of oocytes retrieved. The field of assisted reproduction has begun to apply machine learning techniques to clinical decision support for ovarian stimulation, but no studies have specifically investigated optimizing starting FSH dose selection. Study design, size, duration We performed a retrospective analysis of patients undergoing autologous, non-cancelled IVF cycles from 2014 - 2020 (n = 18,591) at three different IVF clinics in the United States. The primary outcomes were the average number of MIIs, 2PNs, and usable blastocysts in relation to starting and total doses of FSH. Participants/materials, setting, methods A K-nearest neighbor similarity model was trained on all cycles and used to identify the 100 most similar patients to a patient-of-interest using age, BMI, baseline anti-mullerian hormone (AMH), and baseline antral follicle count (AFC). For each patient, a patient-specific dose response curve was created by fitting a constrained second order polynomial to the number of MII oocytes relative to the starting dose of FSH across all of the neighbors. Main results and the role of chance For each patient, their individual dose response curve was used to determine if there was an optimal dose that maximizes the prediction of MIIs (called dose-responsive patients), or if the dose response curve shows no optimal dose (called non-responsive patients). 30% of cycles were identified as dose-responsive, 64% were identified as non-responsive, and 6% were inconclusive and excluded from analysis. Dose-responsive patients who received an optimal starting dose had, on average, 1.5 more MIIs, 1.0 more 2PNs, and 0.5 more usable blastocysts using 10 IU’s less of starting FSH and 195 IU’s less of total FSH compared to propensity-matched patients with non-optimal doses. Non-responsive patients who received a low starting dose had, on average, 0.3 more MIIs, 0.4 more 2PNs, and 0.3 more usable blastocysts using 150 IU’s less of FSH and 1375 IU’s less of total FSH compared to propensity-matched patients with a high starting dose. Limitations, reasons for caution The primary limitation is the retrospective nature of this study. Further, our calculations of starting FSH combined the contribution of pure FSH plus the FSH component of FSH/LH medication, rather than evaluating each separately. We also did not differentiate between types of protocols, as the majority were antagonist cycles. Wider implications of the findings Our results suggest that a patient similarity model for selecting starting FSH can help increase MII outcomes while reducing the amount of FSH given to a patient. Future work will include continuing to increase the diversity of our dataset and performing validation studies to show improved outcomes with model use. Trial registration number Not applicable

Category-Specific Nuance Exploration Network for Fine-Grained Object Retrieval

Employing additional prior knowledge to model local features as a final fine-grained object representation has become a trend for fine-grained object retrieval (FGOR). A potential limitation of these methods is that they only focus on common parts across the dataset (e.g. head, body or even leg) by introducing additional prior knowledge, but the retrieval of a fine-grained object may rely on category-specific nuances that contribute to category prediction. To handle this limitation, we propose an end-to-end Category-specific Nuance Exploration Network (CNENet) that elaborately discovers category-specific nuances that contribute to category prediction, and semantically aligns these nuances grouped by subcategory without any additional prior knowledge, to directly emphasize the discrepancy among subcategories. Specifically, we design a Nuance Modelling Module that adaptively predicts a group of category-specific response (CARE) maps via implicitly digging into category-specific nuances, specifying the locations and scales for category-specific nuances. Upon this, two nuance regularizations are proposed: 1) semantic discrete loss that forces each CARE map to attend to different spatial regions to capture diverse nuances; 2) semantic alignment loss that constructs a consistent semantic correspondence for each CARE map of the same order with the same subcategory via guaranteeing each instance and its transformed counterpart to be spatially aligned. Moreover, we propose a Nuance Expansion Module, which exploits context appearance information of discovered nuances and refines the prediction of current nuance by its similar neighbors, leading to further improvement on nuance consistency and completeness. Extensive experiments validate that our CNENet consistently yields the best performance under the same settings against most competitive approaches on CUB Birds, Stanford Cars, and FGVC Aircraft datasets.