Appearance Variations Research Articles

Pattern Analysis of Benign and Malignant Atypical Melanocytic Skin Lesions of Palms and Soles: Variations of Dermoscopic Features According to Anatomic Site and Personal Experience.

Background: The differential diagnosis of atypical melanocytic skin lesions localized on palms and soles represents a diagnostic challenge: indeed, this spectrum encompasses atypical nevi (AN) and early-stage melanomas (EN) displaying overlapping clinical and dermoscopic features. This often generates unnecessary excisions or delayed diagnosis. Investigations to date were mostly carried out in specific populations, focusing either on acrolentiginous melanomas or morphologically typical acquired nevi. Aims: To investigate the dermoscopic features of atypical melanocytic palmoplantar skin lesions (aMPPLs) as evaluated by variously skilled dermatologists and assess their concordance; to investigate the variations in dermoscopic appearance according to precise location on palms and soles; to detect the features with the strongest association with malignancy/benignity in each specific site. Methods: A dataset of 471 aMPPLs-excised in the suspect of malignancy-was collected from 10 European Centers, including a standardized dermoscopic picture (17×) and lesion/patient metadata. An anatomical classification into 17 subareas was considered, along with an anatomo-functional classification considering pressure/friction, (4 macroareas). A total of 156 participants (95 with less than 5 years of experience in dermoscopy and 61 with ≥than 5 years) from 17 countries performed a blinded tele-dermoscopic pattern analysis over 20 cases through a specifically realized web platform. Results: A total of 37,440 dermoscopic evaluations were obtained over 94 (20%) EM and 377 (80%) AN. The areas with the highest density of EM compared to AN were the heel (40.3% EM/aMPPLs) of the sole and the "fingers area" (33%EM/aMPPLs) of the palm, both characterized by intense/chronic traumatism/friction. Globally, the recognition rates of 12 dermoscopic patterns were non statistically different between 95 dermatology residents and 61 specialists: aMPPLs in the plantar arch appeared to be the most "difficult" to diagnose, the parallel ridge pattern was poorly recognized and irregular/regular fibrillar patterns often misinterpreted. Regarding the aMPPL of the "heel area", the parallel furrow pattern (p = 0.014) and lattice-like pattern (p = 0.001) significantly discriminated benign cases, while asymmetry of colors (p = 0.002) and regression structures (p = 0.025) malignant ones. In aMPPLs of the "plantar arch", the lattice-like pattern (p = 0.012) was significant for benignity and asymmetry of structures, asymmetry of colors, regression structures, or blue-white veil for malignancy. In palmar lesions, no data were significant in the discrimination between malignant and benign aMPPLs. Conclusions: This study highlights that (i) the pattern analysis of aMPPLs is challenging for both experienced and novice dermoscopists; (ii) the histological distribution varies according to the anatomo-functional classification; and (iii) different dermoscopic patterns are able to discriminate malignant from benign aMPPLs within specific plantar and palmar areas.

DLA-Net: dual lesion attention network for classification of pneumoconiosis using chest X-ray images

Accurate and early detection of pneumoconiosis using chest X-rays (CXR) is important for preventing the progression of this incurable disease. It is also a challenging task due to large variations in appearance, size and location of lesions in the lung regions as well as inter-class similarity and intra-class variance. Compared to traditional methods, Convolutional Neural Networks-based methods have shown improved results; however, these methods are still not applicable in clinical practice due to limited performance. In some cases, limited computing resources make it impractical to develop a model using whole CXR images. To address this problem, the lung fields are divided into six zones, each zone is classified separately and the zone classification results are then aggregated into an image classification score, based on state-of-the-art. In this study, we propose a dual lesion attention network (DLA-Net) for the classification of pneumoconiosis that can extract features from affected regions in a lung. This network consists of two main components: feature extraction and feature refinement. Feature extraction uses the pre-trained Xception model as the backbone to extract semantic information. To emphasise the lesion regions and improve the feature representation capability, the feature refinement component uses a DLA module that consists of two sub modules: channel attention (CA) and spatial attention (SA). The CA module focuses on the most important channels in the feature maps extracted by the backbone model, and the SA module highlights the spatial details of the affected regions. Thus, both attention modules combine to extract discriminative and rich contextual features to improve classification performance on pneumoconiosis. Experimental results show that the proposed DLA-Net outperforms state-of-the-art methods for pneumoconiosis classification.

Perceiving Actions via Temporal Video Frame Pairs

Video action recognition aims at classifying the action category in given videos. In general, semantic-relevant video frame pairs reflect significant action patterns such as object appearance variation and abstract temporal concepts like speed, rhythm, and so on. However, existing action recognition approaches tend to holistically extract spatiotemporal features. Though effective, there is still a risk of neglecting the crucial action features occurring across frames with a long-term temporal span. Motivated by this, in this article, we propose to perceive actions via frame pairs directly and devise a novel Nest Structure with frame pairs as basic units. Specifically, we decompose a video sequence into all possible frame pairs and hierarchically organize them according to temporal frequency and order, thus transforming the original video sequence into a Nest Structure. Through naturally decomposing actions, the proposed structure can flexibly adapt to diverse action variations such as speed or rhythm changes. Next, we devise a Temporal Pair Analysis module (TPA) to extract discriminative action patterns based on the proposed Nest Structure. The designed TPA module consists of a pair calculation part to calculate the pair features and a pair fusion part to hierarchically fuse the pair features for recognizing actions. The proposed TPA can be flexibly integrated into existing backbones, serving as a side branch to capture various action patterns from multi-level features. Extensive experiments show that the proposed TPA module can achieve consistent improvements over several typical backbones, reaching or updating CNN-based SOTA results on several challenging action recognition benchmarks.

CRTrack: Learning Correlation-Refine network for visual object tracking

Conducting reliable feature interaction plays a critical role in the visual tracking community, especially in recent dominated Siamese-based tracking paradigm. In general, there are two primary approaches for fusing representations from template and search area in the Siamese setting, i.e., cross-correlation and transformer modeling. The former provides a straightforward interaction solution, which may have limitations in handling complex scenarios, such as appearance variations and occlusion. While the latter offers an effective interaction mechanism, albeit with higher computation complexity and model cost. In contrast to traditional Siamese-based trackers which rely on two mentioned feature cross-correlation operators, this paper proposes a novel Correlation-Refine network to address the issue of lacking semantic information caused by local linear matching in correlation, from both spatial and channel perspectives. Correlation-Refine network (named CR) is solely built on top of fully convolutional layers, without employing intricate transformer mechanisms or complex methods to fuse features from multiple scales. Moreover, we present a concise yet effective convolutional tracking framework based on the correlation-refine network. CR network can increase the discriminative ability of semantic information in a coarse-to-fine manner: it gradually learns the semantic features of the target to be tracked and suppresses interference from similar objects by stacking multiple CR layers. Extensive experiments and comparisons with recent competitive trackers in challenging large-scale benchmarks demonstrate that, our tracker outperforms all previous convolutional trackers and has competitive results with transformer-based method. The code will be made available.

Multi-Camera Multi-Vehicle Tracking Guided by Highway Overlapping FoVs

Multi-Camera Multi-Vehicle Tracking (MCMVT) is a critical task in Intelligent Transportation Systems (ITS). Differently to in urban environments, challenges in highway tunnel MCMVT arise from the changing target scales as vehicles traverse the narrow tunnels, intense light exposure within the tunnels, high similarity in vehicle appearances, and overlapping camera fields of view, making highway MCMVT more challenging. This paper presents an MCMVT system tailored for highway tunnel roads incorporating road topology structures and the overlapping camera fields of view. The system integrates a Cascade Multi-Level Multi-Target Tracking strategy (CMLM), a trajectory refinement method (HTCF) based on road topology structures, and a spatio-temporal constraint module (HSTC) considering highway entry–exit flow in overlapping fields of view. The CMLM strategy exploits phased vehicle movements within the camera’s fields of view, addressing such challenges as those presented by fast-moving vehicles and appearance variations in long tunnels. The HTCF method filters static traffic signs in the tunnel, compensating for detector imperfections and mitigating the strong lighting effects caused by the tunnel lighting. The HSTC module incorporates spatio-temporal constraints designed for accurate inter-camera trajectory matching within overlapping fields of view. Experiments on the proposed Highway Surveillance Traffic (HST) dataset and CityFlow dataset validate the system’s effectiveness and robustness, achieving an IDF1 score of 81.20% for the HST dataset.

Age-Invariant Face Recognition Using FaceNet and Multi-task Cascaded Convolutional Networks (MTCNN)

In an era where face recognition technology serves as a cornerstone for security, surveillance, and human-computer interaction, the persistence of age-related variations in facial appearance presents a significant challenge. This research introduces an innovative solution that amalgamates the capabilities of the FaceNet deep learning model with Multi-task Cascaded Convolutional Networks (MTCNN) to achieve precise and robust face recognition across diverse age groups. Leveraging FaceNet's prowess in extracting distinctive features from facial images and mapping them to a high-dimensional feature space for efficient face matching, our system incorporates MTCNN as a pre-processing step to accurately detect and align faces, effectively mitigating age-related changes in facial geometry. Crucially, this novel approach obviates the need for age-specific databases and age group categorization, rendering it a versatile and practical solution for age-invariant face recognition. Rigorous experimentation on benchmark datasets underscores the system's resilience and accuracy across a broad spectrum of age groups, culminating in state-of-the-art results in age-invariant face recognition. The implications of this proposed approach are profound, promising to fortify security measures and enhance user experiences in applications such as access control, personal identification, and customer service, while ensuring dependable and accurate face recognition, irrespective of an individual's age. Key Words: Face recognition, face aging, generative adversarial networks

Relevance of the Foramen of Vesalius for Preoperative Tumor Embolization in Skull Base Meningioma

ObjectiveTo investigate the role of the foramen of Vesalius (FV) in the pathogenesis of skull base meningioma by analyzing data from various multi-image modalities. MethodsFor this single-center retrospective study, 39 consecutive patients with skull base meningioma who underwent tumor resection between January 2020 and March 2023 were enrolled. The anatomical and pathological characteristics of the FV were evaluated using computed tomography (CT) and three-dimensional (3D)-digital subtraction angiography (DSA). The clinical significance of the FV in tumor hemodynamics and treatment, such as preoperative tumor embolization, was investigated using the 3D-DSA/CT fusion images. ResultsWe identified FV in 52% (17/27) of the finally included patients. In 10 (30%) patients, the FV was found bilaterally with no significant variation in appearance between the healthy and tumor-affected sides (p=0.786). The mean FV diameter was significantly larger on the tumor-affected side (p=0.010). No significant anatomical differences, like duplication and partial assimilation with the foramen ovale, were observed between the two sides. The FV was involved in venous skull base perfusion around the tumor in nine cases. In four cases where it was the pathway for tumor feeders, preoperative tumor embolization via the FV resulted in disappearance of the tumor stain. No complications associated with endovascular treatment were observed. ConclusionsThis study elucidated the anatomical asymmetry of the FV and its role in the hemodynamics of skull base meningioma. Our findings highlight the significance of performing anatomical and pathological evaluations of the FV in determining treatment strategies, including preoperative embolization, for skull base lesions.

One-shot neuroanatomy segmentation through online data augmentation and confidence aware pseudo label

Recently, deep learning-based brain segmentation methods have achieved great success. However, most approaches focus on supervised segmentation, which requires many high-quality labeled images. In this paper, we pay attention to one-shot segmentation, aiming to learn from one labeled image and a few unlabeled images. We propose an end-to-end unified network that joints deformation modeling and segmentation tasks. Our network consists of a shared encoder, a deformation modeling head, and a segmentation head. In the training phase, the atlas and unlabeled images are input to the encoder to get multi-scale features. The features are then fed to the multi-scale deformation modeling module to estimate the atlas-to-image deformation field. The deformation modeling module implements the estimation at the feature level in a coarse-to-fine manner. Then, we employ the field to generate the augmented image pair through online data augmentation. We do not apply any appearance transformations cause the shared encoder could capture appearance variations. Finally, we adopt supervised segmentation loss for the augmented image. Considering that the unlabeled images still contain rich information, we introduce confidence aware pseudo label for them to further boost the segmentation performance. We validate our network on three benchmark datasets. Experimental results demonstrate that our network significantly outperforms other deep single-atlas-based and traditional multi-atlas-based segmentation methods. Notably, the second dataset is collected from multi-center, and our network still achieves promising segmentation performance on both the seen and unseen test sets, revealing its robustness. The source code will be available at https://github.com/zhangliutong/brainseg.

ShellNeRF: Learning a Controllable High‐resolution Model of the Eye and Periocular Region

AbstractEye gaze and expressions are crucial non‐verbal signals in face‐to‐face communication. Visual effects and telepresence demand significant improvements in personalized tracking, animation, and synthesis of the eye region to achieve true immersion. Morphable face models, in combination with coordinate‐based neural volumetric representations, show promise in solving the difficult problem of reconstructing intricate geometry (eyelashes) and synthesizing photorealistic appearance variations (wrinkles and specularities) of eye performances. We propose a novel hybrid representation ‐ ShellNeRF ‐ that builds a discretized volume around a 3DMM face mesh using concentric surfaces to model the deformable ‘periocular’ region. We define a canonical space using the UV layout of the shells that constrains the space of dense correspondence search. Combined with an explicit eyeball mesh for modeling corneal light‐transport, our model allows for animatable photorealistic 3D synthesis of the whole eye region. Using multi‐view video input, we demonstrate significant improvements over state‐of‐the‐art in expression re‐enactment and transfer for high‐resolution close‐up views of the eye region.

Modelling appearance variations in expressive and neutral face image for automatic facial expression recognition

AbstractIn automatic facial expression recognition (AFER) systems, modelling the spatio‐temporal feature information in a specific manner, coalescing, and its effective utilization is challenging. The state‐of‐the‐art studies have examined integrating multiple features to enhance the recognition rate of AFER systems. However, the feature variations between expressive and neutral face images are not fully explored to identify the expression class. The proposed research presents an innovative approach to AFER by modelling appearance variations in both expressive and neutral face images. The prominent contributions of the work are developing a novel and hybrid feature space by integrating the discriminative feature distribution derived from expressive and neutral face images; preserving the highly discriminative latent feature distribution using autoencoders. Local binary pattern (LBP) and histogram of oriented gradients (HOG) are the feature descriptors employed to derive the discriminative texture and shape information, respectively. The component‐based approach is employed, wherein the features are derived from the salient facial regions instead of the whole face. The three‐stage stacked deep convolutional autoencoder (SDCA) and multi‐class support vector machine (MSVM) are employed to address dimensionality reduction and classification, respectively. The efficacy of the proposed model is substantiated by empirical findings, which establish its superiority in terms of accuracy in AFER tasks on widely recognized benchmark datasets.

Advancing Fruit Image Classification with State-of-the-Art Deep Learning Techniques

Fruit image classification technology using deep learning is making significant contributions in the agriculture and food retail sectors, promising to increase efficiency and productivity. However, there is an identified knowledge gap in dealing with the considerable variation in fruit appearance caused by factors such as type, size, color, and lighting conditions, as well as the precise identification of damage or disease. This research focuses on applying the developed Convolutional Neural Network architecture to fill this gap by using it in an extensive and diverse dataset, covering 67,692 image files categorized into 131 fruit classes. The training process showed substantial accuracy improvement, with training accuracy reaching 98.39% and validation accuracy at 90%, while training loss decreased to 0.0430 and validation loss to 0.2991. In the advanced stage of training, the training accuracy peaked at 99.43% in the 59th epoch with a shallow loss of 0.0251. However, the validation loss showed variation, indicating room for improvement in model generalization. The findings provide insight into the potential and challenges of applying Convolutional Neural Network models and fruit image classification with improved fruit sorting accuracy. Contribution to the literature in the field of information technology and agriculture by showing deep learning models can be improved to address the issue of fruit image variability.

Robust ensemble person reidentification via orthogonal fusion with occlusion handling

Occlusion remains one of the major challenges in person reidentification (ReID) due to the diversity of poses and the variation of appearances. Developing novel architectures to improve the robustness of occlusion-aware person Re-ID requires new insights, especially on low-resolution edge cameras. We propose a deep ensemble model that harnesses both CNN and Transformer architectures to generate robust feature representations. To achieve robust Re-ID without manually labeling occluded regions, we propose to take an ensemble learning-based approach derived from the analogy between arbitrarily shaped occluded regions and robust feature representation. Using the orthogonality principle, our developed deep CNN model uses masked autoencoder (MAE) and global–local feature fusion for robust person identification. Furthermore, we present a part occlusion-aware transformer capable of learning feature space that is robust to occluded regions. Experimental results are reported on several Re-ID datasets to show the effectiveness of our developed ensemble model named orthogonal fusion with occlusion handling (OFOH). Compared to competing methods, the proposed OFOH approach has achieved competent rank-1 and mAP performance.

Electroencephalography as an objective method for assessing subjective emotions during the application of cream.

To compare emotional responses elicited by four cosmetic products on different sensory modalities (smell, visual, and touch), and analyze the link between objective instrumental analysis results and subjective evaluation of participants occurring within dimensional valence-arousal model of emotions. In this study, four cream products exhibiting variations in olfactory perception, visual appearance and perception usability were selected. Electroencephalography (EEG) and a subjective emotion scale were used to assess participants' emotional responses during the sensory experience of utilizing the creams. The study revealed that the objective emotional valence and arousal of different cream products exhibited certain variations at distinct stages of usage. The trend of valence differences induced by different products measured by EEG at the same stage was almost as same as measured by subjective evaluation. The correspondence between the valence measured by EEG closely approximated that obtained through subjective evaluation across various products at distinct stages of usage. These findings demonstrate a significant correlation between EEG-based valence and subjective valence, however, no such relationship was observed for arousal. This study demonstrates the feasibility of using EEG as a method to assess emotions elicited by various stages of cosmetics application, including smelling, looking, rubbing, and afterfeel. This technique serves as a valuable supplement to traditional methods for examining emotional responses by providing more objective evidence.

Vulvar Lichenoid Dermatoses With Emphasis on the Distinction Between Lichen Sclerosus and Lichen Planus: A 10-Year Study.

Lichen planus (LP) and lichen sclerosus (LS) are the most common vulvar lichenoid dermatoses. The diagnostic challenges are due to site-specific variation in microscopic appearance and small-sized biopsies. Authentication of diagnostic criteria to distinguish LS and LP to uncover any resemblance or divergence in presentation of these conditions is attempted. Cases of vulvar LP and LS diagnosed between January 2012 to December 2022 were included. The clinical details included age, presenting symptoms, examination findings, and other organ involvement. Histopathological analysis of epidermal, dermal, and adnexal findings was done. There were 28 cases of vulvar LP and 72 cases of LS, with a median age of 51 and 60 years, respectively. Depigmentation and atrophy were the major clinical features in LS, whereas ulcers/erosions and erythema were more prevalent in LP with a significantly higher incidence of oral involvement. The most diagnostic feature in LS was diffuse dermal sclerosis (76.8%) and interstitial pattern of inflammation (81.4%), whereas the characteristic features in LP cases was a lichenoid pattern of inflammation (85.7%), necrotic keratinocytes, and lymphocytic exocytosis. In 44.4% of LS, unconventional features like compact orthokeratosis, parakeratosis, thickened/wedge-shaped hypergranulosis, and sawtooth rete pegs were noted. Lichen sclerosus with lichenoid inflammation (21.4%) mimicked LP, from which it was distinguished by presence of thickened or diminished granular layer with basal melanin absence (60%) and dermal homogenization (80%). Although the classical, well-established variant of LS poses no diagnostic difficulty, the unconventional variant may mimic LP. Identification of the subtle histological clues demonstrated in this study can help to arrive at the correct diagnosis.

A Hierarchical Utilization of Semantic Gradients and Scene Structure for Visual Place Recognition

Visual Place Recognition (VPR) is a fundamental element for long-term Simultaneous Localization and Mapping (SLAM) systems. For long-term VPR, severe appearance and viewpoint variations are inevitable. In this paper, we introduce a novel VPR system named Semantic Scene Structure Place Recognition (3SPR), inspired by the repeatability of semantic gradients and the scene structure of urban environments. Semantic gradients are densely sampled according to the sum of absolute gradients of all channels in the logits layer. Features of the semantic gradients in different layers are concatenated to exploit features’ characteristics at different levels. Based on partitions by vanishing points of road lines and Vector of Locally Aggregated Descriptors (VLAD), the Scene Structure VLAD (SSVLAD) is generated from concatenated features of the semantic gradients. Moreover, a local point group match method is used to enhance the spatial verification. Experimental results show that our method achieves state-of-the-art performance on the Oxford Robotcar dataset and the Synthia dataset.

Fabrication and Characterization of Non-Conventional Starch Based Biofilms Employing Different Plasticizers

Increasing stress on environmental awareness has promoted the use of alternative strategies for common petrochemical based plastic sources to environmentally friendly biofilms and packaging materials. Food by products and underutilised indigenous plants sources are rich in biopolymers which can naturally decompose. The present study focusses on preparations of bioplastics from unconventional starch sources of litchi seed and churkha tuber as novel initiative to combat environmental pollution along with valorisation of undervalued plant sources and agrobased wastes. Ternary blend films of poly vinyl alcohol, starch, plasticizers like (glycerol, sorbitol, mannitol, propylene glycol, polyethylene glycol), citric acid prepared by casting technique produced best result. Comparative study of test samples (TH11, TH12, TH13, TH14 etc) with standardized thin films (TH1, TH2, TH3 etc.) showed variation in appearance, water activity, thickness, since the starch differ in their compositions, gelatinization and gelation time and temperature. High amylose containing litchi seed starch forms firm and more flexible films in comparison to low amylose containing churkha tuber starch. Maximum hygroscopicity was recorded in case of TH1 (77.4%) while thickness was maximum in case of TH12 (0.18 mm). Micro structural analysis clearly showed the nature of crosslinking between starch molecule and different plasticizers. The sensory analysis of food packed with the best among the formularized thin films showed mild change in taste while the other parameters remain same as normal. Better acceptability was in case of TH12 wrapped food rather than TH1. Thus, fabrication of biodegradable packaging is considered as the most sustainable alternative for food preservation. Int. J. Appl. Sci. Biotechnol. Vol 12(1): 29-38.

A Study of YOLO-Based Object Detection for Visually Impaired Individuals

Abstract: An essential area of investigation involves recognizing and tracking objects, given the frequent changes in object motion, scene dimensions, occlusions, appearance, ego-motion, and lighting variations. Effective object tracking heavily relies on feature selection due to its critical role. This process is integral to numerous real-time applications like vehicle detection and video surveillance. To address detection challenges, tracking methods need to be adept at handling object movement and appearance changes. Among these methods, tracking algorithms play a pivotal role in smoothing video sequences and receive significant attention. However, only a few approaches leverage previously gathered data on object attributes such as shape and color. This study delves into a tracking algorithm that encompasses all these object properties. Its aim is to investigate and assess past methodologies for tracking and detecting moving objects across various stages of video sequences. Additionally, it aims to identify existing challenges and propose innovative strategies to enhance object tracking throughout video frames

An Appearance-Semantic Descriptor with Coarse-to-Fine Matching for Robust VPR.

In recent years, semantic segmentation has made significant progress in visual place recognition (VPR) by using semantic information that is relatively invariant to appearance and viewpoint, demonstrating great potential. However, in some extreme scenarios, there may be semantic occlusion and semantic sparsity, which can lead to confusion when relying solely on semantic information for localization. Therefore, this paper proposes a novel VPR framework that employs a coarse-to-fine image matching strategy, combining semantic and appearance information to improve algorithm performance. First, we construct SemLook global descriptors using semantic contours, which can preliminarily screen images to enhance the accuracy and real-time performance of the algorithm. Based on this, we introduce SemLook local descriptors for fine screening, combining robust appearance information extracted by deep learning with semantic information. These local descriptors can address issues such as semantic overlap and sparsity in urban environments, further improving the accuracy of the algorithm. Through this refined screening process, we can effectively handle the challenges of complex image matching in urban environments and obtain more accurate results. The performance of SemLook descriptors is evaluated on three public datasets (Extended-CMU Season, Robot-Car Seasons v2, and SYNTHIA) and compared with six state-of-the-art VPR algorithms (HOG, CoHOG, AlexNet_VPR, Region VLAD, Patch-NetVLAD, Forest). In the experimental comparison, considering both real-time performance and evaluation metrics, the SemLook descriptors are found to outperform the other six algorithms. Evaluation metrics include the area under the curve (AUC) based on the precision-recall curve, Recall@100%Precision, and Precision@100%Recall. On the Extended-CMU Season dataset, SemLook descriptors achieve a 100% AUC value, and on the SYNTHIA dataset, they achieve a 99% AUC value, demonstrating outstanding performance. The experimental results indicate that introducing global descriptors for initial screening and utilizing local descriptors combining both semantic and appearance information for precise matching can effectively address the issue of location recognition in scenarios with semantic ambiguity or sparsity. This algorithm enhances descriptor performance, making it more accurate and robust in scenes with variations in appearance and viewpoint.

Multi-Prototype Space Learning for Commonsense-Based Scene Graph Generation

In the domain of scene graph generation, modeling commonsense as a single-prototype representation has been typically employed to facilitate the recognition of infrequent predicates. However, a fundamental challenge lies in the large intra-class variations of the visual appearance of predicates, resulting in subclasses within a predicate class. Such a challenge typically leads to the problem of misclassifying diverse predicates due to the rough predicate space clustering. In this paper, inspired by cognitive science, we maintain multi-prototype representations for each predicate class, which can accurately find the multiple class centers of the predicate space. Technically, we propose a novel multi-prototype learning framework consisting of three main steps: prototype-predicate matching, prototype updating, and prototype space optimization. We first design a triple-level optimal transport to match each predicate feature within the same class to a specific prototype. In addition, the prototypes are updated using momentum updating to find the class centers according to the matching results. Finally, we enhance the inter-class separability of the prototype space through iterations of the inter-class separability loss and intra-class compactness loss. Extensive evaluations demonstrate that our approach significantly outperforms state-of-the-art methods on the Visual Genome dataset.

A causality-inspired generalized model for automated pancreatic cancer diagnosis

Pancreatic cancer (PC) is a severely malignant cancer variant with high mortality. Since PC has no obvious symptoms, most PC patients are belatedly diagnosed at advanced disease stages. Recently, artificial intelligence (AI) approaches have demonstrated promising prospects for early diagnosis of pancreatic cancer. However, certain non-causal factors (such as intensity and texture appearance variations, also called confounders) tend to induce spurious correlation with PC diagnosis. This undermines the generalization performance and the clinical applicability of the AI-based PC diagnosis approaches. Therefore, we propose a causal intervention based automated method for pancreatic cancer diagnosis with contrast-enhanced computerized tomography (CT) images, where a confounding effects reduction scheme is developed for alleviating spurious correlations to achieve unbiased learning, thereby improving the generalization performance. Specifically, a continuous image generation strategy was developed to simulate wide variations of intensity differences caused by imaging heterogeneities, where Monte Carlo sampling is added to further enhance the continuity of simulated images. Then, to enhance the pancreatic texture variability, a texture diversification method was introduced in conjunction with gradient-based data augmentation. Finally, a causal intervention strategy was proposed to alleviate the adverse confounding effects by decoupling the causal and non-causal factors and combining them randomly. Extensive experiments showed remarkable diagnosis performance on a cross-validation dataset. Also, promising generalization performance with an average accuracy of 0.87 was attained on three independent test sets of a total of 782 subjects. Therefore, the proposed method shows high clinical feasibility and applicability for pancreatic cancer diagnosis.