Object Appearance Research Articles

The Role of Artificial Intelligence in the Culture of the Tourism Industry of the Modern World

The trend in using artificial intelligence to promote business processes and form marketing campaigns is increasingly being used in the travel industry. Artificial intelligence is a set of programs that can simulate human skills, for example, planning, solving specific tasks, learning and improving its functionality as information accumulates. AI technologies are improving and transforming travel in all sorts of ways. Offers of personal recommendations are based on a systematic selection of information and improving the quality of customer service. Virtual assistants and optimization of operational efficiency improve the booking service of travel products. When examining the importance of artificial intelligence in the development of the modern tourism industry, one can note both advantages – this is data analysis and its transformation, improving the quality of service in travel, ensuring the preservation of historical sites, and disadvantages – digital replication, the use of neural networks to change the appearance of objects or even entire cities, which in the future can lead to a distorted view of tourists on a particular tourist destination.

End-to-end multiple object tracking in high-resolution optical sensors of drones with transformer models

Drone aerial imaging has become increasingly important across numerous fields as drone optical sensor technology continues to advance. One critical challenge in this domain is achieving both accurate and efficient multi-object tracking. Traditional deep learning methods often separate object identification from tracking, leading to increased complexity and potential performance degradation. Conventional approaches rely heavily on manual feature engineering and intricate algorithms, which can further limit efficiency. To overcome these limitations, we propose a novel Transformer-based end-to-end multi-object tracking framework. This innovative method leverages self-attention mechanisms to capture complex inter-object relationships, seamlessly integrating object detection and tracking into a unified process. By utilizing end-to-end training, our approach simplifies the tracking pipeline, leading to significant performance improvements. A key innovation in our system is the introduction of a trajectory detection label matching technique. This technique assigns labels based on a comprehensive assessment of object appearance, spatial characteristics, and Gaussian features, ensuring more precise and logical label assignments. Additionally, we incorporate cross-frame self-attention mechanisms to extract long-term object properties, providing robust information for stable and consistent tracking. We further enhance tracking performance through a newly developed self-characteristics module, which extracts semantic features from trajectory information across both current and previous frames. This module ensures that the long-term interaction modules maintain semantic consistency, allowing for more accurate and continuous tracking over time. The refined data and stored trajectories are then used as input for subsequent frame processing, creating a feedback loop that sustains tracking accuracy. Extensive experiments conducted on the VisDrone and UAVDT datasets demonstrate the superior performance of our approach in drone-based multi-object tracking.

IVA-former: invisible–visible query guided amodal mask measurement network for desktop object via hierarchical transformer

Abstract Instance segmentation of desktop objects is important for service robots. Most of the previous works for desktop environments are restricted to measuring the visible area of target objects. However, when a target object is placed behind another, the algorithm that only performs visible area segmentation is unable to provide accurate appearance information for the occluded objects. To solve this problem, we propose the invisible–visible query guided amodal mask measurement network based on a hierarchical transformer for desktop scenes, which can perceive the entire appearance of objects in the presence of occlusions. In this method, the RGB-D backbone is adopted to fuse the features from both RGB and depth images. Then, the pixel decoder is used to generate multi-scale feature maps. We then adopt a hierarchical transformer decoder to predict invisible, visible, and amodal masks simultaneously. To enhance the associations between the three prediction branches, we propose a query transform module to transfer object queries between adjacent branches. Since amodal masks are a combination of invisible and visible masks, we propose an invisible–visible mixture loss that takes masks from both invisible and visible branches to further supervise the network. Our method is trained on synthetic datasets for desktop objects and evaluated on both visible and amodal real-world datasets. Compared to other recent segmentation algorithms, our method achieves competitive performance.

Has the cat got your tongue, or is something obstructing your throat? A review of imaging of ingested and aspirated foreign bodies in the paediatric population.

Children frequently swallow or inhale foreign objects, a situation that can be life-threatening. Radiographic imaging plays a lead role in the early identification and location of inhaled or swallowed objects is essential. Promptly identifying and locating inhaled or swallowed objects are essential, as some items require immediate removal. For example, button batteries in the throat can cause grave harm; magnets can attract each other through the gut and cause perforations; and other objects can obstruct the airway or intestinal tract. Radiologists must understand how these objects appear in images to assist doctors in treating patients effectively. Recognising signs of inhaled objects on radiographs is also crucial, as symptoms may not always be clear, and many inhaled objects are not visible on radiographs. Radiographs are the primary means of checking for swallowed or inhaled objects, although other tests like fluoroscopy and computed tomography may be used in complex cases. Doctors working with children should be acquainted with the appearance of these common objects on images and their clinical importance.

A Hybrid Approach for Robust Object Detection: Integrating Template Matching and Faster R-CNN

Object detection is a critical task in computer vision, with applications ranging from autonomous vehicles to medical imaging. Traditional methods like template matching offer precise localization but struggle with variations in object appearance, while deep learning approaches such as Faster R-CNN excel in handling diverse and complex datasets but often require extensive computational resources and large amounts of labeled data. This paper proposes a hybrid approach that integrates template matching with Faster R-CNN to leverage the strengths of both techniques. By combining the accuracy of template matching with the robustness and generalization capabilities of Faster R-CNN, our method achieves superior performance in challenging scenarios, including objects with occlusions, varying scales, and complex backgrounds. Extensive experiments demonstrate that the hybrid model not only enhances detection accuracy but also reduces computational load, making it a practical solution for real-world applications.

Object and spatial discrimination makes weakly supervised local feature better

Local feature extraction plays a crucial role in numerous critical visual tasks. However, there remains room for improvement in both descriptors and keypoints, particularly regarding the discriminative power of descriptors and the localization precision of keypoints. To address these challenges, this study introduces a novel local feature extraction pipeline named OSDFeat (Object and Spatial Discrimination Feature). OSDFeat employs a decoupling strategy, training descriptor and detection networks independently. Inspired by semantic correspondence, we propose an Object and Spatial Discrimination ResUNet (OSD-ResUNet). OSD-ResUNet captures features from the feature map that differentiate object appearance and spatial context, thus enhancing descriptor performance. To further improve the discriminative capability of descriptors, we propose a Discrimination Information Retained Normalization module (DIRN). DIRN complementarily integrates spatial-wise normalization and channel-wise normalization, yielding descriptors that are more distinguishable and informative. In the detection network, we propose a Cross Saliency Pooling module (CSP). CSP employs a cross-shaped kernel to aggregate long-range context in both vertical and horizontal dimensions. By enhancing the saliency of keypoints, CSP enables the detection network to effectively utilize descriptor information and achieve more precise localization of keypoints. Compared to the previous best local feature extraction methods, OSDFeat achieves Mean Matching Accuracy of 79.4% in local feature matching task, improving by 1.9% and achieving state-of-the-art results. Additionally, OSDFeat achieves competitive results in Visual Localization and 3D Reconstruction. The results of this study indicate that object and spatial discrimination can improve the accuracy and robustness of local feature, even in challenging environments. The code is available at https://github.com/pandaandyy/OSDFeat.

HyperSOR: Context-Aware Graph Hypernetwork for Salient Object Ranking.

Salient object ranking (SOR) aims to segment salient objects in an image and simultaneously predict their saliency rankings, according to the shifted human attention over different objects. The existing SOR approaches mainly focus on object-based attention, e.g., the semantic and appearance of object. However, we find that the scene context plays a vital role in SOR, in which the saliency ranking of the same object varies a lot at different scenes. In this paper, we thus make the first attempt towards explicitly learning scene context for SOR. Specifically, we establish a large-scale SOR dataset of 24,373 images with rich context annotations, i.e., scene graphs, segmentation, and saliency rankings. Inspired by the data analysis on our dataset, we propose a novel graph hypernetwork, named HyperSOR, for context-aware SOR. In HyperSOR, an initial graph module is developed to segment objects and construct an initial graph by considering both geometry and semantic information. Then, a scene graph generation module with multi-path graph attention mechanism is designed to learn semantic relationships among objects based on the initial graph. Finally, a saliency ranking prediction module dynamically adopts the learned scene context through a novel graph hypernetwork, for inferring the saliency rankings. Experimental results show that our HyperSOR can significantly improve the performance of SOR.

Factors contributing to transient-induced fading: Examining the impact of luminance contrasts and subjective contours.

Transient-induced fading is a phenomenon where a peripheral target perceptually fades when a surrounding object is flashed. It has been suggested that the transient-induced fading could be affected not only by the lower-level factors such as the luminance contrast change, but also by the higher-level factors such as Gestalt grouping by similarity. In the present study, Experiment 1 investigated whether the perceptual fading of a visual target could be strongly induced when a ring area surrounding the target with high luminance contrast disappeared rather than appeared. Experiment 2 examined the effect of the (dis)appearance of a higher-level object (Kanizsa-type subjective contour) on the fading perception. Experiment 3 further investigated whether the rating of the perceived effortlessness of a subjective contour could be positively correlated with the fading duration of the target. Our results revealed that perceptual fading was mainly induced by the disappearance of fan areas inside black disks producing a subjective contour surrounding the target. Disappearance of a perceptual object at the representation level does not trigger the transient-induced fading even if a higher-level factor (e.g., grouping by similarity) affects the fading objects.

A semantic guidance-based fusion network for multi-label image classification

Multi-label image classification (MLIC), a fundamental task assigning multiple labels to each image, has been seen notable progress in recent years. Considering simultaneous appearances of objects in the physical world, modeling object correlations is crucial for enhancing classification accuracy. This involves accounting for spatial image feature correlation and label semantic correlation. However, existing methods struggle to establish these correlations due to complex spatial location and label semantic relationships. On the other hand, regarding the fusion of image feature relevance and label semantic relevance, existing methods typically learn a semantic representation in the final CNN layer to combine spatial and label semantic correlations. However, different CNN layers capture features at diverse scales and possess distinct discriminative abilities. To address these issues, in this paper we introduce the Semantic Guidance-Based Fusion Network (SGFN) for MLIC. To model spatial image feature correlation, we leverage the advanced TResNet architecture as the backbone network and employ the Feature Aggregation Module for capturing global spatial correlation. For label semantic correlation, we establish both local and global semantic correlation. We further enrich model features by learning semantic representations across multiple convolutional layers. Our method outperforms current state-of-the-art techniques on PASCAL VOC (2007, 2012) and MS-COCO datasets.

Analysis of Students' Spatial Abilities Reviewed From Gender Differences In Class X

Spatial skills are important for studying geometric shapes. People who have good spatial abilities can easily imagine objects in three dimensions. Factors that influence differences in student abilities include gender differences. The aim of this research is to describe students' spatial abilities in terms of gender differences. The type of research used is quantitative-qualitative research. Spatial ability test results data are calculated and presented in quantitative form and analyzed qualitatively. This research was conducted at SMA Negeri 3 Ambon class X Science 1 with a total of 36 students consisting of 12 male students and 24 female students. In determining the subject of this research, a purposive sampling technique was used, namely taking data sources with certain considerations. The research subjects selected to be interviewed were 6 students with the assessment criteria for spatial ability test questions consisting of 1 male student and 1 female student with high spatial ability, 1 male student and 1 female student with moderate spatial ability, and 1 male student with moderate spatial ability. male and 1 female student with low spatial abilities. The largest percentage of student test results are in the low category. Based on the results of the spatial ability test, it shows that in the element of spatial perception, male subjects are more likely to meet the indicators of the ability to identify objects vertically and horizontally even though the position of the object is manipulated. Meanwhile, female subjects tend to use their spatial abilities as indicators of the ability to distinguish lines, horizontal and vertical planes in spatial figures. In the spatial visualization element, male subjects are more likely to fulfill the indicators than female subjects regarding stacks of unit cubes whose position or shape has been manipulated and visualizing stacks of unit cubes from more than one point of view. For the element of mental rotation, male subjects tend to be more capable than female subjects of rotating, imagining rotation and describing a flat shape or spatial structure correctly after rotation. However, elements of spatial orientation of female subjects are more likely to determine the appearance of objects from different perspectives than male subjects. In the spatial relations element, male subjects are more likely to determine the relationship of an object with other objects than female subjects.

Incidental learning of predictive temporal context within cortical representations of visual shape.

Abstract Incidental learning of spatio-temporal regularities and consistencies— also termed ‘statistical learning’— may be important for discovering the causal principles governing the world. We studied statistical learning of temporal structure simultaneously at two time-scales: the presentation of synthetic visual objects (3 s) and predictive temporal context (30 s) in the order of appearance of such objects. Visual objects were complex and rotated in three dimensions about varying axes. Observers viewed fifteen (15) objects recurring many times each, intermixed with other objects that appeared only once, while whole-brain BOLD activity was recorded. Over three successive days, observers grew familiar with the recurring objects and reliably distinguished them from others. As reported elsewhere (Kakaei & Braun, 2023), representational similarity analysis (RSA) of multivariate BOLD activity revealed 124 ‘object-selective’ brain parcels with selectivity for recurring objects, located mostly in the ventral occipitotemporal cortex and the parietal cortex. Here we extend RSA to the representation of predictive temporal context, specifically “temporal communities” formed by objects that tended to follow each other. After controlling for temporal proximity, we observed 27 ‘community-sensitive’ brain parcels, in which pairwise distances between multivariate responses reflected community structure, either positively (smaller distances within than between communities) or negatively (larger distances within). Among object-selective parcels, 11 parcels were positively community-sensitive in the primary visual cortex (2 parcels), the ventral occipital, lingual, or fusiform cortex (8 parcels), and the inferior temporal cortex (1 parcel). Among non-object-selective parcels, 12 parcels were negatively community-sensitive in the superior, middle, and medial frontal cortex (6 parcels), the insula (2 parcels), the putamen (1 parcel), and in the superior temporal or parietal cortex (3 parcels). We conclude that cortical representations of object shape and of predictive temporal context are largely coextensive along the ventral occipitotemporal cortex.

Optimized Deep LSTM-Based Multi-Object Tracking with Occlusion Handling Mechanism

The multi-object tracking is a basic computer vision process having a huge class of real-life tools that range from monitoring of medical video to surveillance. The goal of tracking numerous items is to place numerous objects in a scene, handle their identities throughout time, and construct trajectories for analysis. However, this is a complex task, because of certain issues like occlusions, complicated object dynamics, and variations in the appearance of objects. In this research, a new technique named TPRO-based Deep LSTM is developed for tracking multi-object with occlusion handling. Here, the videos are considered as input wherein the extraction of frames is done from each video. Each frame undergoes pre-processing with filtering to eliminate noise from frames. By using a sparse Fuzzy c-Means (FCM) and Local Optimal-Oriented Pattern (LOOP) features, the localization of objects is done. Moreover, the visual and spatial trackings are considered for hybrid tracking. The second derivative model and neighborhood search model are used to perform visual tracking. Then the occlusion handling is performed. Concurrently, with the use of Deep Long Short-Term Memory (Deep LSTM) the spatial tracking is performed and the Taylor Poor Rich Optimization (TPRO) algorithm assigns the weight and bias of the Deep LSTM. The TPRO is obtained by the unification of the Taylor series along with the Poor and Rich Optimization algorithm. By combining visual and spatial tracking, the final tracked output is generated. The devised method achieves a performance with the highest value of 88.9% for Multiple Object Tracking Precision (MOTP), smallest tracking distance (TD) of 4.185, average MOTP of 0.889, average TD of 4.201, and highest tracking number (TN) of 14.

AI-ENHANCED TRACKSEGNET AN ADVANCED MACHINE LEARNING TECHNIQUE FOR VIDEO SEGMENTATION AND OBJECT TRACKING

Video segmentation and object tracking are critical tasks in computer vision with applications spanning surveillance, autonomous driving, and interactive media. Traditional methods often struggle with the dynamic nature of video data, where object occlusions, variations in illumination, and complex motion patterns present significant challenges. Existing segmentation and tracking systems frequently suffer from inaccuracies in handling real-time video sequences, particularly in distinguishing and tracking multiple overlapping objects. The limitations of current models in addressing these issues necessitate the development of more advanced techniques that can effectively manage dynamic scenes and improve tracking accuracy. To address these challenges, we propose an advanced machine learning technique, AI-Enhanced TrackSegNet, which integrates deep learning with novel attention mechanisms for improved video segmentation and object tracking. Our method utilizes a combination of Convolutional Neural Networks (CNNs) for feature extraction and Long Short-Term Memory (LSTM) networks for temporal sequence modeling. We introduce an attention-based mechanism to dynamically focus on relevant features, enhancing the model's ability to handle occlusions and varying object appearances. The model was trained on a diverse dataset of video sequences, incorporating both synthetic and real-world footage. The AI-Enhanced TrackSegNet demonstrated significant improvements in performance compared to existing techniques. Our method achieved an average Intersection over Union (IoU) score of 86.7% for segmentation and a tracking precision rate of 91.3% on the MOT17 benchmark dataset. These results represent a 10.2% improvement in IoU and a 7.5% increase in tracking precision compared to state-of-the-art methods. The model also exhibited enhanced robustness in complex scenes, handling occlusions and motion variations with greater accuracy.

Ghost Channels and Ghost Cycles Guiding Long Transients in Dynamical Systems.

Dynamical descriptions and modeling of natural systems have generally focused on fixed points, with saddles and saddle-based phase-space objects such as heteroclinic channels or cycles being central concepts behind the emergence of quasistable long transients. Reliable and robust transient dynamics observed for real, inherently noisy systems is, however, not met by saddle-based dynamics, as demonstrated here. Generalizing the notion of ghost states, we provide a complementary framework that does not rely on the precise knowledge or existence of (un)stable fixed points, but rather on slow directed flows organized by ghost sets in ghost channels and ghost cycles. Moreover, we show that the appearance of these novel objects is an emergent property of a broad class of models typically used for description of natural systems.

Neural SSS: Lightweight Object Appearance Representation

AbstractWe present a method for capturing the BSSRDF (bidirectional scattering‐surface reflectance distribution function) of arbitrary geometry with a neural network. We demonstrate how a compact neural network can represent the full 8‐dimensional light transport within an object including heterogeneous scattering. We develop an efficient rendering method using importance sampling that is able to render complex translucent objects under arbitrary lighting. Our method can also leverage the common planar half‐space assumption, which allows it to represent one BSSRDF model that can be used across a variety of geometries. Our results demonstrate that we can render heterogeneous translucent objects under arbitrary lighting and obtain results that match the reference rendered using volumetric path tracing.

A generative machine learning model for the 3D reconstruction of material microstructure and performance evaluation

The 3D reconstruction is generally defined as the process of capturing the shape and appearance of real objects. By reconstructing 3D digital model from a series of 2D slices, it brings considerable convenience to visualize internal structure and decipher structure-property relation of a material. Nowadays, the 3D reconstruction is becoming a cutting-edge technique in depicting the internal structure and evaluating the physical performance of targeted materials. Recent years, generative machine learning methods, such as generative adversarial networks (GAN), have achieved tremendous success in AI-generated physical content (AIGPC). However, lots of technical challenges remain, including oversimplified models, oversized dataset requirement and inefficient convergence. These difficulties are caused by the insufficient ability to capture detailed features and the inadequacy of the generated model quality. To this end, a novel generative model is developed, which combines the multiscale features of U-net and the synthesis ability of GANs. With the help of the multiscale channel aggregation module, the hierarchical feature aggregation module and the convolutional block attention module, our model is developed to capture the features of the material microstructure better. The loss function is refined by combining the image regularization loss with the Wasserstein distance loss. In addition, the anisotropy index is adopted to measure anisotropic degree of selected samples quantitively. The results demonstrate that the 3D structures generated by the proposed model retain high fidelity with ground truth samples. With remarkable performance, the proposed model not only overtakes traditional GAN, but will shed a brilliant light on AIGPC and physical 3D reconstruction.

Illuminator: Image-based illumination editing for indoor scene harmonization

Illumination harmonization is an important but challenging task that aims to achieve illumination compatibility between the foreground and background under different illumination conditions. Most current studies mainly focus on achieving seamless integration between the appearance (illumination or visual style) of the foreground object itself and the background scene or producing the foreground shadow. They rarely considered global illumination consistency (i.e., the illumination and shadow of the foreground object). In our work, we introduce “Illuminator”, an image-based illumination editing technique. This method aims to achieve more realistic global illumination harmonization, ensuring consistent illumination and plausible shadows in complex indoor environments. The Illuminator contains a shadow residual generation branch and an object illumination transfer branch. The shadow residual generation branch introduces a novel attention-aware graph convolutional mechanism to achieve reasonable foreground shadow generation. The object illumination transfer branch primarily transfers background illumination to the foreground region. In addition, we construct a real-world indoor illumination harmonization dataset called RIH, which consists of various foreground objects and background scenes captured under diverse illumination conditions for training and evaluating our Illuminator. Our comprehensive experiments, conducted on the RIH dataset and a collection of real-world everyday life photos, validate the effectiveness of our method.

SMART: Syntax-Calibrated Multi-Aspect Relation Transformer for Change Captioning.

Change captioning aims to describe the semantic change between two similar images. In this process, as the most typical distractor, viewpoint change leads to the pseudo changes about appearance and position of objects, thereby overwhelming the real change. Besides, since the visual signal of change appears in a local region with weak feature, it is difficult for the model to directly translate the learned change features into the sentence. In this paper, we propose a syntax-calibrated multi-aspect relation transformer to learn effective change features under different scenes, and build reliable cross-modal alignment between the change features and linguistic words during caption generation. Specifically, a multi-aspect relation learning network is designed to 1) explore the fine-grained changes under irrelevant distractors (e.g., viewpoint change) by embedding the relations of semantics and relative position into the features of each image; 2) learn two view-invariant image representations by strengthening their global contrastive alignment relation, so as to help capture a stable difference representation; 3) provide the model with the prior knowledge about whether and where the semantic change happened by measuring the relation between the representations of captured difference and the image pair. Through the above manner, the model can learn effective change features for caption generation. Further, we introduce the syntax knowledge of Part-of-Speech (POS) and devise a POS-based visual switch to calibrate the transformer decoder. The POS-based visual switch dynamically utilizes visual information during different word generation based on the POS of words. This enables the decoder to build reliable cross-modal alignment, so as to generate a high-level linguistic sentence about change. Extensive experiments show that the proposed method achieves the state-of-the-art performance on the three public datasets.

Class-agnostic counting and localization with feature augmentation and scale-adaptive aggregation

Compared to traditional class-specific counting, class-agnostic counting (CAC) enables the counting of arbitrary novel categories with only a few exemplars. The core issue of CAC lies in feature matching between the query image and exemplars. However, the scarce user-given exemplars make it difficult to adequately cover the diverse appearance of target objects, which poses challenges for feature matching. In this paper, we propose a Class-Agnostic Counting and Localization model (CACL) to enrich the exemplars by mining more inherent instances from the query feature. Specifically, CACL consists mainly of Self-Augment Feature Matching (SAFM) and Scale-Adaptive Spatial Aggregation (SASA). SAFM collects features of target instances frequently occurring in the query feature to augment the limited user-given exemplar, thus producing more comprehensive semantic features of the target category. To improve the robustness against scale variation, we design the SASA to enable the model to aggregate multiscale features adaptively. Compared to previous density map-based approaches, our method can locate the target objects which is more practical in real-world applications. Extensive experiments on FSC-147 suggest that our method achieves a competitive counting performance compared to state-of-the-art methods. Moreover, cross-dataset experiments demonstrate superior generalization in class-specific counting tasks.

Triangulated Irregular Network based Seamline Determination for Fast Image Stitching of Multiple UAV Images

Abstract. To maintain the efficiency of UAV (unmanned aerial vehicle) remote sensing, rapid image stitching is essential to make multiple UAV images into a seamless mosaic image. Relief displacement introduces variations in object appearance for each image, causing mismatch errors at mosaic seamlines. Traditional approaches involve orthorectifying images using DSMs (digital surface models). While these approaches allow for accurate image stitching, they do not cope with the advantages of UAVs due to their time consumption. In contrast, fast image stitching techniques that do not use orthorectification are well suited for UAV image processing. Related researches have attempted to optimize seamlines to eliminate the errors caused by relief displacement without the use of DSMs. We propose to utilize a TIN (triangular irregular network) of tiepoints to effectively eliminate errors caused by relief displacement while maintaining the fast speed of image stitching. In this study, a TIN is constructed based image tiepoints whose ground coordinates have been obtained through bundle adjustment. The edges of the TIN are used to generate seamlines for image stitching, and the facets of the TIN are used to select minimal images for image stitching and to optimize seamlines. Image stitching results of our proposed method had small error of 1–2 pixels and the processing time of less than 10 minutes for 97 UAV images. This study showed that the proposed method could stitch multiple images while maintaining stable quality using only geometric clues of a TIN of tiepoints.