High Visual Similarity Research Articles

Scene-aware Foveated Rendering.

We propose a new scene-aware foveated rendering method, which incorporates the scene awareness and characteristics of the human visual system into the mapping-based foveated rendering framework. First, we generate the conservative visual importance map that encodes the visual features of the scene, visual acuity, and gaze motion. Second, we construct the pixel size control map using a convolution kernel method. Third, we utilize the pixel size control map to guide the foveated rendering. At last, a temporal coherent refinement strategy is used to maintain the smooth foveated rendering for the adjacent frames. Compared to the state-of-the-art mapping-based foveated rendering methods using the same compression ratio, our method achieves smaller MSE, higher PSNR, and SSIM in the fovea, periphery, salient regions, and the whole image. We also conducted user studies, and the results proved that the perceptual quality of our method has a high visual similarity with the around truth rendered with the full resolution.

Grid-Based DBSCAN Clustering Accelerator for LiDAR’s Point Cloud

Autonomous robots operate on batteries, rendering power efficiency essential. The substantial computational demands of object detection present a significant burden to the low-power cores employed in these robots. Therefore, we propose a grid-based density-based spatial clustering of applications with a noise (DBSCAN) clustering accelerator for light detection and ranging (LiDAR)’s point cloud to accelerate computational speed and alleviate the operational burden on low-power cores. The proposed method for DBSCAN clustering leverages the characteristics of LiDAR. LiDAR has fixed positions where light is emitted, and the number of points measured per frame is also fixed. These characteristics make it possible to impose grid-based DBSCAN on clustering a LiDAR’s point cloud, mapping the positions and indices where light is emitted to a 2D grid. The designed accelerator with the proposed method lowers the time complexity from O(n2) to O(n). The designed accelerator was implemented on a field programmable gate array (FPGA) and verified by comparing clustering results, speeds, and power consumption across various devices. The implemented accelerator speeded up clustering speeds by 9.54 and 51.57 times compared to the i7-12700 and Raspberry Pi 4, respectively, and recorded a 99% reduction in power consumption compared to the Raspberry Pi 4. Comparisons of clustering results also confirmed that the proposed algorithm performed clustering with high visual similarity. Therefore, the proposed accelerator with a low-power core successfully accelerated speed, reduced power consumption, and effectively conducted clustering.

Data-driven Bayesian Gaussian mixture optimized anchor box model for accurate and efficient detection of green citrus

The demand for harvesting green citrus has increased due to the rapid growth of the green citrus industry. However, real-time detection of green citrus is challenging because of its high visual similarity to the background. To address this issue, a new model called EDGC-YOLO (Efficient Detection of Green Citrus based on the YOLO) has been developed, focusing on detecting green citrus effectively and accurately. Specifically, the proposed model uses a data-driven approach that employs the highest confidence level from the aspect ratio of unobstructed citrus fruits to infer anchor boxes of obscured citrus. This method ensures a high degree of coincidence with bounding boxes based on empirical annotation. Furthermore, analyzing the distribution range of various citrus annotation categories within the dataset and the incorporating the highest confidence level of the normal distribution regarding aspect ratios, the initial cluster centers of the BGMM (Bayesian Gaussian Mixture Model) are dynamically adjusted. This strategy enables the anchor boxes to be inferred from the best suited for detecting green citrus. These improved anchor boxes provide a more accurate and effective solution for centroid positioning of obscured green citrus, significantly enhancing the entire network model’s detection accuracy and lightweight efficiency. To further improve detection accuracy and reduce model size, this study integrated the Refined-EfficientNetV2 network as the backbone, enhanced with a Convolutional Block Attention Module (CBAM) for better feature extraction. This integration allows the model to effectively capture relevant channel information and spatial features, increasing the feature extraction capability for green citrus images. Experimental results demonstrate the model’s superior performance: parameters reduced to 4.52 million, computational demand to 7.8 GFLOPs (64.4 % and 49.4 % of the original model, respectively), and model size decreased to 9.4 MB (65.3 % of the original). Additionally, the optimized model improved Precision (P), Recall (R), and mean Average Precision (mAP) by 0.5 %, 1.6 %, and 3.0 %, respectively, compared to the original model. The proposed model achieves higher detection accuracy with a smaller model size, providing theoretical support for the harvesting decision-making of green citrus harvesting robots.

Camouflaged Object Detection via Dual-branch Fusion and Dual Self-similarity constraints

Camouflaged Object Detection (COD) remains a challenging task due to the inherent difficulty in distinguishing concealed objects from their surroundings, primarily owing to their high visual similarity. Current research efforts have been insufficient in distinguishing the similarity differences between camouflaged objects and their corresponding backgrounds, leading to suboptimal performance, particularly when locating small-scale objects. To solve this issue, we introduce a novel Dual-branch Fusion and Dual Self-similarity Network (DSNet) comprising three modules. The first, a dual-branch fusion, mimics human observation of camouflaged objects and extracts multi-angle information. Dual-branch features are then decoded using a symmetric joint decoder module with channel interaction via multi-stage inter-group interaction. Inspired by natural organisms’ self-similarity, the self-similarity constraint module employs global and mutual constraints to identify subtle foreground-background differences. DSNet shows superior performance in experiments, and the constraint module can enhance other models as a plug-and-play component, further boosting overall performance.

A Multi-Group Multi-Stream attribute Attention network for fine-grained zero-shot learning

Fine-grained visual categorization in zero-shot setting is a challenging problem in the computer vision community. It requires algorithms to accurately identify fine-grained categories that do not appear during the training phase and have high visual similarity to each other. Existing methods usually address this problem by using attribute information as intermediate knowledge, which provides sufficient fine-grained characteristics of categories and can be transferred from seen categories to unseen categories. However, the learning of attribute visual features is not trivial due to the following two reasons: (i) The visual information about attributes of different types may interfere with the visual feature learning of each other. (ii) The visual characteristics of the same attribute may vary in different categories. To solve these issues, we propose a Multi-Group Multi-Stream attribute Attention network (MGMSA), which not only separates the feature learning of attributes of different types, but also isolates the learning of attribute visual features for categories with big differences in attribute appearance. This avoids the interference between uncorrelated attributes and helps to learn category-specific attribute-related visual features. This is beneficial for distinguishing fine-grained categories with subtle visual differences. Extensive experiments on benchmark datasets show that MGMSA achieves state-of-the-art performance on attribute prediction and fine-grained zero-shot learning.

Quality-preserving multilevel mesh generation for building models

ABSTRACT Three-dimensional (3D) high-fidelity building models are crucial for digital twin cities. Multi-scale modeling is crucial for accurately capturing the complexity of urban environments; however, existing methods often lack the flexibility required to satisfy the varied demands of diverse applications. In this paper, we propose a framework to continuously simplify 3D building models, it dynamically adjusts mesh accuracy to meet diverse needs, enabling seamless transitions from high-poly to medium and low-poly meshes. Our method contains three main stages: First, in the mesh repair stage, we introduce a novel mesh repair framework based on an advanced Marching Cubes 33 algorithm to ensure a watertight and clean topology mesh. Then, for generating medium-poly meshes, we employ a feature-preserving mesh simplification algorithm that combines curvature and area-weighted quadratic error metrics to achieve a high-fidelity simplified mesh. For generating low-poly meshes, we use the concept of carving visual hull to generate low-poly mesh with high visual similarity. Our method has been evaluated on building subsets of the Free3D and Thingi10 K datasets, as well as on various building models generated from different observation scales, and its superiority has been validated through extensive comparisons with state-of-the-art techniques.

Phishing webpage detection based on global and local visual similarity

In recent years, phishing websites have constantly evolved, causing traditional URL or HTML-based detection methods less effective. This limitation motivated the development of visual similarity-based detection methods. These methods can identify phishing websites in a rapidly changing phishing landscape since the key factor in deceiving users is the high visual similarity between phishing pages and their corresponding legitimate ones. However, existing similarity-based methods often suffer from high false positive rates. To alleviate the problem, this paper proposed a new webpage similarity phishing detection framework. Using a deep learning model, the framework locates a website logo and integrates the logo area with the full webpage to detect phishing pages. Experimental evaluations on multiple datasets demonstrate the model’s ability to achieve a precision of 91% and a recall of 88% in detecting phishing websites. Comparative experiments against existing methods reveal that the proposed model enhances precision and recall by 17% and 21%, respectively, while reducing the false positive rate by 1%.

Quality Control of DAS VSP Data in Desert Environment Using Simulations and Matching Filters.

The unconsolidated near surface and large, daily temperature variations in the desert environment degrade the vertical seismic profiling (VSP) data, posing the need for rigorous quality control. Distributed acoustic sensing (DAS) VSP data are often benchmarked using geophone surveys as a gold standard. This study showcases a new simulation-based way to assess the quality of DAS VSP acquired in the desert without geophone data. The depth uncertainty of the DAS channels in the wellbore is assessed by calibrating against formation depth based on the concept of conservation of the energy flux. Using the 1D velocity model derived from checkshot data, we simulate both DAS and geophone VSP data via an elastic pseudo-spectral finite difference method, and estimate the source and receiver signatures using matching filters. These field geophone data show high amplitude variations between channels that cannot be replicated in the simulation. In contrast, the DAS simulation shows a high visual similarity with the field DAS first arrival waveforms. The simulated source and receiver signatures are visually indistinguishable from the field DAS data in this study. Since under perfect conditions, the receiver signatures should be invariant with depth, we propose a new DAS data quality control metric based on local variations of the receiver signatures which does not require geophone measurements.

On-the-fly image-level oversampling for imbalanced datasets of manufacturing defects

Visual defect recognition and its manufacturing applications have been an upcoming topic in recent AI research. Defect datasets are often severely imbalanced and can be additionally burdened with separating classes of high visual similarity. Although various methods of data augmentation have been proposed to mitigate the class imbalance, they often fail to cope with tinier minority classes or have fidelity issues with smaller defects while, at the same time, needing significant computational resources to train. Also, augmentation based on vector-based oversampling struggles to produce high-fidelity inputs and is hard to apply on custom CNN architectures, which often perform better for this type of problem. Our work presents an image-level oversampling method based on an instance-based image generator that can be applied to any CNN directly during the training process without increasing the order of training time required. It is based on identifying a small number of the most uncertain base samples close to the estimated class boundaries and using them as seeds for augmentation. The resulting images are of high visual quality preserving small class differences, and they also improve the classifier boundary leading to higher recall scores than other state-of-the-art approaches.

Discrimination of object information by bat echolocation deciphered from acoustic simulations.

High-precision visual sensing has been achieved by combining cameras with deep learning. However, an unresolved challenge involves identifying information that remains elusive for optical sensors, such as occlusion spots hidden behind objects. Compared to light, sound waves have longer wavelengths and can, therefore, collect information on occlusion spots. In this study, we investigated whether bats could perform advanced sound sensing using echolocation to acquire a target's occlusion information. We conducted a two-alternative forced choice test on Pipistrellus abramus with five different targets, including targets with high visual similarity from the front, but different backend geometries, i.e. occlusion spots or textures. Subsequently, the echo impulse responses produced by these targets, which were difficult to obtain with real measurements, were computed using three-dimensional acoustic simulations to provide a detailed analysis consisting of the acoustic cues that the bats obtained through echolocation. Our findings demonstrated that bats could effectively discern differences in target occlusion spot structure and texture through echolocation. Furthermore, the discrimination performance was related to the differences in the logarithmic spectral distortion of the occlusion-related components in the simulated echo impulse responses. This suggested that the bats obtained occlusion information through echolocation, highlighting the advantages of utilizing broadband ultrasound for sensing.

Scene-aware Foveated Neural Radiance Fields.

Foveated rendering provides an idea for improving the image synthesis performance of neural radiance fields (NeRF) methods. In this paper, we propose a scene-aware foveated neural radiance fields method to synthesize high-quality foveated images in complex VR scenes at high frame rates. Firstly, we construct a multi-ellipsoidal neural representation to enhance the neural radiance field's representation capability in salient regions of complex VR scenes based on the scene content. Then, we introduce a uniform sampling based foveated neural radiance field framework to improve the foveated image synthesis performance with one-pass color inference, and improve the synthesis quality by leveraging the foveated scene-aware objective function. Our method synthesizes high-quality binocular foveated images at the average frame rate of 66 frames per second (FPS) in complex scenes with high occlusion, intricate textures, and sophisticated geometries. Compared with the state-of-the-art foveated NeRF method, our method achieves significantly higher synthesis quality in both the foveal and peripheral regions with 1.41-1.46× speedup. We also conduct a user study to prove that the perceived quality of our method has a high visual similarity with the ground truth.

Make some Noise: Acoustic Classification of Manual Work Steps Towards Adaptive Assistance Systems

With 32 million people working in the European manufacturing sector, human work still plays a crucial role in industry. However, due to lot size one manufacturing and increased quality requirements, the complexity of products and processes is growing. Therefore, numerous approaches introduce adaptive assistance systems in assembly to support workers during complex work tasks and adapt their level of assistance to the current situation. To enable adaptivity, human action recognition must be incorporated into the consideration of context. Until now, research has focused on providing context to the machine through wearable sensors or cameras. However, wearable sensors hinder worker’s movements and cameras have difficulties distinguishing between work steps of high visual similarity. To mitigate these challenges, we present a new method to classify manual work steps only by their typical acoustic characteristics. The proposed method uses log-Mel spectrograms of work sounds fed into a convolutional neural network (CNN), thus learning their characteristic structure. Moreover, we present a new public dataset for the acoustic classification of manual work steps. The dataset includes typical sources of sounds in manufacturing, such as working with a bench grinder, cordless screwdriver, fling, or grabbing screws. Before feeding the data to the CNN, we apply various pre-processing and data augmentation techniques to increase generalisation capabilities. Our method can detect work steps with reliable accuracy while requiring less parameters than other techniques, proving that detecting work context through acoustics is possible and feasible.

Visual similarity simulation of slub denim based on image processing

The slub denim has various appearance styles that attract consumers due to the irregular distribution and long cycle of the slubs in the slub yarn. This characteristic of the slub yarn makes the slub denim proofing that relies on artificial visual contrast time-consuming with a long design cycle. A visual similarity simulation method was proposed for slub denim based on slub yarn images to solve this problem. At first, the yarn images were pre-processed by grey-level transformation, image threshold segmentation, and morphological operations to obtain the yarn core images. Secondly, according to the elastic change of yarn in fabric structure, the elliptic and sine curve models were established to treat the string to obtain the actual yarn shape in the fabric. Then, a light intensity curve function, which consists of a radial curve model and an interlacing point curve model, was established to simulate the light intensity distribution on the yarn surface. Finally, the cover relation of warp and weft yarns was controlled by Boolean matrix, and mean filtering was used to adjust the fuzzy denim fabric images. The experimental results show that the slub denims simulated by the method proposed in this article have a high visual similarity with the actual slub denims. Meanwhile, multiple parameters of the simulation model established in this article can be adjusted to enhance the adaptability of the model and the accuracy of the simulation results.

Compression and reconstruction of flotation foam images based on generative adversarial networks

The froth flotation is a beneficiation method that utilizes the differences in the physical and chemical properties of the surface of mineral particles to achieve the effective separation of different minerals. Recently, with the development of image processing and computer technology, machine vision has been widely used in mineral froth flotation. However, due to the high pixels of the flotation froth image and the limited storage space of the industrial computer, it is difficult to save a large amount of image data at the industrial site. To this end, a flotation foam image compression and reconstruction method is presented in this work. First, a lossless froth image is collected from the flotation cell in a gold hydrometallurgical plant. Then, the JPEG image compression algorithm compresses the original images to one in dozens. Under this situation, low-quality and small-volume images can be stored in an industrial computer easily. To reconstruct the compressed image, the generative adversarial networks (GAN) is used in this work, and RRDB and VGG19 are employed as the generative and discriminative models of the network, respectively. The loss function of the model is the sum of three losses, which are L1 loss, perceptual loss, and adversarial loss. Three methods for testing the texture features of the froth image are selected to verify the efficiency of the proposed method. The results show that the reconstructed image can effectively restore the details of the bubbles, reflect the image features, and has less noise and high visual similarity with the original image. The reconstructed images generated by low-resolution images with different compression qualities mostly maintain a restoration degree of more than 90% compared with the original images in three texture features.

Visual similarity with L1 facilitates the neural specialization for scripts of L2

The neural mechanism underlying the acquisition of scripts of a second language (L2) is an open issue. The aim of the present study is to investigate the neural specialization for L2 scripts by focusing on the influence of overall visual similarity between first language (L1) and L2. EEG signals were recorded in native Chinese Han readers at the first and ninth months of learning Korean as L2 when they passively viewed Chinese characters (CC), high Chinese-like Korean characters (HKC) and low Chinese-like Korean characters (LKC). Time-frequency analysis revealed that event-related synchronization in the theta band (θ-ERS) is sensitive to CC and Korean character (KC), with a stronger and more left-lateralized θ-ERS for CC and a clear initial response trend of left-/right-lateralization for HKC/LKC. After nine months of learning, increased θ-ERS was shown for both HKC and LKC, whereas robust left lateralization was observed only for HKC. These results suggest that high visual similarity to native language scripts may facilitate the progress of neural specialization for L2. These findings were discussed in light of the “neural recycling” theory.

LIFRNet: A Novel Lightweight Individual Fish Recognition Method Based on Deformable Convolution and Edge Feature Learning

With the continuous development of industrial aquaculture and artificial intelligence technology, the trend of the use of automation and intelligence in aquaculture is becoming more and more obvious, and the speed of the related technical development is becoming faster and faster. Individual fish recognition could provide key technical support for fish growth monitoring, bait feeding and density estimation, and also provide strong data support for fish precision farming. However, individual fish recognition faces significant hurdles due to the underwater environment complexity, high visual similarity of individual fish and the real-time aspect of the process. In particular, the complex and changeable underwater environment makes it extremely difficult to detect individual fish and extract biological features extraction. In view of the above problems, this paper proposes an individual fish recognition method based on lightweight convolutional neural network (LIFRNet). This proposed method could extract the visual features of underwater moving fish accurately and efficiently and give each fish unique identity recognition information. The method proposed in this paper consists of three parts: the underwater fish detection module, underwater individual fish recognition module and result visualization module. In order to improve the accuracy and real-time availability of recognition, this paper proposes a lightweight backbone network for fish visual feature extraction. This research constructed a dataset for individual fish recognition (DlouFish), and the fish in dataset were manually sorted and labeled. The dataset contains 6950 picture information instances of 384 individual fish. In this research, simulation experiments were carried out on the DlouFish dataset. Compared with YOLOV4-Tiny and YOLOV4, the accuracy of the proposed method in fish detection was increased by 5.12% and 3.65%, respectively. Additionally, the accuracy of individual fish recognition reached 97.8%.

Foveated Stochastic Lightcuts.

Foveated rendering provides an idea for accelerating rendering algorithms without sacrificing the perceived rendering quality in virtual reality applications. In this paper, we propose a foveated stochastic lightcuts method to render high-quality many-lights illumination effects in high perception-sensitive regions. First, we introduce a spatiotemporal-luminance based lightcuts generation method to generate lightcuts with different accuracy for different visual perception-sensitive regions. Then we propose a multi-resolution light samples selection method to select the light sample for each node in the lightcuts more efficiently. Our method supports full-dynamic scenes containing over 250k dynamic light sources and dynamic diffuse/specular/glossy objects. It provides frame rates up to 110fps for high-quality many-lights illumination effects in high perception-sensitive regions of the HVS in VR HMDs. Compared with the state-of-the-art stochastic lightcuts method using the same rendering time, our method achieves smaller mean squared errors in the fovea and periphery. We also conduct user studies to prove that the perceived quality of our method has a high visual similarity with the results of the ground truth rendered by using the stochastic lightcuts with 2048 light samples per pixel.

MULTIPLE OIL PAD DETECTION USING DEEP LEARNING

Abstract. Deep learning (DL) algorithms are widely used in object detection such as roads, vehicles, buildings, etc., in aerial images. However, the object detection task is still considered challenging for detecting complex structures, oil pads are one such example: due to its shape, orientation, and background reflection. A recent study used Faster Region-based Convolutional Neural Network (FR-CNN) to detect a single oil pad from the center of the image of size 256 × 256. However, for real-time applications, it is necessary to detect multiple oil pads from aerial images irrespective of their orientation. In this study, FR-CNN was trained to detect multiple oil pads. We cropped images from high spatial resolution images to train the model containing multiple oil pads. The network was trained for 100 epochs using 164 training images and tested with 50 images under 3 different categories. with images containing: single oil pad, multiple oil pad and no oil pad. The model performance was evaluated using standard metrics: precision, recall, F1-score. The final model trained for multiple oil pad detection achieved a weighted average for 50 images precision of 0.67, recall of 0.80, and f1 score of 0.73. The 0.80 recall score indicates that 80% of the oil pads were able to identify from the given test set. The presence of instances in test images like cleared areas, rock structures, and sand patterns having high visual similarity with the target resulted in a low precision score.

TRANSFER LEARNING APPROACH FOR CLASSIFICATION OF WIDELY USED SPICES

People around the world relish variety of food that are flavourful. Spices add flavours to the food without adding any fat or calories. People have used spices for many centuries and are an integral part of our food. In addition to aroma, spices also have anti-bacterial, anti-inflammatory properties and other health-promoting properties. Recognizing spices from images is a challenging problem for a machine as they come in varying sizes and shapes, different colours, high visual similarity, and texture. The classification of spices presents useful applications in the field of Artificial Intelligence-driven food industry, e-commerce, and health care. In the billion-dollar spice industry, image classification of spices finds applications ranging from receiving, processing, labelling, and packaging them. As there is no dataset currently available for spices, in this work, a Spice10 dataset with 2000 images of spices is first created. This study aims to find out whether the accurate classification of spices is possible using computer vision technology. Instead of building models from scratch, a pre-trained transfer learning approach has been implemented in this work to classify the commonly used spices. The images in the dataset are of different sizes and have to be resized and pre-processed before using it with the transfer learning approach. Few different pre-trained networks are modified and used for the image classification of spices. The best classification average accuracy obtained by the VGG16 model is nearly 93.06% which is better than the other models. The high accuracy of the VGG16 model indicates it can be successfully used for the classification of spices.

Distributed densely connected convolutional network approach on patient’s metadata of dermoscopic images for early melanoma detection

Melanoma is one of the most deadly cancers on the planet, and it has the ability to spread to new places of the body if not discovered early enough. Dermoscopic pictures are routinely used to identify melanoma. Many previous studies, based on standard classification approaches and deep learning models, have been proposed for automated analysis of skin lesions. In traditional classification systems, handcrafted functions are used as input. However, due to the high visual similarity between different classes of skin lesions and complex skin diseases, hand-made features are not discriminating enough and fail in many circumstances. Convolutional networks with fewer connections between the input layers and those close to the output can be significantly deeper, more accurate and more effective for training, according to a recent study. In this study, we accept and offer the application of Hadoop Distributed Tight Convolution Network (HdiDenseNet) for melanoma skin cancer detection and study of failed lesions, which can be connected to the mode of transmission. You can use all feature maps at all input levels as input, and your own maps will be used as inputs in all poster headers.