Vision Applications Research Articles

Bridging the gap: Active learning for efficient domain adaptation in object detection

In practical object detection computer vision applications, training commonly incorporates multiple data sources. Domain adaptation enhances the models’ capacity to generalize across different source and target domain data. Active learning, which serves as a bridge between unsupervised and supervised domain adaptation, is an area that has been underexplored for object detection. Our methods are specifically designed to tackle the challenges of selecting target domain images for annotation, complementing the existing source domain images. First, this paper formally describes active learning in domain adaptation for object detection. We propose three new selection strategies for active learning in domain adaptation for object detection. We utilize multiple machine learning models when selecting target domain data for annotation. In particular, our best-performing selection method utilizes a domain discriminator to predict the target domain likelihood of each sample. Additionally, we introduce an evaluation framework that evaluates per-box and per-image labeling efforts in the unique object detection application. Through extensive experiments we could show that our best-performing method could improve the active learning object detection performance over random selection by about 2.4% while requiring about 13% less annotated bounding boxes on the target domain.

Fabric4show: real-time vision system for fabric defect detection and post-processing

The exploration of computer vision applications for fabric defect detection has immense potential value. However, current relevant research in this area has primarily focused on detection models that aim for high detection accuracy and algorithmic efficiency, while neglecting the practical industrial production requirements. Therefore, we propose a fabric defect detection and post-processing system that integrates an optimized region with convolutional neural network (CNN) features (i.e., Faster R-CNN) for defect detection, defect localization and detection model evaluation. In addition, the proposed intelligent system incorporates novel approaches, such as a rearranged fabric dataset, anomaly detection, recommended clipping region division, and a replenishment device. This study illustrates an example of artificial intelligence (AI)-driven automated technology in fabric manufacturing. The accuracy and detection speed of different detection models under identical hardware conditions are evaluated and compared with related work. Experimental results demonstrate that the proposed approach achieves comparable performance to other models, while significantly reducing computational resource requirements. The potential efficiency of using two-stage networks on hardware systems for fabric defect detection tasks is highlighted, which is likely to have relevant implications for the textile industry.

Underwater Image Enhancement Algorithm Based on Adversarial Training

Ocean observation is the first step in the development of the ocean, whose abundant resources and strategic significance are attracting increasing attention. Observation methods based on visual sensor networks have received great attention from researchers due to their visualization capability and high information capacity. However, below the sea surface, objective factors such as blurriness, turbulence, and underwater color casting can cause image distortion and affect the acquisition of images. In this paper, the enhancement of underwater images is tackled using an adversarial learning-based approach. First, pre-processing is applied to address the significant color casting in the dataset, thus enhancing feature learning for subsequent style transfer. Then, corresponding improvements are made to a generative adversarial network’s structure and loss functions to better restore the features of the network output. Finally, evaluations and comparisons are performed using underwater image quality assessment metrics and several public datasets. Through multidimensional experiments, the proposed algorithm is shown to exhibit excellent performance in both subjective and objective evaluation metrics compared to state-of-the-art algorithms, as well as in practical visual applications.

Opportunities for improving the use of computer vision in municipal solid waste sorting

This article explores the possibilities of improving the application of optical sorting of municipal solid waste on conveyor belts using computer vision and machine learning technologies in order to replace manual sorting lines. Also the relevance of improving waste sorting is emphasized. This work expands the understanding of the application of computer vision in the field of waste management and offers practical recommendations for its implementation in industrial processes. The paper reviews current approaches to sorting municipal solid waste, analyzes their effectiveness, and explores the potential for improving waste management processes and increasing recycling efficiency based on recent research in this area. The most popular equipment used in sorting and its features are considered. A generalization of the morphological composition of waste is carried out, and the minimum and maximum possible shares of each class of waste in the recycling stream are identified. Based on this data, sufficient proportions of images in each category are selected to train a convolutional neural network for waste classification. Applied research in terms of detection and classification of waste objects on conveyor belts using well-known neural network architectures is considered. The analysis showed that most solutions are based on 2018 architectures, while the reference classification accuracy of more modern architectures, such as OmniVec, is 17 % higher, and detection efficiency is similar orders of magnitude higher. The resulting classification accuracy is significantly affected by image augmentation in the set of images for training the neural network — its presence can increase accuracy by 10 to 15 %. The minimum training data set has been determined, which should be about 2 000 images due to the fact that after a further increase in the number, the accuracy does not change significantly.

45‐1: Invited Paper: Design Strategies for NIR Emitting Materials

The development of OLED materials by molecular design is important for high performance organic light emitting diode because tailoring the molecular structures can dramatically tune the properties such as efficiency, color, stability of device. Near‐infrared organic light‐emitting diodes (NIR OLEDs) with heavy metals or TADF materials are regularly reported due to the advantages of their various applications in healthcare services, veil authentication, and night vision displays. In this paper, we are summarized the collection of innovative organic near‐infrared (NIR) emitters which is crucial factor to design specific NIR emitter for NIR OLEDs

Transferring Learned Behaviors between Similar and Different Radios.

Transfer learning (TL) techniques have proven useful in a wide variety of applications traditionally dominated by machine learning (ML), such as natural language processing, computer vision, and computer-aided design. Recent extrapolations of TL to the radio frequency (RF) domain are being used to increase the potential applicability of RFML algorithms, seeking to improve the portability of models for spectrum situational awareness and transmission source identification. Unlike most of the computer vision and natural language processing applications of TL, applications within the RF modality must contend with inherent hardware distortions and channel condition variations. This paper seeks to evaluate the feasibility and performance trade-offs when transferring learned behaviors from functional RFML classification algorithms, specifically those designed for automatic modulation classification (AMC) and specific emitter identification (SEI), between homogeneous radios of similar construction and quality and heterogeneous radios of different construction and quality. Results derived from both synthetic data and over-the-air experimental collection show promising performance benefits from the application of TL to the RFML algorithms of SEI and AMC.

Enhancing accessibility and discoverability of digital archive images through automated image recognition tool

This research paper presents a comprehensive evaluation of the effectiveness of Imagga and Google cloud vision application programming interface (API) as image recognition tools for generating metadata in digital archive images. The assessment encompasses a diverse range of archive images, including those without text, images with text, and both color and black-and-white images. Through the use of evaluation metrics such as cosine similarity, word overlap similarity, recall, precision, and F1 score, the performance of these tools is quantitatively measured. The findings highlight the strong individual performance of both Imagga and Google cloud vision API, with the combined metadata outputs achieving significantly higher scores across all metrics. This emphasizes the potential benefits of employing a combined approach, leveraging the strengths of multiple tools to enhance the reliability and robustness of the metadata extraction process. The findings contribute to the advancement of metadata management in digital archives and underscore the importance of utilizing multiple tools for improved performance in image metadata generation.

Finite-Time Synchronization of Neural Networks With Proportional Delays for RGB-D Image Protection.

Since the depth information of images facilitates the analysis of the spatial distance of objects in computer vision applications, it is necessary to protect the image depth information. Thus this article proposes a novel red-green-blue-depth (RGB-D) image protection algorithm, which is implemented with the finite-time synchronization (FTS) of neural networks (NNs) with proportional delays via the quantized intermittent control to derive the system synchronization criterion based on Lyapunov stability theory. The performance of RGB-D image protection depends on the synchronization error of the system by driving the system sequence to encrypt the RGB-D image and responding to the system sequence to decrypt the encrypted image. Subsequently, the validity of the proposed criteria is verified by simulation examples, and the practical application of RGB-D image protection is verified.

Unsupervised Domain Adaptation of Object Detectors: A Survey.

Recent advances in deep learning have led to the development of accurate and efficient models for various computer vision applications such as classification, segmentation, and detection. However, learning highly accurate models relies on the availability of large-scale annotated datasets. Due to this, model performance drops drastically when evaluated on label-scarce datasets having visually distinct images, termed as domain adaptation problem. There are a plethora of works to adapt classification and segmentation models to label-scarce target dataset through unsupervised domain adaptation. Considering that detection is a fundamental task in computer vision, many recent works have focused on developing novel domain adaptive detection techniques. Here, we describe in detail the domain adaptation problem for detection and present an extensive survey of the various methods. Furthermore, we highlight strategies proposed and the associated shortcomings. Subsequently, we identify multiple aspects of the problem that are most promising for future research. We believe that this survey shall be valuable to the pattern recognition experts working in the fields of computer vision, biometrics, medical imaging, and autonomous navigation by introducing them to the problem, and familiarizing them with the current status of the progress while providing promising directions for future research.

Is human perception reliable? Toward illumination robust food freshness prediction from food appearance — Taking lettuce freshness evaluation as an example

Machine vision-based food quality evaluation systems have achieved great attention in industry and academia in recent years because of their high-throughput and non-invasive properties. In practice, the environmental illumination conditions variations would cause visual evaluation bias for both human and machine perceptions. Compared to other existing studies which take sample images and human visual grading under fixed illumination conditions, this study first investigated the environmental illumination effects on the performance of visual-based food grading for both humans and machines. Taking lettuce samples as an example, an image dataset that considered the environmental illumination variations was first established. This dataset encompasses human visual grading scores obtained from sensory panels as well. In contrast to current studies that utilize a single grader or L*a*b* color features to assess sample freshness, our study reveals a substantial impact of environmental illumination on both human perception (p<0.0001) and L*a*b* color features (p<0.0001). In order to enhance the performance of machine vision-based freshness prediction, multitask learning protocols were incorporated into the proposed new network architectures. This allowed the simultaneous prediction of both sample freshness and illumination conditions. In comparison to commonly used generic convolutional neural network models and vision transformer models, the newly proposed model exhibited superior freshness prediction performance. It minimized the prediction error by 20.36%, outperforming the generic ResNet model. This research represents the first quantitative study addressing human and machine perceptions under varied illumination conditions for food quality evaluation. The findings are anticipated to play a pivotal role in expediting the integration of machine vision applications into food engineering practices.

A single-image GAN model using self-attention mechanism and DenseNets

Image generation from a single natural image using generative adversarial networks (GANs) has attracted extensive attention recently due to the GANs’ practical ability to produce photo-realistic images and their potential applications in computer vision. However, learning a powerful generative model that generates realistic, high-quality images from only a single natural image is still a challenging problem. Training GANs in limited data regimes often causes some issues, such as overfitting, memorization, training divergence, poor image quality, and a long training time. In this study, we investigated the state-of-the-art GAN models in computer vision tasks. We conducted several experiments to deeply understand the challenges of learning a powerful generative model. We introduced a novel unconditional GAN model that produces realistic, high-quality, diverse images based on a single training image. In our model, we employed a self-attention mechanism (SAM), a densely connected convolutional network (DenseNet) architecture, and a relativistic average least-squares GAN with gradient penalty (RaLSGAN-GP) for both the generator and discriminator networks to perform image generation tasks better. SAM controls the global contextual information level. It is complementary to convolutions for large feature maps and gives the generator and discriminator more capability to capture long-range dependencies in feature maps. It compensates for the long training time and low image quality issues. DenseNet connects each layer to every other layer in a feed-forward manner to ensure maximum information flow between layers in the network. It is highly parameter efficient and requires less computation to achieve high performance. It provides improved information and gradient flow throughout the network for easy training. It has a regularizing effect that reduces overfitting in image generation. RaLSGAN-GP further improves data generation quality and the stability of our model at no computational cost and provides much more stable training. Thanks to the appropriate combination of SAM, DenseNet, and RaLSGAN-GP, our model successfully generates realistic, high-quality, diverse images while maintaining the global context of the training image. We conducted experiments, user studies, and model evaluation methods to test our model’s performance and compared it with the previous well-known models on three datasets (Places, LSUN, ImageNet). We demonstrated our model’s capability in image synthesis and image manipulation tasks. In our experiments, we showed that our model utilized parameters more efficiently, prevented overfitting, better captured the internal patch statistics of images with complex structures and textures, achieved comparable performance in single image generation tasks, and had much better visual results than its competitive peers. User studies confirmed that the images generated by our model were commonly confused with the original images. Our model can provide a powerful tool for various image manipulation tasks as well as data augmentation in domains dealing with limited training data.

Agricultural object detection with You Only Look Once (YOLO) Algorithm: A bibliometric and systematic literature review

Vision is a major component in several digital technologies and tools used in agriculture. Object detection plays a pivotal role in digital farming by automating the task of detecting, identifying, and localization of various objects in large-scale agrarian landscapes. The single-stage detection algorithm, You Only Look Once (YOLO), has gained popularity in agriculture in a relatively short span due to its state-of-the-art performance in terms of accuracy, speed, and network size. YOLO offers real-time detection performance with good accuracy and is implemented in various agricultural tasks, including monitoring, surveillance, sensing, automation, and robotics operations. The research and application of YOLO in agriculture are accelerating at a tremendous speed but are fragmented and multidisciplinary in nature. Moreover, the performance characteristics (i.e., accuracy, speed, computation) of the object detector influence the rate of technology implementation and adoption in agriculture. Therefore, this study aimed to collect extensive literature to document and critically evaluate the advances and application of YOLO for agricultural object recognition tasks. First, we conducted a bibliometric review of 257 selected articles to understand the scholarly landscape (i.e., research trends, evolution, global hotspots, and gaps) of YOLO in the broad agricultural domain. Secondly, we conducted a systematic literature review on 30 selected articles to identify current knowledge, critical gaps, and modifications in YOLO for specific agricultural tasks. The study critically assessed and summarized the information on YOLO’s end-to-end learning approach, including data acquisition, processing, network modification, integration, and deployment. We also discussed task-specific YOLO algorithm modification and integration to meet the agricultural object or environment-specific challenges. In general, YOLO-integrated digital tools and technologies showed the potential for real-time, automated monitoring, surveillance, and object handling to reduce labor, production cost, and environmental impact while maximizing resource efficiency. The study provides detailed documentation and significantly advances the existing knowledge on applying YOLO in agriculture, which can greatly benefit the scientific community. The results of this study open the door for implementing YOLO-based solutions in practical agricultural scenarios and add to the expanding corpus of information on computer vision applications in agriculture.

Multi-channel and multi-featured extended orb with gabor filter and non-maximum suppression

This research paper presents an enhanced approach to feature extraction using the Oriented FAST and Rotated BRIEF (ORB) algorithm (Rublee et al., 2014). The proposed method leverages the power of Gabor filters (Mehrotra et al., 1992) in combination with Non-Maximum Suppression (Neuback and Van Gool, 2006) to improve the robustness and efficiency of feature detection in computer vision applications. The process begins with the acquisition of an RGB image, which is then transformed into seven single-channel images representing different color and intensity aspects: red, green, blue, hue, saturation, value, and grayscale. Each single-channel image is independently subjected to Gabor filtering, resulting in seven Gabor-filtered images. These images capture distinct texture and frequency information, enhancing the discriminative power of subsequent feature extraction. Subsequently, the Oriented FAST and Rotated BRIEF (ORB) algorithm is applied to each of the seven Gabor-filtered images to extract key points and descriptors. This step yields a comprehensive set of keypoints (Mallick et al., 2015) and descriptors, capturing rich local feature information across multiple image channels. To further refine the extracted features and eliminate redundant keypoints, Non-Maximum Suppression is employed independently on each single-channel image. This process effectively filters out overlapped and false keypoints, ensuring the selection of only the most salient features for downstream tasks. Experimental results demonstrate the efficacy of the proposed approach in improving feature detection performance compared to traditional methods. The combination of Gabor filtering and Non-Maximum Suppression enhances feature discriminability, robustness to noise, and resistance to image transformations, making it well-suited for various computer vision applications, including object recognition, image matching, and scene understanding.

Perceptual Learning for Promoting Design Thinking in Space Design Education

The application of real-time visualization and virtual reality (VR) technology was examined to promote students’ design thinking in space design education. Learning space design, such as architecture and interior design, takes a series of iterations of training to nurture their imagination of how to translate their idea into a physical format. While design thinking refers to ideating, prototyping, testing, and implementing, educational methods for space design linearly provide students with an opportunity by employing this procedure. Traditionally, academic tools for space design education have been drawing, model making, and, recently, computer rendering. However, these methods differ from prototyping in design thinking regarding physical experience. It lacks the problem-finding, and cannot articulate the feedback system. Thus, this study was carried out to implement perceptual learning and training by data processing via acquired visual experience. With real-time visualization and VR technologies, spatial design prototypes were developed virtually. After the perceptual learning, the result evaluation was carried out with a qualitative analysis of the participants and a quantitative analysis of pre-course and after-course questionnaires. Afterward, the teaching experience of perceptual learning and findings were discussed.

Solution-Processed Ultralow Voltage Organic Transistors With Sharp Switching for Adaptive Visual Perception.

Neuromorphic visual systems can emulate biological retinal systems to perceive visual information under different levels of illumination, making them have considerable potential for future intelligent vehicles and vision automation. However, the complex circuits and high operating voltages of conventional artificial vision systems present great challenges for device integration and power consumption. Here, bioinspired synaptic transistors based on organic single crystal phototransistors are reported, which exhibit excitation and inhibition synaptic plasticity with time-varying. By manipulating the charge dynamics of the trapping centers of organic crystal-electret vertical stacks, organic transistors can operate below 1V with record high on/off ratios close to 108 and sharp switching with a subthreshold swing of 59.8mV dec-1. Moreover, the approach offers visual adaptation with highly localized modulation and over 98.2% recognition accuracy under different illumination levels. These bioinspired visual adaptation transistors offer great potential for simplifying the circuitry of artificial vision systems and will contribute to the development of machine vision applications.

Training Vehicle Detection and Classification Models with Less Data: An Active Learning Approach

The application of computer vision in transportation engineering has facilitated real-time traffic flow optimization, vehicle counting, anomaly detection, and ameliorated transportation safety. Most vision systems are, however, developed through a supervised learning process, which can be data hungry and costly because it requires manual annotation of objects from a variety of sources. The general rule of thumb for building accurate and transferrable vision models has been to increase the quality, diversity, and quantity of the annotated datasets used in model training. This paper presents a simple, yet efficient active learning framework that significantly reduces the number of annotations needed to build a state-of-the-art vehicle detection and classification model. To achieve this, we first leverage a vision transformer that generates embeddings rich with information needed to quantify the similarity and diversity between images in a two-dimensional embedding space. To select which images from the embedding space should be annotated, we propose a scoring and sampling strategy that minimizes class imbalance and model uncertainty through an iterative process. The latest iteration of the You Only Look Once (YOLO) model, YOLOv8, is used as the active learner. We compare the efficacy of our proposed active learning methods with models developed at much higher sampling rates using the mean average precision. The models developed were also integrated with tracking algorithms to evaluate differences in accuracy for vehicle counts and their practical implications for direction counts.

Deep learning for identifying bee species from images of wings and pinned specimens.

One of the most challenging aspects of bee ecology and conservation is species-level identification, which is costly, time consuming, and requires taxonomic expertise. Recent advances in the application of deep learning and computer vision have shown promise for identifying large bumble bee (Bombus) species. However, most bees, such as sweat bees in the genus Lasioglossum, are much smaller and can be difficult, even for trained taxonomists, to identify. For this reason, the great majority of bees are poorly represented in the crowdsourced image datasets often used to train computer vision models. But even larger bees, such as bumble bees from the B. vagans complex, can be difficult to separate morphologically. Using images of specimens from our research collections, we assessed how deep learning classification models perform on these more challenging taxa, qualitatively comparing models trained on images of whole pinned specimens or on images of bee forewings. The pinned specimen and wing image datasets represent 20 and 18 species from 6 and 4 genera, respectively, and were used to train the EfficientNetV2L convolutional neural network. Mean test precision was 94.9% and 98.1% for pinned and wing images respectively. Results show that computer vision holds great promise for classifying smaller, more difficult to identify bees that are poorly represented in crowdsourced datasets. Images from research and museum collections will be valuable for expanding classification models to include additional species, which will be essential for large scale conservation monitoring efforts.

Object Tracking Using Computer Vision: A Review

Object tracking is one of the most important problems in computer vision applications such as robotics, autonomous driving, and pedestrian movement. There has been a significant development in camera hardware where researchers are experimenting with the fusion of different sensors and developing image processing algorithms to track objects. Image processing and deep learning methods have significantly progressed in the last few decades. Different data association methods accompanied by image processing and deep learning are becoming crucial in object tracking tasks. The data requirement for deep learning methods has led to different public datasets that allow researchers to benchmark their methods. While there has been an improvement in object tracking methods, technology, and the availability of annotated object tracking datasets, there is still scope for improvement. This review contributes by systemically identifying different sensor equipment, datasets, methods, and applications, providing a taxonomy about the literature and the strengths and limitations of different approaches, thereby providing guidelines for selecting equipment, methods, and applications. Research questions and future scope to address the unresolved issues in the object tracking field are also presented with research direction guidelines.

Greedy Ensemble Hyperspectral Anomaly Detection.

Hyperspectral images include information from a wide range of spectral bands deemed valuable for computer vision applications in various domains such as agriculture, surveillance, and reconnaissance. Anomaly detection in hyperspectral images has proven to be a crucial component of change and abnormality identification, enabling improved decision-making across various applications. These abnormalities/anomalies can be detected using background estimation techniques that do not require the prior knowledge of outliers. However, each hyperspectral anomaly detection (HS-AD) algorithm models the background differently. These different assumptions may fail to consider all the background constraints in various scenarios. We have developed a new approach called Greedy Ensemble Anomaly Detection (GE-AD) to address this shortcoming. It includes a greedy search algorithm to systematically determine the suitable base models from HS-AD algorithms and hyperspectral unmixing for the first stage of a stacking ensemble and employs a supervised classifier in the second stage of a stacking ensemble. It helps researchers with limited knowledge of the suitability of the HS-AD algorithms for the application scenarios to select the best methods automatically. Our evaluation shows that the proposed method achieves a higher average F1-macro score with statistical significance compared to the other individual methods used in the ensemble. This is validated on multiple datasets, including the Airport-Beach-Urban (ABU) dataset, the San Diego dataset, the Salinas dataset, the Hydice Urban dataset, and the Arizona dataset. The evaluation using the airport scenes from the ABU dataset shows that GE-AD achieves a 14.97% higher average F1-macro score than our previous method (HUE-AD), at least 17.19% higher than the individual methods used in the ensemble, and at least 28.53% higher than the other state-of-the-art ensemble anomaly detection algorithms. As using the combination of greedy algorithm and stacking ensemble to automatically select suitable base models and associated weights have not been widely explored in hyperspectral anomaly detection, we believe that our work will expand the knowledge in this research area and contribute to the wider application of this approach.

DDformer: Dimension decomposition transformer with semi-supervised learning for underwater image enhancement

Vision-guided Autonomous Underwater Vehicles (AUVs) have gradually become significant tools for human exploration of the ocean. However, distorted images severely limit the visual ability, making it difficult to meet the needs of complex underwater environment perception. Fortunately, recent advancements in deep learning have led to rapid developments in underwater image enhancement. The emergence of the Transformer architecture has further enhanced the capabilities of deep learning. However, the direct application of Transformer to underwater image enhancement presents challenges in computing pixel-level global information and extracting local features. In this paper, we present a novel approach that merges dimension decomposition Transformer with semi-supervised learning for underwater image enhancement. To begin, dimension decomposition attention is proposed, which enables Transformer to compute global dependencies directly at the original scale and correct color distortions effectively. Concurrently, we employ convolutional neural networks to compensate for Transformer's limitations in extracting local features, thereby enriching details and textures. Subsequently, a multi-stage Transformer strategy is introduced to divide the network into high- and low-resolution stages for multi-scale global information extraction. It helps correct color distortions while enhancing the network's focus on regions with severe degradation. Moreover, we design a semi-supervised learning framework to reduce the reliance on paired datasets and construct a corresponding multi-scale fusion discriminator to enhance the sensitivity to input data. Experimental results demonstrate that our method outperforms state-of-the-art approaches, showcasing excellent learning and generalization capabilities on subjective perception and overall evaluation metrics. Furthermore, outstanding results highlight the significant improvements it brings to downstream visual engineering applications. The code of the proposed DDformer is available at https://github.com/ZhiGao-hfuu/DDformer.