Computer Vision Research Articles

Image-based Lung Analysis in the Context of Digital Pathology: a Brief Review

Lung cancer is the 2nd most diagnosed cancer worldwide. The corresponding histopathological analysis, being both costly and time-consuming, demands the commitment of skilled professionals who, while engrossed in this task, experience constraints on their ability to attend to other crucial responsibilities. Moreover, as it is a human-driven process, mistakes may lead to incorrect diagnosis and treatment. Given the disease frequency and mortality, automated diagnostic systems, using Artificial Intelligence (AI), represent valuable improvements in diagnostic timing and overall performance. Recently, Deep Learning (DL) has been widely used for extracting features from histopathologic images approaching more accurate and expeditious analysis. With this line of research in mind, a brief review of recent technical/scientific works within the scope of lung Digital Pathology (DP) image analysis is provided in this paper, covering different computer vision tasks including classification, segmentation, and detection. Furthermore, available datasets and open-source annotation tools capable of providing support to the aforementioned DP-related tasks are also overviewed. Afterward, a summary table and a discussion around the reviewed approaches is provided, consolidating critical information such as technique/DL architecture, involved datasets, metrics, etc. From this study, it was observed that the ARA-CNN technique achieved the highest Area Under the Curve (AUC) ranging from 0.72 to 0.99 for classification. On the other hand, the multimodal-based approach, with an AUC of 0.95, performed better for the segmentation task. As for the detection task, the BCNN approach stood out, achieving a high AUC of 0.988. This review work aims to provide a comprehensive overview of recent advancements in lung DP image analysis and serves as a foundation for future research in this critical area.

Effect of L-theanine on selective attention in a traffic-related reaction task in sleep-deprived young adults: a double-blind placebo-controlled, crossover study

ABSTRACT Background L-theanine is a non-protein-forming amino acid found in tea. Limited evidence suggests that it improves selective attention. Sleep deprivation impairs attention and psychomotor reactions, affecting automobile driving. We aimed to determine whether L-theanine improves neurobehavioral measures of visual attention in acutely sleep-deprived healthy adults in a traffic-scene-based attention task. Methods In a double-blind, placebo-controlled, counterbalanced, two-way crossover study, we compared the effects of a 200-mg dose of L-theanine with a placebo (150 ml of distilled water) on a computerised, traffic-scene-based visual recognition reaction task in 24 healthy volunteers (age 20–25 years; 13 males) sleep-deprived overnight. The participants made speeded button-presses to imminent accident scenes (i.e. hits), while ignoring safe scenes. They were tested pre-dose and 45 min post-dose, each treatment administered one week apart. Results Hit rates were more than 90% in all sessions, and were similar in two treatments, pre- vs post-dose. L-theanine significantly reduced false alarms (i.e. responses to safe scenes) (p = 0.014) and increased A′ (i.e. target-distractor discriminability) (p = 0.009), whereas placebo did not (p > 0.05). L-theanine reduced hit reaction time by 38.65 ms (p = 0.007), and placebo by 19.08 ms (p = 0.016), however reaction time changes from baseline were not significantly different between treatments (p > 0.05). Conclusions L-theanine in high doses appears to improve selective visual attention by concurrently improving information processing speed and target-distractor discriminability in acutely sleep-deprived individuals. This is consistent with previous functional neuroimaging findings, where L-theanine suppressed distractor-processing and default-mode-network activity in visual selective attention tasks.

Monitoring of meat quality and change-point detection by a sensor array and profiling of bacterial communities

BackgroundReal-time monitoring of food consumer quality remains challenging due to diverse bio-chemical processes taking place in the food matrices, and hence it requires accurate analytical methods. Thresholds to determine spoiled food are often difficult to set. The existing analytical methods are too complicated for rapid in situ screening of foodstuff. ResultsWe have studied the dynamics of meat spoilage by electronic nose (e-nose) for digitizing the smell associated with volatile spoilage markers of meat, comparing the results with changes in the microbiome composition of the spoiling meat samples. We apply the time series analysis to follow dynamic changes in the gas profile extracted from the e-nose responses and to identify the change-point window of the meat state. The obtained e-nose features correlate with changes in the microbiome composition such as increase in the proportion of Brochothrix and Pseudomonas spp. and disappearance of Mycoplasma spp., and with representative gas sensors towards hydrogen, ammonia, and alcohol vapors with R2 values of 0.98, 0.93, and 0.91, respectively. Integration of e-nose and computer vision into a single analytical panel improved the meat state identification accuracy up to 0.85, allowing for more reliable meat state assessment. SignificanceAccurate identification of the change-point in the meat state achieved by digitalizing volatile spoilage markers from the e-nose unit holds promises for application of smart miniaturized devices in food industry.

Advanced Techniques for Real-time Facial Expression Recognition Using Deep Learning

Abstract—Developing systems that can automatically recognize and interpret human emotions from facial expressions is the aim of the quickly expanding field of facial emotion identification and detection research. This technology finds applications in a wide range of areas, including as healthcare, marketing, security, and human-computer interface. Using computer vision and machine learning algorithms, facial emotion recognition systems analyze a face’s features and classify it into numerous emotional categories, including joyful, sorrowful, angry, fearful, and surprised.The three steps in the multi-step process that goes into identifying facial emotions are face detection, facial feature extraction, and emotion categorization. Thanks to recent advances in deep learn- ing, facial emotion detection systems can now identify emotions with high resilience and precision. Further, the development of real-time face expression recognition systems has opened up new avenues for applications such as sentiment analysis, emotional intelligence, and affective computing. This technology could fundamentally alter human-machine interactions and open the way to more compassionate and personalized relationships.A multitude of applications, including virtual assistants, mental health aids, and human-centered technology, will be greatly impacted by the development of systems for identifying and de- tecting facial expressions. Artificial intelligence (AI) technologies that recognize emotions allow people to interact with the digital world in more intelligent and flexible ways. But the complexity of emotion identification lies in the fact that it requires context and geometric elements in addition to facial expressions.

REDef-DETR: real-time and efficient DETR for industrial surface defect detection

Abstract Industrial surface defect detection is an important part of industrial production, which aims to identify and detecting various defects on the surface of product to ensure quality and meet customer requirements. With the development of deep learning and image processing technologies, the surface defect detection methods based on computer vision has become the mainstream method. However, the prevalent convolutional neural network-based defect detection methods also have many problems. For example, these methods rely on post-processing of Non-Maximum Suppression and have poor detection ability for small targets, which affects the speed and accuracy of surface defect detection in industrial scenarios. Therefore, we propose a novel DEtection TRansformer-based surface defect detection method. Firstly, we propose a Multi-scale Contextual Information Dilated module and fuse it into the backbone. The module is mainly composed of large kernel convolutions, which aims to expand the receptive field of the model, thus reducing the leakage rate of the model. Moreover, we design an efficient encoder which mainly contains two important modules, namely feature enhancement based on cascaded group attention module and efficient feature fusion module based on content-aware. The former module effectively enhances the high-level semantic information extracted by the backbone, thus enabling the model to better interpret features, and it can improve the problem of high computational cost of transformer encoder, thus increasing the detection speed. The latter module performs multi-scale feature fusion across the feature information of various scales, thus improving the detection accuracy of the model for small-size defects. Experimental results show that the proposed method achieves 80.6%mAP and 80.3FPS on NEU-DET, and 98.0%mAP and 79.4FPS on PCB-DET. Our proposed method exhibits excellent detection performance and achieves real-time and efficient surface defect detection capability to meet the needs of industrial surface defect detection.

Centralised visual processing center for remote sensing target detection

Target detection in satellite images is an essential topic in the field of remote sensing and computer vision. Despite extensive research efforts, accurate and efficient target detection in remote sensing images remains unsolved due to the large target scale span, dense distribution, and overhead imaging and complex backgrounds, which result in high target feature similarity and serious occlusion. In order to address the above issues in a comprehensive manner, within this paper, we first propose a Centralised Visual Processing Center (CVPC), this structure is a parallel visual processing center for Transformer encoder and CNN, employing a lightweight encoder to capture broad, long-range interdependencies. Pixel-level Learning Center (PLC) module is used to establish pixel-level correlations and improve the depiction of detailed features. CVPC effectively improves the detection efficiency of remote sensing targets with high feature similarity and severe occlusion. Secondly, we propose a centralised feature cross-layer fusion pyramid structure to fuse the results with the CVPC in a top-down manner to enhance the detailed feature representation capability at each layer. Ultimately, we present a Context Enhanced Adaptive Sparse Convolutional Network (CEASC), which improves the accuracy while ensuring the detection efficiency. Based on the above modules, we designed and conducted a series of experiments. These experiments are conducted on three challenging public datasets, DOTA-v1.0, DIOR, and RSDO, showing that our proposed 3CNet achieves a more advanced detection accuracy while balancing the detection speed (78.62% mAP for DOTA-v1.0, 79.12% mAP for DIOR, and 95.50% mAP for RSOD).

NHi-SEGA: n-Hierarchy SEmantic Guided Attention for few-shot learning

AbstractHumans excel at learning and recognizing objects, swiftly adapting to new concepts with just a few samples. However, current studies in computer vision on few-shot learning have not yet achieved human performance in integrating prior knowledge during the learning process. Humans utilize a hierarchical structure of object categories based on past experiences to facilitate learning and classification. Therefore, we propose a method named n-Hierarchy SEmantic Guided Attention (nHi-SEGA) that acquires abstract superclasses. This allows the model to associate with and pay attention to different levels of objects utilizing semantics and visual features embedded in the class hierarchy (e.g., house finch-bird-animal, goldfish-fish-animal, rose-flower-plant), resembling human cognition. We constructed an nHi-Tree using WordNet and Glove tools and devised two methods to extract hierarchical semantic features, which were then fused with visual features to improve sample feature prototypes.

Dental cavity analysis, prediction, localization, and quantification using computer vision

Dental cavity analysis, prediction, localization, and quantification using computer vision

Symmetry group detection of point clouds in 3D via a decomposition method

Analyzing the symmetries present in point clouds, which represent sets of 3D coordinates, is important for understanding their underlying structure and facilitating various applications. In this paper, we propose a novel decomposition-based method for detecting the entire symmetry group of 3D point clouds. Our approach decomposes the point cloud into simpler shapes whose symmetry groups are easier to find. The exact symmetry group of the original point cloud is then derived from the symmetries of these individual components. The method presented in this paper is a direct extension of the approach recently formulated in Bizzarri et al. (2022a) for discrete curves in plane. The method can be easily modified also for perturbed data. This work contributes to the advancement of symmetry analysis in point clouds, providing a foundation for further research and enhancing applications in computer vision, robotics, and augmented reality.

A permutable MLP-like architecture for disease prediction from gut metagenomic data

Metagenomic data plays a crucial role in analyzing the relationship between microbes and diseases. However, the limited number of samples, high dimensionality, and sparsity of metagenomic data pose significant challenges for the application of deep learning in data classification and prediction. Previous studies have shown that utilizing the phylogenetic tree structure to transform metagenomic abundance data into a 2D matrix input for convolutional neural networks (CNNs) improves classification performance. Inspired by the success of a Permutable MLP-like architecture in visual recognition, we propose Metagenomic Permutator (MetaP), which applied the Permutable MLP-like network structure to capture the phylogenetic information of microbes within the 2D matrix formed by phylogenetic tree. Our experiments demonstrate that our model achieved competitive performance compared to other deep neural networks and traditional machine learning, and has good prospects for multi-classification and large sample sizes. Furthermore, we utilize the SHAP (SHapley Additive exPlanations) method to interpret our model predictions, identifying the microbial features that are associated with diseases.

ETLP: Event-based Three-factor Local Plasticity for online learning with neuromorphic hardware

Abstract Neuromorphic perception with event-based sensors, asynchronous hardware, and spiking neurons shows promise for real-time, energy-efficient inference in embedded systems. Brain-inspired computing aims to enable adaptation to changes at the edge with online learning. However, the parallel and distributed architectures of neuromorphic hardware based on co-localized compute and memory imposes locality constraints to the on-chip learning rules. We propose the Event-based Three-factor Local Plasticity (ETLP) rule that uses the pre-synaptic spike trace, the post-synaptic membrane voltage and a third factor in the form of projected labels with no error calculation, that also serve as update triggers. ETLP is applied to visual and auditory event-based pattern recognition using feedforward and recurrent spiking neural networks. Compared to Back-Propagation Through Time (BPTT), eProp and DECOLLE, ETLP achieves competitive accuracy with lower computational complexity. We also show that when using local plasticity, threshold adaptation in spiking neurons and a recurrent topology are necessary to learn spatio-temporal patterns with a rich temporal structure. Finally, we provide a proof of concept hardware implementation of ETLP on FPGA to highlight the simplicity of its computational primitives and how they can be mapped into neuromorphic hardware for online learning with real-time interaction and low energy consumption.

A Multi-Plant Height Detection Method Based on Ruler-Free Monocular Computer Vision

Plant height is an important parameter of plant phenotype as one indicator of plant growth. In view of the complexity and scale limitation in current measurement systems, a scaleless method is proposed for the automatic measurement of plant height based on monocular computer vision. In this study, four peppers planted side by side were used as the measurement objects. Two color images of the measurement object were obtained by using a monocular camera at different shooting heights. Binary images were obtained as the images were processed by super-green grayscale and the Otsu method. The binarized images were transformed into horizontal one-dimensional data by the statistical number of vertical pixels, and the boundary points of multiple plants in the image were found and segmented into single-plant binarized images by filtering and searching for valleys. The pixel height was extracted from the segmented single plant image and the pixel displacement of the height was calculated, which was substituted into the calculation together with the reference height displacement to obtain the realistic height of the plant and complete the height measurements of multiple plants. Within the range of 2–3 m, under the light condition of 279 lx and 324 lx, this method can realize the rapid detection of multi-plant phenotypic parameters with a high precision and obtain more accurate plant height measurement results. The absolute error of plant height measurement is not more than ±10 mm, and the absolute proportion error is not more than ± 4%.

The Development of a Prototype Solution for Detecting Wear and Tear in Pedestrian Crossings

Crosswalks play a fundamental role in road safety. However, over time, many suffer wear and tear that makes them difficult to see. This project presents a solution based on the use of computer vision techniques for identifying and classifying the level of wear on crosswalks. The proposed system uses a convolutional neural network (CNN) to analyze images of crosswalks, determining their wear status. The design includes a prototype system mounted on a vehicle, equipped with cameras and processing units to collect and analyze data in real time as the vehicle traverses traffic routes. The collected data are then transmitted to a web application for further analysis and reporting. The prototype was validated through extensive tests in a real urban environment, comparing its assessments with manual inspections conducted by experts. Results from these tests showed that the system could accurately classify crosswalk wear with a high degree of accuracy, demonstrating its potential for aiding maintenance authorities in efficiently prioritizing interventions.

Intelligent treatment system based on bioinformatics and neuro-immune-digestive tract diseases

There are currently many clinical issues plaguing digestive oncologists, including the impact of dietary habits on the occurrence of early gastrointestinal cancer, the association between irritable bowel syndrome, ulcerative colitis, and early gastrointestinal cancer, the expected changes in gastrointestinal cancer after IBS, IBD, and susceptibility gene mutations, and the mechanisms by which susceptibility gene mutations in gastrointestinal cancer lead to different diseases. Meanwhile, as a carcinogen, betel nut is also prone to cancer and emotional disorders, and there are also many doctor-patient conflicts in the current clinical treatment of digestive tract tumors. To address the aforementioned issues, the research team analyzed dietary habits data using SOM clustering algorithm and isolated forest algorithm. Based on bioinformatics, new therapies have been developed for the pathogenesis and immune pathways of various cancers in the digestive system. Using network pharmacology methods, explore the toxicological mechanism of betel nut and its mechanism in mediating emotional disorders. Molecular docking is used to elucidate drug prediction. Based on four AI algorithms, the occurrence of gastrointestinal cancer was predicted through dietary habits. Using computer vision technology to address the deficiencies in early clinical reporting of cancer. This article provides clinical doctors with new treatment tools.

LeafNST: an improved data augmentation method for classification of plant disease using object-based neural style transfer

Plant diseases significantly threaten global agriculture, impacting crop yield and food security. Nearly 30% of the crop yield is lost due to plant diseases. Efficient identification and classification of plant diseases through computer vision techniques have become imperative for timely intervention. However, popular plant disease datasets often suffer from data imbalance, with certain classes underrepresented, hindering the performance of machine learning models. Traditional data augmentation methods, such as rotation and flipping, are limited in their effectiveness, especially when faced with imbalanced datasets. To address this limitation, we explore advanced data augmentation techniques, including Generative Adversarial Networks (GANs) such as CycleGAN and LeafGAN, which have shown promise in generating synthetic images. However, we propose an innovative approach of Object-based single Style Transfer on a single neural network for augmenting the plant disease dataset. This technique focuses on mitigating data imbalance issues within datasets, which can adversely affect the model’s ability to generalize across diverse classes. The proposed method is compared with state-of-the-art data augmentation techniques, highlighting its superiority in addressing data imbalance issues. Our approach aims to produce more realistic and diverse synthetic images, leading to improved model generalization and accuracy in plant disease classification tasks validated using different classifiers. The efficiency of our approach is validated through extensive experimentation and benchmarking against existing methods.

Semantic mapping techniques for indoor mobile robots: Review and prospect

With the continuous development of robotics and computer vision technology, mobile robots have been widely applied in various fields. In this process, semantic maps for robots have attracted considerable attention because they provide a comprehensive and anthropomorphic representation of the environment. On the one hand, semantic maps are a tool for robots to depict the environment, which can enhance the robot’s cognitive expression of space and build the communication bond between robots and humans. On the other hand, semantic maps contain spatial location and semantic properties of entities, which helps robots realize intelligent decision-making in human-centered indoor environments. In this paper, we review the primary approaches of semantic mapping proposed over the last few decades, and group them according to the type of information used to extract semantics. First, we give a formal definition of semantic map and describe the techniques of semantic extraction. Then, the characteristics of different solutions are comprehensively analyzed from different perspectives. Finally, the open issues and future trends regarding semantic maps are discussed in detail. We wish this review provides a comprehensive reference for researchers to drive future research in related field.

Impact of face swapping and data augmentation on sign language recognition

AbstractThis study addresses the challenge of improving communication between the deaf and hearing community by exploring different sign language recognition (SLR) techniques. Due to privacy issues and the need for validation by interpreters, creating large-scale sign language (SL) datasets can be difficult. The authors address this by presenting a new Spanish isolated sign language recognition dataset, CALSE-1000, consisting of 5000 videos representing 1000 glosses, with various signers and scenarios. The study also proposes using different computer vision techniques, such as face swapping and affine transformations, to augment the SL dataset and improve the accuracy of the model I3D trained using them. The results show that the inclusion of these augmentations during training leads to an improvement in accuracy in top-1 metrics by up to 11.7 points, top-5 by up to 8.8 points and top-10 by up to 9 points. This has great potential to improve the state of the art in other datasets and other models. Furthermore, the analysis confirms the importance of facial expressions in the model by testing with a facial omission dataset and shows how face swapping can be used to include new anonymous signers without the costly and time-consuming process of recording.

Streamlining electronic reporting of serious adverse events (SAEs) using the REDCap data collection system: the eSAE Project

BackgroundIt is essential that electronic data collection (EDC) systems are both compliant with regulations and the principles of Good Clinical Practice (GCP) to allow for the timely and accurate reporting of data including safety data. For clinical trials of investigational medicinal products (CTIMPs), investigators must immediately report to the sponsor any serious adverse event (SAE) that occurs in a site for which they are responsible. It is therefore expected that sponsors provide systems for timely review and reporting should a SAE be classified as a suspected unexpected serious adverse reaction (SUSAR). Challenges arise when data related to adverse events (AEs) needs to be re-entered for SAEs; this can be prone to error and may delay reporting. Additionally, recognising what has changed from an initial SAE report when an investigator responds to queries raised can cause errors.MethodA multi-disciplinary working group came together from a UK academic clinical trials unit (CTU) to establish if an electronic system could be created in the unit’s open-source EDC system—REDCap, to manage SAEs in an efficient way.ResultsA module has been created in REDCap to facilitate electronic SAE reporting: enabling an AE form to automatically trigger an SAE form for any AE which is also a SAE, prepopulating relevant fields of the SAE form, reducing the risk of delay and error when entering data into the SAE form. The system has also been developed with an embedded code to allow for instant visual recognition of any data updated following reporting to allow the sponsor to immediately review and resolve SAEs in a timely manner, complying with UK regulatory reporting. This functionality ‘The eSAE Project’ is now an active project for all of our new trials where data collection is undertaken using the REDCap system.ConclusionThe eSAE Project coded into REDCap offers a unique way of populating SAE forms with information already entered in the initial AE forms as applicable, coupled with highlighting any updates during the lifetime of the SAE for sponsors to identify any new information that needs to be reassessed to process and report the SAE.

LipPrint: Using Lip Movements as a Silent Password

Authentication and confidentiality are critical elements in many systems, and passwords are used to protect these systems. One way to enter the password into the system is via voice. The problem with this type of password is that people listen to it easily. The general purpose of this study was to make the password based on lip movement instead of sound. To address this problem, a visual speech recognition (VSR) system was proposed that detects speech from lip movement as a silent password. The proposed system consists of preprocessing the captured video via the proposed algorithms and feature extraction and classification of spoken digits via a modified VGG16. Fairness, robustness, and privacy are also considered in the proposed system. Research has focused on Arabic digits because there is little research on Arabic visual speech recognition. The reported results of the model show remarkable performance, with a validation accuracy of 96.74%. It was concluded that the proposed system, which relies on analysing lip movement instead of sound, increases security by preventing others from hearing the spoken password.

An Image-Retrieval Method Based on Cross-Hardware Platform Features

Artificial intelligence (AI) models have already achieved great success in fields such as computer vision and natural language processing. However, deploying AI models based on heterogeneous hardware is difficult to ensure accuracy consistency, especially for precision sensitive feature-based image retrieval. In this article, we realize an image-retrieval method based on cross-hardware platform features, aiming to prove that the features of heterogeneous hardware platforms can be mixed, in which the Huawei Atlas 300V and NVIDIA TeslaT4 are used for experiments. First, we compared the decoding differences of heterogeneous hardware, and used CPU software decoding to help hardware decoding improve the decoding success rate. Then, we compared the difference between the Atlas 300V and TeslaT4 chip architectures and tested the differences between the two platform features by calculating feature similarity. In addition, the scaling mode in the pre-processing process was also compared to further analyze the factors affecting feature consistency. Next, the consistency of capture and correlation based on video structure were verified. Finally, the experimental results reveal that the feature results from the TeslaT4 and Atlas 300V can be mixed for image retrieval based on cross-hardware platform features. Consequently, cross-platform image retrieval with low error is realized. Specifically, compared with the Atlas 300V hard and CPU soft decoding, the TeslaT4 hard decoded more than 99% of the image with a decoding pixel maximum difference of +1/−1. From the average of feature similarity, the feature similarity between the Atlas 300V and TeslaT4 exceeds 99%. The difference between the TeslaT4 and Atlas 300V in recall and mAP in feature retrieval is less than 0.1%.