Complex Object Recognition Research Articles

SNN using color-opponent and attention mechanisms for object recognition

The current spiking neural network (SNN) relies on spike-timing-dependent plasticity (STDP) primarily for shape learning in object recognition tasks, overlooking the equally critical aspect of color information. To address this gap, our study introduces an unsupervised variant of STDP that incorporates principles from color-opponency mechanisms (COM) and classical receptive fields (CRF) found in the biological visual system, facilitating the integration of color information during parameter updates within the SNN architecture. Our approach initially preprocesses images into two distinct feature maps: one for shape and another for color. Then, signals derived from COM and intensity concurrently drive the STDP process, thereby updating parameters associated with both color and shape feature maps. Furthermore, we propose a channel-wise attention mechanism to enhance differentiation among objects sharing similar shapes or colors. Specifically, this mechanism utilizes convolution to generate an output spike-wave, identifying a winner based on earliest spike timing and maximal potential. The winning kernel computes attention, which is then applied via convolution to each input image feature map, generating post-feature maps. A STDP-like normalization rule compares firing times between pre- and post-feature maps, dynamically adjusting channel weights to optimize object recognition during the training phase.We assessed the proposed algorithm using SNN with both single-layer and multi-layer architectures across three datasets. Experimental findings highlight its efficacy and superiority in complex object recognition tasks compared to state-of-the-art (SOTA) algorithms. Notably, our approach achieved a significant 20% performance improvement over the SOTA on the Caltech-101 dataset. Moreover, the algorithm is well-suited for hardware implementation and energy efficiency, leveraging a winner-selection mechanism based on the earliest spike time.

EXPERT BIOINFORMATION SYSTEM FOR DIAGNOSING FORMS OF ACUTE LEUKEMIA BASED ON ANALYSIS OF BIOMEDICAL INFORMATION

The introductory chapter established the context for this paper by stressing the significance of leukemia in healthcare and the challenges associated with both diagnosis and therapy. The paper ultimate objective is to provide an information technology solution to these issues, thereby improving patient care and prognosis. A conceptual model of an expert system for the diagnosis of acute leukemia is proposed, which will reduce the ambiguity in the interpretation of research objects. Factors influencing the correct recognition of complex objects (images of blast and non-blast blood cells) using an expert system based on computer microscopy methods are considered.

Research on Educational Robot System Based on Vision Processing.

Aimed at the poor recognition effect of current educational robots on objects with complex shapes and colors and the single design of related experiments, this paper proposes a robot teaching instrument. The robot adopts a servo motor with an encoder, a drive, and a variety of sensors to realize a motor current loop, speed loop, position loop, and closed-loop control functions. Three experimental schemes were designed: a PID adjustment experiment, a robot obstacle avoidance and object-grasping program writing experiment, and a complex object recognition experiment based on cascade classifiers. The robot is conducive to improving students' self-initiative ability, deepening their understanding of PID closed-loop control, multi-sensor fusion, and deep learning knowledge. It can improve students' programming ability, enabling them to effectively combine theory and practice, as well as to comprehensively apply professional knowledge.

Diabetic Neovascularization Identification from Fundus Images Retinopathy

Proliferative Diabetic Retinopathy (PDR) is a retinal disease that can affect people with diabetes and cause visual loss if left untreated. Detecting neovascularization, an abnormal growth of veins in the retina, can be difficult due to its irregular pattern and small size. To improve detection, deep learning algorithms, such as MobileNet, are being used to automate complex object recognition. In a neovascularization affirmation technique based on transfer learning, multiple pre-trained models were built during the training phase, including MobileNet, CNN with SVM, AlexNet, GoogleNet, ResNet, ResNet18, and ResNet and GoogleNet models. Machine learning models for HOG feature extraction were also implemented, such as Random Forest, Decision Tree, Gradient Boosting, Support Vector Classifier, and Voting Classifier. MobileNet performed the best and was used to build the model for predicting results from user-uploaded images.

Semantic Segmentation and Building Extraction from Airborne LiDAR Data with Multiple Return Using PointNet++

Light detection and ranging (LiDAR) data of 3D point clouds acquired from laser sensors is a crucial form of geospatial data for recognition of complex objects since LiDAR data provides geometric information in terms of 3D coordinates with additional attributes such as intensity and multiple returns. In this paper, we focused on utilizing multiple returns in the training data for semantic segmentation, in particular building extraction using PointNet++. PointNet++ is known as one of the efficient and robust deep learning (DL) models for processing 3D point clouds. On most building boundaries, two returns of the laser pulse occur. The experimental results demonstrated that the proposed approach could improve building extraction by adding two returns to the training datasets. Specifically, the recall value of the predicted building boundaries for the test data was improved from 0.7417 to 0.7948 for the best case. However, no significant improvement was achieved for the new data because the new data had relatively lower point density compared to the training and test data.

Federated Onboard-Ground Station Computing With Weakly Supervised Cascading Pyramid Attention Network for Satellite Image Analysis

With advances in NanoSat (CubeSat) and high-resolution sensors, the amount of raw data to be analyzed by human supervisors has been explosively increasing for satellite image analysis. To reduce the raw data, the satellite onboard AI processing with low-power COTS (Commercial, Off-The-Shelf) HW has emerged from a real satellite mission. It filters the useless data (e.g. cloudy images) that is worthless to supervisors, achieving efficient satellite-ground station communication. In the application for complex object recognition, however, additional explanation is required for the reliability of the AI prediction due to its low performance. Although various eXplainable AI (XAI) methods for providing human-interpretable explanation have been studied, the pyramid architecture in a deep network leads to the background bias problem which visual explanation only focuses on the background context around the object. Missing the small objects in a tiny region leads to poor explainability although the AI model corrects the object class. To resolve the problems, we propose a novel federated onboard-ground station (FOGS) computing with Cascading Pyramid Attention Network (CPANet) for reliable onboard XAI in object recognition. We present an XAI architecture with a cascading attention mechanism for mitigating the background bias for the onboard processing. By exploiting the localization ability in pyramid feature blocks, we can extract high-quality visual explanation covering the both semantic and small contexts of an object. For enhancing visual explainability of complex satellite images, we also describe a novel computing federation with the ground station and supervisors. In the ground station, active learning-based sample selection and attention refinement scheme with a simple feedback method are conducted to achieve the robustness of explanation and efficient supervisor’s annotation cost, simultaneously. Experiments on various datasets show that the proposed system improves accuracy in object recognition and accurate visual explanation detecting small contexts of objects even in a peripheral region. Then, our attention refinement mechanism demonstrates that the inconsistent explanation can be efficiently resolved only with very simple selection-based feedback.

Improved object-based convolutional neural network (IOCNN) to classify very high-resolution remote sensing images

ABSTRACT The land cover classification of very high-resolution (VHR) remote sensing images is a challenging task. VHR images depict many complex objects with various shapes in complicated contexts. The deep learning-based method is a solution for such dif- ficult task and feature extraction. Nevertheless, this method cannot efficiently deal with images with complex scene structures. An improved object-based convolutional neural network (IOCNN) is designed to classify VHR images with zone division and convolutional position sampling techniques in this study. The method can achieve the best performance of each zone at its own optimized scales. Based on multi-scale convolutional deep features extracted from VHR images, the objects with irregular shapes can be classified using the approach. In this study, the zone-level scale adaption and multi-scale recognition of complex objects are achieved. The performance of IOCNN is compared with the state-of-the-art methods for feature extraction, including five object-based CNN approaches and two fully convolutional networks (FCNs). The results show that the classification performance of IOCNN is considerably stronger than that of state-of-the-art methods. The overall accuracies of the land cover classification in IOCNN are 91.65% and 93.49% on two tested images. The results demonstrate the practicability of IOCNN.

An FPGA-Based Energy-Efficient Reconfigurable Convolutional Neural Network Accelerator for Object Recognition Applications

The computational efficiency is the prime concern of a computation-intensive deep convolutional neural network (CNN). In this Brief, we report an FPGA-based computation-efficient reconfigurable CNN accelerator. It innovates in the utilization of a kernel partition technique to substantially reduce the repeated access to the input feature maps and the kernels. As a result, it balances the ability for parallel computing while consuming less system power. Experimental results prove that the proposed CNN accelerator achieves a peak throughput of 220.0 GOP/s with an energy efficiency of 22.9 GOPs/W at 151.4 frames/s for the AlexNet. It is also reconfigurable to process VGG-16 befitting complex object recognition.

Linear Integration of Sensory Evidence over Space and Time Underlies Face Categorization.

Visual object recognition relies on elaborate sensory processes that transform retinal inputs to object representations, but it also requires decision-making processes that read out object representations and function over prolonged time scales. The computational properties of these decision-making processes remain underexplored for object recognition. Here, we study these computations by developing a stochastic multifeature face categorization task. Using quantitative models and tight control of spatiotemporal visual information, we demonstrate that human subjects (five males, eight females) categorize faces through an integration process that first linearly adds the evidence conferred by task-relevant features over space to create aggregated momentary evidence and then linearly integrates it over time with minimum information loss. Discrimination of stimuli along different category boundaries (e.g., identity or expression of a face) is implemented by adjusting feature weights of spatial integration. This linear but flexible integration process over space and time bridges past studies on simple perceptual decisions to complex object recognition behavior.SIGNIFICANCE STATEMENT Although simple perceptual decision-making such as discrimination of random dot motion has been successfully explained as accumulation of sensory evidence, we lack rigorous experimental paradigms to study the mechanisms underlying complex perceptual decision-making such as discrimination of naturalistic faces. We develop a stochastic multifeature face categorization task as a systematic approach to quantify the properties and potential limitations of the decision-making processes during object recognition. We show that human face categorization could be modeled as a linear integration of sensory evidence over space and time. Our framework to study object recognition as a spatiotemporal integration process is broadly applicable to other object categories and bridges past studies of object recognition and perceptual decision-making.

Design of Low-Cost Object Identification Module for Culinary Applications

Using up one’s fridge’s contents often requires touch creativity, which is typically hard to come by in a very busy world. This aim is to style a model that may scan the things in one’s fridge and recommend recipes that supported what one has got, even taking one’s preferences and dietary restrictions under consideration. Computers are commencing to process globe objects without the requirement for codes or human intervention in their language. This project aims to create a module that uses object recognition to detect vegetables and fruits and display recipes that include those foods. This concept uses a camera that is installed in a refrigerator to show it into a sensible one. The system identifies objects that are placed inside using complex object recognition algorithms using the camera. If a vegetable or fruit is placed inside, the system will identify it and displays its name on the screen. It creates an indication of the chances of object recognition. The platform enables users to put any number of various vegetables into the refrigerator. The system then checks a list of recipes that contain the ingredients inside the refrigerator and filters those that feature the vegetables available. Together with this, we also aim to watch the refrigerator’s contents and find the freshness and age using various sensors and provide inventory and warnings when they are on the verge of completion.

Object expectations alter information use during visual recognition

Prior expectations influence how we perceive and recognize objects. However, how they do so remains unclear, especially in the case of real-world complex objects. Expectations of objects may affect which features are used to recognize them subsequently. In this study, we used reverse correlation to reveal with high precision how the use of information across time is modulated by real-world object expectations in a visual recognition task. We show that coarse information leads to accurate responses earlier when an object is expected, indicating that observers use diagnostic features earlier in this situation. We also demonstrate an increased variability in the use of coarse information depending on the expected object, indicating that observers adopt a more specialized recognition strategy when they expect a specific object. In summary, our results reveal potential mechanisms underlying the effect of expectations on the recognition of complex objects.

Clustered functional domains for curves and corners in cortical area V4.

The ventral visual pathway is crucially involved in integrating low-level visual features into complex representations for objects and scenes. At an intermediate stage of the ventral visual pathway, V4 plays a crucial role in supporting this transformation. Many V4 neurons are selective for shape segments like curves and corners; however, it remains unclear whether these neurons are organized into clustered functional domains, a structural motif common across other visual cortices. Using two-photon calcium imaging in awake macaques, we confirmed and localized cortical domains selective for curves or corners in V4. Single-cell resolution imaging confirmed that curve- or corner-selective neurons were spatially clustered into such domains. When tested with hexagonal-segment stimuli, we find that stimulus smoothness is the cardinal difference between curve and corner selectivity in V4. Combining cortical population responses with single-neuron analysis, our results reveal that curves and corners are encoded by neurons clustered into functional domains in V4. This functionally specific population architecture bridges the gap between the early and late cortices of the ventral pathway and may serve to facilitate complex object recognition.

Capacity of Remote Classification Over Wireless Channels

Remote classification involves offloading complex object-recognition tasks from mobile devices to servers at the network edge. It brings to the mobile device the capability of discerning hundreds of object classes by using the computational and storage capabilities of the infrastructure. Remote classification is challenged by the finite and variable data rate of the wireless channel, which affects the capability to transfer high-dimensional features and thus limits the classification resolution. We introduce a set of metrics under the name of classification capacity that are defined as the maximum number of classes that can be discerned over a given communication channel while meeting a target probability for classification error. We treat both the cases of a channel where the instantaneous rate is known and unknown. The objective is to choose a subset of classes from a class library that offers satisfactory performance over a given channel. We treat two different cases of subset selection. First, a device can select the subset by pruning the class library until arriving at a subset that meets the targeted error probability while maximizing the classification capacity. Adopting a subspace data model, we prove the equivalence of classification capacity maximization to the problem of packing on the Grassmann manifold. The results show that the classification capacity grows exponentially with the instantaneous communication rate, and super-exponentially with the dimensions of each data cluster. This also holds for ergodic and outage capacities with fading if the instantaneous rate is replaced with an average rate and a fixed rate, respectively. In the second case, a device has a unique preference of class subset for every communication rate, which is modeled as an instance of uniformly sampling the library. Without class selection, the classification capacity and its ergodic and outage counterparts are proved to scale linearly with their corresponding communication rates instead of the exponential growth in the last case.

Using deep neural networks to evaluate object vision tasks in rats.

In the last two decades rodents have been on the rise as a dominant model for visual neuroscience. This is particularly true for earlier levels of information processing, but a number of studies have suggested that also higher levels of processing such as invariant object recognition occur in rodents. Here we provide a quantitative and comprehensive assessment of this claim by comparing a wide range of rodent behavioral and neural data with convolutional deep neural networks. These networks have been shown to capture hallmark properties of information processing in primates through a succession of convolutional and fully connected layers. We find that performance on rodent object vision tasks can be captured using low to mid-level convolutional layers only, without any convincing evidence for the need of higher layers known to simulate complex object recognition in primates. Our approach also reveals surprising insights on assumptions made before, for example, that the best performing animals would be the ones using the most abstract representations-which we show to likely be incorrect. Our findings suggest a road ahead for further studies aiming at quantifying and establishing the richness of representations underlying information processing in animal models at large.

Automated knowledge-assisted mitosis cells detection framework in breast histopathology images.

Based on the Nottingham Histopathology Grading (NHG) system, mitosis cells detection is one of the important criteria to determine the grade of breast carcinoma. Mitosis cells detection is a challenging task due to the heterogeneous microenvironment of breast histopathology images. Recognition of complex and inconsistent objects in the medical images could be achieved by incorporating domain knowledge in the field of interest. In this study, the strategies of the histopathologist and domain knowledge approach were used to guide the development of the image processing framework for automated mitosis cells detection in breast histopathology images. The detection framework starts with color normalization and hyperchromatic nucleus segmentation. Then, a knowledge-assisted false positive reduction method is proposed to eliminate the false positive (i.e., non-mitosis cells). This stage aims to minimize the percentage of false positive and thus increase the F1-score. Next, features extraction was performed. The mitosis candidates were classified using a Support Vector Machine (SVM) classifier. For evaluation purposes, the knowledge-assisted detection framework was tested using two datasets: a custom dataset and a publicly available dataset (i.e., MITOS dataset). The proposed knowledge-assisted false positive reduction method was found promising by eliminating at least 87.1% of false positive in both the dataset producing promising results in the F1-score. Experimental results demonstrate that the knowledge-assisted detection framework can achieve promising results in F1-score (custom dataset: 89.1%; MITOS dataset: 88.9%) and outperforms the recent works.

Deep Neural Networks Point to Mid-level Complexity of Rodent Object Vision

In the last two decades rodents have been on the rise as a dominant model for visual neuroscience. This is particularly true for earlier levels of information processing, but high-profile papers have suggested that also higher levels of processing such as invariant object recognition occur in rodents. Here we provide a quantitative and comprehensive assessment of this claim by comparing a wide range of rodent behavioral and neural data with convolutional deep neural networks. These networks have been shown to capture the richness of information processing in primates through a succession of convolutional and fully connected layers. We find that rodent object vision can be captured using low to mid-level convolutional layers only, without any convincing evidence for the need of higher layers known to simulate complex object recognition in primates. Our approach also reveals surprising insights on assumptions made before, for example, that the best performing animals would be the ones using the most complex representations - which we show to likely be incorrect. Our findings suggest a road ahead for further studies aiming at quantifying and establishing the richness of representations underlying information processing in animal models at large.

Sulcal morphology of ventral temporal cortex is shared between humans and other hominoids

Hominoid-specific brain structures are of particular importance in understanding the evolution of human brain structure and function, as they are absent in mammals that are widely studied in the extended neuroscience field. Recent research indicates that the human fusiform gyrus (FG), which is a hominoid-specific structure critical for complex object recognition, contains a tertiary, longitudinal sulcus (mid-fusiform sulcus, MFS) that bisects the FG into lateral and medial parallel gyri. The MFS is a functional and architectonic landmark in the human brain. Here, we tested if the MFS is specific to the human FG or if the MFS is also identifiable in other hominoids. Using magnetic resonance imaging and cortical surface reconstructions in 30 chimpanzees and 30 humans, we show that the MFS is also present in chimpanzees. The MFS is relatively deeper and cortically thinner in chimpanzees compared to humans. Additional histological analyses reveal that the MFS is not only present in humans and chimpanzees, but also in bonobos, gorillas, orangutans, and gibbons. Taken together, these results reveal that the MFS is a sulcal landmark that is shared between humans and other hominoids. These results require a reconsideration of the sulcal patterning in ventral temporal cortex across hominoids, as well as revise the compensation theory of cortical folding.

A Sustainable Deep Learning Framework for Object Recognition Using Multi-Layers Deep Features Fusion and Selection

With an overwhelming increase in the demand of autonomous systems, especially in the applications related to intelligent robotics and visual surveillance, come stringent accuracy requirements for complex object recognition. A system that maintains its performance against a change in the object’s nature is said to be sustainable and it has become a major area of research for the computer vision research community in the past few years. In this work, we present a sustainable deep learning architecture, which utilizes multi-layer deep features fusion and selection, for accurate object classification. The proposed approach comprises three steps: (1) By utilizing two deep learning architectures, Very Deep Convolutional Networks for Large-Scale Image Recognition and Inception V3, it extracts features based on transfer learning, (2) Fusion of all the extracted feature vectors is performed by means of a parallel maximum covariance approach, and (3) The best features are selected using Multi Logistic Regression controlled Entropy-Variances method. For verification of the robust selected features, the Ensemble Learning method named Subspace Discriminant Analysis is utilized as a fitness function. The experimental process is conducted using four publicly available datasets, including Caltech-101, Birds database, Butterflies database and CIFAR-100, and a ten-fold validation process which yields the best accuracies of 95.5%, 100%, 98%, and 68.80% for the datasets respectively. Based on the detailed statistical analysis and comparison with the existing methods, the proposed selection method gives significantly more accuracy. Moreover, the computational time of the proposed selection method is better for real-time implementation.

Efficient complex ISAR object recognition using adaptive deep relation learning

Complex inverse synthetic aperture radar (ISAR) object recognition is a critical and challenging problem in computer vision tasks. An efficient complex object recognition method for ISAR images is proposed based on adaptive deep relation learning. (i) An adaptive multimodal mechanism is proposed to greatly improve the multimodal sampling and transformation capabilities of convolutional neural networks and increase the resolutions of the feature maps. In particular, an adaptive ranking and selection strategy for related regions are proposed. (ii) Deep graphical learning is proposed to jointly estimate large numbers of heterogeneous attributes and leverage the relations among features and attributes. Extensive experiments performed on real-world datasets show that the proposed method outperforms other state-of-the-art methods.

Multiscale Time-Sharing Elastography Algorithms and Transfer Learning of Clinicopathological Features of Uterine Cervical Cancer for Medical Intelligent Computing System.

Intelligent medical diagnosis and computing system faces many challenges in complex object recognition, large-scale data imaging and real-time diagnosis, such as poor real-time computing, low efficiency of data storage and low recognition rate of lesions. In order to solve the above problems, this paper proposes a medical intelligent computing system and a series of algorithms for the clinical pathology of cervical cancer based on the multi-scale imaging and transfer learning framework. Firstly, based on data dimensions, imaging errors and other factors, this paper designs a multi-scale time-sharing elastic imaging algorithm based on image reconstruction time and data sample characteristics. Then, taking the burst imaging cohort and the calculation data set of new cervical cancer cases as the objects, based on the difficulties of cervical cancer feature modeling, this paper proposes the transfer learning algorithm of clinical and pathological features of cervical cancer. Finally, a medical intelligent computing system for cervical cancer pathology analysis and calculation with high efficiency and reliability is established. A series of proposed algorithms are compared with single-scale Retinex (SSR), which is based on single-scale Retinex migration learning (SSR-TL). The experimental results show that the proposed algorithm in cervical cancer pathological imaging and scoring, as well as the feature extraction and recognition of lesions, especially the efficiency of system execution, is obviously due to the comparison algorithm.