Human Object Detection Research Articles

Fusing human action recognition and object detection for human-robot collaborative assembly

Fusing human action recognition and object detection for human-robot collaborative assembly

Auxiliary Feature Fusion and Noise Suppression for HOI Detection

In recent years, one-stage HOI (Human–Object Interaction) detection methods tend to divide the original task into multiple sub-tasks by using a multi-branch network structure. However, there is no sufficient attention to information communication between these branches. The inference approach in the cascaded structure is singular, while fully parallel methods will disrupt the associations between different pieces of information. Besides, noise interference may occur during the fusion of different features and thus affect the detection performance. To address these issues, this article proposes a one-stage three-branch parallel HOI detection method, which treats HOI as three separate sub-tasks (human detection, object detection, and interaction detection) and leverages three distinct reasoning relationships to generate richer relational information. Firstly , an auxiliary feature fusion (AFF) module is introduced, which integrates features originally extracted independently to form fused features enriched with supplementary information. This approach strengthens communication between branches in the network while handling the three sub-tasks concurrently, thereby facilitating the exchange of more contextual information. Secondly , to mitigate noise interference generated during the fusion process, a fusion noise suppression (FNS) module is introduced, which effectively suppresses noise and enhances the model’s performance in interaction detection tasks. Finally , experiments are conducted on two major benchmark datasets, and experimental results show that our HOI detection method is superior to previous methods. Also, ablation studies confirm the effectiveness of all the components in our proposed method.

Pose-oriented scene-adaptive matching for abnormal event detection

For intelligent surveillance systems, abnormal event detection automatically analyses surveillance video sequences and detects abnormal objects or unusual human actions at the frame level. Due to the lack of labelled data, most approaches are semi-supervised based on reconstruction or prediction methods. However, these methods may not generalize well to unseen scene contexts. To address this issue, we present a novel self and mutual scene-adaptive matching method for abnormal event detection. In the framework, we propose synergistic pose estimation and object detection, which effectively integrates human pose and object detection information to improve pose estimation accuracy. Then, the poses are resized to reduce the spatial distance between the source and target domains. The improved pose sequences are further fed into a spatio-temporal graph convolutional network to extract the geometric features. Finally, the features are embedded in a clustering layer to classify action types and compute normality scores. The training data is taken from the training part of common video anomaly detection datasets: UCSD PED1 & PED2, CHUK Avenue, and ShanghaiTech Campus. The proposed framework is evaluated on video sequences with unseen scene contexts in the UCSD PED2 and ShanghaiTech Campus datasets. The detection accuracy and efficiency are also evaluated in detail, and the proposed method for abnormal event detection achieves the highest AUC performance, 84.6%, on the ShanghaiTech Campus dataset and relatively high AUC performance, 96.9% and 74.8%, on UCSD PED2 & PED1 datasets. Compared with other state-of-the-art works, the performance analysis and results confirm the robustness and effectiveness of our proposed framework for cross-scene abnormal event detection.

Intelligent Video Analytics for Abnormal Event Detection

Surveillance cameras have become a primary tool for monitoring human movement and preventing unwanted or unintended activities. In today's world, security management professionals increasingly rely on video surveillance to combat crime and mitigate incidents that could adversely affect society. The number of surveillance camera installations has dramatically increased in both the private and public sectors, serving to monitor public activities effectively. Video surveillance is considered one of the most efficient methods for ensuring security. Installing a surveillance camera allows the captured footage to be transmitted to security personnel; however, merely having video footage is insufficient for identifying abnormal activities. To effectively analyze this footage, it is essential to integrate an intelligent system.This paper aims to design and implement an Intelligent Video Analytics Model (IVAM), also known as the Human Object Detection (HOD) method, for analyzing and detecting video-based anomalies and abnormal human activities. IVAM can be deployed alongside surveillance cameras in various public locations such as Institutions, airports, hospitals, shopping malls, and railway stations, allowing for the automatic identification of unusual events. The IVAM was experimented with using MATLAB software, and the results were thoroughly verified. The performance of IVAM was assessed by comparing the obtained results with those from existing approaches, demonstrating that IVAM outperforms contemporary methods concerning accurate anomaly detection, lower error rates, and higher classification accuracy.

Third-Person View Attention Prediction in Natural Scenarios With Weak Information Dependency and Human-Scene Interaction Mechanism

First-person view attention has been widely studied in computer science domain since 1990s while third-person view attention in natural scenarios begins to gain the intensive interest until 2015. This paper focuses on the problem of third-person view attention prediction in natural scenarios where a human freely performs daily activities without constraints. To handle the two insuffiencies of existing methods: (i) assuming some extra information (except for input images) are given in advance and (ii) ignoring the importance of human-scene interaction, this paper proposes a model with weak information dependency, which helps to alleviate annotation costs. In addition, a transformer-based human-scene interaction mechanism is proposed to explore the global and long-dependency contexts between the human and scene. The pipeline of the proposed model is firstly extracting human and scene features, then inferring human attention probability map by fusing human and scene features via a transformer-based network, and finally predicting human attention object based on human attention probability map and object detection. The experiments on two public datasets validate the effectiveness of our model.

UAV Infrared Image Human and Vehicle Object Detection Based on YOLOv5

UAV Infrared Image Human and Vehicle Object Detection Based on YOLOv5

TCFormer: Visual Recognition via Token Clustering Transformer.

Transformers are widely used in computer vision areas and have achieved remarkable success. Most state-of-the-art approaches split images into regular grids and represent each grid region with a vision token. However, fixed token distribution disregards the semantic meaning of different image regions, resulting in sub-optimal performance. To address this issue, we propose the Token Clustering Transformer (TCFormer), which generates dynamic vision tokens based on semantic meaning. Our dynamic tokens possess two crucial characteristics: (1) Representing image regions with similar semantic meanings using the same vision token, even if those regions are not adjacent, and (2) concentrating on regions with valuable details and represent them using fine tokens. Through extensive experimentation across various applications, including image classification, human pose estimation, semantic segmentation, and object detection, we demonstrate the effectiveness of our TCFormer. The code and models for this work are available at https://github.com/zengwang430521/TCFormer.

Peak response regularization for localization

Deep convolutional neural networks approaches often assume that the feature response has a Gaussian distribution with target-centered peak response, which can be used to guide the target location and classification. Nevertheless, such an assumption is implausible when there is progressive interference from other targets and/or background noise, which produces sub-peaks on the tracking response map and causes model drift. In this paper, we propose a feature response regularization approach for sub-peak response suppression and peak response enforcement and aim to handle progressive interference systematically. Our approach, referred to as Peak Response Regularization (PRR), applies simple-yet-efficient method to aggregate and align discriminative features, which convert local extremal response in discrete feature space to extremal response in continuous space, which enforces the localization and representation capability of convolutional features. Experiments on human pose detection, object detection, object tracking, and image classification demonstrate that PRR improves the performance of image tasks with a negligible computational cost.

Insights into aerial intelligence: assessing CNN-based algorithms for human action recognition and object detection in diverse environments

Insights into aerial intelligence: assessing CNN-based algorithms for human action recognition and object detection in diverse environments

HDetect-VS: Tiny Human Object Enhancement and Detection Based on Visual Saliency for Maritime Search and Rescue

Strong sun glint noise is an inevitable obstruction for tiny human object detection in maritime search and rescue (SAR) tasks, which can significantly deteriorate the performance of local contrast method (LCM)-based algorithms and cause high false alarm rates. For SAR tasks in noisy environments, it is more important to find tiny objects than localize them. Hence, considering background clutter and strong glint noise, in this study, a noise suppression methodology for maritime scenarios (HDetect-VS) is established to achieve tiny human object enhancement and detection based on visual saliency. To this end, the pixel intensity value distributions, color characteristics, and spatial distributions are thoroughly analyzed to separate objects from background and glint noise. Using unmanned aerial vehicles (UAVs), visible images with rich details, rather than infrared images, are applied to detect tiny objects in noisy environments. In this study, a grayscale model mapped from the HSV model (HSV-gray) is used to suppress glint noise based on color characteristic analysis, and large-scale Gaussian Convolution is utilized to obtain the pixel intensity surface and suppress background noise based on pixel intensity value distributions. Moreover, based on a thorough analysis of the spatial distribution of objects and noise, two-step clustering is employed to separate objects from noise in a salient point map. Experiments are conducted on the SeaDronesSee dataset; the results illustrate that HDetect-VS has more robust and effective performance in tiny object detection in noisy environments than other pixel-level algorithms. In particular, the performance of existing deep learning-based object detection algorithms can be significantly improved by taking the results of HDetect-VS as input.

Research on a Human Moving Object Detection Method Based on Gaussian Model and Deep Learning

In order to understand human motion object detection methods, the author proposes a research on human motion object detection method based on Gaussian model. Firstly, traditional Gaussian models are unable to detect complex scenes or slow moving targets. Therefore, an improved Gaussian model based moving object detection algorithm is proposed. Secondly, multiple Gaussian models are used to represent the features of each pixel in the moving target image, and based on the matching of each pixel in the image with the Gaussian model, it is considered as a background point. Conversely, it is based on the principle of the foreground, and the Gaussian model is updated. Finally, by updating the foreground model and calculating short-term stability indicators, the detection effect of moving targets is improved. By determining the Gaussian distribution and pixel relationship, new parameters are set to construct the background model and eliminate the impact caused by sudden changes in lighting. The experimental analysis results show that this method can effectively detect and track moving targets, with good noise resistance, high clarity, and an accuracy rate of up to 99%. Compared with traditional Gaussian model methods, the improved method can more effectively detect moving targets and has better robustness.

The improvement of ground truth annotation in public datasets for human detection

The quality of annotations in the datasets is crucial for supervised machine learning as it significantly affects the performance of models. While many public datasets are widely used, they often suffer from annotations errors, including missing annotations, incorrect bounding box sizes, and positions. It results in low accuracy of machine learning models. However, most researchers have traditionally focused on improving model performance by enhancing algorithms, while overlooking concerns regarding data quality. This so-called model-centric AI approach has been predominant. In contrast, a data-centric AI approach, advocated by Andrew Ng at the DATA and AI Summit 2022, emphasizes enhancing data quality while keeping the model fixed, which proves to be more efficient in improving performance. Building upon this data-centric approach, we propose a method to enhance the quality of public datasets such as MS-COCO and Open Image Dataset. Our approach involves automatically retrieving missing annotations and correcting the size and position of existing bounding boxes in these datasets. Specifically, our study deals with human object detection, which is one of the prominent applications of artificial intelligence. Experimental results demonstrate improved performance with models such as Faster-RCNN, EfficientDet, and RetinaNet. We can achieve up to 32% compared to original datasets in the term of mAP after applying both proposed methods to dataset which is transformed the grouped of instances to individual instance. In summary, our methods significantly enhance the model’s performance by improving the quality of annotations at a lower cost with less time than manual improvement employed in other studies.

Two-Stage Method for Clothing Feature Detection

The rapid expansion of e-commerce, particularly in the clothing sector, has led to a significant demand for an effective clothing industry. This study presents a novel two-stage image recognition method. Our approach distinctively combines human keypoint detection, object detection, and classification methods into a two-stage structure. Initially, we utilize open-source libraries, namely OpenPose and Dlib, for accurate human keypoint detection, followed by a custom cropping logic for extracting body part boxes. In the second stage, we employ a blend of Harris Corner, Canny Edge, and skin pixel detection integrated with VGG16 and support vector machine (SVM) models. This configuration allows the bounding boxes to identify ten unique attributes, encompassing facial features and detailed aspects of clothing. Conclusively, the experiment yielded an overall recognition accuracy of 81.4% for tops and 85.72% for bottoms, highlighting the efficacy of the applied methodologies in garment categorization.

ABYOLOv4: improved YOLOv4 human object detection based on enhanced multi-scale feature fusion

The purpose of human object detection is to obtain the number of people and their position in images, which is one of the core problems in the field of machine vision. However, the high missing detection rate from small- and medium-sized human bodies due to the large variety of human scale in human object detection tasks still influences the performance of human object detection. To solve the above problem, this paper proposed an improved ASPP_BiFPN_YOLOv4 (ABYOLOv4) method to detect human object detection. In detail, Atrous Spatial Pyramid Pooling (ASPP) module was used to replace the original Spatial Pyramid Pooling module to increase the receptive field level of the network and improve the perception ability of multi-scale targets. Then, the original Path Aggregation Network (PANet) multi-scale fusion module was replaced by the self-built bi-layer bidirectional feature pyramid network (Bi-FPN). Meanwhile, a new feature was imported into the proposed model to reuse the mid- and low-level features, which could enhance the ability of the network to express the characteristics of small- and medium-sized targets. Finally, the standard convolution in Bi-FPN was replaced by depth-separable convolution to make the network achieve the balance of accuracy and the number of parameters. To identify the performance of the proposed ABYOLOv4 model, the human object detection experiment is carried out by using the public data set of VOC2007 and VOC2012, the improved YOLOv4 algorithm is 0.5% higher than the original AP algorithm, and the weight file size of the model is reduced by 45.3 M. The experimental results demonstrated that the proposed ABYOLOv4 network has higher accuracy and lower computational cost for human target detection.

Fishing risky behavior recognition based on adaptive transformer, reinforcement learning and stochastic configuration networks

The fishing behavior in a high-voltage environment may lead to electric shock accidents. This paper proposes a fishing risky behavior recognition method based on adaptive Transformer, reinforcement learning and stochastic configuration networks (SCNs). Firstly, the fishing behavior image sample training set is evaluated using the actor network based on linear entropy in the reinforcement learning module to select suitable training samples that possess abundant straight-line elements. Subsequently, the selected training samples are fed into a multi-scale spatial deep convolution to extract continuous spatial features of elongated objects. The deformable Transformer network, enhanced with adaptive encoding and decoding layers through SCNs, is used to obtain the position and detection results of the fishing rod. The trained Transformer model is evaluated by a defined credibility evaluation metric for acquiring rewards to update the training sample set iteratively. Then, an adaptive adjustment mechanism is constructed for the encoding and decoding layers of the adaptive Transformer network to establish a library of adaptive Transformer models with different encoding and decoding layer levels to accommodate the feature requirements of training samples in various scenarios. Finally, the blending learning method for combining the detection results from the model library is integrated with human body object detection and pose estimation methods, and a predefined expert system is utilized for logic reasoning on fishing risky behavior. Experimental results demonstrate the effectiveness of the proposed method in this paper.

Physiological characteristics inspired hidden human object detection model

The current target detection algorithms provide the unsatisfactory performance on the task of detecting hidden human targets. In this study, we put forward the physiological characteristics inspired hidden human object detection model considering the spatio-temporal physiological features and their interdependent relationships. The experimental results of homemade hidden human object dataset demonstrate that the proposed model generates the detection accuracy of 64%, 44%, and 54% for indoor scene, outdoor scene, and overall dataset, respectively, outperforming the YOLO v4 models and the models based on HOG, LBP, and Haar features, with at least 22% promotion in detection accuracy. The ablation experiments indicate the effectiveness of each module of the method. In the future, the proposed model or the corresponding modeling idea has the potential to be applied to military rescue, public security investigation and other fields. Once the paper is accepted, we will make the homemade dataset publicly available.

Parallel disentangling network for human–object interaction detection

Human–object interaction (HOI) detection aims to localize and classify triplets of human, object and interaction from a given image. Earlier two-stage methods suffer both from mutually independent training processes and the interference of redundant negative human–object pairs. Prevailing one-stage transformer-based methods are free from the above problems by tackling HOI in an end-to-end manner. However, one-stage transformer-based methods carry the unnecessary entanglements of the query for different tasks, i.e., human–object detection and interaction classification, and thus bring in poor performance. In this paper, we propose a new transformer-based approach that parallelly disentangles human–object detection and interaction classification in a triplet-wise manner. To make each query focus on one specific task clearly, we exhaustively disentangle HOI by parallelly expanding the naive query in vanilla transformer as triple explicit queries. Then, we introduce a semantic communication layer to preserve the consistent semantic association of each HOI through mixing the feature representations of each query triplet of the correspondence constraint. Extensive experiments demonstrate that our proposed framework outperforms the existing methods and achieves the state-of-the-art performance, with significant reduction in parameters and FLOPs.

Detecting Objects Using Haar Cascade for Human Counting Implemented in OpenMV

Sight is a fundamental sense for humans, and individuals with visual impairments often rely on assistance from others or tools that promote independence in performing various tasks. One crucial aspect of aiding visually impaired individuals involves the detection and counting of objects. This paper aims to develop a simulation tool designed to assist visually impaired individuals in detecting and counting human objects. The tool's implementation necessitates a synergy of both hardware and software components, with OpenMV serving as a central hardware device in this study. The research software was developed using the Haar Cascade Classifier algorithm. The research process commences with the acquisition of image data through the OpenMV camera. Subsequently, the image data undergoes several stages of processing, including the utilization of the Haar Cascade classifier method within the OpenMV framework. The resulting output consists of bounding boxes delineating the detection areas and the tally of identified human objects. The results of human object detection and counting using OpenMV exhibit an accuracy rate of 71%. Moreover, when applied to video footage, the OpenMV system yields a correct detection rate of 73% for counting human objects. In summary, this study presents a valuable tool that aids visually impaired individuals in the detection and counting of human objects, achieving commendable accuracy rates through the implementation of OpenMV and the Haar Cascade Classifier algorithm.

Human Object Detection for Real-Time Camera using Mobilenet-SSD

Technology development is very rapid, so all fields are required to develop technology to increase the effectiveness and efficiency of work. One of the focuses is related to image processing technology. We can benefit from this system, so various fields have implemented image processing systems, such as security, health, and education. One of the current obstacles is safety, namely in searching for people, which is still done manually. Searching for teams to find people is often challenging because of the significant search area, low light conditions, and complex search fields. Therefore, we need a tool capable of detecting humans to assist in finding people. Therefore, to detect human objects, the authors try to research human object detection using a simple device for the human object detection system. The authors use the MobilenetV2-SSD, where this algorithm has high detection and accuracy. Using the mobilenetV2-SSD simulation method for human object recognition, a detection rate of 100% is obtained with an FPS value of 5.

Advanced System for Enhancing Location Identification through Human Pose and Object Detection

Location identification is a fundamental aspect of advanced mobile robot navigation systems, as it enables establishing meaningful connections between objects, spaces, and actions. Understanding human actions and accurately recognizing their corresponding poses play pivotal roles in this context. In this paper, we present an observation-based approach that seamlessly integrates object detection algorithms, human pose detection, and machine learning techniques to effectively learn and recognize human actions in household settings. Our method entails training machine learning models to identify the common actions, utilizing a dataset derived from the interaction between human pose and object detection. To validate our approach, we assess its effectiveness using a diverse dataset encompassing typical household actions. The results demonstrate a significant improvement over existing techniques, with our method achieving an accuracy of over 95% in classifying eight different actions within household environments.. Furthermore, we ascertain the robustness of our approach through rigorous testing in real-world environments, demonstrating its ability to perform well despite the various challenges of data collection in such settings. The implications of our method for robotic applications are significant, as a comprehensive understanding of human actions is essential for tasks such as semantic navigation. Moreover, our findings unveil promising opportunities for future research, as our approach can be extended to learn and recognize a wide range of other human actions. This perspective, which highlights the potential leverage of these techniques, provides an encouraging path for future investigations in this field.