Bounding Box Research Articles

YOLO V3 AND CCN FOR THE TRACKING AND CLASSIFICATION OF AERIAL OBJECT AND DRONES

The goal of this study is to give headways in aeronautical article ID that will help with making recognitions that are both more exact and more precise. Specifically, we revamp the meaning of the article recognition anchor enclose request to remember turns for expansion to level and width, and besides, we make it conceivable to have erratic four corner point structures. Furthermore, the consideration of new anchor boxes gives the model additional adaptability to address protests that are focused at a pivot of turn that gives a 45-degree point. By accomplishing these results, we can make an organization that considers negligible tradeoffs about speed and unwavering quality, while likewise giving more exact restrictions. The latest ways to deal with PC vision and article acknowledgment are for the most part dependent on brain organizations and different advances that utilize profound learning. This powerful field of study is utilized in various applications, including military and observation, aeronautical photography, independent driving, and airborne perception. To precisely locate the location of an item, contemporary object identification techniques make use of bounding boxes that are drawn over the object and have a rectangular form (horizontal and vertical). These orthogonal bounding boxes do not consider the posture of the object, which leads to a decrease in the amount of object localization and restricts subsequent tasks such as object comprehension and tracking. We have used the DOTA dataset to present all of the results, demonstrating the value of flexible object boundaries, particularly with rotated and non-rectangular objects. We have also achieved an accuracy of 98.47% for the detection and classification of aerial objects, with forty percent of the data being used for training and the remaining twenty percent being used for testing. There was a minimum of 2.8 seconds of processing time required for the whole program to be executed to categorize all of the aerial items that were parked on the base.

Segmentation Method of Zanthoxylum bungeanum Cluster Based on Improved Mask R-CNN

The precise segmentation of Zanthoxylum bungeanum clusters is crucial for developing picking robots. An improved Mask R-CNN model was proposed in this study for the segmentation of Zanthoxylum bungeanum clusters in natural environments. Firstly, the Swin-Transformer network was introduced into the model’s backbone as the feature extraction network to enhance the model’s feature extraction capabilities. Then, the SK attention mechanism was utilized to fuse the detailed information into the mask branch from the low-level feature map of the feature pyramid network (FPN), aiming to supplement the image detail features. Finally, the distance intersection over union (DIOU) loss function was adopted to replace the original bounding box loss function of Mask R-CNN. The model was trained and tested based on a self-constructed Zanthoxylum bungeanum cluster dataset. Experiments showed that the improved Mask R-CNN model achieved 84.0% and 77.2% in detection mAP50box and segmentation mAP50mask, respectively, representing a 5.8% and 4.6% improvement over the baseline Mask R-CNN model. In comparison to conventional instance segmentation models, such as YOLACT, Mask Scoring R-CNN, and SOLOv2, the improved Mask R-CNN model also exhibited higher segmentation precision. This study can provide valuable technology support for the development of Zanthoxylum bungeanum picking robots.

Enhancing fruit and vegetable detection in unconstrained environment with a novel dataset

Automating the detection of fruits and vegetables using computer vision is essential for modernizing agriculture, improving efficiency, ensuring food quality, and contributing to sustainable and technologically advanced farming practices. This paper presents an end-to-end pipeline for detecting and localizing fruits and vegetables in real-world scenarios. To achieve this, a dataset named FRUVEG67 was curated that includes images of 67 classes of fruits and vegetables captured in unconstrained scenarios, with only a few manually annotated samples per class. A semi-supervised data annotation algorithm (SSDA) was developed that generates bounding boxes for objects to label the remaining nonannotated images. For detection, Fruit and Vegetable Detection Network (FVDNet) was proposed, an ensemble version of YOLOv8n featuring three distinct grid configurations. In addition, an averaging approach for the prediction of the bounding box and a voting mechanism for the prediction of the classes was implemented. Finally Jensen–Shannon Divergence (JSD) in conjunction with focal loss was integrated as the overall loss function for better detection of smaller objects. Experimental results highlight the superiority of FVDNet compared to recent versions of YOLO, showcasing remarkable improvements in detection and localization performance. An impressive mean average precision (mAP) score of 0.82 across all classes was achieved. Furthermore, the efficacy of FVDNet on open-category refrigerator images were evaluated, where it demonstrates promising results.

SPGAN Optimized by Piranha Foraging Optimization for Thyroid Nodule Classification in Ultrasound Images.

In this research work, Semantic-Preserved Generative Adversarial Network optimized by Piranha Foraging Optimization for Thyroid Nodule Classification in Ultrasound Images (SPGAN-PFO-TNC-UI) is proposed. Initially, ultrasound images are gathered from the DDTI dataset. Then the input image is sent to the pre-processing step. During pre-processing stage, the Multi-Window Savitzky-Golay Filter (MWSGF) is employed to reduce the noise and improve the quality of the ultrasound (US) images. The pre-processed output is supplied to the Generalized Intuitionistic Fuzzy C-Means Clustering (GIFCMC). Here, the ultrasound image's Region of Interest (ROI) is segmented. The segmentation output is supplied to the Fully Numerical Laplace Transform (FNLT) to extract the features, such as geometric features like solidity, orientation, roundness, main axis length, minor axis length, bounding box, convex area, and morphological features, like area, perimeter, aspect ratio, and AP ratio. The Semantic-Preserved Generative Adversarial Network (SPGAN) separates the image as benign or malignant nodules. Generally, SPGAN does not express any optimization adaptation methodologies for determining the best parameters to ensure the accurate classification of thyroid nodules. Therefore, the Piranha Foraging Optimization (PFO) algorithm is proposed to improve the SPGAN classifier and accurately identify the thyroid nodules. The metrics, like F-score, accuracy, error rate, precision, sensitivity, specificity, ROC, computing time is examined. The proposed SPGAN-PFO-TNC-UI method attains 30.54%, 21.30%, 27.40%, and 18.92% higher precision and 26.97%, 20.41%, 15.09%, and 18.27% lower error rate compared with existing techniques, like Thyroid detection and classification using DNN with Hybrid Meta-Heuristic and LSTM (TD-DL-HMH-LSTM), Quantum-Inspired convolutional neural networks for optimized thyroid nodule categorization (QCNN-OTNC), Thyroid nodules classification under Follow the Regularized Leader Optimization based Deep Neural Networks (CTN-FRL-DNN), Automatic classification of ultrasound thyroids images using vision transformers and generative adversarial networks (ACUTI-VT-GAN) respectively.

Daymet and DeGAUSS: Development of a Novel Method that Links High Spatial Resolution Gridded Meteorological Data with Geocoded Healthcare Encounters

ObjectiveDue to human-caused climate change, extreme temperature-related morbidity/mortality are increasing global public health concerns. To better understand the burdens of heat exposure, researchers must merge accurate/granular environmental data with clinical data. We developed a privacy-preserving method that links high-spatial-resolution continuous, gridded, meteorological data to geocoded healthcare encounters. ApproachIn an R environment, we developed code that links geocoded healthcare encounters to publicly-available Daymet data via the corresponding Daymet raster cell number and healthcare encounter date. Daymet provides daily 1km x 1km surface meteorological data over North America since 1980. User specifications include the bounding box latitude/longitude of Daymet data download, which Daymet environmental variables are linked, and the day-lag of Daymet data matching to healthcare encounter date. ResultsLinking 500,000 healthcare encounters over a 7-day lag with 7 years of Daymet data comprising minimum/maximum temperature over a bounding box encompassing Cook County, Illinois (5,670 1km raster cells) took 11.92 minutes with 4 CPUs and 8GB RAM. Thirteen years of Daymet data took 23.85 minutes. Linking 7.5 million encounters with 7 years of data took 28.49 minutes. ConclusionsOur newly-developed method for linking Daymet data to geocoded healthcare encounters preserves patient privacy, and processes the data efficiently over a minimal timespan. ImplicationsDaymet data linked to healthcare encounters allows for accurate assessment of the association between environmental variable exposure and any clinical outcome over North America. Packaging the code allows for public distribution within the Decentralized Geomarker Assessment for Multi-Site Studies (DeGAUSS) universe of geoanalytic tools.

A Lightweight Real-Time Recognition Algorithm for Tomato Leaf Disease Based on Improved YOLOv8

To address the real-time detection challenge of deploying deep learning-based tomato leaf disease detection algorithms on embedded devices, an improved tomato leaf disease detection algorithm based on YOLOv8n is proposed in this paper. It is able to achieve the efficient, real-time detection of tomato leaf diseases while maintaining model’s lightweight requirements. The algorithm incorporated the LMSM (lightweight multi-scale module) and ALSA (Attention Lightweight Subsampling Module) to improve the ability to extract lightweight and multi-scale semantic information for the specific characteristics of tomato leaf disease, which include irregular spot size and lush tomato leaves. The head network was redesigned utilizing partial and group convolution along with a parameter-sharing method. Scalable auxiliary bounding box and loss function optimization strategies were introduced to further enhance performance. After undergoing the pruning technique, computation decreased by 61.7%, the model size decreased by 55.6%, and the FPS increased by 44.8%, all while a high level of accuracy was maintained. A detection speed of 19.70FPS on the Jetson Nano was obtained after undergoing TensorRT quantization, showing a 64.85% improvement compared to the initial detection speed. This method met the high real-time performance and small model size requirements for embedded tomato leaf disease detection systems, indirectly reducing the energy consumption of online detection. It provided an effective solution for the online detection of tomato leaf disease.

Advancing Nighttime Object Detection through Image Enhancement and Domain Adaptation

Due to the lack of annotations for nighttime low-light images, object detection in low-light images has always been a challenging problem. Achieving high-precision results at night is also an issue. Additionally, we aim to use a single nighttime dataset to complete the knowledge distillation task while improving the detection accuracy of object detection models under nighttime low-light conditions and reducing the computational cost of the model, especially for small targets and objects contaminated by special nighttime lighting. This paper proposes a Nighttime Unsupervised Domain Adaptation Network (NUDN) based on knowledge distillation to address these issues. To improve the detection accuracy of nighttime images, high-confidence bounding box predictions from the teacher and region proposals from the student are first fused, allowing the teacher to perform better in subsequent training, thus generating a combination of high-confidence and low-confidence pseudo-labels. This combination of feature information is used to guide model training, enabling the model to extract feature information similar to that of source images in nighttime low-light images. Nighttime images and pseudo-labels undergo random size transformations before being used as input for the student, enhancing the model’s generalization across different scales. To address the scarcity of nighttime datasets, we propose a nighttime-specific augmentation pipeline called LightImg. This pipeline enhances nighttime features, transforming them into daytime features and reducing issues such as backlighting, uneven illumination, and dim nighttime light, enabling cross-domain research using existing nighttime datasets. Our experimental results show that NUDN can significantly improve nighttime low-light object detection accuracy on the SHIFT and ExDark datasets. We conduct extensive experiments and ablation studies to demonstrate the effectiveness and efficiency of our work.

Weld seam object detection system based on the fusion of 2D images and 3D point clouds using interpretable neural networks

This study introduces a novel approach that addresses the limitations of existing methods by integrating 2D image processing with 3D point cloud analysis, enhanced by interpretable neural networks. Unlike traditional methods that rely on either 2D or 3D data alone, our approach leverages the complementary strengths of both data types to improve detection accuracy in environments adversely affected by welding spatter and smoke. Our system employs an improved Faster R-CNN model with a ResNet50 backbone for 2D image analysis, coupled with an innovative orthogonal plane intersection line extraction algorithm for 3D point cloud processing. By incorporating explainable components such as visualizable feature maps and a transparent region proposal network, we address the “black box” issue common in deep learning models.This architecture enables a more transparent decision-making process, providing technicians with necessary insights to understand and trust the system’s outputs. The Faster-RCNN structure is designed to break down the object detection process into distinct, understandable steps, from initial feature extraction to final bounding box refinement. This fusion of 2D-3D data analysis and interpretability not only improves detection performance but also sets a new standard for transparency and reliability in automated welding systems, facilitating wider adoption in industrial applications.

Efficient Highlighting of Visual Targets in Complex Natural Scenes

Guided visual search is a common theme in HF applications. In this project we use large-scale databases of wildlife camera-trap imagery as a testbed for optimization of target highlighting. MegaDetector, a generic animal detection deep machine learning model, provides bounding boxes for potential targets within an image. In some cases, human observers are necessary to confirm or further classify the detections. Outlining the bounding box can direct human attention to the target AOI to improve observers’ classification speed and accuracy. However, this outline introduces visual clutter and crowding at the AOI boundary. In a first empirical study we investigate the use of padding to mitigate the effects of local clutter and compared different methods of visual highlighting (colored outline vs. blur outside of AOI). We found support for using padding to improve performance when animals were hard to see. Both colored outlines and blur were effective at directing observers’ attention.

AEPF: Attention-Enabled Point Fusion for 3D Object Detection.

Current state-of-the-art (SOTA) LiDAR-only detectors perform well for 3D object detection tasks, but point cloud data are typically sparse and lacks semantic information. Detailed semantic information obtained from camera images can be added with existing LiDAR-based detectors to create a robust 3D detection pipeline. With two different data types, a major challenge in developing multi-modal sensor fusion networks is to achieve effective data fusion while managing computational resources. With separate 2D and 3D feature extraction backbones, feature fusion can become more challenging as these modes generate different gradients, leading to gradient conflicts and suboptimal convergence during network optimization. To this end, we propose a 3D object detection method, Attention-Enabled Point Fusion (AEPF). AEPF uses images and voxelized point cloud data as inputs and estimates the 3D bounding boxes of object locations as outputs. An attention mechanism is introduced to an existing feature fusion strategy to improve 3D detection accuracy and two variants are proposed. These two variants, AEPF-Small and AEPF-Large, address different needs. AEPF-Small, with a lightweight attention module and fewer parameters, offers fast inference. AEPF-Large, with a more complex attention module and increased parameters, provides higher accuracy than baseline models. Experimental results on the KITTI validation set show that AEPF-Small maintains SOTA 3D detection accuracy while inferencing at higher speeds. AEPF-Large achieves mean average precision scores of 91.13, 79.06, and 76.15 for the car class's easy, medium, and hard targets, respectively, in the KITTI validation set. Results from ablation experiments are also presented to support the choice of model architecture.

Automatic detection and extraction of lost shipping containers based on YOLO and the segment anything model

ABSTRACT The artificial visual method is currently commonly used to decipher multi-beam water column images to obtain the position and state of lost shipping containers. However, the recognition efficiency and accuracy of this method need to be improved. The You Only Look Once (YOLO) series model has strong real-time target detection capability. Meanwhile, the Segment Anything Model (SAM) has strong zero-shot transferability. A detection and extraction method for lost shipping containers, which combines the two models mentioned above, is proposed in this study. First, the YOLO series model is employed to detect lost shipping container targets in a single-frame water column image. On this basis, the output of bounding box positions by the optimal target detection model is used as a prompt for the SAM. Finally, the SAM is used to extract lost shipping container targets in images through zero-shot transferability. Experimental results in the Pearl River estuary show that the combined modelling method of YOLOv5-n and EdgeSAM-3× achieves the best overall performance. The precision and recall for the detection of lost shipping containers by this method is better than 95%. In terms of target extraction, YOLOv5-n and EdgeSAM-3× have the best Intersection over Union, recall, and F1 scores.

Six-Degree-of-Freedom Pose Estimation Method for Multi-Source Feature Points Based on Fully Convolutional Neural Network

An object’s six-degree-of-freedom (6DoF) pose information has great importance in various fields. Existing methods of pose estimation usually detect two-dimensional (2D)-three-dimensional (3D) feature point pairs, and directly estimates the pose information through Perspective-n-Point (PnP) algorithms. However, this approach ignores the spatial association between pixels, making it difficult to obtain high-precision results. In order to apply pose estimation based on deep learning methods to real-world scenarios, we hope to design a method that is robust enough in more complex scenarios. Therefore, we introduce a method for 3D object pose estimation from color images based on farthest point sampling (FPS) and object 3D bounding box. This method detects the 2D projection of 3D feature points through a convolutional neural network, matches it with the 3D model of the object, and then uses the PnP algorithm to restore the feature point pair to the object pose. Due to the global nature of the bounding box, this approach can be considered effective even in partially occluded or complex environments. In addition, we propose a heatmap suppression method based on weighted coordinates to further improve the prediction accuracy of feature points and the accuracy of the PnP algorithm in solving the pose position. Compared with other algorithms, this method has higher accuracy and better robustness. Our method yielded 93.8% of the ADD(-s) metrics on the Linemod dataset and 47.7% of the ADD(-s) metrics on the Occlusion Linemod dataset. These results show that our method is more effective than existing methods in pose estimation of large objects.

Human–object interaction detection algorithm based on graph structure and improved cascade pyramid network

Aiming at the problem of insufficient use of human–object interaction (HOI) information and spatial location information in images, we propose a human–object​ interaction detection network based on graph structure and improved cascade pyramid. This network is composed of three branches, namely, graph branch, human–object branch and human pose branch. In graph branch, we propose a Graph-based Interactive Feature Generation Algorithm (GIFGA) to address the inadequate utilization of interaction information. GIFGA constructs an initial dense graph model by taking humans and objects as nodes and their interaction relationships as edges. Then, by traversing each node, the graph model is updated to generate the final interaction features. In human pose branch, we propose an Improved Cascade Pyramid Network (ICPN) to tackle the underutilization of spatial location information. ICPN extracts human pose features and maps both the object bounding boxes and extracted human pose maps onto the global feature map to capture the most discriminative interaction-related region features within the global context. Finally, the features from the three branches are fed into a Multi-Layer Perceptron (MLP) for fusion and then classified for recognition. Experimental results demonstrate that our network achieves mAP of 54.93% and 28.69% on the V-COCO and HICO-DET datasets, respectively.

GooseDetectlion: A Fully Annotated Dataset for Lion-head Goose Detection in Smart Farms

Large datasets are required to develop Artificial Intelligence (AI) models in AI powered smart farming for reducing farmers’ routine workload, this paper contributes the first large lion-head goose dataset GooseDetectlion, which consists of 2,660 images and 98,111 bounding box annotations. The dataset was collected with 6 cameras deployed in a goose farm in Chenghai district of Shantou city, Guangdong province, China. Images sampled from videos collected during July 9 -10 in 2022 were fully annotated by a team of fifty volunteers. Compared with another 6 well known animal datasets in literature, our dataset has higher capacity and density, which provides a challenging detection benchmark for main stream object detectors. Six state-of-the-art object detectors have been selected to be evaluated on the GooseDetectlion, which includes one two-stage anchor-based detector, three one-stage anchor-based detectors, as well as two one-stage anchor-free detectors. The results suggest that the one-stage anchor-based detector You Only Look Once version 5 (YOLO v5) achieves the best overall performance in terms of detection precision, model size and inference efficiency.

Employing 3D laser scanner for automated morphological assessment of aggregates produced from diverse sources and by different crushing methods

This study introduced an automated approach employing 3D scanner for quick and precise aggregates shape analysis. Recognizing existing gaps, there is still an industry demand to propose a standardized method for measuring material characteristics using 3D scanning. By addressing the limitations of previous studies, this research considered unexplored aspects of 3D scanning to improve the speed and efficiency of that. This research comprehensively examined nearly 1500 aggregates from five natural sources and five types of slag. Four statistical tests named Mann-Whitney U, Kolmogorov-Smirnov, Bootstrap, and Permutation were utilized to determine the optimal sample size for scanning. The findings suggest that scanning just 100 aggregates from each source is sufficient to be a small representative of the whole dataset and it can significantly reduce resource use while maintaining data integrity. As a practical insight, it has tried to scan a batch of 25 aggregates simultaneously in one session which could improve efficiency of scanning by reducing the processing time to just 0.6 min per particle. In MATLAB® software, some common bounding volume techniques such as Axis-Aligned, Oriented, Ellipsoid, and Sphere Bounding Box were used to analyze morphological properties. The findings showed that the aggregates sourced from Basalt, Moraine, and most of slags have more cubical shapes, while Porphyrit and Diabase aggregates are more elongated, flaky, and angular. This study also provided a comparative study of how various techniques for stone crushing affect the morphology of aggregates. The results indicate that a combination of jaw, cone, and horizontal impact crusher produces more spherical and cubical aggregates. Replacing of the cone crusher with a gyratory crusher in the same combination can improve elongation and sphericity slightly but may increase flaky particles. These insights offer valuable guidelines for future research and practical applications in construction material optimization.

Shuffle Attention-Based Pavement-Sealed Crack Distress Detection.

To enhance the detection of pavement-sealed cracks and ensure the long-term stability of pavement performance, a novel approach called the shuffle attention-based pavement-sealed crack detection is proposed. This method consists of three essential components: the feature extraction network, the detection head, and the Wise Intersection over Union loss function. Within both the feature extraction network and the detection head, the shuffle attention module is integrated to capture the high-dimensional semantic information of pavement-sealed cracks by combining spatial and channel attention in parallel. The two-way detection head with multi-scale feature fusion efficiently combines contextual information for pavement-sealed crack detection. Additionally, the Wise Intersection over Union loss function dynamically adjusts the gradient gain, enhancing the accuracy of bounding box fitting and coverage area. Experimental results highlight the superiority of our proposed method, with higher mAP@0.5 (98.02%), Recall (0.9768), and F1-score (0.9680) values compared to the one-stage state-of-the-art methods, showcasing improvements of 0.81%, 1.8%, and 2.79%, respectively.

End-to-End Implicit Object Pose Estimation.

To accurately estimate the 6D pose of objects, most methods employ a two-stage algorithm. While such two-stage algorithms achieve high accuracy, they are often slow. Additionally, many approaches utilize encoding-decoding to obtain the 6D pose, with many employing bilinear sampling for decoding. However, bilinear sampling tends to sacrifice the accuracy of precise features. In our research, we propose a novel solution that utilizes implicit representation as a bridge between discrete feature maps and continuous feature maps. We represent the feature map as a coordinate field, where each coordinate pair corresponds to a feature value. These feature values are then used to estimate feature maps of arbitrary scales, replacing upsampling for decoding. We apply the proposed implicit module to a bidirectional fusion feature pyramid network. Based on this implicit module, we propose three network branches: a class estimation branch, a bounding box estimation branch, and the final pose estimation branch. For this pose estimation branch, we propose a miniature dual-stream network, which estimates object surface features and complements the relationship between 2D and 3D. We represent the rotation component using the SVD (Singular Value Decomposition) representation method, resulting in a more accurate object pose. We achieved satisfactory experimental results on the widely used 6D pose estimation benchmark dataset Linemod. This innovative approach provides a more convenient solution for 6D object pose estimation.

LSR-Det: A Lightweight Detector for Ship Detection in SAR Images Based on Oriented Bounding Box

Convolutional neural networks (CNNs) have significantly advanced in recent years in detecting arbitrary-oriented ships in synthetic aperture radar (SAR) images. However, challenges remain with multi-scale target detection and deployment on satellite-based platforms due to the extensive model parameters and high computational complexity. To address these issues, we propose a lightweight method for arbitrary-oriented ship detection in SAR images, named LSR-Det. Specifically, we introduce a lightweight backbone network based on contour guidance, which reduces the number of parameters while maintaining excellent feature extraction capability. Additionally, a lightweight adaptive feature pyramid network is designed to enhance the fusion capability of the ship features across different layers with a low computational cost by incorporating adaptive ship feature fusion modules between the feature layers. To efficiently utilize the fused features, a lightweight rotating detection head is designed, incorporating the idea of sharing the convolutional parameters, thereby improving the network’s ability to detect multi-scale ship targets. The experiments conducted on the SAR ship detection dataset (SSDD) and the rotating ship detection dataset (RSDD-SAR) demonstrate that LSR-Det achieves an average precision (AP50) of 98.5% and 97.2% with 3.21 G floating point operations (FLOPs) and 0.98 M parameters, respectively, outperforming the current popular SAR arbitrary-direction ship target detection methods.

A novel multi-stream hand-object interaction network for assembly action recognition

PurposeAssembly action recognition plays an important role in assembly process monitoring and human-robot collaborative assembly. Previous works overlook the interaction relationship between hands and operated objects and lack the modeling of subtle hand motions, which leads to a decline in accuracy for fine-grained action recognition. This paper aims to model the hand-object interactions and hand movements to realize high-accuracy assembly action recognition.Design/methodology/approachIn this paper, a novel multi-stream hand-object interaction network (MHOINet) is proposed for assembly action recognition. To learn the hand-object interaction relationship in assembly sequence, an interaction modeling network (IMN) comprising both geometric and visual modeling is exploited in the interaction stream. The former captures the spatial location relation of hand and interacted parts/tools according to their detected bounding boxes, and the latter focuses on mining the visual context of hand and object at pixel level through a position attention model. To model the hand movements, a temporal enhancement module (TEM) with multiple convolution kernels is developed in the hand stream, which captures the temporal dependences of hand sequences in short and long ranges. Finally, assembly action prediction is accomplished by merging the outputs of different streams through a weighted score-level fusion. A robotic arm component assembly dataset is created to evaluate the effectiveness of the proposed method.FindingsThe method can achieve the recognition accuracy of 97.31% and 95.32% for coarse and fine assembly actions, which outperforms other comparative methods. Experiments on human-robot collaboration prove that our method can be applied to industrial production.Originality/valueThe author proposes a novel framework for assembly action recognition, which simultaneously leverages the features of hands, objects and hand-object interactions. The TEM enhances the representation of dynamics of hands and facilitates the recognition of assembly actions with various time spans. The IMN learns the semantic information from hand-object interactions, which is significant for distinguishing fine assembly actions.

A Hybrid Segmentation Algorithm for Rheumatoid Arthritis Diagnosis Using X-ray Images

Rheumatoid Arthritis (RA) is a chronic autoimmune illness that occurs in the joints, resulting in inflammation, pain, and stiffness. X-ray examination is one of the most common diagnostic procedures for RA, but manual X-ray image analysis has limitations because it is a time-consuming procedure and is prone to errors. A specific algorithm aims to a lay stable and accurate segmenting of carpal bones from hand bone images, which is vitally important for identifying rheumatoid arthritis. The algorithm demonstrates several stages, starting with Carpal bone Region of Interest (CROI) specification, dynamic thresholding, and Gray Level Co-occurrence Matrix (GLCM) application for texture analysis. To get the clear edges of the image, the component is first converted to the greyscale function and thresholding is carried out to separate the hand from the background. The pad region is identified to obtain the contours of it, and the CROI is defined by the bounding box of the largest contour. The threshold value used in the CROI method is given a dynamic feature that can separate the carpal bones from the surrounding tissue. Then the GLCM texture analysis is carried out, calculating the number of pixel neighbors, with the specific intensity and neighbor relations of the pixels. The resulting feature matrix is then employed to extract features such as contrast and energy, which are later used to categorize the images of the affected carpal bone into inflamed and normal. The proposed technique is tested on a rheumatoid arthritis image dataset, and the results show its contribution to diagnosis of the disease. The algorithm efficiently divides carpal bones and extracts the signature parameters that are critical for correct classification of the inflammation in the cartilage images.