Region-based Convolutional Neural Network Model Research Articles

Comparative Study of Lightweight Target Detection Methods for Unmanned Aerial Vehicle-Based Road Distress Survey.

Unmanned aerial vehicles (UAVs) are effective tools for identifying road anomalies with limited detection coverage due to the discrete spatial distribution of roads. Despite computational, storage, and transmission challenges, existing detection algorithms can be improved to support this task with robustness and efficiency. In this study, the K-means clustering algorithm was used to calculate the best prior anchor boxes; Faster R-CNN (region-based convolutional neural network), YOLOX-s (You Only Look Once version X-small), YOLOv5-s, YOLOv7-tiny, YOLO-MobileNet, and YOLO-RDD models were built based on image data collected by UAVs. YOLO-MobileNet has the most lightweight model but performed worst in accuracy, but greatly reduces detection accuracy. YOLO-RDD (road distress detection) performed best with a mean average precision (mAP) of 0.701 above the Intersection over Union (IoU) value of 0.5 and achieved relatively high accuracy in detecting all four types of distress. The YOLO-RDD model most successfully detected potholes with an AP of 0.790. Significant or severe distresses were better identified, and minor cracks were relatively poorly identified. The YOLO-RDD model achieved an 85% computational reduction compared to YOLOv7-tiny while maintaining high detection accuracy.

Application of mask R-CNN for building detection in UAV remote sensing images

This study aims to tackle the challenges of low accuracy in building feature extraction and insufficient details in three-dimensional (3D) modeling faced by traditional methods, particularly in complex backgrounds. To address these issues, a method for building feature extraction based on Mask Region-Convolutional Neural Network (Mask R-CNN) is proposed. This approach combines deep learning techniques with aerial images to ensure precise and efficient automatic detection and feature extraction. Urban building images are captured through aerial photography, and building outlines are annotated to create a comprehensive dataset of building features. The Mask R-CNN-based method efficiently processes and classifies the features of the dataset, generating candidate regions for further analysis. Additionally, this method demonstrates significant advantages in building feature extraction by employing the Mask R-CNN model to generate adaptive features. Comparative analysis with models such as Convolutional Neural Network (CNN), Region-based Convolutional Neural Network (R-CNN), Fast Region-based Convolutional Neural Network (Fast R-CNN), Faster Region-based Convolutional Neural Network (Faster R-CNN), and Generative Adversarial Network (GAN) indicates that Mask R-CNN exhibits superior performance in building feature extraction. The Mask R-CNN-based approach achieved approximately 95 % classification accuracy, while also showcasing strong stability and generalization capabilities. This study provides new methodologies and insights for enhancing feature extraction in aerial building imagery, offering significant reference value for the fields of architectural design and urban planning.

Breast cancer characterization using region-based convolutional neural network with screening and diagnostic mammogram

Background: Detection and classification of microcalcifications in breast tissues is crucial for early breast cancer diagnosis and long-term treatment. Objective: This paper aims to propose a robust model capable of detection and classification of breast cancer calcifications in digital mammogram images using Deep Convolutional Neural Networks (DCNN). Materials and methods: An expert breast radiologist annotated the 3,265 clinical mammogram images to create a comprehensive ground truth dataset comprising 2,500 annotations for malignant and benign calcifications. This dataset was utilized to train our model, a two-stage detection system incorporating a Region-based Convolutional Neural Network (RCNN) with AlexNet and support vector machines to enhance the system’s robustness. The proposed model was compared to the one-stage detection, utilizing YOLOv4 combined with the Cross-Stage Partial Darknet53 (CSPDarknet53) architecture. A separate dataset of 504 mammogram images was explicitly set aside for model testing. The efficacy of the proposed model was evaluated based on key performance metrics, including precision, recall, F1 score, and mean average precision (mAP). Results: The results showed that the proposed RCNN-2 model could automatically identify and categorize calcifications as malignant or benign, outperforming the YOLOv4 models. The RCNN-2’s overall effectiveness, as evaluated by precision, recall, F1 score, and mean average precision (mAP), achieved scores of 0.82, 0.85, 0.83, and 0.74, respectively. Conclusion: The proposed RCNN-2 model demonstrates very effective detection and classification of calcification in mammogram images, especially in high-dense breast images. The performance of the proposed model was compared to that of YOLOv4, and it can be concluded that the proposed RCNN model yields outstanding performance. The model can be a helpful tool for radiologists.

Enhanced marine fish small sample image recognition with RVFL in Faster R-CNN model

Addressing the challenge of insufficient underwater marine life images leading to diminished accuracy in existing target recognition models, this paper introduces Random Vector Functional Link (RVFL), a suitable model for small sample training, to reduce the dependency of traditional visual recognition models on data volume by replacing the Softmax classifier in the Faster Region-Based Convolutional Neural Network (R-CNN) model. The RVFL network assigns random weights to enhance the model's adaptability to diverse data distributions, thereby improving its generalization performance. Furthermore, the RVFL model incorporates multiple nonlinear activation functions, thereby enhancing the neural network's representational capacity for high-dimensional data. The application of the proposed model to the task of marine fish small sample target recognition demonstrates its effectiveness in mitigating the low accuracy issue caused by data scarcity. It significantly improved recognition accuracy and offered new insights into addressing underwater small sample image recognition challenges.

Nutrient Stress Symptom Detection in Cucumber Seedlings Using Segmented Regression and a Mask Region-Based Convolutional Neural Network Model

The health monitoring of vegetable and fruit plants, especially during the critical seedling growth stage, is essential to protect them from various environmental stresses and prevent yield loss. Different environmental stresses may cause similar symptoms, making visual inspection alone unreliable and potentially leading to an incorrect diagnosis and delayed corrective actions. This study aimed to address these challenges by proposing a segmented regression model and a Mask R-CNN model for detecting the initiation time and symptoms of nutrient stress in cucumber seedlings within a controlled environment. Nutrient stress was induced by applying two different treatments: an indicative nutrient deficiency with an electrical conductivity (EC) of 0 dSm−1, and excess nutrients with a high-concentration nutrient solution and an EC of 6 dSm−1. Images of the seedlings were collected using an automatic image acquisition system two weeks after germination. The early initiation of nutrient stress was detected using a segmented regression analysis, which analyzed morphological and textural features extracted from the images. For the Mask R-CNN model, 800 seedling images were annotated based on the segmented regression analysis results. Nutrient-stressed seedlings were identified from the initiation day to 4.2 days after treatment application. The Mask R-CNN model, implemented using ResNet-101 for feature extraction, leveraged transfer learning to train the network with a smaller dataset, thereby reducing the processing time. This study identifies the top projected canopy area (TPCA), energy, entropy, and homogeneity as prospective indicators of nutritional deficits in cucumber seedlings. The results from the Mask R-CNN model are promising, with the best-fit image achieving an F1 score of 93.4%, a precision of 93%, and a recall of 94%. These findings demonstrate the effectiveness of the integrated statistical and machine learning (ML) methods for the early and accurate diagnosis of nutrient stress. The use of segmented regression for initial detection, followed by the Mask R-CNN for precise identification, emphasizes the potential of this approach to enhance agricultural practices. By facilitating the early detection and accurate diagnosis of nutrient stress, this approach allows for quicker and more precise treatments, which improve crop health and productivity. Future research could expand this methodology to other crop types and field conditions to enhance image processing techniques, and researchers may also integrate real-time monitoring systems.

Novel Deep Learning Application: Recognizing Inconsistent Characters on Pharmaceutical Packaging

Background Machine vision faces significant challenges when applied to text recognition on cardboard packaging particularly due to multiple printing methods, irregular character shapes, and curved packaging surfaces. Methods This research introduces a novel deep learning application for recognizing binarized expiration date and batch code characters printed using multiple printing methods. The method, based on Region-based Convolutional Neural Networks (R-CNN), enables character recognition directly from in the images without the need for extracting handcrafted features. In detail, this approach performs character recognition by using the whole image as input, extracting and learning salient character features directly from the packaging surface images. Results The R-CNN model, with a precision of 91.1% and an F1 score of 80.9%, effectively recognizes manufacturing markings on pharmaceutical packages, with inconsistencies in the characters’ shapes. In a comparative experiment using the same dataset of images, the R-CNN model significantly outperformed Tesseract OCR, achieving much higher precision, recall, and F1 scores. Conclusions The results of this study reveal that the deep learning method outperforms the well-established optical character recognition method in recognizing text characters printed with different printing methods. Presented in this study, the deep learning method recognizes text characters with high precision. It is also suitable for recognizing text printed on curved surfaces, provided proper preprocessing is applied. The problem investigated in the study differs from previous research in the field, focusing on the recognition of texts printed with different printing methods. The research thus fills a gap in text recognition that existed in the research of the field. Furthermore, the study presents new ideas that will be utilized in our future research.

Manipulation of a Complex Object Using Dual-Arm Robot with Mask R-CNN and Grasping Strategy

Hot forging is one of the common manufacturing processes for producing brass workpieces. However forging produces flash which is a thin metal part around the desired part formed with an excessive material. Using robots with vision system to manipulate this workpiece has encountered several challenging issues, e.g. the uncertain shape of flash, color, reflection of brass surface, different lighting condition, and the uncertainty surrounding the position and orientation of the workpiece. In this research, Mask region-based convolutional neural network together with image processing is used to resolve these issues. The depth camera can provide images for visual detection. Machine learning Mask region-based convolutional neural network model was trained with color images and the position of the object is determined by the depth image. A dual arm 7 degree of freedom collaborative robot with proposed grasping strategy is used to grasp the workpiece that can be in inappropriate position and pose. Eventually, experiments were conducted to assess the visual detection process and the grasp planning of the robot.

Development and validation of a self-attention network-based algorithm to detect mediastinal lesions on computed tomography images.

Diagnosis of mediastinal lesions on computed tomography (CT) images is challenging for radiologists, as numerous conditions can present as mass-like lesions at this site. This study aimed to develop a self-attention network-based algorithm to detect mediastinal lesions on CT images and to evaluate its efficacy in lesion detection. In this study, two separate large-scale open datasets [National Institutes of Health (NIH) DeepLesion and Medical Image Computing and Computer Assisted Intervention (MICCAI) 2022 Mediastinal Lesion Analysis (MELA) Challenge] were collected to develop a self-attention network-based algorithm for mediastinal lesion detection. We enrolled 921 abnormal CT images from the NIH DeepLesion dataset into the pretraining stage and 880 abnormal CT images from the MELA Challenge dataset into the model training and validation stages in a ratio of 8:2 at the patient level. The average precision (AP) and confidence score on lesion detection were evaluated in the validation set. Sensitivity to lesion detection was compared between the faster region-based convolutional neural network (R-CNN) model and the proposed model. The proposed model achieved an 89.3% AP score in mediastinal lesion detection and could identify comparably large lesions with a high confidence score >0.8. Moreover, the proposed model achieved a performance boost of almost 2% in the competition performance metric (CPM) compared to the faster R-CNN model. In addition, the proposed model can ensure an outstanding sensitivity with a relatively low false-positive rate by setting appropriate threshold values. The proposed model showed excellent performance in detecting mediastinal lesions on CT. Thus, it can drastically reduce radiologists' workload, improve their performance, and speed up the reporting time in everyday clinical practice.

Fusion of Mask RCNN and multi-augmentation for rice recognition

In the real world, the appearance of similar rice varieties depends on various factors such as resolution, angle, lighting conditions, and perspective. Additionally, complex environmental factors and characteristics of each rice type, such as enhanced light intensity, cross-polarization, and shading, rice background color, and image similarity, play a role. This indicates that the data augmentation process may enhance the accuracy of crop identification, particularly in the context of self-supervised machine learning. The aim of this research is to develop a precise rice segmentation method based on the improved Mask R-CNN (Region-based Convolutional Neural Network) with multitask data augmentation. The Mask R-CNN model is enhanced by incorporating multitask input to improve feature extraction for rice. Experimental results demonstrate that the improved Mask R-CNN model can accurately segment various rice types under different conditions, such as different background colors and varying sizes of rice grains. The achieved precision, recall, F1 score, and segmentation mean Average Precision (mAP) are 95.5%, 96.3%, 95.9%, and 0.924, respectively. The average runtime on the test set is 0.35 seconds per image. Our method outperforms two comparative approaches, showcasing its ability to accurately segment rice in the market deployment phase with near real-time performance. This study establishes the foundation for the accurate detection of valuable agricultural products.

Deep Learning-Assisted Quantitative Measurement of Thoracolumbar Fracture Features on Lateral Radiographs.

This study aimed to develop and validate a deep learning (DL) algorithm for the quantitative measurement of thoracolumbar (TL) fracture features, and to evaluate its efficacy across varying levels of clinical expertise. Using the pretrained Mask Region-Based Convolutional Neural Networks model, originally developed for vertebral body segmentation and fracture detection, we fine-tuned the model and added a new module for measuring fracture metrics-compression rate (CR), Cobb angle (CA), Gardner angle (GA), and sagittal index (SI)-from lumbar spine lateral radiographs. These metrics were derived from six-point labeling by 3 radiologists, forming the ground truth (GT). Training utilized 1,000 nonfractured and 318 fractured radiographs, while validations employed 213 internal and 200 external fractured radiographs. The accuracy of the DL algorithm in quantifying fracture features was evaluated against GT using the intraclass correlation coefficient. Additionally, 4 readers with varying expertise levels, including trainees and an attending spine surgeon, performed measurements with and without DL assistance, and their results were compared to GT and the DL model. The DL algorithm demonstrated good to excellent agreement with GT for CR, CA, GA, and SI in both internal (0.860, 0.944, 0.932, and 0.779, respectively) and external (0.836, 0.940, 0.916, and 0.815, respectively) validations. DL-assisted measurements significantly improved most measurement values, particularly for trainees. The DL algorithm was validated as an accurate tool for quantifying TL fracture features using radiographs. DL-assisted measurement is expected to expedite the diagnostic process and enhance reliability, particularly benefiting less experienced clinicians.

Feasibility of predicting functional decline in the elderly through key posture information during sit-to-stand movement

BackgroundEarly detection of functional decline in the elderly in day care centres facilitates timely implementation of preventive and treatment measures. Research questionWhether or not a predictive model can be developed by applying image recognition to analyze elderly individuals' posture during the sit-to-stand (STS) manoeuvre. MethodsWe enrolled sixty-six participants (24 males and 42 females) in an observational study design. To estimate posture key point information, we employed a region-based convolutional neural network model and utilized nine key points and their coordinates to calculate seven eigenvalues (X1-X7) that represented the motion curve features during the STS manoeuvre. One-way analysis of variance was performed to evaluate four STS strategies and four types of compensation strategies for three groups with different capacities (college students, community-dwelling elderly, and day care center elderly). Finally, a machine learning predictive model was established. ResultsSignificant differences (p < 0.05) were observed in all eigenvalues except X2 (momentum transfer phase, p = 0.168) between participant groups; significant differences (p < 0.05) were observed in all eigenvalues except X2 (p = 0.219) and X3 (hip-rising phase, p = 0.286) between STS patterns; significant differences (p < 0.05) were observed in all eigenvalues except X2 (p = 0.842) and X3 (p = 0.074) between compensation strategies. The motion curve eigenvalues of the seven posture key points were used to build a machine learning model with 85% accuracy in capacity detection, 70% accuracy in pattern detection, and 85% accuracy in compensation strategy detection. SignificanceThis study preliminarily demonstrates that eigenvalues can be used to detect STS patterns and compensation strategies adopted by individuals with different capacities. Our machine learning model has excellent predictive accuracy and may be used to develop inexpensive and effective systems to help caregivers to continuously monitor STS patterns and compensation strategies of elderly individuals as warning signs of functional decline.

Optimization of Targeted Differential Interferometric Measurements for Wellpads Detected by Mask Region-Based Convolutional Neural Network in the Tengiz Oilfield of the Caspian Sea Coast

Many previous studies have primarily focused on the use of deep learning for interferometric processing or separate recognition purposes rather than targeted measurements of detected wellpads. The present study centered around the integration of deep learning recognition and interferometric measurements for Tengiz oilfield wellpads. This study proposes the optimization, automation, and acceleration of targeted ground deformation wellpad monitoring. Mask Region-based Convolutional Neural Network (R-CNN)-based deep learning wellpad recognition and consequent Small Baseline Subset Synthetic Aperture Radar Interferometry (SBAS-InSAR) analyses were used for the assessment of ground deformation in the wellpads. The Mask R-CNN technique allowed us to detect 159 wells with a confidence level of more than 95%. The Mask R-CNN model achieved a precision value of 0.71 and a recall value of 0.91. SBAS-InSAR interferometric measurements identified 13 wells for Sentinel-1 (SNT1), 8 wells for COSMO-SkyMed (CSK), and 20 wells for TerraSAR-X (TSX) located within the −54–−40 mm/y class of vertical displacement (VD) velocity. Regression analyses for the annual deformation velocities and cumulative displacements (CD) of wells derived from SNT1, CSK, and TSX satellite missions showed a good agreement with R2 &gt; 95. The predictions for cumulative displacements showed that the vertical subsidence processes will continue and reach −339 mm on 31 December 2023, with increasing spatial coverage and the potential to impact a higher number of wells. The hydrological analyses in the Tengiz oilfield clearly demonstrated that water flow has been moving towards the detected hotspot of subsidence and that its accumulation will increase with increasing subsidence. This detected subsidence hotspot was observed at a crossing with a seismic fault that might always be subject to reactivation. The role of this seismic fault should also be investigated as one of the ground deformation-controlling factors, even though this area is not considered seismically active. The primary practical and scientific values of these studies were identified for the operational risk assessment and maintenance needs of oilfield and gas field operators.

A timely and accurate approach to nearshore oil spill monitoring using deep learning and GIS

Oil spill accidents are a key contributor to marine pollution worldwide. Therefore, timely and effective oil spill detection is crucial for reducing marine pollution and enhancing environmental protection. Against this backdrop, this study explored two methods for performing nearshore on-site oil spill detection and segmentation, namely the U-net and Mask region-based convolutional neural network (R-CNN) methods. The U-net and Mask R-CNN models were revealed to exhibit acceptable and favorable performance, achieving overall accuracy of 77.01 % and 89.02 %, respectively. Subsequently, a verification system based on the Geographic Information System (GIS) was developed to improve the performance of the deep-learning model. With the integration of the verification system, the Mask R-CNN model achieved higher overall accuracy of 90.78 %. The feasibility of applying deep-learning methods to nearshore on-site oil spill monitoring was demonstrated through this study. In addition, the integration of the GIS not only assisted in the provision of oil spill information but also in the improvement of the deep-learning models. The timely, accurate, and effective method for nearshore on-site oil spill monitoring that this study explored can be applied to considerably improve traditional on-site oil spill monitoring, which has received limited academic attention in the last two decades.

Detection of the Corn Kernel Breakage Rate Based on an Improved Mask Region-Based Convolutional Neural Network

Real-time knowledge of kernel breakage during corn harvesting plays a significant role in the adjustment of operational parameters of corn kernel harvesters. (1) Transfer learning by initializing the DenseNet121 network with pre-trained weights for training and validating a dataset of corn kernels was adopted. Additionally, the feature extraction capability of DenseNet121 was improved by incorporating the attention mechanism of a Convolutional Block Attention Module (CBAM) and a Feature Pyramid Network (FPN) structure. (2) The quality of intact and broken corn kernels and their pixels were found to be coupled, and a linear regression model was established using the least squares method. The results of the test showed that: (1) The MAPb50 and MAPm50 of the improved Mask Region-based Convolutional Neural Network (RCNN) model were 97.62% and 98.70%, in comparison to the original Mask Region-based Convolutional Neural Network (RCNN) model, which were improved by 0.34% and 0.37%, respectively; the backbone FLOPs and Params were 3.09 GMac and 9.31 M, and the feature extraction time was 206 ms; compared to the original backbone, these were reduced by 3.87 GMac and 17.32 M, respectively. The training of the obtained prediction weights for the detection of a picture of the corn kernel took 76 ms, so compared to the Mask RCNN model, it was reduced by 375 ms; based on the concept of transfer learning, the improved Mask RCNN model converged twice as quickly with the loss function using pre-training weights than the loss function without pre-training weights during training. (2) The coefficients of determination R2 of the two models, when the regression models of the pixels and the quality of intact and broken corn kernels were analyzed, were 0.958 and 0.992, respectively. These findings indicate a strong correlation between the pixel characteristics and the quality of corn kernels. The improved Mask RCNN model was used to segment mask pixels to calculate the corn kernel breakage rate. The verified error between the machine vision and the real breakage rate ranged from −0.72% to 0.65%, and the detection time of the corn kernel breakage rate was only 76 ms, which could meet the requirements for real-time detection. According to the test results, the improved Mask RCNN method had the advantages of a fast detection speed and high accuracy, and can be used as a data basis for adjusting the operation parameters of corn kernel harvesters.

Improvement of Road Instance Segmentation Algorithm Based on the Modified Mask R-CNN

Although the Mask region-based convolutional neural network (R-CNN) model possessed a dominant position for complex and variable road scene segmentation, some problems still existed, including insufficient feature expressive ability and low segmentation accuracy. To address these problems, a novel road scene segmentation algorithm based on the modified Mask R-CNN was proposed. The multi-scale backbone network, Res2Net, was utilized to replace the ResNet network, and aimed to improve the feature extraction capability. The soft non-maximum suppression algorithm with attenuation function (soft-NMS) was adopted to improve detection efficiency in the case of a higher overlap rate. The comparison analyses of partition accuracy for various models were performed on the adopted Cityscapes dataset. The results demonstrated that the modified Mask R-CNN effectively increased the segmentation accuracy, especially for small and highly overlapping objects. The adopted Res2Net and soft-NMS can effectively enhance the feature extraction and improve segmentation performance. The average accuracy of the modified Mask R-CNN model reached up to 0.321, and was 0.054 higher than Mask R-CNN. This work provides important guidance to design a more efficient road scene instance segmentation algorithm for further promoting the actual application in automatic driving systems.

A Method for Detecting Key Points of Transferring Barrel Valve by Integrating Keypoint R-CNN and MobileNetV3

Industrial robots need to accurately identify the position and rotation angle of the handwheel of chemical raw material barrel valves during the process of opening and closing, in order to avoid interference between the robot gripper and the handwheel. This paper proposes a handwheel keypoint detection algorithm for fast and accurate acquisition of handwheel position and rotation pose. The algorithm is based on the Keypoint R-CNN (Region-based Convolutional Neural Network) keypoint detection model, which integrates the lightweight mobile network MobileNetV3, the Coordinate Attention module, and improved BiFPN (Bi-directional Feature Pyramid Network) structure to improve the detection speed of the model, enhance the feature extraction performance of the handwheel, and improve the expression capability of small targets at keypoint locations. Experimental results on a self-built handwheel dataset demonstrate that the proposed algorithm outperforms the Keypoint R-CNN model in terms of detection speed and accuracy, with a speed improvement of 54.6%. The detection accuracy and keypoint detection accuracy reach 93.3% and 98.7%, respectively, meeting the requirements of the application scenario and enabling accurate control of the robot’s rotation of the valve handwheel.

Improved Faster Region-Based Convolutional Neural Networks (R-CNN) Model Based on Split Attention for the Detection of Safflower Filaments in Natural Environments

The accurate acquisition of safflower filament information is the prerequisite for robotic picking operations. To detect safflower filaments accurately in different illumination, branch and leaf occlusion, and weather conditions, an improved Faster R-CNN model for filaments was proposed. Due to the characteristics of safflower filaments being dense and small in the safflower images, the model selected ResNeSt-101 with residual network structure as the backbone feature extraction network to enhance the expressive power of extracted features. Then, using Region of Interest (ROI) Align improved ROI Pooling to reduce the feature errors caused by double quantization. In addition, employing the partitioning around medoids (PAM) clustering was chosen to optimize the scale and number of initial anchors of the network to improve the detection accuracy of small-sized safflower filaments. The test results showed that the mean Average Precision (mAP) of the improved Faster R-CNN reached 91.49%. Comparing with Faster R-CNN, YOLOv3, YOLOv4, YOLOv5, and YOLOv6, the improved Faster R-CNN increased the mAP by 9.52%, 2.49%, 5.95%, 3.56%, and 1.47%, respectively. The mAP of safflower filaments detection was higher than 91% on a sunny, cloudy, and overcast day, in sunlight, backlight, branch and leaf occlusion, and dense occlusion. The improved Faster R-CNN can accurately realize the detection of safflower filaments in natural environments. It can provide technical support for the recognition of small-sized crops.

Automatic borehole fracture detection and characterization with tailored Faster R-CNN and simplified Hough transform

Understanding the geological information of fractures from borehole images is essential for conducting a thorough geological analysis in engineering. Recently, the convolutional neural network (CNN) has been employed to extract high-level features of fractures from images. We propose an improved Faster R-CNN (region-based CNN) model for fracture detection, which is suitably equipped with a tailored region proposal network f-RPN for borehole fractures. The anchors generated by f-RPN are specifically designed with a fixed width and varying heights, and only the anchors with the maximum intersection-over-union values are labeled as positive. By taking advantage of the cyclic feature of borehole images, the images are expanded in four orientations to increase the amount and diversity of the model input. The detection results of the model can further be utilized to simplify the Hough transform for characterizing dip angles and dip directions of fractures. The proposed method is validated with borehole images from multiple projects in hydraulic engineering. Experiments reveal that the average detection accuracy is approximately 91.5%, which exhibits an improvement of 11.06% compared to the original Faster R-CNN. Further analysis indicates that both f-RPN and the multi-orientation expansion contribute to improving the prediction accuracy. It is also shown that this method demonstrates robustness against blurred image background and outperforms traditional image processing methods in the presence of noise. The fracture characterization by the simplified Hough transform turns out to be computationally efficient and accurate, with the absolute errors less than 5° compared to the manual identification results.

Moving Object Detection using Intelligent Spectaculars

Abstract: Moving Object detection systems are used to identify and locate objects in images or videos. When used spectacles, an object detection system can allow the user to see information about the objects in their field of view. This can be useful in a variety of applications, such as helping blind people navigate their environment, or providing augmented reality information to workers. Region-based Convolutional Neural Networks (RCNN) is a type of machine learning model that is commonly used for object detection. The RCNN model first uses a convolutional neural network (CNN) to extract features from the input image, and then applies a region proposal algorithm to identify potential object regions in the image. These regions are then fed into a second CNN, which classifies the regions as objects or background. The RCNN model has been shown to be effective at detecting a wide range of objects in images and videos.

Affective Feedback Synthesis Towards Multimodal Text and Image Data

In this article, we have defined a novel task of affective feedback synthesis that generates feedback for input text and corresponding images in a way similar to humans responding to multimodal data. A feedback synthesis system has been proposed and trained using ground-truth human comments along with image–text input. We have also constructed a large-scale dataset consisting of images, text, Twitter user comments, and the number of likes for the comments by crawling news articles through Twitter feeds. The proposed system extracts textual features using a transformer-based textual encoder. The visual features have been extracted using a Faster region-based convolutional neural networks model. The textual and visual features have been concatenated to construct multimodal features that the decoder uses to synthesize the feedback. We have compared the results of the proposed system with baseline models using quantitative and qualitative measures. The synthesized feedbacks have been analyzed using automatic and human evaluation. They have been found to be semantically similar to the ground-truth comments and relevant to the given text–image input.