Image Cropping Research Articles

Continuous Growth Monitoring and Prediction with 1D Convolutional Neural Network Using Generated Data with Vision Transformer

Crop growth information is collected through destructive investigation, which inevitably causes discontinuity of the target. Real-time monitoring and estimation of the same target crops can lead to dynamic feedback control, considering immediate crop growth. Images are high-dimensional data containing crop growth and developmental stages and image collection is non-destructive. We propose a non-destructive growth prediction method that uses low-cost RGB images and computer vision. In this study, two methodologies were selected and verified: an image-to-growth model with crop images and a growth simulation model with estimated crop growth. The best models for each case were the vision transformer (ViT) and one-dimensional convolutional neural network (1D ConvNet). For shoot fresh weight, shoot dry weight, and leaf area of lettuce, ViT showed R2 values of 0.89, 0.93, and 0.78, respectively, whereas 1D ConvNet showed 0.96, 0.94, and 0.95, respectively. These accuracies indicated that RGB images and deep neural networks can non-destructively interpret the interaction between crops and the environment. Ultimately, growers can enhance resource use efficiency by adapting real-time monitoring and prediction to feedback environmental controls to yield high-quality crops.

Rice Leaf Disease Classification—A Comparative Approach Using Convolutional Neural Network (CNN), Cascading Autoencoder with Attention Residual U-Net (CAAR-U-Net), and MobileNet-V2 Architectures

Classifying rice leaf diseases in agricultural technology helps to maintain crop health and to ensure a good yield. In this work, deep learning algorithms were, therefore, employed for the identification and classification of rice leaf diseases from images of crops in the field. The initial algorithmic phase involved image pre-processing of the crop images, using a bilateral filter to improve image quality. The effectiveness of this step was measured by using metrics like the Structural Similarity Index (SSIM) and the Peak Signal-to-Noise Ratio (PSNR). Following this, this work employed advanced neural network architectures for classification, including Cascading Autoencoder with Attention Residual U-Net (CAAR-U-Net), MobileNetV2, and Convolutional Neural Network (CNN). The proposed CNN model stood out, since it demonstrated exceptional performance in identifying rice leaf diseases, with test Accuracy of 98% and high Precision, Recall, and F1 scores. This result highlights that the proposed model is particularly well suited for rice leaf disease classification. The robustness of the proposed model was validated through k-fold cross-validation, confirming its generalizability and minimizing the risk of overfitting. This study not only focused on classifying rice leaf diseases but also has the potential to benefit farmers and the agricultural community greatly. This work highlights the advantages of custom CNN models for efficient and accurate rice leaf disease classification, paving the way for technology-driven advancements in farming practices.

An Extended Method Based on the Geometric Position of Salient Image Features: Solving the Dataset Imbalance Problem in Greenhouse Tomato Growing Scenarios

Machine vision has significant advantages in a wide range of agricultural applications; however, acquiring a large number of high-quality image resources is often challenging in actual agricultural production due to environmental and equipment conditions. Therefore, crop image augmentation techniques are particularly important in crop growth analysis. In this paper, greenhouse tomato plants were used as research subjects to collect images of their different fertility stages with flowers and fruits. Due to the different durations of each fertility period, there is a significant difference in the number of images collected. For this reason, this paper proposes a method for balanced amplification of significant feature information in images based on geometric position. Through the geometric position information of the target in the image, different segmentation strategies are used to process the image and supervised and unsupervised methods are applied to perform balanced augmentation of the image, which is combined with the YOLOv7 algorithm to verify the augmentation effect. In terms of the image dataset, the mixed image dataset (Mix) is supplemented with mobile phone images on top of in situ monitoring images, with precision increased from 70.33% to 82.81% and recall increased from 69.15% to 81.25%. In terms of image augmentation, after supervised balanced amplification, the detection accuracy is improved from 70.33% to 77.29%, which is suitable for supervised balanced amplification. For the mobile phone dataset (MP), after amplification, it was found that better results could be achieved without any amplification method. The detection accuracy of the mixed dataset with different data sources matching the appropriate amplification method increased slightly from 82.81% to 83.59%, and accurate detection could be achieved when the target was shaded by the plant, and in different environments and light conditions.

CTARR: A fast and robust method for identifying anatomical regions on CT images via atlas registration

Medical image analysis tasks often focus on regions or structures located in a particular location within the patient’s body. Often large parts of the image may not be of interest for the image analysis task. When using deep-learning based approaches, this causes an unnecessary increases the computational burden during inference and raises the chance of errors. In this paper, we introduce CTARR, a novel generic method for CT Anatomical Re- gion Recognition. The method serves as a pre-processing step for any deep learning-based CT image analysis pipeline by automatically identifying the pre-defined anatomical region that is relevant for the follow-up task and removing the rest. It can be used in (i) image segmentation to prevent false positives in anatomically implausible regions and speeding up the inference, (ii) image classification to produce image crops that are consistent in their anatomical context, and (iii) image registration by serving as a fast pre-registration step. Our proposed method is based on atlas registration and provides a fast and robust way to crop any anatomical region encoded as one or multiple bounding box(es) from any unlabeled CT scan of the brain, chest, abdomen and/or pelvis. We demonstrate the utility and robustness of the proposed method in the context of medical image segmentation by evaluating it on six datasets of public segmentation challenges. The foreground voxels in the regions of interest are preserved in the vast majority of cases and tasks (97.45-100%) while taking only fractions of a seconds to compute (0.1-0.21s) on a deep learning worksta- tion and greatly reducing the segmentation runtime (2.0-12.7x). Our code is available at <a href='https://github.com/ThomasBudd/ctarr'>https://github.com/ThomasBudd/ctarr</a>

Skin feature point tracking using deep feature encodings

AbstractFacial feature tracking is a key component of imaging ballistocardiography (BCG) where accurate quantification of the displacement of facial keypoints is needed for good heart rate estimation. Skin feature tracking enables video-based quantification of motor degradation in Parkinson’s disease. While traditional computer vision algorithms like Scale Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), and Lucas-Kanade method (LK) have been benchmarks due to their efficiency and accuracy, they often struggle with challenges like affine transformations and changes in illumination. In response, we propose a pipeline for feature tracking, that applies a convolutional stacked autoencoder to identify the most similar crop in an image to a reference crop containing the feature of interest. The autoencoder learns to represent image crops into deep feature encodings specific to the object category it is trained upon. We train the autoencoder on facial images and validate its ability to track skin features in general using manually labelled face and hand videos of small and large motion recorded in our lab. Our evaluation protocol is comprehensive, including quantification of errors in human annotation. The tracking errors of distinctive skin features (moles) are so small that we cannot exclude the fact that they stem from the manual labelling based on a $$\chi ^2$$ χ 2 -test. With a mean error of 0.6–3.3 pixels, our method outperformed the other methods in all but one scenario. More importantly, our method was the only one that did not diverge. We also compare our method with the latest state-of-the-art transformer for feature matching by Google—Omnimotion. Our results indicate that our method is superior at tracking different skin features under large motion conditions and that it creates better feature descriptors for tracking, matching, and image registration compared to both traditional algorithms and the latest Omnimotion.

An Improved Reptile Search Algorithm with Multiscale Adaptive Deep Learning Technique and Atrous Spatial Pyramid Pooling for IoT-based Smart Agriculture Management

Smart farming is an enhanced option for increasing food production, resource management and labour. Existing prediction methods need trained experts to analyse the data, which is a time-consuming process, and thus, there is a need for smart farming with the Internet of Things (IoT). Hence, a well-developed IoT-assisted smart agriculture model is proposed for managing agricultural needs to improve the economic value of farmers. The proposed framework constitutes four major aspects like Crop prediction, Crop yield prediction, Plant disease prediction and Smart irrigation. Initially, the crop images are taken as the input and fed to the Multi-scale Adaptive and Attention-based Convolution Neural Network with Atrous Spatial Pyramid Pooling (MACNN-ASPP), where the dissimilar crops are classified and obtained. In the second aspect, the crop image as well as crop-related data, are fetched from the data sources and given as input. Further, the deep features of the image are extracted that are added with soil and environment condition data. Then this feature is subjected to the Multi-scale Adaptive and Attention-based One-Dimensional Convolution Neural Network with Atrous Spatial Pyramid Pooling (MA1DCNN-ASPP) for crop yield prediction. While in the third aspect, the leaf images are assembled from the data file and fed as input to the MACNN-ASPP for detecting the variety of diseases affecting the plants. In the final aspect, smart irrigation is done by collecting field images with related data. Further, the deep features retrieved from the field images are applied to MA1DCNN-ASPP, where the different conditions of the field will be attained. In order to develop the model adaptively, the hyper-parameters in the network are optimised using the Improved Reptile Search Algorithm (IRSA). Finally, the investigation is done over the proposed methodology using multiple evaluation metrics. In contrast with other approaches, the proposed smart agriculture outperforms the performance of managing crops without any hazardous effects. Accuracy validation executed in the implemented IRSA-MACNN-ASSP-based crop yield prediction model accomplished better efficiency as 6.89%, 4.45%, 2.19% and 1.08% better than the classical techniques like CNN, LSTM, 1DCNN and MA1DCNN-ASPP.

FG-UNet: fine-grained feature-guided UNet for segmentation of weeds and crops in UAV images.

Semantic segmentation of weed and crop images is a key component and prerequisite for automated weed management. For weeds in unmanned aerial vehicle (UAV) images, which are usually characterized by small size and easily confused with crops at early growth stages, existing semantic segmentation models have difficulties to extract sufficiently fine features. This leads to their limited performance in weed and crop segmentation of UAV images. We proposed a fine-grained feature-guided UNet, named FG-UNet, for weed and crop segmentation in UAV images. Specifically, there are two branches in FG-UNet, namely the fine-grained feature branch and the UNet branch. In the fine-grained feature branch, a fine feature-aware (FFA) module was designed to mine fine features in order to enhance the model's ability to segment small objects. In the UNet branch, we used an encoder-decoder structure to realize high-level semantic feature extraction in images. In addition, a contextual feature fusion (CFF) module was designed for the fusion of the fine features and high-level semantic features, thus enhancing the feature discrimination capability of the model. The experimental results showed that our proposed FG-UNet, achieved state-of-the-art performance compared to other semantic segmentation models, with mean intersection over union (MIOU) and mean pixel accuracy (MPA) of 88.06% and 92.37%, respectively. The proposed method in this study lays a solid foundation for accurate detection and intelligent management of weeds. It will have a positive impact on the development of smart agriculture. © 2024 Society of Chemical Industry.

Application of Sentinel-2 imagery for total suspended solids mapping off the Bodri River, Kendal Regency, Indonesia

Total suspended solids (TSS) is one of the key parameters in water quality monitoring. TSS dynamics off the seas of Bodri River can be observed using Sentinel-2. This research validates Wirasatriya and Laili's algorithm and redevelops a TSS retrieval algorithm based on in situ TSS and Sentinel-2 reflectance (Rrs). During the satellite's passing time, water sampling for TSS was conducted at 64 stations at a depth of 0.5m in October 2023. Bottom of Atmosphere (BoA) Rrs data were obtained from SNAP software, which had previously performed image cropping and resampling. Rrs data from all bands were initially correlated with in situ TSS. The validation results of Wirasatriya and Laili's algorithms obtained very high of RMSE values reaching 272.23 mg/L and 33.08 mg//L. While the results of generating the algorithm produced better the performance of the algorithmic model with RMSE = 7.34 mg/L, MAPE = 14.24%, and bias = 0.99. The development of the algorithms in the study based on the green band (B3) resulted in the equation: TSS (mg/L) = 4.6698exp10.609∗B3, which was further used for temporal mapping. High TSS was found in nearshore areas and river mouths influenced by shallow waters. Temporally, high TSS was found in October (55.65 mg/L) and low in May (11 mg/L). The temporal variation pattern is influenced by precipitation and tides. The findings of this algorithm are very important for analyzing the evolution of land accretion in the Bodri River estuary and as a review for land management in upland areas and can be used to monitor other shallow areas.

Camellia oleifera Tree Detection and Counting Based on UAV RGB Image and YOLOv8

The detection and counting of Camellia oleifera trees are important parts of the yield estimation of Camellia oleifera. The ability to identify and count Camellia oleifera trees quickly has always been important in the context of research on the yield estimation of Camellia oleifera. Because of their specific growing environment, it is a difficult task to identify and count Camellia oleifera trees with high efficiency. In this paper, based on a UAV RGB image, three different types of datasets, i.e., a DOM dataset, an original image dataset, and a cropped original image dataset, were designed. Combined with the YOLOv8 model, the detection and counting of Camellia oleifera trees were carried out. By comparing YOLOv9 and YOLOv10 in four evaluation indexes, including precision, recall, mAP, and F1 score, Camellia oleifera trees in two areas were selected for prediction and compared with the real values. The experimental results show that the cropped original image dataset was better for the recognition and counting of Camellia oleifera, and the mAP values were 8% and 11% higher than those of the DOM dataset and the original image dataset, respectively. Compared to YOLOv5, YOLOv7, YOLOv9, and YOLOv10, YOLOv8 performed better in terms of the accuracy and recall rate, and the mAP improved by 3–8%, reaching 0.82. Regression analysis was performed on the predicted and measured values, and the average R2 reached 0.94. This research shows that a UAV RGB image combined with YOLOv8 provides an effective solution for the detection and counting of Camellia oleifera trees, which is of great significance for Camellia oleifera yield estimation and orchard management.

Phenotyping of Panicle Number and Shape in Rice Breeding Materials Based on Unmanned Aerial Vehicle Imagery.

The number of panicles per unit area (PNpA) is one of the key factors contributing to the grain yield of rice crops. Accurate PNpA quantification is vital for breeding high-yield rice cultivars. Previous studies were based on proximal sensing with fixed observation platforms or unmanned aerial vehicles (UAVs). The near-canopy images produced in these studies suffer from inefficiency and complex image processing pipelines that require manual image cropping and annotation. This study aims to develop an automated, high-throughput UAV imagery-based approach for field plot segmentation and panicle number quantification, along with a novel classification method for different panicle types, enhancing PNpA quantification at the plot level. RGB images of the rice canopy were efficiently captured at an altitude of 15 m, followed by image stitching and plot boundary recognition via a mask region-based convolutional neural network (Mask R-CNN). The images were then segmented into plot-scale subgraphs, which were categorized into 3 growth stages. The panicle vision transformer (Panicle-ViT), which integrates a multipath vision transformer and replaces the Mask R-CNN backbone, accurately detects panicles. Additionally, the Res2Net50 architecture classified panicle types with 4 angles of 0°, 15°, 45°, and 90°. The results confirm that the performance of Plot-Seg is comparable to that of manual segmentation. Panicle-ViT outperforms the traditional Mask R-CNN across all the datasets, with the average precision at 50% intersection over union (AP50) improved by 3.5% to 20.5%. The PNpA quantification for the full dataset achieved superior performance, with a coefficient of determination (R 2) of 0.73 and a root mean square error (RMSE) of 28.3, and the overall panicle classification accuracy reached 94.8%. The proposed approach enhances operational efficiency and automates the process from plot cropping to PNpA prediction, which is promising for accelerating the selection of desired traits in rice breeding.

Next‐Gen Crop Monitoring: MTEG‐RTU Algorithm and UAV Synergy for Precise Disease Diagnosis

ABSTRACTThe rapidly changing climatic scenarios are highly favorable for the rising diseases that lead to increasing threats to food production and supply. Various scholars and scientists make long steps to hasten the process of making innovations in farming for managing these issues. In this context, UAV is applied for the purpose of managing and monitoring plant health. The abiotic stresses available in plant diagnosis through traditional strategies are highly labor‐intensive and unfit for large‐scale deployment. Conversely, UAVs designed with mobile sensors, multispectral, radar, and so on make them flexible, affordable, and more effective. Thus, this study proposes a novel meta ensemble transfer extreme gradient‐based random tactical unit (MTEG‐RTU) algorithm for diagnosing crop illnesses precisely. The proposed MTEG‐RTU methodology entails three methods such as transfer learning, adaptive boost, and meta‐ensemble, and the hyper parameters are tuned using random tactical unit algorithm. Healthier and disordered crop images gained from the crop disease dataset comprise 8000 images and are preprocessed. The more optimal features from the preprocessed images are learned through the ResNet method, and these features enter into the classification phase. Random tactical unit algorithm enhanced the performance by optimizing the hyperparameters of MTEG classifier. The experimental results conducted based on the various assessment components and validation dataset indicate that the developed method outperformed the other chosen models, achieving precision, recall, and accuracy of 98.5%, 97.9%, and 98.6%, respectively. The other achievements made by the model are offering technical guidance for conducting the precise diagnosis and treatment of plant pathologies with less time of 9 s.

Deep-learning based discrimination of pathologic complete response using MRI in HER2-positive and triple-negative breast cancer

Distinguishing between pathologic complete response and residual cancer after neoadjuvant chemotherapy (NAC) is crucial for treatment decisions, but the current imaging methods face challenges. To address this, we developed deep-learning models using post-NAC dynamic contrast-enhanced MRI and clinical data. A total of 852 women with human epidermal growth factor receptor 2 (HER2)-positive or triple-negative breast cancer were randomly divided into a training set (n = 724) and a validation set (n = 128). A 3D convolutional neural network model was trained on the training set and validated independently. The main models were developed using cropped MRI images, but models using uncropped whole images were also explored. The delayed-phase model demonstrated superior performance compared to the early-phase model (area under the receiver operating characteristic curve [AUC] = 0.74 vs. 0.69, P = 0.013) and the combined model integrating multiple dynamic phases and clinical data (AUC = 0.74 vs. 0.70, P = 0.022). Deep-learning models using uncropped whole images exhibited inferior performance, with AUCs ranging from 0.45 to 0.54. Further refinement and external validation are necessary for enhanced accuracy.

Aesthetic image cropping meets VLP: Enhancing good while reducing bad

Aesthetic Image Cropping (AIC) enhances the visual appeal of an image by adjusting its composition and aesthetic elements. People make these adjustments based on these elements, aiming to enhance appealing aspects while minimizing detrimental factors. Motivated by these observations, we propose a novel approach called CLIPCropping, which simulates the human decision-making process in AIC. CLIPCropping leverages Contrastive Language-Image Pre-training (CLIP) to align visual perception with textual description. It consists of three branches: composition embedding, aesthetic embedding, and image cropping. The composition embedding branch learns principles based on Composition Knowledge Embedding (CKE), while the aesthetic embedding branch learns principles based on Aesthetic Knowledge Embedding (AKE). The image cropping branch evaluates the quality of candidate crops by aggregating knowledge from CKE and AKE; an MLP produces the best result. Extensive experiments on three benchmark datasets—GAICD-1236, GAICD-3336, and FCDB—show that CLIPCropping outperforms state-of-the-art methods and provides insightful interpretations.

Deep learning neural network-assisted badminton movement recognition and physical fitness training optimization

This work aims to solve the problem of low accuracy in recognizing the trajectory of badminton movement. This work focuses on the visual system in badminton robots and conducts side detection and tracking of flying badminton in two-dimensional image plane video streams. Then, the cropped video images are input into a convolutional neural network frame by frame. By adding an attention mechanism, it helps identify the badminton movement trajectory. Finally, to address the detection challenge of flying badminton as a small target in video streams, the deep learning one-stage detection network, Tiny YOLOv2, is improved from both the loss function and network structure perspectives. Moreover, it is combined with the Unscented Kalman Filter algorithm to predict the trajectory of badminton movement. Simulation results show that the improved algorithm performs excellently in tracking and predicting badminton trajectories compared with the existing algorithms. The average accuracy of the proposed method for tracking badminton trajectories is 91.40 %, and the recall rate is 84.60 %. The average precision, recall, and frame rate of the measured trajectories in four simple and complex scenarios of badminton flight video streams are 96.7 %, 95.7 %, and 29.2 frames/second, respectively. They are all superior to other classic algorithms. It is evident that the proposed method can provide powerful support for badminton trajectory recognition and help improve the accuracy of badminton movement recognition.

Automatic location and recognition of horse freezing brand using rotational YOLOv5 deep learning network

Individual livestock identification is of great importance to precision livestock farming. Liquid nitrogen freezing labeled horse brand is an effective way for livestock individual identification. Along with various technological developments, deep-learning-based methods have been applied in such individual marking recognition. In this research, a deep learning method for oriented horse brand location and recognition was proposed. Firstly, Rotational YOLOv5 (R-YOLOv5) was adopted to locate the oriented horse brand, then the cropped images of the brand area were trained by YOLOv5 for number recognition. In the first step, unlike classical detection methods, R-YOLOv5 introduced the orientation into the YOLO framework by integrating Circle Smooth Label (CSL). Besides, Coordinate Attention (CA) was added to raise the attention to positional information in the network. These improvements enhanced the accuracy of detecting oriented brands. In the second step, number recognition was considered as a target detection task because of the requirement of accurate recognition. Finally, the whole brand number was obtained according to the sequences of each detection box position. The experiment results showed that R-YOLOv5 outperformed other rotating target detection algorithms, and the AP (Average Accuracy) was 95.6 %, the FLOPs were 17.4 G, the detection speed was 14.3 fps. As for the results of number recognition, the mAP (mean Average Accuracy) was 95.77 %, the weight size was 13.71 MB, and the detection speed was 68.6 fps. The two-step method can accurately identify brand numbers with complex backgrounds. It also provides a stable and lightweight method for livestock individual identification.

Wheat leaf localization and segmentation for yellow rust disease detection in complex natural backgrounds

Wheat yellow rust disease poses a significant threat to global wheat yield and grain quality. Early detection of this disease will help to minimize the loss caused by its effects. Existing models work well on images taken in a controlled environment, whereas a uniform background is placed behind the leaf, but these models fail to produce good results in natural settings. Previous research also involves manual interventions in the pipeline to achieve good classification results such as cropping the images, using uniform backgrounds, etc. These systems are not practical to use in natural environments where there will be a lot of background noise to the image and manual cropping becomes an extra step for the farmer. Moreover, the unavailability of the dataset in which images of leaves are taken in a natural setting became another challenge. In this research, a dataset is curated and leaves are annotated for object detection, object segmentation further the leaves are classified into 3 classes ie healthy, resistant, and susceptible. A novel unsupervised image rotation algorithm is proposed that takes input from YOLOv8 to align the leave in such a way that maximum background can be removed by a rectangular bounding box . Then the comparison between multiple state-of-the-art segmentation models ie. UNET, Segment-Anything (SAM), Segnet, LinkNet, PSPNet, FPN, Deep-Labv3+ (Xception), and DeepLabv3+ (Mo-bileNet) has shown that UNET has outperformed all the other segmentation models with an IOU score of 0.9563. Lastly for classification, the performance of multiple convolution neural networks ie. VGG16, Resnet 101(v2), Xception, Mo-bileNetV2, and Transformer-based models ie. Swin trans-former and MobileVit have been compared. Swin transformer has outperformed the state-of-the-art CNN models with an accuracy of 95.8%. This paper proposes a complete robust pipeline that can be deployed in natural environment and does not need any manual intervention to produce good results. This research shows that good localization of leaves and removal of unwanted background noise at the earliest stage of the pipeline will assist the segmentation model to effectively segment the leaf from the background which will enable classification models to achieve high classification accuracy, even when dealing with very small datasets.

TRANSFORMER NETWORKS TO CLASSIFY WEEDS AND CROPS IN HIGH-RESOLUTION AERIAL IMAGES FROM NORTH-EAST SERBIA

The intricate backgrounds present in crop and field images, coupled with the minimal contrast between weed-infested areas and the background, can lead to considerable ambiguity. This, in turn, poses a significant challenge to the resilience and precision of crop identification models. Identifying and mapping weeds are pivotal stages in weed control, essential for maintaining crop health. A multitude of research efforts underscore the significance of leveraging remote sensing technologies and sophisticated machine learning algorithms to enhance weed management strategies. Deep learning techniques have demonstrated impressive effectiveness in a range of agricultural remote sensing applications, including plant classification and disease detection. High-resolution imagery was collected using a UAV equipped with a high-resolution camera, which was strategically deployed over weed, sunflower, tobacco and maize fields to collect data. The VIT models achieved commendable levels of accuracy, with test accuracies of 92.97% and 90.98% in their respective evaluations. According to the experimental results, transformers not only excel in crop classification accuracy, but also achieve higher accuracy with a smaller sample size. Swin-B16 achieved an accuracy of 91.65% on both the training and test datasets. Compared to the other two ViT models, the loss value is significantly lower by half, at 0.6450.

Fishing event detection and species classification using computer vision and artificial intelligence for electronic monitoring

Fisheries regulations require detailed catch reporting on commercial fishing vessels. Vital components for the sustainable management of fish stocks include a robust estimate of the number of fish caught and the species composition. Catch recording is often done manually by human observers on fishing vessels. Human observers are costly, and consistent data streams can be subject to observer availability and the weather. On-vessel cameras (electronic monitoring, EM) are a growing alternative to human observers. However, on-land human auditors are required to review hundreds of hours of videos recorded during fishing trips that can last for weeks. In this paper, a framework is presented to automatically detect fish in EM videos, count the total fishing events, and classify the fish species. For this purpose, a deep learning and computer vision-based model is developed to efficiently detect fish and fishers onboard a vessel. Secondly, a vision-based tracking pipeline tracks the detected fish and counts the total fishing events in the videos. Thirdly, the extracted fishing events are classified through a deep learning-based fish species classifier, to provide the distribution of different fish species caught for a fishing trip. For our experiments, the datasets were prepared using the electronic monitoring data of multiple fishing trips of a fishing vessel. The videos were recorded on Australian longline vessels targeting tunas and billfish. For the fish detection task, video frames were extracted and labelled manually to provide a digital ground-truth. For the fish species classification task, hundreds of fish images of multiple species were cropped to provide a training dataset for the fish classifier. For the fish counting task, manual counts for the fishing events of individual fish species were generated for the test fishing trips. The developed fish and fisher detector achieves a mean Average Precision of 87.0 % for fish and 94.0 % for fishers on test video frames. The fishing event detection pipeline achieves an Average Precision of 81.0 % and an Average Recall of 74.5 % on test videos. The fish species classifier achieves an Accuracy (Top-1) of 91.11 % for the classification of cropped fish images and 89.05 % for the classification of extracted fishing events from the videos. Experimental results show that our proposed computer vision and artificial intelligence-based solution for video analysis has great potential to automate the auditing process from electronic monitoring footage and contribute to the sustainable management of fish stocks.

Facial image analysis for automated suicide risk detection with deep neural networks

Accurately assessing suicide risk is a critical concern in mental health care. Traditional methods, which often rely on self-reporting and clinical interviews, are limited by their subjective nature and may overlook non-verbal cues. This study introduces an innovative approach to suicide risk assessment using facial image analysis. The Suicidal Visual Indicators Prediction (SVIP) Framework leverages EfficientNetb0 and ResNet architectures, enhanced through Bayesian optimization techniques, to detect nuanced facial expressions indicating mental state. The models’ interpretability is improved using GRADCAM, Occlusion Sensitivity, and LIME, which highlight significant facial regions for predictions. Using datasets DB1 and DB2, which consist of full and cropped facial images from social media profiles of individuals with known suicide outcomes, the method achieved 67.93% accuracy with EfficientNetb0 on DB1 and up to 76.6% accuracy with a Bayesian-optimized Support Vector Machine model using ResNet18 features on DB2. This approach provides a less intrusive, accessible alternative to video-based methods and demonstrates the substantial potential for early detection and intervention in mental health care.

Enhancing Pléiades-based crop mapping with multi-temporal and texture information

Accurate crop mapping using satellite imagery is crucial for improving the monitoring of agricultural landscapes. Very high resolution (VHR) satellite imagery offers unique capabilities in this respect, allowing for even small fields to be discerned and image texture analysis to be performed. Additionally, satellite imagery has greater efficiency than unmanned aerial vehicles due to its extensive coverage. Moreover, the operation flexibility of VHR satellites means that timely image acquisition is possible several times during the growing season. This study investigates the potential of VHR Pléiades images and the random forest classifier for accurate crop mapping. Four images acquired on April 9th, May 12th, May 31st, and June 20th were used to test 16 classification scenarios, including single-date and multi-temporal combinations of spectral bands, texture features, and vegetation Indices (VIs). The classification using the spectral bands from all four images achieved the highest overall accuracy, 93.9% and 96.3% at field and pixel levels, respectively. The bitemporal classifications had lower accuracy. Nevertheless, the combination of the May 12th and June 20th spectral bands had 90% accuracy, which indicated that two images may be sufficient for reliable mapping if the periods with phenological differences between crops are considered. Adding texture features to the spectral bands significantly enhanced the accuracy (up to 8%) of single-date classifications, making it highly recommended when only one image is available. However, the impact of texture was more pronounced on the later dates. It showed the most marked benefit for vineyards and alfalfa, with minimal or no improvement observed for other classes like winter barley. An additional increase in overall accuracy was achieved in three of the four dates by supplementing the spectral and texture bands with VIs. This study highlights the importance of considering image acquisition dates and crop types when designing satellite-based crop mapping strategies for optimal accuracy.