Outdoor Images Research Articles

Diffusion-driven Incomplete Multimodal Learning for Air Quality Prediction

Predicting air quality using multimodal data is crucial to comprehensively capture the diverse factors influencing atmospheric conditions. Therefore, this study introduces a multimodal learning framework that integrates outdoor images with traditional ground-based observations to improve the accuracy and reliability of air quality predictions. However, aligning and fusing these heterogeneous data sources pose a formidable challenge, further exacerbated by pervasive data incompleteness issues in practice. In this paper, we propose a novel incomplete multimodal learning approach (iMMAir) to recovery missing data for robust air quality prediction. Specifically, we first design a shallow feature extractor to capture modal-specific features within the latent representation space. Then we develop a conditional diffusion-driven recovery module to mitigate the distribution gap between the recovered and true data. This module further incorporates two conditional constraints of temporal correlation and semantic consistency for effective modal completion. Finally, we reconstruct incomplete modalities and fuse available data using a multimodal transformer network to predict the air quality. To alleviate the modality imbalance problem, we employ an adaptive gradient modulation strategy to adjust the optimization of each modality. Experimental results demonstrate that iMMAir significantly reduces prediction errors, outperforming baseline models by an average of 5.6% and 2.5% in air quality regression and classification tasks. Our source code and data are available at https://github.com/pestasu/IMMAir.

Evaluating Methods for Violence Classification and Firearm Detection in Indoor CCTV Environment

The adoption of security systems based on computer vision for violence detection has the potential to significantly improve safety in various public and private properties. However, developing these systems can be extremely challenging.We can choose to use classification models to identify violence in images or also use object detection models to identify firearms, which may indicate robbery. Additionally, when developing such systems focused on private environments, we encounter specific challenges, such as obtaining appropriate datasets to train the algorithms. Many publicly available datasets for violence detection consist of outdoor images, with elements such as streets and cars, which do not adequately reflect the nuances and unique characteristics of private properties. In this work, we evaluate both learned and handcrafted features to classify videos as 'violence' or 'non-violence' across a variety of datasets, including a new dataset composed exclusively of closed-circuit television (CCTV) images. Additionally, we propose a new dataset for firearm detection in CCTV images and conduct some experiments using YoloV8. In this way, we hope to provide a clearer insight into the possible decisions when developing a security system for indoor environments.

Retraction notice: Indoor and outdoor image classification: a mixture of brightness, straight line, Euclidean shapes and recursive shapes based approach

Retraction notice: Indoor and outdoor image classification: a mixture of brightness, straight line, Euclidean shapes and recursive shapes based approach

Enhancing computer image recognition with improved image algorithms

Advances in computer image recognition have significantly impacted many industries, including healthcare, security and autonomous systems. This paper aims to explore the potential of improving image algorithms to enhance computer image recognition. Specifically, we will focus on regression methods as a means to improve the accuracy and efficiency of identifying images. In this study, we will analyze various regression techniques and their applications in computer image recognition, as well as the resulting performance improvements through detailed examples and data analysis. This paper deals with the problems related to visual image processing in outdoor unstructured environment. Finally, the heterogeneous patterns are converted into the same pattern, and the heterogeneous patterns are extracted from the fusion features of data modes. The simulation results show that the perception ability and recognition ability of outdoor image recognition in complex environment are improved.

DehazeModel: Enhancing Image Clarity with an Encoder-Decoder CNN Approach

We introduce DehazeModel, a Convolutional Neural Network (CNN) tailored for the purpose of enhancing single images by removing haze. DehazeModel comprises several components, including pre-processing, image dehazing, and post-processing modules. The pre-processing module, which is trainable, generates enhanced inputs featuring a wider array of characteristics compared to manually chosen pre-processing methods. The Image Dehazing module employs a novel encoder-decoder framework, overcoming common issues found in traditional multi-scale methods. Moreover, the post-processing module aids in minimizing artifacts in the resultant output. Experimental findings indicate that DehazeModel surpasses existing state-of-the-art techniques on the RESIDE dataset. Our experiments, conducted on both indoor and outdoor images, illustrate the robustness of our approach across various scenarios, independent of atmospheric scattering effects.

Densely integrated multi-branch attentive net for image dehazing

Single image dehazing is a fundamental but challenging task in image processing. Various deep learning-based methods have achieved great dehazing performance. However, there are still hazy residues, even color distortion and texture loss when removing haze from complex outdoor images in dense hazy scenes. A densely integrated multi-branch attentive net for image dehazing is proposed in the paper to address the above problems. The network includes a multi-scale feature extraction module and a dense-feature fusion module. The multi-scale feature extraction module adopts a multi-branch structure composed of residual channel attention blocks, which can expand the receptive field and filter the extracted features of diverse scales by weighting for fusion. It raises network learning accuracy. The dense-feature fusion module contains a multi-level feature fusion module for front and back layers, a color information renovation module, and a feature enhancement module. It achieves dynamically adjusting the channel weights of features at diverse scales, learning rich context information and suppressing redundant information, and compensating for the lack of color and texture information. The dense-feature fusion module bolsters the generalization capability of the network. The proposed method achieves superior objective and subjective evaluation results via quantitative and qualitative experiments on synthetic hazy images and natural hazy images, with better generalization ability and dehazing effect than the current SOTA dehazing algorithms, and effectively ameliorates the color distortion and incomplete dehazing.

Scene recovery: Combining visual enhancement and resolution improvement

Visibility enhancement of outdoor images under complex imaging conditions has been a crucial task for computer vision and received growing attention. However, existing image enhancement methods could result in typical block-like artifacts or color distortion. The undesirable impurities might also be significantly magnified after the enhancement task, further reducing the image quality. For enhancing and super-resolving complex real-world degradation, we propose a simultaneous visual enhancement and resolution improvement (VERI) variational scene recovery model for jointly enhancing image visibility and improving the resolution of the degraded image. Particularly, we estimate the scattering light map for degradation images to achieve clean scene radiance and simultaneously seek a high-quality image through a deep super-resolution network. The semi-proximal alternating direction method of multipliers (sPADMM) algorithm is employed for efficiently solving the minimization problems in the proposed model. Extensive experiments illustrate the effectiveness and robustness of the proposed method in dealing with various scenes, such as haze, sandstorm, underwater or low illumination.

Depth Sensing Imaging System Autonomous Restored Fog

Edge-preserving smoothing is an image processing technique that smooths away textures while retaining sharp edges. Image de-noising is the technique to reduce noises from corrupted images. The aim of the image denoising is to improve the contrast of the image or perception of information in images for human viewers or to provide better output for other automated image processing techniques. Outdoor images taken in hazy climate often get degraded due to the effect of haze. There are several methods to remove haze from hazy images. Most of them over saturates the dehazed images. This degrades the quality of images. The color attenuation prior technique is one of the best algorithm to remove haze from images. Based on this technique and by making use of MATLAB software, this paper suggests a simple method to remove haze from Images. The core of DSIS lies in its fusion of depth information with traditional imaging data. By leveraging depth maps obtained from sensors such as LiDAR or structured light cameras, DSIS accurately delineates objects in the scene and their respective depths. This enables selective fog removal, prioritizing objects closer to the camera for clearer representation while preserving depth cues for improved scene understanding. Our project aims to implement and evaluate DSIS in real-world scenarios, such as outdoor surveillance and automotive vision systems. Through comprehensive testing and performance analysis, we seek to demonstrate the effectiveness of DSIS in restoring visibility under varying fog densities and lighting conditions. Additionally, we will explore optimization techniques to enhance DSIS's computational efficiency, ensuring its practical feasibility for deployment in resource-constrained environments.

Anomaly Detection on Small Wind Turbine Blades Using Deep Learning Algorithms

Wind turbine blade maintenance is expensive, dangerous, time-consuming, and prone to misdiagnosis. A potential solution to aid preventative maintenance is using deep learning and drones for inspection and early fault detection. In this research, five base deep learning architectures are investigated for anomaly detection on wind turbine blades, including Xception, Resnet-50, AlexNet, and VGG-19, along with a custom convolutional neural network. For further analysis, transfer learning approaches were also proposed and developed, utilizing these architectures as the feature extraction layers. In order to investigate model performance, a new dataset containing 6000 RGB images was created, making use of indoor and outdoor images of a small wind turbine with healthy and damaged blades. Each model was tuned using different layers, image augmentations, and hyperparameter tuning to achieve optimal performance. The results showed that the proposed Transfer Xception outperformed other architectures by attaining 99.92% accuracy on the test data of this dataset. Furthermore, the performance of the investigated models was compared on a dataset containing faulty and healthy images of large-scale wind turbine blades. In this case, our results indicated that the best-performing model was also the proposed Transfer Xception, which achieved 100% accuracy on the test data. These accuracies show promising results in the adoption of machine learning for wind turbine blade fault identification.

Dark channel prior in low frequency domain for time-efficient single image dehazing

ABSTRACT Dehazing plays a vital role in enhancing the quality of outdoor images for computer-based applications. While dehazing algorithms significantly improve outdoor image quality for computer vision tasks, their effectiveness hinges on two crucial factors: 1. Speed of dehazing operations 2. Platform-agnostic nature of the chosen algorithm. This study introduces a novel Dark Channel Prior-Low Frequency Domain (DCP-LFD) method designed to effectively and efficiently eliminate haze from single hazy images. The method employs a hybrid strategy, combining DCP for haze reduction in the low-frequency domain and Soft-thresholding for enhancing high-frequency coefficients. This dual-pronged approach enables improvement of image quality in low and high-frequency domains, contributing to better dehazing outcomes taking less computation time. The proposed method’s effectiveness is accessed using metrics like PSNR and SSIM, comparing its performance against state-of-the-art dehazing methods across five benchmark datasets: SOTS, RTTS and HSTS subsets from RESIDE, HazeRD, and Middlebury datasets. No-Reference Image Quality Assessments viz. BRISQUE and FADE are performed on RTTS and private dataset to objectively evaluate dehazed image quality. A comparison of dehazing time is executed, highlighting the method’s proficiency in achieving faster results. This analysis accentuates the method’s capacity to efficiently remove haze from outdoor images while emphasising its time-saving efficiency.

Uncovering local aggregated air quality index with smartphone captured images leveraging efficient deep convolutional neural network

The prevalence and mobility of smartphones make these a widely used tool for environmental health research. However, their potential for determining aggregated air quality index (AQI) based on PM2.5 concentration in specific locations remains largely unexplored in the existing literature. In this paper, we thoroughly examine the challenges associated with predicting location-specific PM2.5 concentration using images taken with smartphone cameras. The focus of our study is on Dhaka, the capital of Bangladesh, due to its significant air pollution levels and the large population exposed to it. Our research involves the development of a Deep Convolutional Neural Network (DCNN), which we train using over a thousand outdoor images taken and annotated. These photos are captured at various locations in Dhaka, and their labels are based on PM2.5 concentration data obtained from the local US consulate, calculated using the NowCast algorithm. Through supervised learning, our model establishes a correlation index during training, enhancing its ability to function as a Picture-based Predictor of PM2.5 Concentration (PPPC). This enables the algorithm to calculate an equivalent daily averaged AQI index from a smartphone image. Unlike, popular overly parameterized models, our model shows resource efficiency since it uses fewer parameters. Furthermore, test results indicate that our model outperforms popular models like ViT and INN, as well as popular CNN-based models such as VGG19, ResNet50, and MobileNetV2, in predicting location-specific PM2.5 concentration. Our dataset is the first publicly available collection that includes atmospheric images and corresponding PM2.5 measurements from Dhaka. Our codes and dataset are available at https://github.com/lepotatoguy/aqi.

AG-YOLO: A Rapid Citrus Fruit Detection Algorithm with Global Context Fusion

Citrus fruits hold pivotal positions within the agricultural sector. Accurate yield estimation for citrus fruits is crucial in orchard management, especially when facing challenges of fruit occlusion due to dense foliage or overlapping fruits. This study addresses the issues of low detection accuracy and the significant instances of missed detections in citrus fruit detection algorithms, particularly in scenarios of occlusion. It introduces AG-YOLO, an attention-based network designed to fuse contextual information. Leveraging NextViT as its primary architecture, AG-YOLO harnesses its ability to capture holistic contextual information within nearby scenes. Additionally, it introduces a Global Context Fusion Module (GCFM), facilitating the interaction and fusion of local and global features through self-attention mechanisms, significantly improving the model’s occluded target detection capabilities. An independent dataset comprising over 8000 outdoor images was collected for the purpose of evaluating AG-YOLO’s performance. After a meticulous selection process, a subset of 957 images meeting the criteria for occlusion scenarios of citrus fruits was obtained. This dataset includes instances of occlusion, severe occlusion, overlap, and severe overlap, covering a range of complex scenarios. AG-YOLO demonstrated exceptional performance on this dataset, achieving a precision (P) of 90.6%, a mean average precision (mAP)@50 of 83.2%, and an mAP@50:95 of 60.3%. These metrics surpass existing mainstream object detection methods, confirming AG-YOLO’s efficacy. AG-YOLO effectively addresses the challenge of occlusion detection, achieving a speed of 34.22 frames per second (FPS) while maintaining a high level of detection accuracy. This speed of 34.22 FPS showcases a relatively faster performance, particularly evident in handling the complexities posed by occlusion challenges, while maintaining a commendable balance between speed and accuracy. AG-YOLO, compared to existing models, demonstrates advantages in high localization accuracy, minimal missed detection rates, and swift detection speed, particularly evident in effectively addressing the challenges posed by severe occlusions in object detection. This highlights its role as an efficient and reliable solution for handling severe occlusions in the field of object detection.

A Dual-Channel Dehaze-Net for Single Image Dehazing in Visual Internet of Things Using PYNQ-Z2 Board

A large number of emerging applications, such as autonomous navigation, space exploration, surveillance, military target detection, and remote sensing, use outdoor images to monitor various activities of interest. However, images acquired under unfavorable weather conditions usually suffer from atmospheric scattering due to environmental pollution causing color-shift and low-contrast images. Dehazing is an emerging research area in the computer vision domain that intends to restore the visibility of images by eliminating the latter types of degradation. Single image dehazing, on the other hand, is more challenging since it necessitates a precise assessment of atmospheric light and transmission map. This study aims to design a dual-channel deep neural network (DCD-Net) for estimating the transmission map, further utilized to compute atmospheric light. Finally, a dehazed image is generated using the obtained atmospheric light and the transmission map. The experimental results are compared qualitatively and quantitatively with eight existing dehazing approaches based on ten metrics on six publicly available standard datasets: Foggy Road Image DAtabase, HazeRD, REalistic Single Image DEhazing, NYU-Depth, O-HAZE, I-HAZE, a few natural hazy images, and underwater images. The DCD-Net outperforms conventional techniques, according to extensive studies. Moreover, a range of relative improvements of the proposed method over other approaches is calculated for better analysis of the results. A visual internet of things (VIoT) framework employing a PYNQ-Z2 board is presented in addition to the DCD-Net. It can be applied in real-time applications, particularly in the transportation and surveillance industries. The DCD-Net is suitable for image dehazing by virtue of its multilayered structure. The VIoT uses the DCD-Net for dehazing, while the PYNQ-Z2 board serves as the central processing unit. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper is motivated by the problems occurring due to haze. Haze reduces the visibility of a scene, causing major concerns in transportation and surveillance. Existing approaches have attempted to address this issue, albeit the methods are limited. As a result, this study proposes a new dual-channel CNN model with two modules, where the first module calculates fine details of the image and the second module estimates the transmission map. Furthermore, both features are combined to produce a more reliable transmission map. The training process highly influences the resulting output of the network. Therefore, an algorithm explaining the training instructions for the practitioners is given in the appendix. The obtained transmission map is further used to estimate atmospheric light. The images are then dehazed using atmospheric light and transmission maps. In addition, we have designed a framework for image dehazing using VIoT with a PYNQ-Z2 board. Experimental results suggest that this approach gives expected results, yet, there is one limitation. This method requires a haze image and a corresponding transmission map, which is not always possible. Therefore, we will attempt to design a semi-supervised learning approach in the future.

Blackberry Fruit Classification in Underexposed Images Combining Deep Learning and Image Fusion Methods

Berry production is increasing worldwide each year; however, high production leads to labor shortages and an increase in wasted fruit during harvest seasons. This problem opened new research opportunities in computer vision as one main challenge to address is the uncontrolled light conditions in greenhouses and open fields. The high light variations between zones can lead to underexposure of the regions of interest, making it difficult to classify between vegetation, ripe, and unripe blackberries due to their black color. Therefore, the aim of this work is to automate the process of classifying the ripeness stages of blackberries in normal and low-light conditions by exploring the use of image fusion methods to improve the quality of the input image before the inference process. The proposed algorithm adds information from three sources: visible, an improved version of the visible, and a sensor that captures images in the near-infrared spectra, obtaining a mean F1 score of 0.909±0.074 and 0.962±0.028 in underexposed images, without and with model fine-tuning, respectively, which in some cases is an increase of up to 12% in the classification rates. Furthermore, the analysis of the fusion metrics showed that the method could be used in outdoor images to enhance their quality; the weighted fusion helps to improve only underexposed vegetation, improving the contrast of objects in the image without significant changes in saturation and colorfulness.

MWA-MNN: Multi-patch Wavelet Attention Memristive Neural Network for image restoration

Adverse weather conditions can severely reduce the image quality captured by outdoor imaging devices. However, most existing image restoration algorithms are designed for specific tasks, lack generalization capabilities, and are unable to balance spatial details with contextual feature. Moreover, outdoor images are typically captured using edge devices with limited computing resources, rendering it challenging to deploy image restoration algorithms on edge devices. To tackle these problems, we propose a memristive-based image restoration algorithm, named Multi-patch Wavelet Attention Memristive Neural Network (MWA-MNN). Multiple image restoration tasks can be achieved by implementing Convolutional Neural Network (CNN) based on memristor crossbar arrays, as well as specified memristive circuits. Our proposed network comprises three sub-networks with multi-patch design enabling each sub-network to focus on different scales, thereby preserving more spatial details while fusing rich contextual feature. We propose the Discrete Wavelet Attention Block (DWAB) and the Coordinate Channel Attention Block (CCAB) to extract rich contextual information and the Supervised Selective Kernel Block (SSKB) to effectively filter the extracted features. Our proposed memristor-based circuit implementation provides an effective solution for deploying image restoration algorithms on edge devices. Experiment results on multiple benchmark datasets for image deraining, dehazing, and low-light image enhancement show that our proposed method outperforms over 20 state-of-the-art methods.

Image deraining based on dual-channel component decomposition

Image deraining aims to remove rain streaks from images and reduce information loss in outdoor images caused by rain. As a fundamental task in image processing, image deraining not only enhances the visibility of images but also provides necessary image restoration for advanced vision tasks. Existing image deraining models mostly train end-to-end models by minimizing the similarity between the output image of the model and the rain-free ground truth. Although these methods have achieved significant results, they often perform poorly in the face of dense and changing rain streak scenes. In this paper, we propose a novel method, called Dual-Channel Component Decomposition Network (DCD-Net). The basic idea of DCD-Net is to leverage the separability prior of rainy images, treats the rain-free background layer and the rain streak mask layer as two parallel component extraction tasks. To this end, it builds a dual-branch parallel networks that extract the rain-free background image and decouple the reconstruction information of the rain streak mask, respectively. It finally applies a composite multi-level contrastive supervision to the output of the above dual-branch parallel network, thereby achieving rain streak removal. Extensive experiments on various datasets demonstrate that the proposed model outperforms existing methods in deraining dense rain streak images.

TED-Face: Texture-Enhanced Deep Face Reconstruction in the Wild.

We present TED-Face, a new method for recovering high-fidelity 3D facial geometry and appearance with enhanced textures from single-view images. While vision-based face reconstruction has received intensive research in the past decades due to its broad applications, it remains a challenging problem because human eyes are particularly sensitive to numerically minute yet perceptually significant details. Previous methods that seek to minimize reconstruction errors within a low-dimensional face space can suffer from this issue and generate close yet low-fidelity approximations. The loss of high-frequency texture details is a key factor in their process, which we propose to address by learning to recover both dense radiance residuals and sparse facial texture features from a single image, in addition to the variables solved by previous work-shape, appearance, illumination, and camera. We integrate the estimation of all these factors in a single unified deep neural network and train it on several popular face reconstruction datasets. We also introduce two new metrics, visual fidelity (VIF) and structural similarity (SSIM), to compensate for the fact that reconstruction error is not a consistent perceptual metric of quality. On the popular FaceWarehouse facial reconstruction benchmark, our proposed system achieves a VIF score of 0.4802 and an SSIM score of 0.9622, improving over the state-of-the-art Deep3D method by 6.69% and 0.86%, respectively. On the widely used LS3D-300W dataset, we obtain a VIF score of 0.3922 and an SSIM score of 0.9079 for indoor images, and the scores for outdoor images are 0.4100 and 0.9160, respectively, which also represent an improvement over those of Deep3D. These results show that our method is able to recover visually more realistic facial appearance details compared with previous methods.

A new fractional-order variational approach for joint de-hazing and segmentation of images

Segmenting outdoor images in the presence of haze, fog or smog (which fades the colors and diminishes the contrast of the observed objects) has been a challenging task in image processing with several important applications. In this paper, we propose a new fractional-order variational model that will be able to de-haze and segment a given image simultaneously. The proposed method incorporates the atmospheric veil estimation based on the dark channel prior (DCP). This transmission map can reduce significantly the edge artifacts and enhance estimation precision in the resulting image. The transmission map is then changed over to the high-quality depth map, with which the new fractional-order variational model can be framed to look for the haze free segmenting image for both grey and color outdoor images. An explicit gradient descent scheme is employed to find efficiently the minimizer of the proposed energy functional. Experimental tests on real world scenes show that the proposed method can jointly de-haze and segment hazy or foggy images effectively and efficiently.

MODE: Monocular omnidirectional depth estimation via consistent depth fusion

Monocular depth estimation has seen significant progress in recent years, especially in outdoor scenes. However, depth estimation results are not satisfying in omnidirectional images. As compared to perspective images, estimating the depth map from an omnidirectional image captured in the outdoor scene, using neural networks, has two additional challenges: (i) the depth range of outdoor images varies a lot across different scenes, making it difficult for the depth network to predict accurate depth results for training with an indoor dataset, besides the maximum distance in outdoor scenes mostly stays the same as the camera sees the sky, but depth labels in this region are entirely missing in existing datasets; (ii) a standard representation of omnidirectional images introduces spherical distortion, which causes difficulties for the vanilla network to predict accurate relative structural depth details. In this paper, we propose a novel network-MODE by giving special considerations to those challenges and designing a set of flexible modules for improving the performance of omnidirectional depth estimation. First, a consistent depth structure module is proposed to estimate a consistent depth structure map, and the predicted structural map can improve depth details. Second, to suit the characteristics of spherical sampling, we propose a strip convolution fusion module to enhance long-range dependencies. Third, rather than using a single depth decoder branch as in previous methods, we propose a semantics decoder branch to estimate sky regions in the omnidirectional image. The proposed method is validated on three widely used datasets, demonstrating the state-of-the-art performance. Moreover, the effectiveness of each module is shown through an ablation study on real-world datasets. Our code is available at https://github.com/lkku1/MODE.© 2017 Elsevier Inc. All rights reserved.

A new Gaussian curvature of the image surface based variational model for haze or fog removal.

Outdoor images are usually affected by haze which limits the visibility and reduces the contrast of the images. Removal of haze from real-world images is always a challenging task. Recently, many mathematical models have been proposed for the effective removal of haze from real-world images. However, these models may produce staircase effects or lower the image contrast or smooth the edges of the object. In this paper, we propose a model based on Gaussian curvature for the de-hazing of images. The atmospheric veil estimate is included based on dark channel prior (DCP), which can significantly reduce the artifacts on the edge of the image and increase the accuracy. The transmission map then changes to a high-quality map to reduce haze or fog from gray and color images. DCP combined with Gaussian curvature is done for the first time for image de-hazing/de-fogging. The augmented Lagrangian method is used to find the minimizer of the proposed functional, which will be a system of partial differential equations. To get fast convergence, fast Fourier transforms (FFT) is used to solve the system of PDEs. The performance of the proposed model is compared with other state-of-the-art models qualitatively and quantitatively. The proposed model is tested on various real and synthetic images which show better efficiency in staircase effects reduction, haze/fog removal, image contrast, corners, and sharp edges conservation respectively.