Neural Images Research Articles

About the Possibility of Using Fixation Microsaccades to Improve the Quality of Visible Images in the Foveal Zone

The article is devoted to the description and analysis of a computer model that was created by D. S. Lebedev to demonstrate the possibility of a positive effect of fixation microsaccadic eye movements on the perception of small stimuli. The model is based on the assumption that in the process of fixing the gaze on the test stimulus, several “neural images” of this stimulus, resulting from microsaccades, are summed up in the brain. The series of summed neural images correspond to a sequence of shifted positions of the optical image of a stimulus on the retina. To accurately superimpose neural images on each other, a mechanism for compensating fixation saccadic microshifts is introduced into the model, identical to the mechanism that ensures the constancy of spatial perception in the case of macrosaccades, i.e. when turning the eyes to view large objects or scenes. The author of the model assessed the possibility of improving the quality of visible images by increasing the signal-to-noise ratio, which can be achieved using realistic spatiotemporal parameters of test images, neural noise and eye micromovements, selected bу means of literature analysis. Results of model calculation obtained for the specific parameters of the retina and eye movements showed that the considered summation mechanism with compensation for saccadic shifts can progressively improve the quality of visible test stimuli when the number of summed neural images increases to approximately seven or eight, after which the positive effect practically does not increase. In this article, based on the material of recordings of eye movements in relevant experiments, the degree of realism of this model is discussed.

Efficient and user-friendly visualization of neural relightable images for cultural heritage applications

We introduce an innovative multiresolution framework for encoding and interactively visualizing large relightable images using a neural reflectance model derived from a state-of-the-art technique. The framework is seamlessly integrated into a scalable multi-platform framework that supports adaptive streaming and exploration of multi-layered relightable models in web settings. To enhance efficiency, we optimized the neural model, simplified decoding, and implemented a custom WebGL shader specific to the task, eliminating the need for deep-learning library integration in the code. Additionally, we introduce an efficient level-of-detail management system supporting fine-grained adaptive rendering through on-the-fly resampling in latent feature space. The resulting viewer facilitates interactive neural relighting of large images. Its modular design allows the incorporation of functionalities for Cultural Heritage analysis, such as loading and simultaneous visualization of multiple relightable layers with arbitrary rotations.

Aggregated Pattern Classification Method for improving neural disorder stage detection.

Neurological disorders pose a significant health challenge, and their early detection is critical for effective treatment planning and prognosis. Traditional classification of neural disorders based on causes, symptoms, developmental stage, severity, and nervous system effects has limitations. Leveraging artificial intelligence (AI) and machine learning (ML) for pattern recognition provides a potent solution to address these challenges. Therefore, this study focuses on proposing an innovative approach-the Aggregated Pattern Classification Method (APCM)-for precise identification of neural disorder stages. The APCM was introduced to address prevalent issues in neural disorder detection, such as overfitting, robustness, and interoperability. This method utilizes aggregative patterns and classification learning functions to mitigate these challenges and enhance overall recognition accuracy, even in imbalanced data. The analysis involves neural images using observations from healthy individuals as a reference. Action response patterns from diverse inputs are mapped to identify similar features, establishing the disorder ratio. The stages are correlated based on available responses and associated neural data, with a preference for classification learning. This classification necessitates image and labeled data to prevent additional flaws in pattern recognition. Recognition and classification occur through multiple iterations, incorporating similar and diverse neural features. The learning process is finely tuned for minute classifications using labeled and unlabeled input data. The proposed APCM demonstrates notable achievements, with high pattern recognition (15.03%) and controlled classification errors (CEs) (10.61% less). The method effectively addresses overfitting, robustness, and interoperability issues, showcasing its potential as a powerful tool for detecting neural disorders at different stages. The ability to handle imbalanced data contributes to the overall success of the algorithm. The APCM emerges as a promising and effective approach for identifying precise neural disorder stages. By leveraging AI and ML, the method successfully resolves key challenges in pattern recognition. The high pattern recognition and reduced CEs underscore the method's potential for clinical applications. However, it is essential to acknowledge the reliance on high-quality neural image data, which may limit the generalizability of the approach. The proposed method allows future research to refine further and enhance its interpretability, providing valuable insights into neural disorder progression and underlying biological mechanisms.

Sensor-Fused Nighttime System for Enhanced Pedestrian Detection in ADAS and Autonomous Vehicles.

Ensuring a safe nighttime environmental perception system relies on the early detection of vulnerable road users with minimal delay and high precision. This paper presents a sensor-fused nighttime environmental perception system by integrating data from thermal and RGB cameras. A new alignment algorithm is proposed to fuse the data from the two camera sensors. The proposed alignment procedure is crucial for effective sensor fusion. To develop a robust Deep Neural Network (DNN) system, nighttime thermal and RGB images were collected under various scenarios, creating a labeled dataset of 32,000 image pairs. Three fusion techniques were explored using transfer learning, alongside two single-sensor models using only RGB or thermal data. Five DNN models were developed and evaluated, with experimental results showing superior performance of fused models over non-fusion counterparts. The late-fusion system was selected for its optimal balance of accuracy and response time. For real-time inferencing, the best model was further optimized, achieving 33 fps on the embedded edge computing device, an 83.33% improvement in inference speed over the system without optimization. These findings are valuable for advancing Advanced Driver Assistance Systems (ADASs) and autonomous vehicle technologies, enhancing pedestrian detection during nighttime to improve road safety and reduce accidents.

42‐1: Artificial Retina‐Based Metaverse with Bionic Vision Processing

An artificial retina featuring bionic vision processing is presented as a paradigm shift of metaverse. The main contribution is to provide a methodology to better understand the human vision and to reproduce images along the visual pathway. Simulation results include retinal images, bilateral neural images, depth maps, etc.

Sweet Pepper Leaf Area Estimation Using Semantic 3D Point Clouds Based on Semantic Segmentation Neural Network

In the field of agriculture, measuring the leaf area is crucial for the management of crops. Various techniques exist for this measurement, ranging from direct to indirect approaches and destructive to non-destructive techniques. The non-destructive approach is favored because it preserves the plant's integrity. Among these, several methods utilize leaf dimensions, such as width and length, to estimate leaf areas based on specific models that consider the unique shapes of leaves. Although this approach does not damage plants, it is labor-intensive, requiring manual measurements of leaf dimensions. In contrast, some indirect non-destructive techniques leveraging convolutional neural networks can predict leaf areas more swiftly and autonomously. In this paper, we propose a new direct method using 3D point clouds constructed by semantic RGB-D (Red Green Blue and Depth) images generated by a semantic segmentation neural network and RGB-D images. The key idea is that the leaf area is quantified by the count of points depicting the leaves. This method demonstrates high accuracy, with an R2 value of 0.98 and a RMSE (Root Mean Square Error) value of 3.05 cm2. Here, the neural network's role is to segregate leaves from other plant parts to accurately measure the leaf area represented by the point clouds, rather than predicting the total leaf area of the plant. This method is direct, precise, and non-invasive to sweet pepper plants, offering easy leaf area calculation. It can be implemented on laptops for manual use or integrated into robots for automated periodic leaf area assessments. This innovative method holds promise for advancing our understanding of plant responses to environmental changes. We verified the method's reliability and superior performance through experiments on individual leaves and whole plants.

Analyzing and Improving Existing Neural Network-Based Approaches to Identify AI Generated Images

Images produced by AI software, particularly human faces, have the potential to spread misinformation, thus detection tools are essential to tell facts from fiction. However, current neural network-based tools misclassify images with extreme photography conditions, facial obstructions, or post-processing. This project aimed to address the limitations of existing detection models, especially regarding poorly-photographed or edited images. A MobileNet-v2-based neural network to identify synthetic images was analyzed for possible shortcomings. After obtaining its baseline accuracy, it was stress tested on validation data with brightness, contrast, hue, and saturation edited by set increments. The model performed poorly on processed images and faces with glasses or shadows. Possible limitations were insufficient data augmentation, the Global Average Pooling Layer, too few epochs trained, and inadequate regularization strength or dropout rate. The model was then modified to address these shortcomings by altering the brightness, contrast, hue, and saturation of training data, adding Gaussian noise to training images, implementing a Global Max Pooling Layer, augmenting the number of epochs trained, and changing the regularization strength and dropout rate. The modified model was evaluated on augmented and non-augmented images. Overall, the model improved when Gaussian noise was added with standard deviation 0.1, it was trained for 15 epochs, and there were no dropout layers. This was because the Gaussian noise layer approximated degraded image quality, and the model was too simple to learn image features when trained for fewer epochs or when neurons were disabled. This research could be expanded by testing multiple modifications at once.

Explainable artificial intelligence in deep learning neural nets-based digital images analysis

This review shows the capabilities of artificial intelligence in the analysis of digital images in the field of medicine using convolutional neural networks of deep learning. A new generation of artificial intelligence systems is described with an explanation of decision-making algorithms to the user — explainable artificial intelligence (XAI). The taxonomy of the methods of explanation and the description of the methods themselves are given. The substantiation of the need to use explainable artificial intelligence in classification tasks is given on the example of ophthalmic diseases. The study of the components of deep learning methods used in the reviewed works (neural network architecture, accuracy, characteristics of data sets) and explainable artificial intelligence (methods of explanation, criteria for the accuracy of explanation). As an example, the problem of recognizing two of the most commonly diagnosed eye diseases: diabetic retinopathy and glaucoma by artificial neural networks is considered.

Selection of Distance Measure for Visual and Long Wave Infrared Image Region Similarity using CNN Features

Similarity computation between images or image regions is a necessary precursor for several vision-based applications, such as retrieval, registration, change detection etc. A two-channel convolutional neural network architecture is designed to retrieve an appropriate visual (VS) image representative from the repository, given as query a long-wave infrared (LWIR) image patch of the same region. Both the VS and LWIR image regions are described using pretrained convolution neural network models and images are ranked by computing the dis/similarity between the feature vectors. It is essential to evaluate and identify the suitable combination of feature descriptor and distance measure, when applied to LWIR and visual image similarity, as pre-trained CNNs such as VGG16, VGG19, MobileNet are not trained for LWIR images. RoadScene dataset which contains a pair of aligned long-wave infrared and visible images is used and performance of CNN features and distance measures is objectively evaluated using computation time, number of patches in top 5 and also in computing how close the retrieved LWIR patch is to the visual patch. Results demonstrate that Cosine similarity measure is relatively better when compared to all the other distance measures in addressing the spectral variations.

Unsupervised Joint Domain Adaptation for Decoding Brain Cognitive States From tfMRI Images.

Recent advances in large model and neuroscience have enabled exploration of the mechanism of brain activity by using neuroimaging data. Brain decoding is one of the most promising researches to further understand the human cognitive function. However, current methods excessively depends on high-quality labeled data, which brings enormous expense of collection and annotation of neural images by experts. Besides, the performance of cross-individual decoding suffers from inconsistency in data distribution caused by individual variation and different collection equipments. To address mentioned above issues, a Join Domain Adapative Decoding (JDAD) framework is proposed for unsupervised decoding specific brain cognitive state related to behavioral task. Based on the volumetric feature extraction from task-based functional Magnetic Resonance Imaging (tfMRI) data, a novel objective loss function is designed by the combination of joint distribution regularizer, which aims to restrict the distance of both the conditional and marginal probability distribution of labeled and unlabeled samples. Experimental results on the public Human Connectome Project (HCP) S1200 dataset show that JDAD achieves superior performance than other prevalent methods, especially for fine-grained task with 11.5%-21.6% improvements of decoding accuracy. The learned 3D features are visualized by Grad-CAM to build a combination with brain functional regions, which provides a novel path to learn the function of brain cortex regions related to specific cognitive task in group level.

Classification of Alzheimer’s Disease Based on White Matter Connectivity Network

Alzheimer’s disease (AD) is one of the most common irreversible brain diseases in the elderly. Mild cognitive impairment (MCI) is an early symptom of AD, and the early intervention of MCI may slow down the progress of AD. However, due to the subtle neuroimaging differences between MCI and normal control (NC), the clinical diagnosis is subjective and easy to misdiagnose. Machine learning can extract depth features from neural images, and analyze and label them to assist the diagnosis of diseases. This paper combines diffusion tensor imaging (DTI) and support vector machine (SVM) to classify AD, MCI, and NC. First, the white matter connectivity network was constructed based on DTI. Second, the nodes with significant differences between groups were screened out by the two-sample t-test. Third, the optimal feature subset was selected as the classification feature by recursive feature elimination (RFE). Finally, the Gaussian kernel support vector machine was used for classification. The experiment tested and verified the data downloaded from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database, and the area under the curve (AUC) of AD/MCI and MCI/NC are 0.94 and 0.95, respectively, which have certain competitive advantages compared with other methods.

Forest Gap Extraction Based on Convolutional Neural Networks and Sentinel-2 Images

As a type of small-scale disturbance, forest gap and its accurate extraction are of great significance to monitor forest long-term dynamics, to choose forest recovery mode and to predict forest recovery pace. Currently, airborne LiDAR and high-resolution multi-spectral data are commonly used to accurately classify forest gaps, but they are costly to acquire and have limited time and space availability. In contrast, the Sentinel-2 multi-spectral data with a 10 m spatial resolution overcomes these drawbacks in forest gap extraction. In this work, an integrated framework that combines multi-source remote sensing, machine learning and deep learning to extract forest gap in wide regions was proposed and tested in three sites. First, LiDAR, Sentinel series and random forest (RF) algorithm were synergized to produce a canopy height product in model training site. On this basis, samples for forest canopy, forest gap and non-such were identified from LiDAR-derived canopy height model (CHM) and Sentinel-based canopy height inversion (HI) data to train forest gap extraction models by applying the Deep Forest (DF) and Convolutional Neural Networks (CNN) algorithms, followed by a comparison of the accuracy and the transferability among the four models (DF-CHM, DF-HI, CNN-CHM and CNN-HI). The results indicated that the R2 and RMSE of Sentinel-based canopy height retrievals were estimated at 0.63, and 7.85 m respectively, the difference in the mean height and standard deviation between HI and CHM was 0.03 m and 4.7 m respectively. And there was a spatial agreement of about 98.60% between the HI-identified samples and the CHM-identified samples, with an agreement of 54.89% for the forest gap class. The CNN-HI model had the highest accuracy in both transfer learning test sites, with an overall accuracy (OA) of 0.85 and 0.87, Kappa coefficient at 0.78 and 0.81, respectively, proving that it has good transferability. Conversely, the DF-based models generally gave poorer accuracy and transferability. This study demonstrates that combining Sentinel-2 multi-spectral data and CNN algorithm is feasible and effective in forest gap extraction applications over wide regions.

IBL‐NeRF: Image‐Based Lighting Formulation of Neural Radiance Fields

AbstractWe propose IBL‐NeRF, which decomposes the neural radiance fields (NeRF) of large‐scale indoor scenes into intrinsic components. Recent approaches further decompose the baked radiance of the implicit volume into intrinsic components such that one can partially approximate the rendering equation. However, they are limited to representing isolated objects with a shared environment lighting, and suffer from computational burden to aggregate rays with Monte Carlo integration. In contrast, our prefiltered radiance field extends the original NeRF formulation to capture the spatial variation of lighting within the scene volume, in addition to surface properties. Specifically, the scenes of diverse materials are decomposed into intrinsic components for rendering, namely, albedo, roughness, surface normal, irradiance, and prefiltered radiance. All of the components are inferred as neural images from MLP, which can model large‐scale general scenes. Especially the prefiltered radiance effectively models the volumetric light field, and captures spatial variation beyond a single environment light. The prefiltering aggregates rays in a set of predefined neighborhood sizes such that we can replace the costly Monte Carlo integration of global illumination with a simple query from a neural image. By adopting NeRF, our approach inherits superior visual quality and multi‐view consistency for synthesized images as well as the intrinsic components. We demonstrate the performance on scenes with complex object layouts and light configurations, which could not be processed in any of the previous works.

Modern Neural Network Technologies Text-to-Image

This paper discusses state-of-the-art graphical text-to-image neural networks and methods for text-to-image conversion, analyzing the results achieved and samples created to date for text-to-image conversion tasks. Ways of applying neural network approaches to text-to-image transformation for environmental monitoring, infrastructure and medical data analysis tasks are proposed. In this paper the results of neural network generation and its correlation with the user input linguistic constructions of text queries are reviewed, and the typical flaws and artifacts typical of the neural network generated images are identified and classified. The rapid development of neural network technologies in this field could have a significant impact on society, the professional market and the media, which makes the task of studying neural network images and identifying them among other graphic content particularly relevant.

Residual Network with Attention to Neural Cells Segmentation

Many neuroscience applications, including understanding the evolution of the brain, rely on neural cell instance segmentation, which seeks to integrate the identification and segmentation of neuronal cells in microscopic imagery. However, the task is complicated by cell adhesion, deformation, vague cell outlines, low-contrast cell protrusion structures, and background imperfections. On the other hand, existing segmentation approaches frequently produce inaccurate findings. As a result, an effective strategy for using the residual network with attention to segment cells is suggested in this paper. The segmentation mask of neural cells may be accurately predicted. This method is built on U-net, with EfficientNet serving as the encoder's backbone. The attention approach is employed in the detection and segmentation modules to guide the model's attention to the most valuable features. A massive collection of neural cell microscopic images tests the proposed method. According to the findings of the experiments, this technology can accurately detect and segment neuronal cell occurrences with an intersection over the union IoU of 95.47 and a Dice-Coeff of 98.34, which is superior to current state-of-the-art approaches.

Convolutional neural network-based apple images classification and image quality measurement by light colors using the color-balancing approach

Convolutional neural network-based apple images classification and image quality measurement by light colors using the color-balancing approach

Prediction of lake chlorophyll concentration using the BP neural network and Sentinel-2 images based on time features.

One of the most important indicators of lake eutrophication is chlorophyll-a (Chl-a) concentration, which is also an essential component of lake water quality monitoring. It is an efficient, economical and convenient method to monitor the Chl-a concentration through remote sensing images. Taking the Wuliangsuhai Lake as an example, the relevant bands of Sentinel-2 images were used as the input and the Chl-a concentration as the output to build neural network models. In the process of building the model, we mainly studied and tested the impact of adding time features to the model input on the model accuracy. Through the experiment, it was found that the month and day difference features of remote sensing images and Chl-a measurement could significantly improve the prediction accuracy of Chl-a concentration in varying degrees. Finally, it was determined that the neural network prediction model with 12 bands of Sentinel-2 images combined month features as inputs and one hidden layer, eight neurons and Chl-a concentration as outputs was the best. Then, the accuracy of the model was validated when the test set accounts for 20 and 30%, and good results were obtained.

Mapping soil available copper content in the mine tailings pond with combined simulated annealing deep neural network and UAV hyperspectral images

Improper discharge of slag from mining will pollute the surrounding soil, thereby affecting the ecology and becoming an important global problem. The available copper (ACu) content in polluted soil is an important factor affecting plant growth and development. When investigating a large area of soil with ACu, manual sampling by points and inspection are mainly used, due to the heterogeneity of soil, the efficiency and accuracy are lower. The Unmanned aerial vehicle (UAV) equipped with a hyperspectral sensor as a remote sensing technology is widely used in soil indicator monitoring because of its rapid and convenience. Meanwhile, using the relationship between soil organic matter and available copper has the potential to predict available copper. In this study, we selected the study area with tailings area in the Jianghan Plain of China and used a UAV equipped with a hyperspectral sensor to predict ACu and soil organic matter (SOM) in the soil with two datasets. Firstly, 74 soil samples were collected in the study area, and the ACu and SOM of the soil samples were determined. Second, a hyperspectral image of the study area is obtained using a UAV equipped with a hyperspectral sensor. Thirdly, we combine hyperspectral data with competitive adaptive reweighted sampling (CARS) to obtain feature bands and utilize simulated annealing deep neural network (SA-DNN) to generate estimation models. Finally, maps of the distribution of ACu and SOM in the area were generated using the model. In two datasets, the model of ACu with R2 values both are 0.89, and R2 on the model of SOM is 0.89 and 0.88. The results show that the combination of UAV hyperspectral imagery with the SA-DNN model has good performance in the prediction of organic matter and available copper, which is helpful for soil environmental monitoring.

Application of an Improved U2-Net Model in Ultrasound Median Neural Image Segmentation.

To investigate whether an improved U2-Net model could be used to segment the median nerve and improve segmentation performance, we performed a retrospective study with 402 nerve images from patients who visited Huashan Hospital from October 2018 to July 2020; 249 images were from patients with carpal tunnel syndrome, and 153 were from healthy volunteers. From these, 320 cases were selected as training sets, and 82 cases were selected as test sets. The improved U2-Net model was used to segment each image. Dice coefficients (Dice), pixel accuracy (PA), mean intersection over union (MIoU) and average Hausdorff distance (AVD) were used to evaluate segmentation performance. Results revealed that the Dice, MIoU, PA and AVD values of our improved U2-Net were 72.85%, 79.66%, 95.92% and 51.37 mm, respectively, which were comparable to the actual ground truth; the ground truth came from the labeling of clinicians. However, the Dice, MIoU, PA and AVD values of U-Net were 43.19%, 65.57%, 86.22% and 74.82 mm, and those of Res-U-Net were 58.65%, 72.53%, 88.98% and 57.30 mm. Overall, our data suggest our improved U2-Net model might be used for segmentation of ultrasound median neural images.

COVID-19 Disease Detection using Pre-Trained Deep Convolutional Neural Network (GoogleNet) on Cloud Platform

Background and Purpose: The COVID -19 epidemics are causing the main rash in more than 151 countries around the whole world.Covid-19 has a bad effect on human life worldwide. One of the critical steps in fighting COVID-19 is finding the contaminated patients early enough and putting these infected people under special care. Our main aim is to separate COVID-19 patients from other patients. Materials and Methods:In this research article, we used GoogleNet as a learning network. GoogleNet is a deep convolutional neural network of 22 layers deep. We have used a pre-trained version of the GoogleNet trained on ImageNet. The pre-trained GoogleNet image input size is 224 x 224.GoogleNet; the deep convolutional neural network model cananalyze X-ray images to classify the patient’s condition of the affected disease. Result:Experiments and evaluation of the GoogleNet have been effectively done based on 80% of X-ray pictures for training and 20% of X-ray pictures for testing phases respectively. GoogleNet shows a good result for disease classification with 91.40% of accuracy in 2.49 minutes. Conclusion:In this research paper, we have usedthe deep CNN model to classify COVID-19 disease using X-ray images based on the projected GoogleNet. Scientific studies will be the next goal of this research article