Neural Image Research Articles

Endogenous tagging using split mNeonGreen in human iPSCs for live imaging studies

Endogenous tags have become invaluable tools to visualize and study native proteins in live cells. However, generating human cell lines carrying endogenous tags is difficult due to the low efficiency of homology-directed repair. Recently, an engineered split mNeonGreen protein was used to generate a large-scale endogenous tag library in HEK293 cells. Using split mNeonGreen for large-scale endogenous tagging in human iPSCs would open the door to studying protein function in healthy cells and across differentiated cell types. We engineered an iPS cell line to express the large fragment of the split mNeonGreen protein (mNG21-10) and showed that it enables fast and efficient endogenous tagging of proteins with the short fragment (mNG211). We also demonstrate that neural network-based image restoration enables live imaging studies of highly dynamic cellular processes such as cytokinesis in iPSCs. This work represents the first step towards a genome-wide endogenous tag library in human stem cells.

Endogenous tagging using split mNeonGreen in human iPSCs for live imaging studies.

Endogenous tags have become invaluable tools to visualize and study native proteins in live cells. However, generating human cell lines carrying endogenous tags is difficult due to the low efficiency of homology-directed repair. Recently, an engineered split mNeonGreen protein was used to generate a large-scale endogenous tag library in HEK293 cells. Using split mNeonGreen for large-scale endogenous tagging in human iPSCs would open the door to studying protein function in healthy cells and across differentiated cell types. We engineered an iPS cell line to express the large fragment of the split mNeonGreen protein (mNG21-10) and showed that it enables fast and efficient endogenous tagging of proteins with the short fragment (mNG211). We also demonstrate that neural network-based image restoration enables live imaging studies of highly dynamic cellular processes such as cytokinesis in iPSCs. This work represents the first step towards a genome-wide endogenous tag library in human stem cells.

Neuraltalk+: neural image captioning with visual assistance capabilities

Neuraltalk+: neural image captioning with visual assistance capabilities

Lateral ionic-gated graphene synaptic transistor with transition from paired-pulse facilitation to depression for filtering and image recognition

The transition from paired-pulse facilitation to paired-pulse depression is of significant importance for regulating biologic neural network. Here, we fabricate and investigate a lateral ionic-gated graphene synaptic transistor (GST) with short gate length. We control the channel current via e-field-dependent movement and diffusion of ions for realizing the transition from paired-pulse facilitation to depression. This phenomenon comes from the over-diffusion and trapping of a finite number of ions and the short ion-diffusion length under large negative gate pulse stimuli. This device successfully emulates facilitation and depression synaptic functions, for leveraging the accumulation, migration and diffusion of ions. Moreover, enhanced high-pass filtering and conversion from long-term depression to long-term potentiation are achieved by changing pulses time interval. Our results indicate that our device possesses excellent frequency-dependent synaptic plasticity, making it highly suitable for information filtering up to 33 Hz in neuromorphic systems, and image edge enhancement for neural image processing. In additional, a simulated artificial neural network built on lateral ionic-gated GSTs for handwritten digit recognition can achieve a learning accuracy of 82.4 %.

Novel virtual sensors development based on machine learning combined with convolutional neural-network image processing-translation for feedback control systems of internal combustion engines

Physical sensors are commonly used to record performance data of internal combustion engines (ICEs) for online feedback control and calibration, but they are prone to diagnostic and increased development costs. Lookup tables are commonly used in conventional calibration and feedback control; however, the table parameters increase with the advancement of ICE technologies under transient operations. Consequently, the calibration and control systems are time-consuming. This work proposes novel virtual sensors to address these issues by predicting the combustion, performance, and emission of ICEs using neural networks and image processing/translation. The novel sensors are targeted for onboard feedback control systems under transient driving. Firstly, a virtual diesel engine (VDE) was developed and calibrated against experimental data taken from a production 2.2 L turbocharged diesel engine. The VDE was calibrated under WLTC, JC08, and NEDC transient operations and was used to generate teaching data. Next, the virtual sensors are developed using five machine learning (ML) regressors. The result shows that the coefficient of determination R2 from all ML regressors exceeded 0.94, and the XG-Boost outperforms other ML techniques with R2 > 0.977. XG-Boost parameter estimations were 8 times faster than that on a desktop simulation. Then, an image classification model using a deep convolutional neural network (D-CNN) is constructed, and the dependency of performance parameters and exhaust emissions with the rate of heat release (R.H.R) and in-cylinder pressure profile is confirmed. The performance parameters and emissions dependency was compared individually with R.H.R. and the in-cylinder pressure profile. As a result, a strong correlation between the performance and R.H.R. was observed. Finally, a generative adversarial network (GAN) model was constructed to translate the in-cylinder pressure profile to R.H.R. profile. A novel method to develop virtual sensors for advanced feedback control of any type of ICEs is proposed for the first time.

Quantitative analysis of trabecular bone tissue cryosections via a fully automated neural network-based approach.

Cryosectioning is known as a common and well-established histological method, due to its easy accessibility, speed, and cost efficiency. However, the creation of bone cryosections is especially difficult. In this study, a cryosectioning protocol for trabecular bone that offers a relatively cheap and undemanding alternative to paraffin or resin embedded sectioning was developed. Sections are stainable with common histological dying methods while maintaining sufficient quality to answer a variety of scientific questions. Furthermore, this study introduces an automated protocol for analysing such sections, enabling users to rapidly access a wide range of different stainings. Therefore, an automated 'QuPath' neural network-based image analysis protocol for histochemical analysis of trabecular bone samples was established, and compared to other automated approaches as well as manual analysis regarding scattering, quality, and reliability. This highly automated protocol can handle enormous amounts of image data with no significant differences in its results when compared with a manual method. Even though this method was applied specifically for bone tissue, it works for a wide variety of different tissues and scientific questions.

Dynamic Unstructured Pruning Neural Network Image Super-resolution Reconstruction

Dynamic Unstructured Pruning Neural Network Image Super-resolution Reconstruction

Unmasking DeepFakes Using Machine Learning

Our task objectives to deal with the trouble of Deepfake videos, which can be inflicting fear because of their capacity to unfold fake records and manage human beings. Deepfake veideos appearance so actual that it`s tough to inform them other than actual ones, making them a severe danger. This sensible look can result in sizable damage .We are using Machine Learning Algorithms discover Deepfake videos. This entails the usage of era to differentiate among actual and manipulated content. By growing this, we lessen the effect of incorrect information and manipulation, safeguarding human beings from cappotential damage as a result of deceptive content. Keywords: Deepfake Detection, Machine Learning, Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Facial Analysis, Anomaly Detection, Synthetic Media, Image and Video Analysis, Temporal and Spatial Patterns, Training Datasets, Real-time Detection, Visual Content Security

Convolutional neural network-based magnetic resonance image differentiation of filum terminale ependymomas from schwannomas

PurposePreoperative diagnosis of filum terminale ependymomas (FTEs) versus schwannomas is difficult but essential for surgical planning and prognostic assessment. With the advancement of deep-learning approaches based on convolutional neural networks (CNNs), the aim of this study was to determine whether CNN-based interpretation of magnetic resonance (MR) images of these two tumours could be achieved.MethodsContrast-enhanced MRI data from 50 patients with primary FTE and 50 schwannomas in the lumbosacral spinal canal were retrospectively collected and used as training and internal validation datasets. The diagnostic accuracy of MRI was determined by consistency with postoperative histopathological examination. T1-weighted (T1-WI), T2-weighted (T2-WI) and contrast-enhanced T1-weighted (CE-T1) MR images of the sagittal plane containing the tumour mass were selected for analysis. For each sequence, patient MRI data were randomly allocated to 5 groups that further underwent fivefold cross-validation to evaluate the diagnostic efficacy of the CNN models. An additional 34 pairs of cases were used as an external test dataset to validate the CNN classifiers.ResultsAfter comparing multiple backbone CNN models, we developed a diagnostic system using Inception-v3. In the external test dataset, the per-examination combined sensitivities were 0.78 (0.71–0.84, 95% CI) based on T1-weighted images, 0.79 (0.72–0.84, 95% CI) for T2-weighted images, 0.88 (0.83–0.92, 95% CI) for CE-T1 images, and 0.88 (0.83–0.92, 95% CI) for all weighted images. The combined specificities were 0.72 based on T1-WI (0.66–0.78, 95% CI), 0.84 (0.78–0.89, 95% CI) based on T2-WI, 0.74 (0.67–0.80, 95% CI) for CE-T1, and 0.81 (0.76–0.86, 95% CI) for all weighted images. After all three MRI modalities were merged, the receiver operating characteristic (ROC) curve was calculated, and the area under the curve (AUC) was 0.93, with an accuracy of 0.87.ConclusionsCNN based MRI analysis has the potential to accurately differentiate ependymomas from schwannomas in the lumbar segment.

Machine learning aided single cell image analysis improves understanding of morphometric heterogeneity of human mesenchymal stem cells

The multipotent stem cells of our body have been largely harnessed in biotherapeutics. However, as they are derived from multiple anatomical sources, from different tissues, human mesenchymal stem cells (hMSCs) are a heterogeneous population showing ambiguity in their in vitro behavior. Intra-clonal population heterogeneity has also been identified and pre-clinical mechanistic studies suggest that these cumulatively depreciate the therapeutic effects of hMSC transplantation. Although various biomarkers identify these specific stem cell populations, recent artificial intelligence-based methods have capitalized on the cellular morphologies of hMSCs, opening a new approach to understand their attributes. A robust and rapid platform is required to accommodate and eliminate the heterogeneity observed in the cell population, to standardize the quality of hMSC therapeutics globally.Here, we report our primary findings of morphological heterogeneity observed within and across two sources of hMSCs namely, stem cells from human exfoliated deciduous teeth (SHEDs) and human Wharton jelly mesenchymal stem cells (hWJ MSCs), using real-time single-cell images generated on immunophenotyping by imaging flow cytometry (IFC). We used the ImageJ software for identification and comparison between the two types of hMSCs using statistically significant morphometric descriptors that are biologically relevant. To expand on these insights, we have further applied deep learning methods and successfully report the development of a Convolutional Neural Network-based image classifier. In our research, we introduced a machine learning methodology to streamline the entire procedure, utilizing convolutional neural networks and transfer learning for binary classification, achieving an accuracy rate of 97.54%. We have also critically discussed the challenges, comparisons between solutions and future directions of machine learning in hMSC classification in biotherapeutics.

A Convolutional Neural Network Image Compression Algorithm for UAVs

In the task of power line inspection, Unmanned Aerial Vehicles (UAVs) are frequently used for capturing images. With the rapid advancement of sensor technology, the spatial, radiometric, and spectral resolutions of UAV images are constantly improving, leading to an increased storage requirement for individual images. Given that UAVs usually operate with limited computational resources, transmission capability and storage space, there are significant challenges in image compression, storage and transmission. This underscores the importance of a high-performance image compression technique. To solve the above problem, we unveil a compression strategy for images that have been acquired through learning utilizing discrete Gaussian mixture-based probability distributions to increase the efficiency of image compression and the fidelity of reconstruction. In addition, to speed up decoding, we employ a parallel context model, which facilitates decoding in a highly parallel manner. Experimental evidence indicates that our approach attains performance that is at the forefront of the field while significantly expediting the decoding process (speeding up the decoding process by more than 49.78%) in our experiments, outpacing traditional coding standards and existing learned compression approaches by 5.75[Formula: see text]dB and 1.23[Formula: see text]dB in PSNR.

NAWQ-SR: A Hybrid-Precision NPU Engine for Efficient On-Device Super-Resolution

In recent years, image and video delivery systems have begun integrating deep learning super-resolution (SR) approaches, leveraging their unprecedented visual enhancement capabilities while reducing reliance on networking conditions. Nevertheless, deploying these solutions on mobile devices still remains an active challenge as SR models are excessively demanding with respect to workload and memory footprint. Despite recent progress on on-device SR frameworks, existing systems either penalize visual quality, lead to excessive energy consumption or make inefficient use of the available resources. This work presents NAWQ-SR, a novel framework for the efficient on-device execution of SR models. Through a novel hybrid-precision quantization technique and a runtime neural image codec, NAWQ-SR exploits the multi-precision capabilities of modern mobile NPUs in order to minimize latency, while meeting user-specified quality constraints. Moreover, NAWQ-SR selectively adapts the arithmetic precision at run time to equip the SR DNN's layers with wider representational power, improving visual quality beyond what was previously possible on NPUs. Altogether, NAWQ-SR achieves an average speedup of 7.9×, 3× and 1.91× over the state-of-the-art on-device SR systems that use heterogeneous processors (MobiSR), CPU (SplitSR) and NPU (XLSR), respectively. Furthermore, NAWQ-SR delivers an average of 3.2× speedup and 0.39 dB higher PSNR over status-quo INT8 NPU designs, but most importantly mitigates the negative effects of quantization on visual quality, setting a new state-of-the-art in the attainable quality of NPU-based SR.

Pelvic bone tumor segmentation fusion algorithm based on fully convolutional neural network and conditional random field

Background and objectivePelvic bone tumors represent a harmful orthopedic condition, encompassing both benign and malignant forms. Addressing the issue of limited accuracy in current machine learning algorithms for bone tumor image segmentation, we have developed an enhanced bone tumor image segmentation algorithm. This algorithm is built upon an improved full convolutional neural network, incorporating both the fully convolutional neural network (FCNN-4s) and a conditional random field (CRF) to achieve more precise segmentation. MethodologyThe enhanced fully convolutional neural network (FCNN-4s) was employed to conduct initial segmentation on preprocessed images. Following each convolutional layer, batch normalization layers were introduced to expedite network training convergence and enhance the accuracy of the trained model. Subsequently, a fully connected conditional random field (CRF) was integrated to fine-tune the segmentation results, refining the boundaries of pelvic bone tumors and achieving high-quality segmentation. ResultsThe experimental outcomes demonstrate a significant enhancement in segmentation accuracy and stability when compared to the conventional convolutional neural network bone tumor image segmentation algorithm. The algorithm achieves an average Dice coefficient of 93.31 %, indicating superior performance in real-time operations. ConclusionIn contrast to the conventional convolutional neural network segmentation algorithm, the algorithm presented in this paper boasts a more intricate structure, proficiently addressing issues of over-segmentation and under-segmentation in pelvic bone tumor segmentation. This segmentation model exhibits superior real-time performance, robust stability, and is capable of achieving heightened segmentation accuracy.

Detection of Potential Faults in the Electricity Distribution Network Using Unmanned Aerial Vehicles and Thermal Cameras Through Deep Learning Methods

In this article, a software-based unmanned aerial vehicle (UAV) prototype has been developed for exploration purposes, replacing the current manual inspection of energy transmission lines (ETLs) by human teams. The UAV's capabilities address the limitations of human viewing angles, ensuring more efficient exploration activities. Real-time exploration is now achieved through data collected with UAVs and processed using Convolutional Neural Networks (CNNs) and image processing techniques. The UAV’s path and detected objects are autonomously processed in the Cartesian coordinate system. The autonomous conductor tracking system provides bidirectional tracking of conductors in the distribution network through color space range and Faster R-CNN conductor detection. Incorporating a proportional-integral-derivative (PID) design, the UAV is capable of operating in adverse weather conditions. An additional overheating warning system protects equipment in the electricity distribution network (EDN) from potential issues related to uneven load distribution, economic life, and labor errors. Through modeling studies with convolutional neural networks and field data, impressive F-1 scores have been achieved for detecting various anomalies: 77% for trees in contact with the line, 67% for concrete equipment exceeding its economic life, 88% for bird nests, 55% for cracked porcelain insulators, 89% for foreign objects, and 97% for nonuse of original fuses.

Case-based selection of explanation methods for neural network image classifiers

Deep learning is especially remarkable in terms of image classification. However, the outcomes of models are not explainable to users due to their complex nature, having an impact on the users’ trust in the provided classifications. To solve this problem, several explanation techniques have been proposed, but they greatly depend on the nature of the images being classified and the users’ perception of the explanations.In this work, we present Case-Based Reasoning as a learning-based solution to the problem of selecting the best explanation method for the image classifications obtained by models. We propose the elicitation of a case base that reflects the human perception of the quality of the explanations and how to reuse this knowledge to select the best explanation approach for a given image classification.

Hybrid Quantum Neural Network Image Anti-Noise Classification Model Combined with Error Mitigation

In this study, we present an innovative approach to quantum image classification, specifically designed to mitigate the impact of noise interference. Our proposed method integrates key technologies within a hybrid variational quantum neural network architecture, aiming to enhance image classification performance and bolster robustness in noisy environments. We utilize a convolutional autoencoder (CAE) for feature extraction from classical images, capturing essential characteristics. The image information undergoes transformation into a quantum state through amplitude coding, replacing the coding layer of a traditional quantum neural network (QNN). Within the quantum circuit, a variational quantum neural network optimizes model parameters using parameterized quantum gate operations and classical–quantum hybrid training methods. To enhance the system’s resilience to noise, we introduce a quantum autoencoder for error mitigation. Experiments conducted on FashionMNIST datasets demonstrate the efficacy of our classification model, achieving an accuracy of 92%, and it performs well in noisy environments. Comparative analysis with other quantum algorithms reveals superior performance under noise interference, substantiating the effectiveness of our method in addressing noise challenges in image classification tasks. The results highlight the potential advantages of our proposed quantum image classification model over existing alternatives, particularly in noisy environments.

Weighted Fusion of Pre-processing Techniques for Neural Network-based Image Haze Removal

Haze is the natural phenomenon, which affects an image's air light and visibility. It creates a layer that hides the information in an acquired hazy image and decreases its visibility. Hazy scenarios are mostly seen in the transportation sector and remote sensing. It affects the quality of an image captured. Haze is one of the major hurdles in several computer vision applications. This paper observes and analyses different methods of haze removal via image enhancement techniques. Proposes the weighted average of the image enhancement methods to generate the enhanced hazy input image as the initial step. These enhanced images do train the neural network to estimate transmission map as well as atmospheric light, used for haze removal from images. The proposed method is experimented with 135 hazy images from three standard datasets, alias I-Haze, NH-Haze, and O-Haze (45 images from each total 135 hazy images). It gives clearer results than a few similar existing haze removal techniques. Also, the experimental results tested with performance metrics Entropy PSNR, and SSIM have demonstrated the effectiveness of the proposed haze removal method having weighted fusion of pre-processing techniques.

Diagnosing osteoporosis using deep neural networkassisted optical image processing method

Diagnosing osteoporosis using deep neural networkassisted optical image processing method

Understanding skin color bias in deep learning-based skin lesion segmentation

Background: The field of dermatological image analysis using deep neural networks includes the semantic segmentation of skin lesions, pivotal for lesion analysis, pathology inference, and diagnoses. While biases in neural network-based dermatoscopic image classification against darker skin tones due to dataset imbalance and contrast disparities are acknowledged, a comprehensive exploration of skin color bias in lesion segmentation models is lacking. It is imperative to address and understand the biases in these models.Methods: Our study comprehensively evaluates skin tone bias within prevalent neural networks for skin lesion segmentation. Since no information about skin color exists in widely used datasets, to quantify the bias we use three distinct skin color estimation methods: Fitzpatrick skin type estimation, Individual Typology Angle estimation as well as manual grouping of images by skin color. We assess bias across common models by training a variety of U-Net-based models on three widely-used datasets with 1758 different dermoscopic and clinical images. We also evaluate commonly suggested methods to mitigate bias.Results: Our findings expose a significant and large correlation between segmentation performance and skin color, revealing consistent challenges in segmenting lesions for darker skin tones across diverse datasets. Using various methods of skin color quantification, we have found significant bias in skin lesion segmentation against darker-skinned individuals when evaluated both in and out-of-sample. We also find that commonly used methods for bias mitigation do not result in any significant reduction in bias.Conclusions: Our findings suggest a pervasive bias in most published lesion segmentation methods, given our use of commonly employed neural network architectures and publicly available datasets. In light of our findings, we propose recommendations for unbiased dataset collection, labeling, and model development. This presents the first comprehensive evaluation of fairness in skin lesion segmentation.

Historical changes in tree and impervious surface cover following urban renewal in a small postindustrial city.

Changes in tree cover and impervious surfaces have been observed across many cities in the United States over the past 70 years. Many municipalities are implementing tree planting programs in efforts to increase tree cover. A detailed understanding of historical changes in land cover can inform urban forest management. I applied a convolutional neural network image segmentation approach to historical aerial imagery to delineate changes in land cover in 1957, 1974, and 2017 in Utica, New York, a small, postindustrial city. The model predicted tree, pavement, and building land cover in each year with overall accuracies ranging from 82-87%. From 1957 to 2017, tree cover declined in many areas and impervious surface cover (buildings and pavement) increased. Tree cover gains largely occurred in uninhabited, natural areas; whereas, the greatest declines in tree coverage occurred in many residential areas following the start of the urban renewal efforts in 1957. Current tree planting efforts targeted at homeowners could drive disparities in future tree cover since several areas of Utica with low tree have a high proportion of renter occupied homes and a low median household income. Convolutional Neural Network approaches for image segmentation of aerial imagery are a helpful tool in understanding patterns in changes in tree and impervious surfaces. A better understanding of the legacies of historical policies and neighborhood-scale changes in land cover can assist in highlighting priorities for urban forest management and justice-oriented urban forestry approaches to urban tree planting.