Computer Image Research Articles

Enhancing Brain Tumour Identification through Convolutional Neural Network

A tumor is carried on by rapid and uncontrolled cell proliferation in the brain. If it is not treated in the early stages, it could prove fatal. Despite multiple significant efforts and encouraging outcomes, accurate segmentation and classification still pose a difficulty. Detection of brain tumors is extremely complicated by the distinctions in tumor position, structure, and proportions. Using computational intelligence and statistical image processing techniques, proposed in our project provide multiple approaches to detect brain cancer and tumors. Also shows an evaluation matrix for a specific system using particular systems and type of dataset. Also explains the morphology of brain tumors, accessible data sets, augmentation techniques, component extraction, and classification of Deep Learning (DL), Transfer Learning (TL), and Machine Learning (ML) models. Key Words: brain tumour, image classification, image sementation, convolutional neural networks.

Detail-preserving Joint Image Upsampling

Image operators can be instrumental to computational imaging and photography. However, many of them are computationally intensive. In this paper, we propose an effective yet efficient joint upsampling method to accelerate various image operators. We show that edge-preserving filtering can be facilitated with a downsampling-and-upsampling process. Moreover, when the extent of smoothing is mild, the process is detail-preserving, i.e., the fine details lost in the low-resolution (LR) images can be accurately restored in the high-resolution (HR) images. Given an HR input and an LR output of an operator, we downsample the HR input and calculate its affinities to the HR input. By applying the affinities to the LR output, we promote its resolution. Due to the strong detail-preserving property, the HR output derived in the previous step may exhibit aliasing artifacts around the salient edges. We further refine it based on the linear relations in a small neighborhood to rid the artifacts. Experiments on various image operators show that our method achieves superior quality over the state-of-the-art joint upsampling methods. Furthermore, the running time of our method is linear to the number of pixels. Our naive implementation derives 1080P images in real time (24 fps) on an NVIDIA GTX 3070 GPU.

Multimodal 2D and 3D microscopic mapping of growth cartilage by computational imaging techniques - a short review including new research.

Being able to image the microstructure of growth cartilage is important for understanding the onset and progression of diseases such as osteochondrosis and osteoarthritis, as well as for developing new treatments and implants. Studies of cartilage using conventional optical brightfield microscopy rely heavily on histological staining, where the added chemicals provide tissue-specific colours. Other microscopy contrast mechanisms include polarization, phase- and scattering contrast, enabling non-stained or "label-free" imaging that significantly simplifies the sample preparation, thereby also reducing the risk of artefacts. Traditional high-performance microscopes tend to be both bulky and expensive. Computational imaging denotes a range of techniques where computers with dedicated algorithms are used as an integral part of the image formation process. Computational imaging offers many advantages like 3D measurements, aberration correction and quantitative phase contrast, often combined with comparably cheap and compact hardware. X-ray microscopy is also progressing rapidly, in certain ways trailing the development of optical microscopy. In this study, we first briefly review the structures of growth cartilage and relevant microscopy characterization techniques with an emphasis on FPM and advanced X-ray microscopies. We next demonstrate with our own results computational imaging through Fourier ptychographic microscopy (FPM) and compare the images with hematoxylin eosin and saffron (HES)-stained histology. Zernike phase contrast, and the nonlinear optical microscopy techniques of second harmonic generation (SHG) and two-photon excitation fluorescence (TPEF) are reported. X-ray attenuation-, phase- and diffraction-contrast computed tomography (CT) images of the very same sample are also presented for comparisons. Future perspectives on the links to artificial intelligence, dynamic studies and in vivo possibilities conclude the article.

Imaging through thick scattering media based on envelope-informed learning with a simulated training dataset

Computational imaging faces significant challenges in dealing with multiple scattering through thick complex media. While deep learning has addressed some ill-posed problems in scattering imaging, its practical application is limited by the acquisition of the training dataset. In this study, the Gaussian-distributed envelope of the speckle image is employed to simulate the point spread function (PSF), and the training dataset is obtained by the convolution of the handwritten digits with the PSF. This approach reduces the requirement of time and conditions for constructing the training dataset and enables a neural network trained on this dataset to reconstruct objects obscured by an unknown scattering medium in real experiments. The quality of reconstructed objects is negatively correlated with the thickness of the scattering medium. Our proposed method provides a new way, to the best of our knowledge, to apply deep learning in scattering imaging by reducing the time needed for constructing the training dataset.

Bio-inspired foveal super-resolution method for multi-focal-length images based on local gradient constraints

Most existing super-resolution (SR) imaging systems, inspired by the bionic compound eye, utilize image registration and reconstruction algorithms to overcome the angular resolution limitations of individual imaging systems. This article introduces a multi-aperture multi-focal-length imaging system and a multi-focal-length image super-resolution algorithm, mimicking the foveal imaging of the human eye. Experimental results demonstrate that with the proposed imaging system and an SR imaging algorithm inspired by the human visual system, the proposed method can enhance the spatial resolution of the foveal region by up to 4 × compared to the original acquired image. These findings validate the effectiveness of the proposed imaging system and computational imaging algorithm in enhancing image texture and spatial resolution.

Keratin 17 modulates the immune topography of pancreatic cancer

BackgroundThe immune microenvironment impacts tumor growth, invasion, metastasis, and patient survival and may provide opportunities for therapeutic intervention in pancreatic ductal adenocarcinoma (PDAC). Although never studied as a potential modulator of the immune response in most cancers, Keratin 17 (K17), a biomarker of the most aggressive (basal) molecular subtype of PDAC, is intimately involved in the histogenesis of the immune response in psoriasis, basal cell carcinoma, and cervical squamous cell carcinoma. Thus, we hypothesized that K17 expression could also impact the immune cell response in PDAC, and that uncovering this relationship could provide insight to guide the development of immunotherapeutic opportunities to extend patient survival.MethodsMultiplex immunohistochemistry (mIHC) and automated image analysis based on novel computational imaging technology were used to decipher the abundance and spatial distribution of T cells, macrophages, and tumor cells, relative to K17 expression in 235 PDACs.ResultsK17 expression had profound effects on the exclusion of intratumoral CD8+ T cells and was also associated with decreased numbers of peritumoral CD8+ T cells, CD16+ macrophages, and CD163+ macrophages (p < 0.0001). The differences in the intratumor and peritumoral CD8+ T cell abundance were not impacted by neoadjuvant therapy, tumor stage, grade, lymph node status, histologic subtype, nor KRAS, p53, SMAD4, or CDKN2A mutations.ConclusionsThus, K17 expression correlates with major differences in the immune microenvironment that are independent of any tested clinicopathologic or tumor intrinsic variables, suggesting that targeting K17-mediated immune effects on the immune system could restore the innate immunologic response to PDAC and might provide novel opportunities to restore immunotherapeutic approaches for this most deadly form of cancer.

Computational study of diffraction image formation from XFEL irradiated single ribosome molecule

Single particle imaging at atomic resolution is perhaps one of the most desired goals for ultrafast X-ray science with X-ray free-electron lasers. Such a capability would create great opportunity within the biological sciences, as high-resolution structural information of biosamples that may not crystallize is essential for many research areas therein. In this paper, we report on a comprehensive computational study of diffraction image formation during single particle imaging of a macromolecule, containing over one hundred thousand non-hydrogen atoms. For this study, we use a dedicated simulation framework, SIMEX, available at the European XFEL facility. Our results demonstrate the full feasibility of computational single-particle imaging studies for biological samples of realistic size. This finding is important as it shows that the SIMEX platform can be used for simulations to inform relevant single-particle-imaging experiments and help to establish optimal parameters for these experiments. This will enable more focused and more efficient single-particle-imaging experiments at XFEL facilities, making the best use of the resource-intensive XFEL operation.

Selection of the Discriming Feature Using the BEMD’s BIMF for Classification of Breast Cancer Mammography Image

Mammogram exam images are useful in identifying diseases, such as breast cancer, which is one of the deadliest cancers, affecting adult women around the world. Computational image analysis and machine learning techniques can help experts identify abnormalities in these images. In this work we present a new system to help diagnose and analyze breast mammogram images. To do this, the system a method the Selection of the Most Discriminant Attributes of the images preprocessed by BEMD “SMDA-BEMD”, this entails picking the most pertinent traits from the collection of variables that characterize the state under study. A reduction of attribute based on a transformation of the data also called an extraction of characteristics by extracting the Haralick attributes from the Co-occurrence Matrices Methods “GLCM” this reduction which consists of replacing the initial set of data by a new reduced set, constructed at from the initial set of features extracted by images decomposed using Bidimensional Empirical Multimodal Decomposition “BEMD”, for discrimination of breast mammogram images (healthy and pathology) using BEMD. This decomposition makes it possible to decompose an image into several Bidimensional Intrinsic Mode Functions “BIMFs” modes and a residue. The results obtained show that mammographic images can be represented in a relatively short space by selecting the most discriminating features based on a supervised method where they can be differentiated with high reliability between healthy mammographic images and pathologies, However, certain aspects and findings demonstrate how successful the suggested strategy is to detect the tumor. A BEMD technique is used as preprocessing on mammographic images. This suggested methodology makes it possible to obtain consistent results and establishes the discrimination threshold for mammography images (healthy and pathological), the classification rate is improved (98.6%) compared to existing cutting-edge techniques in the field. This approach is tested and validated on mammographic medical images from the Kenitra-Morocco reproductive health reference center (CRSRKM) which contains breast mammographic images of normal and pathological cases.

Quantitative study of solid-state mechanical mixing by high-pressure torsion based on image processing

Mechanical alloying through the process of severe plastic deformation can significantly enhance the mechanical properties of dissimilar metals. However, uneven mixing of these materials can adversely affect their mechanical performance and may even lead to material failure. To quantitatively assess the degree of mixing between dissimilar materials, a mechanical mixing efficiency index (η) has been introduced. This index is calculated accurately by combining an improved Non-dominated Sorting Genetic Algorithm II (NSGA-II) with computer image processing techniques. The study focuses on the Al-Zn mechanical mixture, establishing a parametric correlation between the mechanical mixing efficiency index and the microstructural evolution of mechanical mixing. The findings indicate that when the η value falls below 0.27865, the material transitions from the mechanical mixing stage to the chemical fusion stage. This research facilitates the controllable preparation of mechanical alloying, providing theoretical support and practical evidence for the process.

Lens-free reflective topography for high-resolution wafer inspection

The demand for high-resolution and large-area imaging systems for non-destructive wafer inspection has grown owing to the increasing complexity and extremely fine nature of semiconductor processes. Several studies have focused on developing high-resolution imaging systems; however, they were limited by the tradeoff between image resolution and field of view. Hence, computational imaging has arisen as an alternative method to conventional optical imaging, aimed at enhancing the aforementioned parameters. This study proposes a method for improving the resolution and field of view of an image in a lens-less reflection-type system. Our method was verified by computationally restoring the final image from diffraction images measured at various illumination positions using a visible light source. We introduced speckle illumination to expand the numerical aperture of the entire system, simultaneously improving image resolution and field of view. The image reconstruction process was accelerated by employing a convolutional neural network. Using the reconstructed phase images, we implemented high-resolution topography and demonstrated its applicability in wafer surface inspection. Furthermore, we demonstrated an ideal diffraction-limited spatial resolution of 1.7 μm over a field of view of 1.8 × 1.8 mm2 for the topographic imaging of targets with various surface roughness. The proposed approach is suitable for applications that simultaneously require high throughput and resolution, such as wafer-wide integrated metrology, owing to its compact design, cost-effectiveness, and mechanical robustness.

Single-pixel imaging based on self-supervised conditional mask classifier-free guidance

Reconstructing high-quality images at a low measurement rate is a pivotal objective of Single-Pixel Imaging (SPI). Currently, deep learning methods achieve this by optimizing the loss between the target image and the original image, thereby constraining the potential of low measurement values. We employ conditional probability to ameliorate this, introducing the classifier-free guidance model (CFG) for enhanced reconstruction. We propose a self-supervised conditional masked classifier-free guidance (SCM-CFG) for single-pixel reconstruction. At a 10% measurement rate, SCM-CFG efficiently completed the training task, achieving an average peak signal-to-noise ratio (PSNR) of 26.17 dB on the MNIST dataset. This surpasses other methods of photon imaging and computational ghost imaging. It demonstrates remarkable generalization performance. Moreover, thanks to the outstanding design of the conditional mask in this paper, it can significantly enhance the accuracy of reconstructed images through overlay. SCM-CFG achieved a notable improvement of an average of 7.3 dB in overlay processing, in contrast to only a 1 dB improvement in computational ghost imaging. Subsequent physical experiments validated the effectiveness of SCM-CFG.

High-speed 4-dimensional scanning transmission electron microscopy using compressive sensing techniques.

Here we show that compressive sensing allows 4-dimensional (4-D) STEM data to be obtained and accurately reconstructed with both high-speed and reduced electron fluence. The methodology needed to achieve these results compared to conventional 4-D approaches requires only that a random subset of probe locations is acquired from the typical regular scanning grid, which immediately generates both higher speed and the lower fluence experimentally. We also consider downsampling of the detector, showing that oversampling is inherent within convergent beam electron diffraction (CBED) patterns and that detector downsampling does not reduce precision but allows faster experimental data acquisition. Analysis of an experimental atomic resolution yttrium silicide dataset shows that it is possible to recover over 25 dB peak signal-to-noise ratio in the recovered phase using 0.3% of the totaldata. Lay abstract: Four-dimensional scanning transmission electron microscopy (4-D STEM) is a powerful technique for characterizing complex nanoscale structures. In this method, a convergent beam electron diffraction pattern (CBED) is acquired at each probe location during the scan of the sample. This means that a 2-dimensional signal is acquired at each 2-D probe location, equating to a 4-D dataset. Despite the recent development of fast direct electron detectors, some capable of 100kHz frame rates, the limiting factor for 4-D STEM is acquisition times in the majority of cases, where cameras will typically operate on the order of 2kHz. This means that a raster scan containing 256^2 probe locations can take on the order of 30s, approximately 100-1000 times longer than a conventional STEM imaging technique using monolithic radial detectors. As a result, 4-D STEM acquisitions can be subject to adverse effects such as drift, beam damage, and sample contamination. Recent advances in computational imaging techniques for STEM have allowed for faster acquisition speeds by way of acquiring only a random subset of probe locations from the field of view. By doing this, the acquisition time is significantly reduced, in some cases by a factor of 10-100 times. The acquired data is then processed to fill-in or inpaint the missing data, taking advantage of the inherently low-complex signals which can be linearly combined to recover the information. In this work, similar methods are demonstrated for the acquisition of 4-D STEM data, where only a random subset of CBED patterns are acquired over the raster scan. We simulate the compressive sensing acquisition method for 4-D STEM and present our findings for a variety of analysis techniques such as ptychography and differential phase contrast. Our results show that acquisition times can be significantly reduced on the order of 100-300 times, therefore improving existing frame rates, as well as further reducing the electron fluence beyond just using a faster camera.

FPM-WSI: Fourier ptychographic whole slide imaging via feature-domain backdiffraction

Fourier ptychographic microscopy (FPM) theoretically provides a solution to the trade-off between spatial resolution and field of view (FOV), and has promising prospects in digital pathology. However, block reconstruction and then stitching has become an unavoidable procedure for reconstruction of large FOV due to vignetting artifacts. This introduces digital stitching artifacts, as the existing image-domain optimization algorithms are highly sensitive to systematic errors. Such obstacles significantly impede the advancement and practical implementation of FPM, explaining why, despite a decade of development, FPM has not gained widespread recognition in the field of biomedicine. We report a feature-domain FPM (FD-FPM) based on the structure-aware forward model to realize stitching-free, full-FOV reconstruction. The loss function is uniquely formulated in the feature domain of images, which bypasses the troublesome vignetting effect and algorithmic vulnerability via feature-domain backdiffraction. Through massive simulations and experiments, we show that FD-FPM effectively eliminates vignetting artifacts for full-FOV reconstruction, and still achieves impressive reconstructions despite the presence of various systematic errors. We also found it has great potential in recovering the data with a lower spectrum overlapping rate, and in realizing digital refocusing without a prior defocus distance. With FD-FPM, we achieved full-color and high-throughput imaging (4.7 mm diameter FOV, 336 nm resolution in the blue channel) free of blocking-and-stitching procedures on a self-developed Fourier ptychographic microscopy whole slide imaging platform. The reported FD-FPM shows the value of FPM for various experimental circumstances, and offers physical insights useful for the developments of models for other computational imaging techniques. The reported platform demonstrates high-quality, high-speed imaging and low cost, and could find applications in many fields of biomedical research, as well as in clinical applications.

Multilayer lensless camera for improving the condition number

Computational lensless imaging technology implements the thinning of the optical system by replacing optical imaging with image reconstruction processing. The conventional optical design uses a single coded mask and an image sensor; however, researchers have recently proposed optical designs incorporating multiple stacked coded apertures for multidimensional and wide-field imaging. Here, we investigate the effects of multilayering the coded aperture on the performance of two-dimensional spatial imaging. Through simulations and optical experiments, we demonstrate that multilayering the coded aperture enhances the condition number of the optical system’s transmission matrix and consequently improves the accuracy of image reconstruction in lensless imaging.

Anti-noise computational ghost imaging based on wavelet threshold denoising

Noise can inevitably affect imaging quality of optical imaging system. In this paper, a method of computational ghost imaging (CGI) with better anti-noise performance is proposed by using wavelet threshold denoising (WTD). The simulation and experimental results show that this method exhibits better robustness against both the light source noise and the path noise. To explain this phenomenon, the decline rate of the correlation coefficients between speckle patterns (CCSP) under different noise levels are analyzed. As a post-processing method, our method is simple to implement and is applicable to both random speckle CGI and orthogonal modulated CGI, potentially broadening application prospects in practical applications of ghost imaging.

Improving the reliability of deep learning computational ghost imaging with prediction uncertainty based on neighborhood feature maps

Defect inspection is required in various fields, and many researchers have attempted deep-learning algorithms for inspections. Deep-learning algorithms have advantages in terms of accuracy and measurement time; however, the reliability of deep-learning outputs is problematic in precision measurements. This study demonstrates that iterative estimation using neighboring feature maps can evaluate the uncertainty of the outputs and shows that unconfident error predictions have higher uncertainties. In ghost imaging using deep learning, the experimental results show that removing outputs with higher uncertainties improves the accuracy by approximately 15.7%.

Computational imaging with randomness

AbstractImaging is a longstanding research topic in optics and photonics and is an important tool for a wide range of scientific and engineering fields. Computational imaging is a powerful framework for designing innovative imaging systems by incorporating signal processing into optics. Conventional approaches involve individually designed optical and signal processing systems, which unnecessarily increased costs. Computational imaging, on the other hand, enhances the imaging performance of optical systems, visualizes invisible targets, and minimizes optical hardware. Digital holography and computer-generated holography are the roots of this field. Recent advances in information science, such as deep learning, and increasing computational power have rapidly driven computational imaging and have resulted in the reinvention these imaging technologies. In this paper, I survey recent research topics in computational imaging, where optical randomness is key. Imaging through scattering media, non-interferometric quantitative phase imaging, and real-time computer-generated holography are representative examples. These recent optical sensing and control technologies will serve as the foundations of next-generation imaging systems in various fields, such as biomedicine, security, and astronomy.

A Novel Fractional Edge Detector Based on Generalized Fractional Operator

This work pioneers a novel approach in image edge detection through the utilization of the generalized fractional operator. By harnessing the global attributes inherent in fractional derivatives, it aims to enhance the extraction of intricate edge details.This is accomplished by creating the mask by using fractional derivative and adapt the mask by another parameter, yielding compelling and informative edge representations, as validated by experimental results. This advancement not only augments computer vision and image analysis techniques but also holds promise for refining image processing methodologies. Future endeavors may explore its adaptability across diverse imaging domains like medical and satellite imagery, while integration into deep learning frameworks could elevate its potential for advanced feature extraction and deeper image understanding. Additionally, optimizing its computational efficiency would broaden its scope for real-time deployment in fields such as robotics and autonomous systems.

Ultrasonic phased array imaging of multi-layered structures using full-matrix migration and normalized cross-correlation matching technique

Multi-layered structures play a critical role in aerospace, automotive and electrical industries, so the non-destructive evaluation and maintenance of such structures must be accurate and rapid. The application of conventional ultrasonic phased array imaging methods is limited due to computational complexity, image quality and image size. In this paper, we proposed a cost-effective, high-quality imaging method that uses full-matrix migration total focus method (FM-TFM) and produces large-area scanned image with the aid of normalized cross-correlation (NCC) template matching technique. The FM-TFM method eliminated the need for iterative refraction point calculations by using full matrix migration extrapolation, and the optimization of NCC was also investigated in the study. The verification was divided into two parts, fixed transducer experiment and manual scanned experiment, to demonstrate the advantages of FM-TFM with improved SNR (22 % or greater) and one order of magnitude reduction in computational time when compared to ray tracing total focus method (RT-TFM) and NCC matching accuracy of multilayer specimen, respectively. The proposed method can help eliminate the need for costly scanning equipment and shows great potential for extension to other TFM-based approaches.

IOT Based Monitoring Model to Identify and Classify the Grading of Fruits and Vegetables

Thirty to forty percent of the fruits and vegetables harvested are discarded at this time due to the large number of employees with no training. A growing problem throughout reaping is fruit and vegetable waste because human judgment is subjective when it comes to crop recognition, categorization, and assessment. The fruit and vegetable business is in serious require of the launch and deployment of a robotic system in order to classify and grade fruits and vegetables based on their level of ripeness. Machine learning techniques can produce a sophisticated machine learning framework that can differentiate among fruits and vegetables based upon their category, variation, development, and preservation using relevant and suitable computational imaging ideas. We analyzed, contrasted, and proposed an Internet of Things-based monitoring approach in this work for identifying, classifying, and grading fruits and vegetables.