Image Sensor System Research Articles

7. An algorithm for using deep learning convolutional neural networks with 3-dimensional depth sensor imaging in scoliosis detection

<h3>BACKGROUND CONTEXT</h3> Timely intervention in growing individuals, such as brace treatment, relies on early detection of adolescent idiopathic scoliosis (AIS). To this end, several screening methods have been implemented. However, these methods have limitations in predicting the Cobb angle. <h3>PURPOSE</h3> This study aimed to evaluate the performance of a 3-dimensional depth sensor imaging system with a deep learning algorithm (DLA), in predicting the Cobb angle in AIS. <h3>STUDY DESIGN/SETTING</h3> Retrospective analysis of prospectively collected, consecutive, nonrandomized series of patients at 5 scoliosis centers in Japan. <h3>PATIENT SAMPLE</h3> One hundred sixty human subjects suspected to have AIS were included. <h3>OUTCOME MEASURES</h3> Patient demographics, radiographic measurements, and predicted Cobb angle derived from the DLA were the outcome measures for this study. <h3>METHODS</h3> One hundred sixty data files were shuffled into 5 datasets with 32 data files at random (dataset 1, 2, 3, 4 and 5) and 5-fold cross validation was performed. The relationships between the actual and predicted Cobb angles were calculated using Pearson's correlation coefficient analyses. The prediction performances of the network models were evaluated using mean absolute error (MAE) and root mean square error (RMSE) between the actual and predicted Cobb angles. The shuffling into five datasets and five-fold cross validation was conducted ten times. There were no study-specific biases related to conflicts of interest. <h3>RESULTS</h3> The mean age was 14.7 ± 2.4 years, and the average Cobb angle was 30° (range: 0˚ to 64˚). The range of correlation coefficients was 0.87 to 0.89 in the five-fold cross validation with 10 repeats. The correlation between the actual Cobb angle and the mean predicted Cobb angle with 10 repeats was 0.91. The range of MAEs was 4.4˚ to 4.7˚in the 5-fold cross validation with 10 repeats. The range of RMSEs was 5.8˚ to 6.3˚. The MAE and RMSE between the actual Cobb angle and the mean predicted Cobb angle with 10 repeats was 4.0˚ and 5.4˚ respectively. At a Cobb angle of 10˚, which confirms a diagnosis of scoliosis, the mean predicted Cobb angle with 10 repeats showed a sensitivity of 0.99, a specificity of 0.42, PPV of 0.95, NPV of 0.71, accuracy of 0.94, PLR of 1.69, and an NLR of 0.03. <h3>CONCLUSIONS</h3> The 3D depth sensor imaging system with its newly innovated CNN for regression is objective and has significant ability to predict the Cobb angle in children and adolescents. This system is expected to be used for screening scoliosis in clinics or physical examination at schools. In addition, this is also a possible surrogate for radiographs to monitor curve progression, preventing unnecessary X-rays for mild case of scoliosis. <h3>FDA DEVICE/DRUG STATUS</h3> This abstract does not discuss or include any applicable devices or drugs.

Mapping Kuwait Oil Fields With Leica Lidar and Imaging Sensor System

Mapping Kuwait Oil Fields With Leica Lidar and Imaging Sensor System

An algorithm for using deep learning convolutional neural networks with three dimensional depth sensor imaging in scoliosis detection

BACKGROUND CONTEXTTimely intervention in growing individuals, such as brace treatment, relies on early detection of adolescent idiopathic scoliosis (AIS). To this end, several screening methods have been implemented. However, these methods have limitations in predicting the Cobb angle. PURPOSEThis study aimed to evaluate the performance of a three-dimensional depth sensor imaging system with a deep learning algorithm, in predicting the Cobb angle in AIS. STUDY DESIGNRetrospective analysis of prospectively collected, consecutive, nonrandomized series of patients at five scoliosis centers in Japan. PATIENT SAMPLEOne hundred and-sixty human subjects suspected to have AIS were included. OUTCOME MEASURESPatient demographics, radiographic measurements, and predicted Cobb angle derived from the deep learning algorithm were the outcome measures for this study. METHODSOne hundred and sixty data files were shuffled into five datasets with 32 data files at random (dataset 1, 2, 3, 4, and 5) and five-fold cross validation was performed. The relationships between the actual and predicted Cobb angles were calculated using Pearson's correlation coefficient analyses. The prediction performances of the network models were evaluated using mean absolute error and root mean square error between the actual and predicted Cobb angles. The shuffling into five datasets and five-fold cross validation was conducted ten times. There were no study-specific biases related to conflicts of interest. RESULTSThe correlation between the actual and the mean predicted Cobb angles was 0.91. The mean absolute error and root mean square error were 4.0° and 5.4°, respectively. The accuracy of the mean predicted Cobb angle was 94% for identifying a Cobb angle of ≥10° and 89% for that of ≥20°. CONCLUSIONSThe three-dimensional depth sensor imaging system with its newly innovated convolutional neural network for regression is objective and has significant ability to predict the Cobb angle in children and adolescents. This system is expected to be used for screening scoliosis in clinics or physical examination at schools.

A Reliable, Label Free Quality Control Method for the Production of DNA Microarrays with Clinical Applications.

The manufacture of a very high-quality microarray support is essential for the adoption of this assay format in clinical routine. In fact, poorly surface-bound probes can affect the diagnostic sensitivity or, in worst cases, lead to false negative results. Here we report on a reliable and easy quality control method for the evaluation of spotted probe properties in a microarray test, based on the Interferometric Reflectance Imaging Sensor (IRIS) system, a high-resolution label free technique able to evaluate the variation of the mass bound to a surface. In particular, we demonstrated that the IRIS analysis of microarray chips immediately after probe immobilization can detect the absence of probes, which recognizably causes a lack of signal when performing a test, with clinical relevance, using fluorescence detection. Moreover, the use of the IRIS technique allowed also to determine the optimal concentration of the probe, that has to be immobilized on the surface, to maximize the target recognition, thus the signal, but to avoid crowding effects. Finally, through this preliminary quality inspection it is possible to highlight differences in the immobilization chemistries. In particular, we have compared NHS ester versus click chemistry reactions using two different surface coatings, demonstrating that, in the diagnostic case used as an example (colorectal cancer) a higher probe density does not reflect a higher binding signal, probably because of a crowding effect.

An image sensor system with color constancy using the Retinex theory and the Michaelis-Menten equation

An image sensor system with color constancy using the Retinex theory and the Michaelis-Menten equation

Research of Intelligent Temperature Measuring Robot System

With the outbreak of novel coronavirus pneumonia worldwide, a kind of epidemic prevention robot is urgently needed in society. The intelligent temperature measurement robot system mainly includes the robot’s high stability multi-attitude omnidirectional movement module, robot path planning and autonomous navigation module, robot environment information acquisition and recognition module, robot remote control module, and robot system integration. According to the design principle of the wheeled robot, high stability and multi attitude omni-directional mobile platform of temperature measuring robot are constructed; path planning and autonomous navigation of robot are realized based on sensor data fusion; temperature detection and remote care function are realized by thermal imaging sensor system by capturing the thermal radiation emitted by the human body, and based on open modular control system architecture. Based on the existing research foundation, the intelligent temperature measurement robot system is established by integrating the robot multi attitude omni-directional mobile system, robot path planning, and autonomous navigation system, robot environment information acquisition and recognition system, and robot remote control system.

Destriping and evaluating FY-3D MERSI-2 data with the moment matching method based on synchronous reference image

Medium Resolution Spectral Imager II (MERSI-2) is a payload of the China meteorological satellite FY-3D. The sensor bands 24 (10.3 to 11.3 μm) and 25 (11.5 to 12.5 μm) images, which are most suitable for land surface temperature (LST) retrieval, have higher spatial resolution than that of similar sensors. However, bands 24/25 images with spatial resolution of 250 m (hereafter bands 24/25 250-m images) have considerable horizontal stripe noise caused by the limitation of the sensor imaging system’s multi-detector parallel scanning and the multiple interferences from space. Thus, the application value of LST retrieval using bands 24/25 250-m images is severely affected. At present, existing methods used to remove stripe noise often cannot retain the spectral information of the original image. Our research considers that MERSI-2 bands 24/25 images with spatial resolution of 1000 m (hereafter bands 24/25 1000-m images) and no stripe noise can provide reasonable reference statistics for bands 24/25 250-m images. Thus, the moment matching method considering synchronous reference image is applied to remove the stripe noise of bands 24/25 250-m images in the land area. Results shows that the moment matching method considering the synchronous reference image can effectively remove the stripe noise in the land area. In addition, quantitative evaluation criteria, such as information entropy H, mean-absolute-error (MAE), and peak signal-to-noise ratio, are better than those of the traditional moment matching method. After destriping, the LST values using MERSI-2 data are retrieved with better effect and decrease by 0.339 and 0.504 K using MAE and root-mean-square error, respectively, in contrast to the Moderate Resolution Imaging Spectroradiometer LST product.

Position-rate control for the time delay control system of tip-tilt mirror

In the image-based tip-tilt mirror control system, the closed-loop performance and bandwidth of the system and are limited due to the influence of sensor sampling frequency and system delay. Under the condition of limited bandwidth, this paper proposes to use linear encoder to measure the position, and get the rate signal by differ-ence. The position-rate feedback control based on the image sensor system is realized to improve the error sup-pression ability of the tip-tilt mirror control system. Because of the addition of rate feedback, the control system has differential characteristics. When the rate feedback closed-loop is completed, the image position loop has integral characteristic. At this time, a PI controller is used to stabilize the system, which makes the system rise from zero type to two type system, and improves the error suppression ability of the system. Simulation and experiment show that this method can effectively improve the closed-loop performance of the tracking control system in low frequency domain.

Reduced Tilting Effect of Smartphone CMOS Image Sensor in Visible Light Indoor Positioning

Visible light positioning (VLP) using complementary metal–oxide–semiconductor (CMOS) image sensors is a cost-effective solution to the increasing demand for an indoor positioning system. However, in most of the existing VLP systems with an image sensor, researchers assume that the receiving image sensor is positioned parallel to the indoor floor without any tilting and, thus, have only focused on the high-precision positioning algorithm and ignored the proper light-emitting diode (LED)-ID recognition. To address these limitations, we present, herein, a smartphone CMOS image sensor and visible light-based indoor localization system for a receiver device in a tilted position, and we have applied a machine learning approach for optimized LED-ID detection. For detection of the LED-ID, we generated different features for different LED-IDs and utilize a machine learning method to identify each ID as opposed to using the conventional coding and decoding method. An image processing method was used for the image features extraction and selection. We utilized the rolling shutter mechanism of the smartphone CMOS image sensor in our indoor positioning system. Additionally, to improve the LED-ID detection and positioning accuracy with the tilting of the receiver, we utilized the embedded fusion sensors of the smartphone (e.g., accelerometer, gyroscope, and magnetometer, which can be used to extract the yaw, pitch, and roll angles). The experimental results for the proposed positioning system show that it can provide 2.49, 4.63, 8.46, and 12.20 cm accuracy with angles of 0, 5, 10, and 15°, respectively, within a 2 m × 2 m × 2 m positioning area.

Tolerancing and characterization of curved image sensor systems.

Curved image sensors, not having to correct the field curvature, are considered a relevant solution for improving the vast majority of optical systems. They offer the possibility of designing compact aberration-free optical systems. In this work, we explain the advantage of the curved sensor system using the aberration theory. A complete procedure was developed to produce functional curved sensors and functional prototypes were carried out. This paper focuses on the tolerancing process of curved sensors and its inclusion in optical design. A compact objective prototype designed and produced demonstrates the advantage of curvature and the impact of tolerancing.

A Seven-Layer Convolutional Neural Network for Chest CT-Based COVID-19 Diagnosis Using Stochastic Pooling.

(Aim) COVID-19 pandemic causes numerous death tolls till now. Chest CT is an effective imaging sensor system to make accurate diagnosis. (Method) This article proposed a novel seven layer convolutional neural network based smart diagnosis model for COVID-19 diagnosis (7L-CNN-CD). We proposed a 14-way data augmentation to enhance the training set, and introduced stochastic pooling to replace traditional pooling methods. (Results) The 10 runs of 10-fold cross validation experiment show that our 7L-CNN-CD approach achieves a sensitivity of 94.44±0.73, a specificity of 93.63±1.60, and an accuracy of 94.03±0.80. (Conclusion) Our proposed 7L-CNN-CD is effective in diagnosing COVID-19 in chest CT images. It gives better performance than several state-of-the-art algorithms. The data augmentation and stochastic pooling methods are proven to be effective.

Evaporated Sex Te1- x Thin Films with Tunable Bandgaps for Short-Wave Infrared Photodetectors.

Semiconducting absorbers in high-performance short-wave infrared (SWIR) photodetectors and imaging sensor arrays are dominated by single-crystalline germanium and III-V semiconductors. However, these materials require complex growth and device fabrication procedures. Here, thermally evaporated Sex Te1- x alloy thin films with tunable bandgaps for the fabrication of high-performance SWIR photodetectors are reported. From absorption measurements, it is shown that the bandgaps of Sex Te1- x films can be tuned continuously from 0.31 eV (Te) to 1.87 eV (Se). Owing to their tunable bandgaps, the peak responsivity position and photoresponse edge of Sex Te1- x film-based photoconductors can be tuned in the SWIR regime. By using an optical cavity substrate consisting of Au/Al2 O3 to enhance its absorption near the bandgap edge, the Se0.32 Te0.68 film (an optical bandgap of ≈0.8 eV)-based photoconductor exhibits a cut-off wavelength at ≈1.7 μm and gives a responsivity of 1.5 AW-1 and implied detectivity of 6.5 × 1010 cm Hz1/2 W-1 at 1.55 μm at room temperature. Importantly, the nature of the thermal evaporation process enables the fabrication of Se0.32 Te0.68 -based 42 × 42 focal plane arrays with good pixel uniformity, demonstrating the potential of this unique material system used for infrared imaging sensor systems.

Instrument and method for measuring the dose rates in 60Co [formula omitted] rays environment with a 2K[formula omitted]2K CMOS image sensor imaging system

The Instrument and method for measuring the dose rates in 60Co γ rays environment with a 2K×2K CMOS image sensor (CIS) imaging system are presented. The experiments were carried out at 60Co γ rays facility. The output signal increase, the frequency of effective spike were measured by the CIS imaging system under the γ rays environment at various dose rates. The dose rates were also measured by the PTW-UNIDOS. Besides, the relationship between mean output signal increase, frequency of effective spike, and dose rates are analyzed and used to measure the dose rates. The dose rates measured by our measurement system are in good agreement with the PTW-UNIDOS results. Usually, using the frequency of effective spikes to measure the dose rates would be more efficient but it is also more complex. The measuring ranges, the life span of the CIS imaging system are also discussed. This work can provide a great way to measure dose rates in 60Co γ rays environment.

Semantic segmentation of sparse 3D point cloud based on geometrical features for trellis-structured apple orchard

Orchard operations such as mechanical pruning and spraying are heavily affected by tree architectures. Quantified inputs (e.g., cutting locations for mechanical pruning, and canopy distribution and density for variable-rate precision spraying) are necessary information for achieving precise control of these orchard operations. Even in planar orchard systems, trees grow differently. Therefore, it is essential to measure the canopy at the individual tree level. A three-dimensional (3D) light detection and ranging (LiDAR) sensor imaging system was developed to estimate the main canopy specifications. The LiDAR sensor was installed on a utility vehicle and driven alongside tree rows in an apple orchard. A total of 1,138 frames of point cloud data were acquired from 69 apple trees in a tall spindle architecture. An algorithm was developed in the MATLAB environment to segment trellis wires, support poles, and tree trunks in these point cloud images. The results indicated that the proposed algorithm achieved overall accuracy values of 88.6%, 82.1%, and 94.7%, respectively, in identifying the corresponding three objects. Furthermore, canopy density and depth maps were created with the distribution of points in the point cloud images. The outcomes from this study provide baseline information for precision orchard operations such as mechanical pruning and precision spraying.

Rigorous Co-Registration of KOMPSAT-3 Multispectral and Panchromatic Images for Pan-Sharpening Image Fusion.

KOMPSAT-3, a Korean earth observing satellite, provides the panchromatic (PAN) band and four multispectral (MS) bands. They can be fused to obtain a pan-sharpened image of higher resolution in both the spectral and spatial domain, which is more informative and interpretative for visual inspection. In KOMPSAT-3 Advanced Earth Imaging Sensor System (AEISS) uni-focal camera system, the precise sensor alignment is a prerequisite for the fusion of MS and PAN images because MS and PAN Charge-Coupled Device (CCD) sensors are installed with certain offsets. In addition, exterior effects associated with the ephemeris and terrain elevation lead to the geometric discrepancy between MS and PAN images. Therefore, we propose a rigorous co-registration of KOMPSAT-3 MS and PAN images based on physical sensor modeling. We evaluated the impacts of CCD line offsets, ephemeris, and terrain elevation on the difference in image coordinates. The analysis enables precise co-registration modeling between MS and PAN images. An experiment with KOMPSAT-3 images produced negligible geometric discrepancy between MS and PAN images.

Multi-source remote sensing image big data classification system design in cloud computing environment

Due to the problems of poor classification and time-consuming in traditional multi-source remote sensing image big data classification system, it cannot meet the standard requirements for image big data classification in related fields. To solve the above problems, the multi-source remote sensing image data classification system under cloud computing environment is optimised. Following the line string transmission protocol architecture, relevant information is processed, transformed and fused. Data are transported to the host through protocol transmission. Based on above principle, the system hardware and software are designed. Detailed, designing hardware system refers to designing image sensor interface and system processing interface. The design of the system software part can be divided into two parts, including the two-wire serial protocol formulation and the image big data classification algorithm that provides users with initialisation operations. At the same time, the image is sharpened and the pixels are improved. Experimental verification results show that the system has good processing effect and short time consumption.

Recent Developments of High-Resolution Chemical Imaging Systems Based on Light-Addressable Potentiometric Sensors (LAPSs).

A light-addressable potentiometric sensor (LAPS) is a semiconductor electrochemical sensor based on the field-effect which detects the variation of the Nernst potential on the sensor surface, and the measurement area is defined by illumination. Thanks to its light-addressability feature, an LAPS-based chemical imaging sensor system can be developed, which can visualize the two-dimensional distribution of chemical species on the sensor surface. This sensor system has been used for the analysis of reactions and diffusions in various biochemical samples. In this review, the LAPS system set-up, including the sensor construction, sensing and substrate materials, modulated light and various measurement modes of the sensor systems are described. The recently developed technologies and the affecting factors, especially regarding the spatial resolution and temporal resolution are discussed and summarized, and the advantages and limitations of these technologies are illustrated. Finally, the further applications of LAPS-based chemical imaging sensors are discussed, where the combination with microfluidic devices is promising.

Design of weak signal detection circuit of CMOS detector with large dynamic range by twice sampling method

In the field of detecting weak signals with large dynamic range, CMOS image sensor, as an optical sensor component, has the advantages of large dynamic range and wide spectral response range, and is often used to detect weak signals. The commercial CMOS image sensor system usually adopts the ADC with 16-bit accuracy, which can not satisfy the detection of weak signal in some cases. In this study, CMOS image sensor S10122 chip from Hammatsu company of Japan was used as the core device. ADI company’s 16-bit ADC chip LTC2182 and 16-bit DAC chip LTC1668 were used. The circuit was designed according to twice sampling principles to improve ADC accuracy. The results show that the accuracy of ADC can be improved from 16 to 18 bits under certain conditions.

Scalable visible light communications with a micro-LED array projector and high-speed smartphone camera.

Solid-state camera systems are now widely available in portable consumer electronics, providing potential receivers for visible light communications in every device. Typically, data rates with camera receivers are limited by the 60 fps frame rate of both the image sensor and projector systems. Recent developments in high-frame rate microdisplays and slow-motion cameras for smartphones now permit high-speed, spatially structured signals to be transmitted and captured. Here, we present a method for transmitting data to a smartphone using a CMOS-controlled micro-LED projector system. Spatial patterns are projected onto a wall at a refresh rate of 480 Hz, which can be captured by the smartphone's 960 fps camera. Data transfer is performed over meter scale distances, and the use of an alignment frame gives the system a level of tolerance to motion and misalignment. The current system allows data transmission at a peak rate of 122.88 kb/s using a 16 × 16 micro-LED array, which can be readily scaled to Mb/s rates with a higher resolution transmitter.

Preventing image information loss of imaging sensors in case of laser dazzle

We present an optical concept for imaging sensor systems, designed to considerably reduce the sensor’s image information loss in cases of laser dazzle, based on the principle of complementary bands. For this purpose, the sensor system’s spectral range is split in several (at least two) spectral channels, where each channel possesses its own imaging sensor. This long-known principle is applied, for example, in high-quality three-sensor color cameras. However, in such camera systems, the spectral separation between the different spectral bands is too poor to prevent complete sensor saturation when illuminated with intense laser radiation. We increased the channel separation by orders of magnitude by implementing advanced optical elements. Thus, monochromatic radiation of a dazzle laser mainly influences the dedicated transmitting spectral channel. The other (out-of-band) spectral channels are not or—depending on the laser power—only hardly affected. We present our system design as well as a performance evaluation of the sensor concerning laser dazzle.