Real-time Multispectral Imaging System Research Articles

Spectral imaging enables contrast agent–free real-time ischemia monitoring in laparoscopic surgery

Laparoscopic surgery has evolved as a key technique for cancer diagnosis and therapy. While characterization of the tissue perfusion is crucial in various procedures, such as partial nephrectomy, doing so by means of visual inspection remains highly challenging. We developed a laparoscopic real-time multispectral imaging system featuring a compact and lightweight multispectral camera and the possibility to complement the conventional surgical view of the patient with functional information at a video rate of 25 Hz. To enable contrast agent-free ischemia monitoring during laparoscopic partial nephrectomy, we phrase the problem of ischemia detection as an out-of-distribution detection problem that does not rely on data from any other patient and uses an ensemble of invertible neural networks at its core. An in-human trial demonstrates the feasibility of our approach and highlights the potential of spectral imaging combined with advanced deep learning-based analysis tools for fast, efficient, reliable, and safe functional laparoscopic imaging.

Self-referencing biosensors using Fano resonance in periodic aluminium nanostructures.

Surface plasmon resonance (SPR) is an important technique for real-time and label-free detection of specific binding biomolecules. However, conventional SPR signals come from both the surface binding biomolecules and the variation in the bulk refractive index. This work demonstrates that Fano resonance in an aluminum capped nanoslit array has the ability to remove the signal of bulk refractive index changes from the SPR signal. As compared to gold nanostructures, the aluminum nanostructure provides an asymmetrical Fano resonance with clear peak and dip wavelengths. The peak wavelength is close to the grating resonance condition. The evanescent depth at the peak wavelength is up to several microns. The dip wavelength comes from the SPR effect. The evanescent depth at the dip wavelength is about 300 nm. By simultaneously measuring the shifts of peaks and the dip wavelengths, the variation in the bulk refractive index can be removed and only the biolayer thickness is measured. The finite-difference time-domain calculation shows that the 470 nm-period nanoslit array with 90 and 70 nm slit depths has the optimal thickness sensitivity. In this experiment, a simple multispectral imaging system is developed for multiple bio-interaction measurements. The measured results verify that the bulk refractive index changes can be removed and the surface biomolecular interactions can be directly obtained without the need of a reference channel.

Conceptual design and proof-of-principle testing of the real-time multispectral imaging system MANTIS

The Multispectral Advanced Narrowband Tokamak Imaging System (MANTIS) is proposed to resolve the steep temperature and density gradients in the scrape-off layer of tokamaks in real-time. The initial design is to deliver two-dimensional distributions of key plasma parameters of the TCV tokamak to a real-time control system in order to enable novel control strategies, while providing new insights into power exhaust physics in the full offline analysis. This paper presents the conceptual system design, the mechanical and optical design of a prototype that was built to assess the optical performance, and the results of the first proof-of-principle tests of the prototype. These demonstrate a central resolving power of 50-46 line pairs per millimeter (CTF50) in the first four channels. For the additional channels, the sharpness is a factor two worse for the odd channels (likely affected by sub-optimal alignment), while the even channels continue the trend observed for the first four channels of 3% degradation per channel. This is explained by the self-cancellation of off-axis aberrations, which is an attractive property of the chosen optical design. The results show that at least a 10-channel real-time multispectral imaging system is feasible.

基于多通道窄带滤光片和彩色图像传感器的实时多光谱成像系统

基于多通道窄带滤光片和彩色图像传感器的实时多光谱成像系统

In-plant Test of In-line Multispectral Imaging System For Fecal Detection during Poultry Processing

Food Safety Inspection Service established new requirements for the zero tolerance for fecal contaminated broiler carcasses entering the chiller. The hyperspectral imaging research accomplished at ARS has demonstrated the real-time multispectral imaging system was able to effectively detect fecal contaminants during poultry processing. The imaging system with two-band (517 and 565 nm) was able to inspect every bird in real-time, in-line mode at commercial processing speed. With several real-time image processing methods including binning, cuticle removal filter, median filter, and morphological analysis, the system performance was improved by maximizing detection accuracy and minimizing false positive errors. Most false positives were found at wing bone, thigh, vent, and boundary. Since the number of true positives confirmed organoleptically by human inspector was only 14 out of 29,821 carcasses (0.05%), more samples need to be tested to confirm the performance of detection accuracy. Overall, the ARS multispectral imaging system was able to detect contaminants with high detection accuracy (approximately 91%) and 3.3% false positive errors at the processing speed of 150 birds/min from the two-day in-plant trials. However, the performance of ARS real-time in-line imaging system for fecal contaminant detection on broiler carcasses can be improved by adopting a better camera which has higher signal-to-noise ratio such as line-scan real-time hyperspectral imaging system currently being investigated for its applicability for real-time processing.

Adaptive Image Processing Methods for Improving Contaminant Detection Accuracy on Poultry Carcasses

A real-time multispectral imaging system has demonstrated a science-based tool for fecal and ingesta contaminant detection during poultry processing. In order to implement this imaging system in the commercial poultry processing industry, the false positives must be removed. To do this, we tested and implemented additional image processing algorithms including binning, cuticle removal filter, median filtering, and morphological analysis in real-time mode to maximize detection accuracy and minimize false positives (FPs). The median filtering and binning processes were able to reduce FPs up to 98.7% and 95.2%, respectively, by eliminating most of the salt-and-pepper noise from the raw images. The detection accuracy varied with the parameter values of the image processing algorithms, including binning, thresholding, median filtering, and morphological analysis. Overall contaminant detection accuracy on moving birds varied from 84.3% to 97.8%, and FP errors were 1.9% and 41.8%, respectively. Although neither the overall detection accuracy nor FP errors were affected by the camera gain, the detection accuracy results changed slightly, from 87.4% to 95.1%. In this case, the FP errors were 1.8% and 15.9%, respectively. Thus, the USDA-ARS multispectral imaging system was able to detect contaminants with 91.6% accuracy and 3.3% FP errors by selecting optimum image processing methods at a processing speed of 140 birds per minute.

Real-time multispectral imaging system for online poultry fecal inspection using unified modeling language

A prototype real-time multispectral imaging system for fecal and ingesta contaminant detection on broiler carcasses has been developed. The prototype system includes a common aperture camera with three optical trim filters (517, 565 and 802-nm wavelength), which were selected by visible/NIR spectroscopy and validated by a hyperspectral imaging system with decision tree algorithm. The on-line testing results showed that the multispectral imaging technique can be used effectively for detecting feces (from duodenum, ceca, and colon) and ingesta on the surface of poultry carcasses with a processing speed of 140 birds per minute. This paper demonstrated both multispectral imaging hardware and real-time image processing software. For the software development, the Unified Modeling Language (UML) design approach was used for on-line application. The UML models included class, object, activity, sequence, and collaboration diagram. User interface model included 17 inputs and 6 outputs. A window-based real-time image processing software composed of 11 components, which represented class, architecture, and activity. Both hardware and software for a real-time fecal detection were tested at the pilot-scale poultry processing plant. The run-time of the software including online calibration was fast enough to inspect carcasses on-line with an industry requirement. Based on the preliminary test at the pilot-scale processing line, the system was able to acquire poultry images in real-time. According to the test results, the imaging system is reliable for the harsh environments and UML-based image processing software is flexible and easy to be updated when additional parameters are needed for in-plant trials.

Real time multispectral high temperature measurement: Application to control in the industry

Many devices are used to realize non-contact temperature measurements. Whenever the body to be controlled behaves as a black body, all the devices inferring the true temperature from the body radiation are accurate and reliable. On the other hand, when it exhibits a behavior different from the black body, emissivity compensations need to be done. In case of a known emissivity, the spectral system (single wavelength) is used whereas, for gray body (unknown but constant emissivity in a narrow bandwidth) a bicolor system is more likely to be utilized. For all other cases, assumptions about the emissivity variations as a function of the wavelength and the temperature have to be ascertained and multispectral system or “hybrid-systems” which are a mix between spectral and multispectral systems have been built and used in particular cases. In this paper, a real time multispectral imaging system based on two CCD cameras is presented. The system is herein carefully characterized and applications such as vision control system are presented. An extension of the system with three cameras is also exposed in the perspectives.