Real-time Image Processing Algorithms Research Articles

Image Link Through Adaptive Encoding Data Base and Optimized GPU Algorithm for Real-time Image Processing of Artificial Intelligence

This paper presents the latest Ethernet standardization of in-vehicle network and the future trends of automotive ethernet technology. The proposed system provides a design and optimization algorithm of in-vehicle networking technologies related Ethernet Audio Video Bridge (AVB) technology. We present a design of in-vehicle network system as well as the optimization of AVB for automotive. A proposal of Reduced Latency of Machin to Machine (RLMM) plays a significant role in reducing the latency between devices. The approach of RLMM on realistic test cases indicated that there was a latency reduction about 30.41% It is expected that the optimized settings for the actual automotive network environment can greatly shorten the time period in the development and design process. The results achieved from the experiments on the latency present in each function are trustworthy since average values are obtained via repeated tests for several months. It would considerably benefit the industry because analyzing the delay between each function in a short period of time is tremendously significant. In addition, through the proposed real-time camera and video streaming via optimized settings of AVB system, it is expected that AI (Artificial Intelligence) algorithms in autonomous driving will be of great help in understanding and analyzing images in real time.

Artificial Intelligence and Democratization of the Use of Lung Ultrasound in COVID-19: On the Feasibility of Automatic Calculation of Lung Ultrasound Score

During the COVID-19 pandemic, lung ultrasound has been revealed as a powerful technique for diagnosis and follow-up of pneumonia, the principal complication of SARS-CoV-2 infection. Nevertheless, being a relatively new and unknown technique, the lack of trained personnel has limited its application worldwide. Computer-aided diagnosis could possibly help to reduce the learning curve for less experienced physicians, and to extend such a new technique such as lung ultrasound more quickly. This work presents the preliminary results of the ULTRACOV (Ultrasound in Coronavirus disease) study, aimed to explore the feasibility of a real-time image processing algorithm for automatic calculation of the lung ultrasound score (LUS). A total of 28 patients positive on COVID-19 were recruited and scanned in 12 thorax zones following the lung score protocol, saving a 3 s video at each probe position. Those videos were evaluated by an experienced physician and by a custom developed automated detection algorithm, looking for A-Lines, B-Lines, consolidations, and pleural effusions. The agreement between the findings of the expert and the algorithm was 88.0% for B-Lines, 93.4% for consolidations and 99.7% for pleural effusion detection, and 72.8% for the individual video score. The standard deviation of the patient lung score difference between the expert and the algorithm was ±2.2 points over 36. The exam average time with the ULTRACOV prototype was 5.3 min, while with a conventional scanner was 12.6 min. Conclusion: A good agreement between the algorithm output and an experienced physician was observed, which is a first step on the feasibility of developing a real-time aided-diagnosis lung ultrasound equipment. Additionally, the examination time was reduced to less than half with regard to a conventional ultrasound exam. Acquiring a complete lung ultrasound exam within a few minutes is possible using fairly simple ultrasound machines that are enhanced with artificial intelligence, such as the one we propose. This step is critical to democratize the use of lung ultrasound in these difficult times.

Improving Sub-Pixel Estimation of Laser Stripe Reflection Center by Autoconvolution on FPGA

Modern laser scanners perform high-speed real-time image processing algorithms while operating in harsh industrial environments. Their performance goal is to extract the central position of the laser line reflection with Gaussian distribution. Traditional algorithms for sub-pixel estimation, such as the Center of Gravity (CG) or Parabolic Fit (PF), show poor performances under low SNR or if the pixels are saturated. Data pre-processing usually has a key role in suppressing the effects of various noise sources and dynamic environment, especially when the images are overexposed and the top of Gaussian pulse is flattened. Both in simulation and in experiment, this study explains a method that improves the accuracy of estimation of the laser stripe reflection center, by using an autoconvolution for extending the bit-width of pixel intensity. Autoconvolution of the image line is an efficient real-time pre-processing filtering method for improving the accuracy of CG calculation. The proposed algorithm is implemented on Field-Programmable Gate Arrays (FPGAs) and experimentally validated at real operational environment. It is shown that this method can reduce the error of CG laser reflection center estimation for more than one pixel in size when the image is highly affected by external noise sources and ambient light.

Visual Intelligence: Prediction of Unintentional Surgical-Tool-Induced Bleeding during Robotic and Laparoscopic Surgery

Unintentional vascular damage can result from a surgical instrument’s abrupt movements during minimally invasive surgery (laparoscopic or robotic). A novel real-time image processing algorithm based on local entropy is proposed that can detect abrupt movements of surgical instruments and predict bleeding occurrence. The uniform nature of the texture of surgical tools is utilized to segment the tools from the background. By comparing changes in entropy over time, the algorithm determines when the surgical instruments are moved abruptly. We tested the algorithm using 17 videos of minimally invasive surgery, 11 of which had tool-induced bleeding. Our preliminary testing shows that the algorithm is 88% accurate and 90% precise in predicting bleeding. The average advance warning time for the 11 videos is 0.662 s, with the standard deviation being 0.427 s. The proposed approach has the potential to eventually lead to a surgical early warning system or even proactively attenuate tool movement (for robotic surgery) to avoid dangerous surgical outcomes.

The system design based on long-wave uncooled infrared detector

According to the requirement of high-definition surveillance of the status of space targets in low illuminance, a high SNR (Signal to Noise Ratio) uncooled long-wave infrared system is designed. Based on the detector of UL04371 with 640 × 480 array, the system scheme, acquisition system and real-time image processing algorithms are realized. On the basis of system implementation, the paper mainly focuses on how to reduce the noises from the aspect of hardware circuits and how to realize the real-time processing in the way of specified infrared image algorithms. Finally, the paper gives the test result which shows that the system has achieved a relatively good SNR characteristic with the average NETD (Noise Equivalent Temperature Difference) value of 220mK under the condition of the integration time of 64us and the radiation of 300K black body.

CONSTRUCTION AND TEST OF BIO-INSPIRED IMAGING POLARIZATION NAVIGATION PROTOTYPE

Abstract. Bio-inspired polarization navigation is a promising navigation method inspired by insects’ autonomous foraging and homing behaviour. Many insects acquire their spatial orientation by sensing the polarization pattern of the skylight. We propose utilization of solar meridian in the polarized skylight as an orientation cue because of its significant features. Using its features, we then design and construct an imaging polarization navigation prototype. The prototype consists of a field-division polarization imaging sensor, the corresponding software, an interface, and the solar-meridian recognizing and measurement algorithm. The field-division polarization imaging sensor is the core component of the prototype and acquires polarized intensity images. To adapt to the demand of real-time on navigation system, we then propose an optimized real-time polarization image processing and pattern recognition algorithm based on Hough transform. The azimuth measurement accuracy of the sensor is then calibrated using a facility that is able to get higher azimuth accuracy by measurement of the star light. To verify the navigation capability of the developed system, we use a dynamic experiment, where the prototype is installed on the top of a vehicle and its navigation performance is compared with GNSS.

Image-Based Gimbal Control in a Drone for Centering Photovoltaic Modules in a Thermal Image

Recently, there have been many types of research applying drones with a thermal camera to detect deteriorations in photovoltaic (PV) modules. A thermal camera can measure temperatures on the surface of PV modules and find the deteriorated area. However, a thermal camera generally has a lower resolution than a visible camera because of the limitations of cost. Due to different resolutions between the visible and thermal cameras, there are often invalid frames from a thermal camera. In this paper, we describe a gimbal controller with a real-time image processing algorithm to control the angle of the camera to position the region of interest (ROI) in the center of target PV modules to solve this problem. We derived the horizontal angle and vertical position of ROI in visible images using image processing algorithms such as the Hough transform. These values are converted into a PID control signal for controlling the gimbal. This process makes the thermal camera capture the effective area of target PV modules. Finally, experimental results showed that the photovoltaic module’s control area was properly located at the center of the thermal image.

Detection of Breast Cancer from Thermography Images

Breast cancer is now the most common cancer in most cities in India, and the second most common in rural areas. Early detection of breast cancer by systematic evaluation of the individual may improve survival rate. Infrared thermography one of the imaging technique that produce high resolution infrared images shows the heat pattern based on the temperature changes in breast with respect to tshe progression of the cancer cells. Increased metabolic activity and the blood flow due to the multiplication of cancer cells induces more heat on the skin layer which are captured by the thermal camera to produce the thermal images. This paper discuss on the real time image processing algorithm to detect the presence of cancer from the acquired thermal images. The methodology includes the preprocessing the acquired image and segmenting the region of interest, extracting the features from the segmented image followed by feature selection and classification. From the results, it is inferred that ANN classifiers yields better classification accuracy of 92% and minimum error rate (0.08) in K-means segmentation method when compared with SVM, KNN classifiers.

A Simulation Environment for Validation and Verification of Real Time Hyperspectral Processing Algorithms on-Board a UAV

The utilization of hyperspectral imaging sensors has gained a significant relevance among many different applications due to their capability for collecting a huge amount of information across the electromagnetic spectrum. These sensors have been traditionally mounted on-board satellites and airplanes in order to extract information from the Earth’s surface. Fortunately, the progressive miniaturization of these sensors during the last lustrum has enabled their use in other remote sensing platforms, such as drones equipped with hyperspectral cameras which bring advantages in terms of higher spatial resolution of the acquired images, more flexible revisit times and lower cost of the flight campaigns. However, when these drones are autonomously flying and taking real-time critical decisions from the information contained in the captured images, it is crucial that the whole process takes place in a safe and predictable manner. In order to deal with this problem, a simulation environment is presented in this work to analyze the virtual behavior of a drone equipped with a pushbroom hyperspectral camera used for assisting harvesting applications, which enables an exhaustive and realistic validation and verification of the drone real-time hyperspectral imaging system prior to its launch. To the best of the authors’ knowledge, the proposed environment represents the only solution in the state-of-the-art that allows the virtual verification of real-time hyperspectral image processing algorithms under realistic conditions.

Multimorphological top-hat-based multiscale target classification algorithm for real-time image processing.

The traditional top-hat method is a commonly used method that quickly separates targets from a background. It is used for its fast processing speed and wide range of applications on programmable hardware. However, in some important fields such as microfluidic control, medicine, aerospace, and optical measurement, the observed targets are often spotted with different sizes. The formation mechanism of multiscale spots varies from each other so that they can not be successfully extracted and classified by the traditional top-hat method. To ensure the integrity of targets with a specific size and suppressed noise, the imaging mechanism of different types of spots are studied, and an improved top-hat method with a gray-scale value-based transform is proposed. Compared with the traditional top-hat method, the proposed algorithm is more effective in completely removing unwanted spots. The calculated results of the simulated and real images verify the effectiveness of the double top-hat method in extracting targets with a specific size. Additionally, the resolution of this method is up to the parameter k, which has been discussed in this paper. Furthermore, a multi-top-hat algorithm is presented to distinguish spots of different sizes, and it could be used for real-time multiscale target detection and tracking, as well as real-time multiscale target detection and tracking.

105. Upgrading Philips Allura Xper FD20 angiography system to Allura Clarity: Analysis of dose reduction and image quality

Purpose Our Allura Xper FD20 system was installed in 2013 and mainly used for stroke treatment. To reduce the radiation dose delivered to patients though maintaining the required image quality, the system was recently upgraded to Allura Clarity. The core characteristics of Allura Clarity family are new real-time image processing algorithms to reduce image noise. The aim of this work is to show the advantages obtained in terms of dose reduction for cardiac, vascular and electrophysiological studies following such upgrade. Methods For both angiographic systems, the patient dose was assessed by means of K a,e (Entrance Air Kerma) measurements, using a PMMA phantom to simulate patient attenuation. The measurement set up was the referred method indicated in the RP162 report [1] and extensively described by Martin [2] . For each investigated procedure, the measurements were carried out for most selectable fields of view and different PMMA thicknesses. The image quality was evaluated using Leeds TOR 18FG and FluoroMesh phantom for low contrast resolution and spatial resolution respectively. Results Depending on the procedure, the dose reduction is meaningful and ranges from 20% up to a factor of around 4 for fluorographic acquisitions. All K a,e values measured on Allura Clarity system are compliant with the Suspension Levels set out in the RP162 report [1] also for digital fluorographic acquisition mode. The image quality obtained with Allura Clarity is equivalent to that of the previous system. Conclusions The Allura Clarity delivers much lower dose to the patient for all clinical applications while maintaining an equivalent image quality.

Guess What?: Towards Understanding Autism from Structured Video Using Facial Affect.

Autism Spectrum Disorder (ASD) is a condition affecting an estimated 1 in 59 children in the United States. Due to delays in diagnosis and imbalances in coverage, it is necessary to develop new methods of care delivery that can appropriately empower children and caregivers by capitalizing on mobile tools and wearable devices for use outside of clinical settings. In this paper, we present a mobile charades-style game, Guess What?, used for the acquisition of structured video from children with ASD for behavioral disease research. We then apply face tracking and emotion recognition algorithms to videos acquired through Guess What? game play. By analyzing facial affect in response to various prompts, we demonstrate that engagement and facial affect can be quantified and measured using real-time image processing algorithms: an important first-step for future therapies, at-home screenings, and outcome measures based on home video. Our study of eight subjects demonstrates the efficacy of this system for deriving highly emotive structured video from children with ASD through an engaging gamified mobile platform, while revealing the most efficacious prompts and categories for producing diverse emotion in participants.

High-precision absolute linear encoder based on a standard calibrated scale

This paper presents the full research and development cycle of a high-precision absolute linear encoder based on a standard calibrated scale. Already available and used in industry standard scales of invar alloy were employed in the study. The scales have incremental indexes with 1 mm spacing intervals without an absolute code. However, the existing technical task is to measure the absolute position in the range of 2 m with accuracy less than 5 μm. For that, the developed encoder rationally combines magnetic measuring channel for the index numbering and an optical channel for the precise estimation of the encoder position. First, for the development, a simulation was performed to synthesize and analyze an image of an index. This image was used to develop a real-time double-threshold image processing algorithm to estimate the index position. Later the developed image processing algorithm was verified by preliminary testing and supported by a presented three-stage calibration procedure. The final measurements proved that the designed and developed encoder has the accuracy of 1.65 μm (3 standard deviations) at the speed up to 3 m/s. The possibility of use of standard calibrated scales with the presented encoder to solve existing and new industrial tasks forms the value of this paper. A possible use of the existing scales also provides unification and compatibility with conventional metrology equipment.

A Mobile Ad hoc Cloud Computing and Networking Infrastructure for Automated Video Surveillance System

Mobile automated video surveillance system involves application of real-time image and video processing algorithms which require a vast quantity of computing and storage resources. To support the execution of mobile automated video surveillance system, a mobile ad hoc cloud computing and networking infrastructure is proposed in which multiple mobile devices interconnected through a mobile ad hoc network are combined to create a virtual supercomputing node. An energy efficient resource allocation scheme has also been proposed for allocation of realtime automated video surveillance tasks. To enable communication between mobile devices, a Wi-Fi Direct based mobile ad hoc cloud networking infrastructure has been developed. More specifically, a routing layer has been developed to support communication between Wi-Fi Direct devices in a group and multi-hop communication between devices across the group. The proposed system has been implemented on a group of Wi-Fi Direct-enabled Samsung mobile devices.

Ultraprecise Three-Axis Visual Motion Tracking of Microscopic Objects

Real-time image processing algorithms and associated mathematical formulas that render ultraprecise three-axis visual motion tracking are presented. The principle of three-axis motion tracking is based on lateral sampled white light interferometry (LSWLI) and real-time unbiased image registration. Two distinct regions of interest (ROIs), i.e., in-plane ROI and out-of-plane ROI, in a single 2-D image, taken at a single time instant, are processed in real time. Real-time continuous image registration (CIR) is realized and applied to 2-D shadow projection image content of the object to locate the object’s in-plane position, whereas adaptive 1-D CIR (A1-CIR) is proposed and applied to an LSWLI fringe pattern on the object to determine its out-of-plane position. Processing of images is synchronized with real-time image acquisition to realize three-axis motion tracking. Computer simulations are used to demonstrate bias caused by image registration and to examine its effect on measurement precision. The ability of the proposed approach to eliminate measurement bias is validated. Experimental results are presented and used to examine measurement resolution for positioning and to illustrate measurement precision for long-range motion tracking.

Real-Time GPU-Based Image Processing for a 3-D THz Radar

A real-time image processing algorithm, using a GPU with NVIDIA’s CUDA technology, for a 3-D through-clothes detection radar in the THz band has been developed. Traditional CPU-based architectures lack the computing power and parallelization needed to meet the image refresh rates imposed by the radar’s scanning. This solution presents a low-cost and flexible alternative that allows image refresh at a rate of more than 8 fps for images composed of 6,000 pixels, corresponding to a scanned area of 50 $\times$ 90 cm $^2$ . The performance of this code has been profiled, with comparison against a previous CPU-executed version, and some optimizations that would enable even faster refresh have been analyzed.

IMPLEMENTATION OF GRADIENT BASED ADAPTIVE INTERPOLATION ON FPGA

FPGA (Field Programmable Gate Array) structure can exploit the temporal and spatial parallelism of the system, subject to hardware constraints and processing mode of the system. Due to this property, real time image processing algorithms can implemented using this platform. Now a days High level Synthesis Tools (HLS) available. Which can used to configure FPGA. This paper presents a real time implementation of gradient-based adaptive interpolation using HLS. We received a speedup of 14X over general purpose processor.

A novel global methodology to analyze the embeddability of real-time image processing algorithms

Advanced driver assistance systems applications increasingly use cameras and image processing algorithms. To embed and achieve real-time execution of these algorithms, semiconductor companies propose heterogeneous systems-on-chip (SoCs). Embedding algorithms on this type of hardware is not trivial: One needs to determine how to partition the computational load on the different processing units. In addition, it is not easy to predict whether a given algorithm can be executed on a given heterogeneous SoC while meeting real-time constraints. We propose a novel global methodology to assist with embedding image processing algorithms on heterogeneous SoC while meeting real-time constraints (using a soft real-time analysis). Our approach proposes several heuristics predicting delays and execution times and is based on a set of multi-level test vectors which extract key features of heterogeneous architectures.

Real Time Computer Vision Systems for Rice Kernels

Objectives: This paper presents a Morphological operations based software code in MATLAB for counting connected rice kernels from digital image and to grade the quality of samples by a methodology used by the Bureau of Indian standards. Methods/Statistical Analysis: The proposed code may further be transferred on to the reprogrammable hardware devices like FPGA by converting the entire code into VHDL (VHSIC Hardware Description Language) with the help of Simulink HDL Coder; making it a hardware compatible code. The efficiency of proposed code is tested over the digital images of rice grain samples with complex backgrounds or captured under poor illumination conditions. Findings: 100% accuracy has been observed in the counting efficiency of software and hardware codes by successfully separating touching kernels. The automatically generated VHDL code through HDL Coder is also successfully synthesized over the FPGA in Xilinx ISE and is compared for its accuracy and processing time with simulated results. It is concluded that a real time image processing algorithm, when design effectively to get synthesized over the FPGAs, may yield faster results with processing time of few seconds, whereas the manual methods or simulated methods are much slower in terms of their speed. Application/ Improvements: The proposed approach may further be utilized to design a portable FPGA based hardware prototype to grade the quality of rice samples by completing eliminating the manual investigation done by humans as well as by computer based simulated inspection.

Enhancing Sensitivity of a Miniature Spectrometer Using a Real-Time Image Processing Algorithm.

A real-time image processing algorithm is developed to enhance the sensitivity of a planar single-mode waveguide miniature spectrometer with integrated waveguide gratings. A novel approach of averaging along the arcs in a curved coordinate system is introduced which allows for collecting more light, thereby enhancing the sensitivity. The algorithm is tested using CdSeS/ZnS quantum dots drop casted on the surface of a single-mode waveguide. Measurements indicate that a monolayer of quantum dots is expected to produce guided mode attenuation approximately 11 times above the noise level.