On-line Image Processing Research Articles

Enhancing the data processing speed of a deep-learning-based three-dimensional single molecule localization algorithm (FD-DeepLoc) with a combination of feature compression and pipeline programming

Three-dimensional (3D) single molecule localization microscopy (SMLM) plays an important role in biomedical applications, but its data processing is very complicated. Deep learning is a potential tool to solve this problem. As the state of art 3D super-resolution localization algorithm based on deep learning, FD-DeepLoc algorithm reported recently still has a gap with the expected goal of online image processing, even though it has greatly improved the data processing throughput. In this paper, a new algorithm Lite-FD-DeepLoc is developed on the basis of FD-DeepLoc algorithm to meet the online image processing requirements of 3D SMLM. This new algorithm uses the feature compression method to reduce the parameters of the model, and combines it with pipeline programming to accelerate the inference process of the deep learning model. The simulated data processing results show that the image processing speed of Lite-FD-DeepLoc is about twice as fast as that of FD-DeepLoc with a slight decrease in localization accuracy, which can realize real-time processing of [Formula: see text] pixels size images. The results of biological experimental data processing imply that Lite-FD-DeepLoc can successfully analyze the data based on astigmatism and saddle point engineering, and the global resolution of the reconstructed image is equivalent to or even better than FD-DeepLoc algorithm.

Dominant Color Detection For Online Fashion Retrievals

This paper introduces a novel approach aimed at efficiently extracting dominant colors from online fashion images. The method addresses challenges related to detecting overlapping objects and computationally expensive methods by combining K-means clustering and graph-cut techniques into a framework. This framework incorporates an adaptive weighting strategy to enhance color extraction accuracy. Additionally, it introduces a two-phase fashion apparel detection method called YOLOv4, which utilizes U-Net architecture for clothing segmentation to precisely separate clothing items from the background or other elements. Experimental results show that K-means with YOLOv4 outperforms K-means with the U-Net model. These findings suggest that the U-Net architecture and YOLOv4 models can be effective methods for complex image segmentation tasks in online fashion retrieval and image processing, particularly in the rapidly evolving e-commerce environment.

Long-term monitoring of huddling behavior in mice using online image processing.

Many animal species, including mice, form societies of numerous individuals for survival. Understanding the interactions between individual animals is crucial for elucidating group behavior. One such behavior in mice is huddling, yet its analysis has been limited. In this study, we propose a cost-effective method for monitoring long-term huddling behavior in mice using online image processing with OpenCV. This method treats a single mouse or a group of mice as a cluster of pixels (a 'blob') in video images, extracting and saving only essential information such as areas, coordinates, and orientations. This approach reduces data storage needs to 1/200000th of what would be required if the video were recorded in its compressed form, thereby enabling long-term behavioral analysis. To validate the performance of our algorithm, ~2000 video frames were randomly chosen. We manually counted the number of clusters of mice in these frames and compared them with the number of blobs automatically detected by the algorithm. The results indicated a high level of consistency, exceeding 90% across the selected video frames. Initial observations of both male and female groups suggested some variations in huddling behavior among male and female groups; however, these results should be interpreted cautiously due to a small sample. Group behavior is known to be disrupted in several neuropsychiatric disorders, such as autism. Various mouse models of these disorders have been proposed. Our measurement system, when combined with drug or genetic modification screening, could provide a valuable tool for high-throughput analyses of huddling behavior.

A Novel Standardized Collaborative Online Model for Processing and Analyzing Remotely Sensed Images in Geographic Problems

In recent years, remote sensing image processing technology has developed rapidly, and the variety of remote sensing images has increased. Solving a geographic problem often requires multiple remote sensing images to be used together. For an image processing analyst, it is difficult to become proficient in the image processing of multiple types of remote sensing images. Therefore, it is necessary to have multiple image processing analysts collaborate to solve geographic problems. However, as a result of the naturally large volumes of data and the computer resources they consume for analysis, remote sensing images present a barrier in the collaboration of multidisciplinary remote sensing undertakings and analysts. As a result, during the development of the collaborative analysis process, it is necessary to achieve the online processing and analysis of remote sensing images, as well as to standardize the online remote sensing image collaborative analysis process. To address the above issues, a hierarchical collaborative online processing and analysis framework was developed in this paper. This framework defined a clear collaborative analysis structure, and it identifies what kinds of online image processing and analysis activities participants can engage in to successfully conduct collaborative processes. In addition, a collaborative process construction model and an online remote sensing image processing analysis model were developed to assist participants in creating a standard collaborative online image processing and analysis process. In order to demonstrate the feasibility and effectiveness of the framework and model, this paper developed a collaborative online post-disaster assessment process that utilizes radar images and optical remote sensing images for a real forest fire event. This process was based on the BPMN2.0 and OGC dual standards. Based on the results, the proposed framework provides a hierarchical collaborative remote sensing image processing and analysis process with well-defined stages and activities to guide the participants’ mutual collaboration. Additionally, the proposed model can help participants to develop a standardized collaborative online image processing process in terms of process structure and information interactions.

ON-LINE VIDEO PROCESSING EMBEDDED SYSTEM FOR AUTOMOTIVE

Real-time processing for images is increasingly used in automotive industry. Video signals are nowadays used for different applications on traffic, such as measuring average speed, monitoring traffic congestion, detect red light crossing, but also for sign recognition, lane assist etc. The paper proposes an embedded system for image processing used for controlling climatization in cars. This system is also useful in case of an accident to activate the airbag if a person is on the seat or not to do it if a heavier object is placed on the seat, for example a bag. For this aim, a Raspberry Pi3 based system is proposed for real-time detecting of the number of passengers existing in a vehicle. The Raspberry Pi3 is used for acquiring frames from a video camera connected via USB interface and for on-line processing of images. The system uses an Haar cascade classifier algorithm (HCCA) for human face detection.

A Digital Twin concept for the prescriptive maintenance of protective coating systems on wind turbine structures

The application of protective coating systems is the major measure against the corrosion of steel for tower sections of wind turbines. The inspection, condition monitoring and maintenance of protective coating system is a demanding and time-consuming procedure and requires high human effort. The paper introduces for the first time a Digital Twin concept for the condition monitoring and prescriptive maintenance planning for surface protection systems on wind turbine towers. The initial point of the concept is an in-situ Virtual Twin for the generation of reference areas for condition monitoring. The paper describes the integration of an online image annotation and processing tool, a maintenance model, corrosive resistance parameters, structural load parameters, and sensor data into the Digital Twin concept. The prospects of the concept and its practical relevance are shown for tower structures of large onshore wind turbines.

Displacement feedback control of actuators for structural testing using image processing and analysis

AbstractExternal displacement control (EDC) is essential to a structural test in which a servo‐hydraulic actuator is connected to the specimen through loading fixtures, steel plates, and bolts in order to ensure that the specimen deformation meets the predefined displacement profile accurately. However, external displacement transducers for actuator feedback control could be detached from the specimen due to concrete spalling or steel buckling, leading to potential risks associated with actuator control stability and safety. In this study, image‐based displacement feedback control of actuators for structural testing is proposed and validated, which provides a noncontact approach to measure specimen deformation. In the proposed control scheme, a high‐speed camera is used to capture the images of a feature on the specimen during the test. The specimen deformation is calculated by using the captured images through online image processing and analysis. The actuator controller utilizes the calculated displacement as the feedback signal for displacement control. Finally, the proposed control scheme is validated in the laboratory. Two types of loading protocols are adopted for experimental validation including quasi‐static cyclic testing and dynamic loading testing. Experimental results demonstrate that the proposed image‐based displacement feedback control of the actuator completes quasi‐static cyclic tests and dynamic loading tests successfully and stably with satisfactory displacement tracking performance. The proposed method provides an alternative for EDC in a structural test.

Automatic Detection of Colorectal Polyps Using Transfer Learning.

Simple SummaryColorectal cancer represents one of the major health problems due to high incidence and mortality rates. A diversity of treatment options as well as a rising population require novel diagnostic tools. The main goal of the research was to develop a novel complex colorectal cancer decision support system based on artificial intelligence. The developed system can classify eight classes of tissue and can identify the malignant areas. In order to allow the easiest and most intuitive interaction with clinicians, the corresponding application was also built.Colorectal cancer is the second leading cause of cancer death and ranks third worldwide in diagnosed malignant pathologies (1.36 million new cases annually). An increase in the diversity of treatment options as well as a rising population require novel diagnostic tools. Current diagnostics involve critical human thinking, but the decisional process loses accuracy due to the increased number of modulatory factors involved. The proposed computer-aided diagnosis system analyses each colonoscopy and provides predictions that will help the clinician to make the right decisions. Artificial intelligence is included in the system both offline and online image processing tools. Aiming to improve the diagnostic process of colon cancer patients, an application was built that allows the easiest and most intuitive interaction between medical staff and the proposed diagnosis system. The developed tool uses two networks. The first, a convolutional neural network, is capable of classifying eight classes of tissue with a sensitivity of 98.13% and an F1 score of 98.14%, while the second network, based on semantic segmentation, can identify the malignant areas with a Jaccard index of 75.18%. The results could have a direct impact on personalised medicine combining clinical knowledge with the computing power of intelligent algorithms.

Detection of Gasoline Adulteration Using Modified Distillation Curves and Artificial Neural Network

AbstractTo detect adulteration in gasoline, an automatic distillation apparatus was set up to measure the recovered volume and temperature simultaneously. The level metering was performed by online image processing instead of the conventional visual operator‐based measurement. To investigate the effect of additives in super gasoline, regular gasoline and diesel were added and the distillation curves were analyzed. The principal component analysis model was employed to reduce the obtained data. Finally, an artificial neural network was applied to predict the volume percentage of adulterants in super gasoline. Statistical analysis showed that the proposed model has a mean relative error and correlation coefficient of 4.6 % and 0.995, respectively.

Мониторинг скорости кровотока в венулах брыжейки крыс

Development of systems for recording and automatic processing of information about direct microvascular responses is an important task for a preclinical study of drugs influencing the microcirculation. The aim was to modify the method for on-line recording and processing of intravital images of microvessels to study changes in the blood flow velocity under disturbing impacts. Method. This article presents an improved technique for recording linear blood flow velocity in microvessels and for processing intravital images. Macroscope (Wild M420, Switzerland), lens (Makrozoom 6.3-32x), and high-speed digital camera (Basler acA2000-165uc USB3.0, Germany) with a CMOSIS CMV2000 matrix were used for visual tracking of changes in blood flow velocity. The camera allows obtaining full-frame, 2000x1000 pixel images with a frequency of 165 fps, and linear images with a frequency of 8000 fps to capture a wide range of linear blood flow velocities. The scale of an image obtained with the microscope-camera system is 0.7 microns per pixel, which makes it possible to measure velocities of 0.1 to 150 mm/s. Digitizing and analyzing images were performed with a MultiMedia Catalog software, Version 2.3 (MMCatalog, Russia). The analysis of blood flow velocity is based on direct measurement of the blood cell flow velocity. The velocity is measured by estimation of the shift of blood cell flow for a predetermined time period between two measurements. To this purpose, a linear marker oriented along the vessel is applied to the blood flow image in a selected section of the microvessel. Following the measurement onset, the camera switches to a linear mode along the marker every second. A series of 5–6 measurements of linear images is taken at definite time intervals. Then measurements are used for construction of optical slices and measurement of the relative shift of the curve. The curve shift is determined by estimating the sum of differences between two arrays of curve brightness in scanning from –20 to +20% of the current position. The position corresponding to the minimum sum of differences is considered the value of curve shift. This procedure is performed for all 5-6 pairs of measurements. From the obtained values of curve shifts a median is selected, which is used as a result of measuring the velocity. Then the camera is again switched to the full-frame shooting mode, which allows you to control the position of the marker in real time. During the experiment, the object may move due to movements or breathing of the animal. To adjust the marker position on the image, it is possible to change its position during the measurements. The program captures the obtained values of curve shits and, thereby, the blood flow velocity. Based on the blood flow velocity, the program allows adjusting the number of measurements per unit time, which makes it possible to maintain stability of velocity measurements. To reduce the effect of high-frequency noise the curves are smoothed using moving averages. The averaging intensity can be adjusted in the process of measurement. Recording of blood flow velocity in a selected area is performed continuously during the entire experiment. Experimental protocol: – recording of the initial blood flow for 180 s – recording of blood flow during laser irradiation for 300 s – recording of blood flow after the exposure for 300 s. Results. This method of recording blood flow velocity allows recording increases in venular blood flow velocity both during and after an exposure to laser irradiation (662 nm). Conclusion. The proposed technique for processing intravital images of blood flow makes it possible to record fast changes in blood flow velocity in microvessels under disturbing impacts in real time.

In-situ Online Measurement of Rhombic Distortion in Billets

DD (diagonal-difference) is considered as measure of rhombic distortion, aka Rhomboidity, which is a shape related defect in square cross-section billets. Rhomboidity in billets starts with non-uniform shell solidification in the mold primarily due to inconsistent cooling causing irregular heat transfer. The higher diagonal difference greatly impacts the quality of billets to be rolled at various mills. Rhomboidity at or over 4% leads to billet twisting in the roughing stands of the rolling mill. Currently, billet rhomboidity is measured manually at end of casting operation. The presented work describes an optical, online & real-time image processing based method to determine the rhomboidity induced in each strand and alerts the operator to take corrective actions. The online Rhomboidity Measurement System employs sophisticated image acquisition & processing techniques to determine face contours of the billet with sub-pixel accuracy. The key features of RMS are the construction of a gaussian penalty function for selection of suitable 4-lines combination that precisely fits the billet face and use of a highly efficient and accurate statistical indicator, based on KL-Divergence measure, to estimate the rhomboidity even in presence of partial occlusion of billet face by scales. The expected savings are to the tune of 0.27 Million USD.

System Architecture for Real-Time Surface Inspection Using Multiple UAVs

This paper presents a real-time control system for surface inspection using multiple unmanned aerial vehicles (UAVs). The UAVs are coordinated in a specific formation to collect data of the inspecting objects. The communication platform for data transmission is based on the Internet of Things (IoT). In the proposed architecture, the UAV formation is established via using the angle-encoded particle swarm optimisation to generate an inspecting path and redistribute it to each UAV where communication links are embedded with an IoT board for network and data processing capabilities. Data collected are transmitted in real time through the network to remote computational units. To detect potential damage or defects, an online image processing technique is proposed and implemented based on histograms. Extensive simulation, experiments and comparisons have been conducted to verify the validity and performance of the proposed system.

WindSTORM: Robust online image processing for high-throughput nanoscopy.

High-throughput nanoscopy becomes increasingly important for unraveling complex biological processes from a large heterogeneous cell population at a nanoscale resolution. High-density emitter localization combined with a large field of view and fast imaging frame rate is commonly used to achieve a high imaging throughput, but the image processing speed and the presence of heterogeneous background in the dense emitter scenario remain a bottleneck. Here, we present a simple non-iterative approach, referred to as WindSTORM, to achieve high-speed high-density emitter localization with robust performance for various image characteristics. We demonstrate that WindSTORM improves the computation speed by two orders of magnitude on CPU and three orders of magnitude upon GPU acceleration to realize online image processing, without compromising localization accuracy. Further, WindSTORM is highly robust to maximize the localization accuracy and minimize the image artifacts in the presence of nonuniform background. WindSTORM paves the way for next generation high-throughput nanoscopy.

Ultraviolet laser vision system

An ultraviolet (UV) laser vision system developed at the All-Russian Scientific Research Institute for Optical and Physical Measurements (VNIIOFI) is presented. The system is based on an electro-optical camera and a laser illumination source at a wavelength of 351 nm. The UV detector is an electro-optical converter with an S25 photocathode on uviol glass. The system also includes objectives, an interference UV filter, and a pulse generator. The software offers the following functionalities: the observation of the image on the computer display while in online mode, image processing and analysis, and the saving of video data.

Rapid and non-invasive surface crack detection for pressed-panel products based on online image processing

Crack detection during the manufacturing process of pressed-panel products is an important aspect of quality management. Traditional approaches for crack detection of those products are subjective and expensive because they are usually performed by experienced human inspectors. Therefore, the development and implementation of an automated and accurate inspection system is required for the manufacturing process. In this article, a crack detection technique based on image processing is proposed that utilizes the images of panel products captured by a regular camera system. First, the binary panel object image is extracted from various backgrounds after considering the color factor. Edge lines are then generated from a binary image using a percolation process. Finally, crack detection and localization is performed with a unique edge-line evaluation. In order to demonstrate the capability of the proposed technique, lab-scale experiments were carried out with a thin aluminum plate. In addition, a test was performed with the panel images acquired at an actual press line. Experimental results show that the proposed technique could effectively detect panel cracks at an improved rate and speed. The experimental results also demonstrate that the proposed technique could be an extension of structural health monitoring frameworks into a new manufacturing application.

Application of qualitative biospeckle methods for the identification of scar region in a green orange

This study demonstrates the feasibility of a view-based method, the motion history image (MHI) to map biospeckle activity around the scar region in a green orange fruit. The comparison of MHI with the routine intensity-based methods validated the effectiveness of the proposed method. The results show that MHI can be implementated as an alternative online image processing tool in the biospeckle analysis.

An investigation on the relationship among marbling features, physiological age and Warner-Bratzler Shear force of steer longissimus dorsi muscle.

Researchers nowadays have paid much attention to the relationships between tenderness and marbling, or physiological age. While the marbling was mainly evaluated qualitatively with scores or grades, and rarely related with physiological age. Present study was carried out to analyze the marbling features of longissimus dorsi muscle between the 12th and 13th ribs from 18, 36, 54 and 72months old Simmental steers were quantitatively described with area and perimeter using computer vision technique. Relationship between Warner-Bratzler Shear force (WBSF), physiological age and the marbling features were examined performing regression analysis. The results revealed that WBSF positively correlated with physiological age, but negatively with marbling area and perimeter. Regression analysis showed that the relationship between the shear force and the steers' age was more close to the quadratic curve (R2=0.996) and exponential curve (R2=0.957). It was observed during study that marbling grew with steers age. Marbling features were in linear correlation with the steers' age, with R2=0.927 for marbling area and R2=0.935 for marbling perimeter. The industries in future may speculate beef tenderness and physiological age based on the marbling features (area and perimeter), which can be determined through the online image acquisition system and image processing.

Online Spatial Data Analysis and Algorithm Development for Geo-scientific Applications Using Remote Sensing Data

Availability of geospatial data from online data repositories and geoportals are increasing with rapid pace. For better utilization of these heterogeneous data sets, it is required to give more attention to develop web based algorithms as well as geocomputation models that allows high-level information extraction from distributed nature of geodata. The Open Geospatial Consortium specification such as Web Processing Service (WPS) provides the solutions for integrating remote sensing data for the computation of algorithms from distributed sources in multi-user environment. The main purpose of present study is to develop and implement a WPS based geoprocesses and analysis approach using Online Image Processing System. The software framework developed during this study is a platform independent solution which is developed using free and open source technology. As a demonstrative case study, the Normalized Difference Vegetation Index and Normalized Difference Built-up Index algorithms are annotated for automatic deployment in geoscientific processing workflow environment. The Sentinel-2 data sets and polygon geometry as region of interest are stored and published as web services for input dataset in application server repository. A case study for Champawat district of Uttarakhand state of India has been demonstrated to identify vegetation greenness and built-up sites during the summer season. In order to access the functionalities of the online application, an interactive and responsive Graphical User Interface is developed in web GIS environment. The proposed approach can be very effective and useful to address the key challenges in commercial desktop based software for geoprocessing and analysis of remote sensing data.

Real-Time Intravascular Ultrasound and Photoacoustic Imaging.

Combined intravascular ultrasound and intravascular photoacoustic (IVUS/IVPA) imaging is an emerging hybrid modality being explored as a means of improving the characterization of atherosclerotic plaque anatomical and compositional features. While initial demonstrations of the technique have been encouraging, they have been limited by catheter rotation and data acquisition, displaying, and processing rates on the order of several seconds per frame as well as the use of off-line image processing. Herein, we present a complete IVUS/IVPA imaging system and method capable of real-time IVUS/IVPA imaging, with online data acquisition, image processing, and display of both IVUS and IVPA images. The integrated IVUS/IVPA catheter is fully contained within a 1-mm outer diameter torque cable coupled on the proximal end to a custom-designed spindle enabling optical and electrical coupling to system hardware, including a nanosecond-pulsed laser with a controllable pulse repetition frequency capable of greater than 10 kHz, motor and servo drive, a US pulser/receiver, and a 200-MHz digitizer. The system performance is characterized and demonstrated on a vessel-mimicking phantom with an embedded coronary stent intended to provide IVPA contrast within content of an IVUS image.

An experimental study on the vision-based control and identification of planar cable-driven parallel robots

This paper presents the implementation of a vision-based position control for planar cable-driven parallel robots. The main contribution of this paper contains three objectives. First, a method is used toward kinematic modeling of the robot using four-bar linkage kinematic concept, which could be used in online control approaches for real-time purposes. In order to ensure positive tension in cables, as a basic essential property, static and dynamic equations of robot are also obtained. Second, in order to track the position of End-Effector, an online image processing procedure is developed and implemented. Finally, as the third contribution, different control approaches are applied in order to validate the model with plant and obtain the most promising controller. As classic controller, pole placement approach is suggested and results reveal weaknesses in modeling the uncertainties. Due to the latter incapability, sliding mode controller is utilized and experimental tests represent effectiveness of this method. However, the chattering phenomenon in the beginning of the robot operation is the main insufficiency of this controller. Hence, in order to present a more accurate controller, adaptive sliding mode controller working alongside with an identification method on the model is applied. The provided identification procedure is based on Recursive Least Square approach, which rebates the effects of uncertainties in the parameters of the model. Moreover, results of these controllers confirm accommodation between the model and robot. The proposed procedure could be well applied for any kind of planar cable-driven parallel robot.