High-resolution Video Imaging Research Articles

Hydromechanical properties of metachronal swimming in polychaetes

Free-swimming polychaetes are common in marine habitats and exhibit a unique form of swimming whereby a metachronal wave occurs simultaneously with a bending body wave. This body wave is unusual among swimming animals because it travels in the same direction as the animal’s swimming direction. However, we currently lack a mechanistic understanding of this unusual form of locomotion. In this study we use a combination of high-speed, high-resolution video and particle image velocimetry (PIV) to quantify kinematics and fluid dynamics for three species of swimming polychaetes, spanning two orders of magnitude in size. We find that in all species, flows generated by metachronal waves of parapodia dominate while typical flows associated with body bending is absent. However, the parapodia are less flexible than propulsive structures in other metachronal swimmers. This creates a localized, but substantial upstream flow during the recovery stroke. Using body bending, the recovery stroke can occur mostly beneath the bulk flow from the power strokes, resulting in minimal inference while the subsequent power stroke can benefit from the pressure field generated during recovery. These results may have implications for future vehicle designs that incorporate metachronal locomotion.

FlyVISTA, an Integrated Machine Learning Platform for Deep Phenotyping of Sleep in Drosophila.

Animal behavior depends on internal state. While subtle movements can signify significant changes in internal state, computational methods for analyzing these "microbehaviors" are lacking. Here, we present FlyVISTA, a machine-learning platform to characterize microbehaviors in freely-moving flies, which we use to perform deep phenotyping of sleep. This platform comprises a high-resolution closed-loop video imaging system, coupled with a deep-learning network to annotate 35 body parts, and a computational pipeline to extract behaviors from high-dimensional data. FlyVISTA reveals the distinct spatiotemporal dynamics of sleep-associated microbehaviors in flies. We further show that stimulation of dorsal fan-shaped body neurons induces micromovements, not sleep, whereas activating R5 ring neurons triggers rhythmic proboscis extension followed by persistent sleep. Importantly, we identify a novel microbehavior ("haltere switch") exclusively seen during quiescence that indicates a deeper sleep stage. These findings enable the rigorous analysis of sleep in Drosophila and set the stage for computational analyses of microbehaviors.

Novel insights into mechanisms of inhibition of colonic motility by loperamide.

It is well known that opiates slow gastrointestinal (GI) transit, via suppression of enteric cholinergic neurotransmission throughout the GI tract, particularly the large intestine where constipation is commonly induced. It is not clear whether there is uniform suppression of enteric neurotransmission and colonic motility across the full length of the colon. Here, we investigated whether regional changes in colonic motility occur using the peripherally-restricted mu opioid agonist, loperamide to inhibit colonic motor complexes (CMCs) in isolated mouse colon. High-resolution video imaging was performed to monitor colonic wall diameter on isolated whole mouse colon. Regional changes in the effects of loperamide on the pattern generator underlying cyclical CMCs and their propagation across the full length of large intestine were determined. The sensitivity of CMCs to loperamide across the length of colon varied significantly. Although there was a dose-dependent inhibition of CMCs with increasing concentrations of loperamide (10 nM - 1 μM), a major observation was that in the mid and distal colon, CMCs were abolished at low doses of loperamide (100 nM), while in the proximal colon, CMCs persisted at the same low concentration, albeit at a significantly slower frequency. Propagation velocity of CMCs was significantly reduced by 46%. The inhibitory effects of loperamide on CMCs were reversed by naloxone (1 μM). Naloxone alone did not change ongoing CMC characteristics. The results show pronounced differences in the inhibitory action of loperamide across the length of large intestine. The most potent effect of loperamide to retard colonic transit occurred between the proximal colon and mid/distal regions of colon. One of the possibilities as to why this occurs is because the greatest density of mu opioid receptors are located on interneurons responsible for neuro-neuronal transmission underlying CMCs propagation between the proximal and mid/distal colon. The absence of effect of naloxone alone on CMC characteristics suggest that the mu opioid receptor has little ongoing constitutive activity under our recording conditions.

Productivity Analysis of Mid-Size Cable Yarding Operation based on UAV-captured Video Sequences

Cable yarding is one of the most preferred timber extraction methods in steep and mountainous terrain conditions. In recent years in Turkiye, mid-size cable yarders have begun to be used frequently in steep forest areas where intensive forestry is carried out. In order to make more effective use of cable yarders and achieve maximum productivity, timber extraction operations must be planned most appropriately and implemented in the field accordingly. The productivity of mechanized equipment used in harvesting operations is mainly determined by using time study analysis. In traditional methods, time measurements are carried out by observing the work stages with time recording devices (stopwatches) in the area where the machine operates. Since timber extraction by using a cable yarder occur in steep and rugged terrain conditions, difficulties may be encountered in applying time analysis in the field. In this study, it was aimed to obtain high-resolution digital video images of the activities of cable yarding operation with an Unmanned Aerial Vehicle (UAV) and to calculate productivity through time study analysis on these images. For the study, the Tajfun MOZ 500 GR model mid-size cable yarder was evaluated during a timber extraction operation in Köyceğiz Forest Enterprise Chief (FEC), located in Muğla City, Turkiye. Besides, statistical analysis was performed to estimate the effects of factors such as average product length, product diameter, lateral yarding distance, number of pieces, and total product volume per turn on the productivity of the cable yarder. The results indicated that UAV video images can be an effective alternative method to conduct time study analysis and productivity estimations. It was found that the productivity of the mid-size cable yarder was 11.76 m3/hour. During the operation, the pulling mainline to the product and choker setting were the most time-consuming work stages, followed by the time-loaded carriage reaching the landing. The statistical analysis indicated a significant relationship (p<0.001) between product length, product diameter, total volume and productivity of the cable yarder. The results showed that there was a negative relationship between lateral yarding distance and productivity. The regression model regarding the productivity of the cable yarder and the independent variables was also significant.

High-performance Reconfigurable DNN Accelerator on a Bandwidth-limited Embedded System

Deep convolutional neural networks (DNNs) have been widely used in many applications, particularly in machine vision. It is challenging to accelerate DNNs on embedded systems because real-world machine vision applications should reserve a lot of external memory bandwidth for other tasks, such as video capture and display, while leaving little bandwidth for accelerating DNNs. In order to solve this issue, in this study, we propose a high-throughput accelerator, called reconfigurable tiny neural network accelerator (ReTiNNA), for the bandwidth-limited system and present a real-time object detection system for the high-resolution video image. We first present a dedicated computation engine that takes different data mapping methods for various filter types to improve data reuse and reduce hardware resources. We then propose an adaptive layer-wise tiling strategy that tiles the feature maps into strips to reduce the control complexity of data transmission dramatically and to improve the efficiency of data transmission. Finally, a design space exploration (DSE) approach is presented to explore design space more accurately in the case of insufficient bandwidth to improve the performance of the low-bandwidth accelerator. With a low bandwidth of 2.23 GB/s and a low hardware consumption of 90.261K LUTs and 448 DSPs, ReTiNNA can still achieve a high performance of 155.86 GOPS on VGG16 and 68.20 GOPS on ResNet50, which is better than other state-of-the-art designs implemented on FPGA devices. Furthermore, the real-time object detection system can achieve a high object detection speed of 19 fps for high-resolution video.

The hydrodynamics and kinematics of the appendicularian tail underpin peristaltic pumping.

Planktonic organisms feed while suspended in water using various hydrodynamic pumping strategies. Appendicularians are a unique group of plankton that use their tail to pump water over mucous mesh filters to concentrate food particles. As ubiquitous and often abundant members of planktonic ecosystems, they play a major role in oceanic food webs. Yet, we lack a complete understanding of the fluid flow that underpins their filtration. Using high-speed, high-resolution video and micro particle image velocimetry, we describe the kinematics and hydrodynamics of the tail in Oikopleura dioica in filtering and free-swimming postures. We show that sinusoidal waves of the tail generate peristaltic pumping within the tail chamber with fluid moving parallel to the tail when filtering. We find that the tail contacts attachment points along the tail chamber during each beat cycle, serving to seal the tail chamber and drive pumping. When we tested how the pump performs across environmentally relevant temperatures, we found that the amplitude of the tail was invariant but tail beat frequency increased threefold across three temperature treatments (5°C, 15°C and 25°C). Investigation into this unique pumping mechanism gives insight into the ecological success of appendicularians and provides inspiration for novel pump designs.

Long range synchronization within the enteric nervous system underlies propulsion along the large intestine in mice

How the Enteric Nervous System (ENS) coordinates propulsion of content along the gastrointestinal (GI)-tract has been a major unresolved issue. We reveal a mechanism that explains how ENS activity underlies propulsion of content along the colon. We used a recently developed high-resolution video imaging approach with concurrent electrophysiological recordings from smooth muscle, during fluid propulsion. Recordings showed pulsatile firing of excitatory and inhibitory neuromuscular inputs not only in proximal colon, but also distal colon, long before the propagating contraction invades the distal region. During propulsion, wavelet analysis revealed increased coherence at ~2 Hz over large distances between the proximal and distal regions. Therefore, during propulsion, synchronous firing of descending inhibitory nerve pathways over long ranges aborally acts to suppress smooth muscle from contracting, counteracting the excitatory nerve pathways over this same region of colon. This delays muscle contraction downstream, ahead of the advancing contraction. The mechanism identified is more complex than expected and vastly different from fluid propulsion along other hollow smooth muscle organs; like lymphatic vessels, portal vein, or ureters, that evolved without intrinsic neurons.

A Singular Molecule-to-Molecule Transformation on Video: The Bottom-Up Synthesis of Fullerene C60 from Truxene Derivative C60H30.

Singular reaction events of small molecules and their dynamics remain a hardly understood territory in chemical sciences since spectroscopy relies on ensemble-averaged data, and microscopic scanning probe techniques show snapshots of frozen scenes. Herein, we report on continuous high-resolution transmission electron microscopic video imaging of the electron-beam-induced bottom-up synthesis of fullerene C60 through cyclodehydrogenation of tailor-made truxene derivative 1 (C60H30), which was deposited on graphene as substrate. During the reaction, C60H30 transformed in a multistep process to fullerene C60. Hereby, the precursor, metastable intermediates, and the product were identified by correlations with electron dose-corrected molecular simulations and single-molecule statistical analysis, which were substantiated with extensive density functional theory calculations. Our observations revealed that the initial cyclodehydrogenation pathway leads to thermodynamically favored intermediates through seemingly classical organic reaction mechanisms. However, dynamic interactions of the intermediates with the substrate render graphene as a non-innocent participant in the dehydrogenation process, which leads to a deviation from the classical reaction pathway. Our precise visual comprehension of the dynamic transformation implies that the outcome of electron-beam-initiated reactions can be controlled with careful molecular precursor design, which is important for the development and design of materials by electron beam lithography.

Application Research of Key Frames Extraction Technology Combined with Optimized Faster R-CNN Algorithm in Traffic Video Analysis

The intelligent transportation system under the big data environment is the development direction of the future transportation system. It effectively integrates advanced information technology, data communication transmission technology, electronic sensing technology, control technology, and computer technology and applies them to the entire ground transportation management system to establish a real-time, accurate, and efficient comprehensive transportation management system that works on a large scale and in all directions. Intelligent video analysis is an important part of smart transportation. In order to improve the accuracy and time efficiency of video retrieval schemes and recognition schemes, this article firstly proposes a segmentation and key frame extraction method for video behavior recognition, using a multi-time scale dual-stream network to extract video features, improving the efficiency and efficiency of video behavior detection. On this basis, an improved algorithm for vehicle detection based on Faster R-CNN is proposed, and the Faster R-CNN network feature extraction layer is improved by using the principle of residual network, and a hole convolution is added to the network to filter out the redundant features of high-resolution video images to improve the problem of vehicle missed detection in the original algorithm. The experimental results show that the key frame extraction technology combined with the optimized Faster R-CNN algorithm model greatly improves the accuracy of detection and reduces the leakage. The detection rate is satisfactory.

Laparoscopic system for simultaneous high-resolution video and rapid hyperspectral imaging in the visible and near-infrared spectral range.

.Significance: Hyperspectral imaging (HSI) can support intraoperative perfusion assessment, the identification of tissue structures, and the detection of cancerous lesions. The practical use of HSI for minimal-invasive surgery is currently limited, for example, due to long acquisition times, missing video, or large set-ups.Aim: An HSI laparoscope is described and evaluated to address the requirements for clinical use and high-resolution spectral imaging.Approach: Reflectance measurements with reference objects and resected human tissue from 500 to 1000 nm are performed to show the consistency with an approved medical HSI device for open surgery. Varying object distances are investigated, and the signal-to-noise ratio (SNR) is determined for different light sources.Results: The handheld design enables real-time processing and visualization of HSI data during acquisition within 4.6 s. A color video is provided simultaneously and can be augmented with spectral information from push-broom imaging. The reflectance data from the HSI system for open surgery at 50 cm and the HSI laparoscope are consistent for object distances up to 10 cm. A standard rigid laparoscope in combination with a customized LED light source resulted in a mean SNR of 30 to 43 dB (500 to 950 nm).Conclusions: Compact and rapid HSI with a high spatial- and spectral-resolution is feasible in clinical practice. Our work may support future studies on minimally invasive HSI to reduce intra- and postoperative complications.

DEEP LEARNING BASED OPTICAL FLOW ESTIMATION FOR CHANGE DETECTION: A CASE STUDY IN INDONESIA EARTHQUAKE

Abstract. Real-time change detection and analysis of natural disasters is of great importance to emergency response and disaster rescue. Recently, a number of video satellites that can record the whole process of natural disasters have been launched. These satellites capture high resolution video image sequences and provide researchers with a large number of image frames, which allows for the implementation of a rapid disaster procedure change detection approach based on deep learning. In this paper, pixel change in image sequences is estimated by optical flow based on FlowNet 2.0 for quick change detection in natural disasters. Experiments are carried out by using image frames from Digital Globe WorldView in Indonesia Earthquake took place on Sept. 28, 2018. In order to test the efficiency of FlowNet 2.0 on natural disaster dataset, 7 state-of-the-art optical flow estimation methods are compared. The experimental results show that FlowNet 2.0 is not only robust to large displacements but small displacements in natural disaster dataset. Two evaluation indicators: Root Mean Square Error (RMSE) and Mean Value are used to record the accuracy. For estimation error of RMSE, FlowNet 2.0 achieves 0.30 and 0.11 pixels in horizontal and vertical direction, respectively. The error in horizontal error is similar to other algorithms but the value in vertical direction is significantly lower than them. And the Mean Value are 1.50 and 0.09 pixels in horizontal and vertical direction, which are most close to the ground truth comparing to other algorithms. Combining the superiority of computing time, the paper proves that only the approach based on FlowNet 2.0 is able to achieve real-time change detection with higher accuracy in the case of natural disasters.

Powder bed irregularity and hot-spot detection in electron beam melting by means of in-situ video imaging

The electron beam melting process has been successfully applied in various sectors to produce high-value-added products. Being a hot process operating in vacuum environment with x-rays and material vaporization among by-products, in-situ sensing and monitoring presents more challenges than in laser powder bed fusion. However, an automated and robust detection of unstable process conditions represents a key capability to meet challenging qualification requirements imposed by industry. This study presents novel in-situ monitoring methods based on high spatial resolution imaging for powder bed homogeneity monitoring and high temporal resolution video-imaging for hot-spot detection, i.e., the detection of anomalous local heat accumulations.

A Technique for Panorama-Creation using Multiple Images

Image stitching, which is a process of integration of multiple images to create a panoramic image using all contents fitted into one frame, finds wide-spread applications in medical, high resolution digital map, satellite and video imaging. This paper proposes a framework to develop panorama image with multiple images. The framework is an automatic process that takes multiple images, checks correlation of the sequential images and removes overlapping area if exists and creates the panorama.We have done experimentations using different image-sets consisting multiple images with and without overlapping and got satisfactory results.

Multi-Semi-Couple Super-Resolution Method for Edge Computing

Video analyses based on edge computing typically need high-resolution video images, while, in practice, the resolutions of captured video images may not high enough. Thus, super-resolution techniques are possible solutions for such purpose with input low-resolution images. Sparse-coding-based super-resolution methods are well known for their efficiency. However, the current sparse-coding-based methods suffer from two major problems. First, the sparse coefficient of a low-resolution patch is assumed to be same as the sparse coefficient of its corresponding high-resolution patch, which is too strict to deal with various patterns; and second, the current methods only learn one pair of high-resolution and low-resolution dictionaries, while, since patches in image are of diversity in real world, it is difficult to use only one pair of dictionaries to cover all possible patches. In this paper, to overcome these two issues, we propose a super-resolution method to: 1) relax the assumption by linearizing the sparse coefficient of a low-resolution patch to that of high-resolution patch and 2) minimize super-resolution errors by jointly partitioning training patches into several clusters and learning dictionaries. Experimental results validate that our algorithm achieve more faithful reconstructions.

Determination of Shift/Bias in Digital Aerial Triangulation of UAV Imagery Sequences

Currently UAV Photogrammetry is characterized a largely automated and efficient data processing. Depicting from the low altitude more often gains on the meaning in the uses of applications as: cities mapping, corridor mapping, road and pipeline inspections or mapping of large areas e.g. forests. Additionally, high-resolution video image (HD and bigger) is more often use for depicting from the low altitude from one side it lets deliver a lot of details and characteristics of ground surfaces features, and from the other side is presenting new challenges in the data processing. Therefore, determination of elements of external orientation plays a substantial role the detail of Digital Terrain Models and artefact-free ortophoto generation. Parallel a research on the quality of acquired images from UAV and above the quality of products e.g. orthophotos are conducted. Despite so fast development UAV photogrammetry still exists the necessity of accomplishment Automatic Aerial Triangulation (AAT) on the basis of the observations GPS/INS and via ground control points. During low altitude photogrammetric flight, the approximate elements of external orientation registered by UAV are burdened with the influence of some shift/bias errors. In this article, methods of determination shift/bias error are presented. In the process of the digital aerial triangulation two solutions are applied. In the first method shift/bias error was determined together with the drift/bias error, elements of external orientation and coordinates of ground control points. In the second method shift/bias error was determined together with the elements of external orientation, coordinates of ground control points and drift/bias error equals 0. When two methods were compared the difference for shift/bias error is more than ±0.01 m for all terrain coordinates XYZ.

Insights into the mechanisms underlying colonic motor patterns.

In recent years there have been significant technical and methodological advances in our ability to record the movements of the gastrointestinal tract. This has led to significant changes in our understanding of the different types of motor patterns that exist in the gastrointestinal tract (particularly the large intestine) and in our understanding of the mechanisms underlying their generation. Compared with other tubular smooth muscle organs, a rich variety of motor patterns occurs in the large intestine. This reflects a relatively autonomous nervous system in the gut wall, which has its own unique population of sensory neurons. Although the enteric nervous system can function independently of central neural inputs, under physiological conditions bowel motility is influenced by the CNS: if spinal pathways are disrupted, deficits in motility occur. The combination of high resolution manometry and video imaging has improved our knowledge of the range of motor patterns and provided some insight into the neural and mechanical factors underlying propulsion of contents. The neural circuits responsible for the generation of peristalsis and colonic migrating motor complexes have now been identified to lie within the myenteric plexus and do not require inputs from the mucosa or submucosal ganglia for their generation, but can be modified by their activity. This review will discuss the recent advances in our understanding of the different patterns of propagating motor activity in the large intestine of mammals and how latest technologies have led to major changes in our understanding of the mechanisms underlying their generation.

Realization of CUDA-based real-time registration and target localization for high-resolution video images

High-resolution video images contain huge amount of data so that the real-time capability of image registration and target localization algorithm is difficult to be achieved when operated on central processing units (CPU). In this paper, improved ORB (Oriented FAST and Rotated BRIEF, FAST, which means “Features from Accelerated Segment Test”, is a corner detection method used for feature points extraction. BRIEF means “Binary Robust Independent Elementary Features”, and it’s a binary bit string used to describe features) based real-time image registration and target localization algorithm for high-resolution video images is proposed. We focus on the parallelization of three of the most time-consuming parts: improved ORB feature extraction, feature matching based on Hamming distance for matching rough points, and Random Sample Consensus algorithm for precise matching and achieving transformation model parameters. Realizing Compute Unified Device Architecture (CUDA)-based real-time image registration and target localization parallel algorithm for high-resolution video images is also emphasized on. The experimental results show that when the registration and localization effect is similar, image registration and target localization algorithm for high-resolution video images achieved by CUDA is roughly 20 times faster than by CPU implementation, meeting the requirement of real-time processing.

Crowdsource and web-published videos for 3D documentation of cultural heritage objects

Abstract Presently, the techniques of automated image-based modelling and camera technology are an efficient tool for cultural heritage documentation and preservation. One option is to use high-resolution video imaging to have 3D models of architectures, statues and monuments. Video imaging is a preferred choice when compared to still image shooting in modeling techniques because the latter needs expertise and a thorough planning. For cultural heritage documentation, crowdsourcing stock photos and images was investigated and showed some promising results. On the other hand, there is a limited number of tries to use the publicly published video imaging for the same purpose of 3D documentation. This limitation is caused by the low resolution of the published video images, the need to process a large number of video images and the effects of blur on a significant number of images. An efficient procedure is demonstrated in this case study paper to use the documentary web-published video images for 3D documentation of cultural heritage objects. Three different web-published videos with a high definition HD resolution are used to create 3D models of the Babylon Lion statue in Iraq (605 BC), the Aphrodite–Venus statue of the British Museum and the carillon tower of Virginia in USA (1926). The developed video based models show suitability for visualization, preservation, virtual museums and for mid-detailed documentation.

Capturing and displaying microscopic images used in medical diagnostics and forensic science using 4K video resolution - an application in higher education.

To analyze, interpret and evaluate microscopic images, used in medical diagnostics and forensic science, video images for educational purposes were made with a very high resolution of 4096 × 2160 pixels (4K), which is four times as many pixels as High-Definition Video (1920 × 1080 pixels). The unprecedented high resolution makes it possible to see details that remain invisible to any other video format. The images of the specimens (blood cells, tissue sections, hair, fibre, etc.) are recorded using a 4K video camera which is attached to a light microscope. After processing, this resulted in very sharp and highly detailed images. This material was then used in education for classroom discussion. Spoken explanation by experts in the field of medical diagnostics and forensic science was also added to the high-resolution video images to make it suitable for self-study.

Pharmacological characterization of an antisense knockdown zebrafish model of Dravet syndrome: inhibition of epileptic seizures by the serotonin agonist fenfluramine.

Dravet syndrome (DS) is one of the most pharmacoresistant and devastating forms of childhood epilepsy syndromes. Distinct de novo mutations in the SCN1A gene are responsible for over 80% of DS cases. While DS is largely resistant to treatment with existing anti-epileptic drugs, promising results have been obtained in clinical trials with human patients treated with the serotonin agonist fenfluramine as an add-on therapeutic. We developed a zebrafish model of DS using morpholino antisense oligomers (MOs) targeting scn1Lab, the zebrafish ortholog of SCN1A. Zebrafish larvae with an antisense knockdown of scn1Lab (scn1Lab morphants) were characterized by automated behavioral tracking and high-resolution video imaging, in addition to measuring brain activity through local field potential recordings. Our findings reveal that scn1Lab morphants display hyperactivity, convulsive seizure-like behavior, loss of posture, repetitive jerking and a myoclonic seizure-like pattern. The occurrence of spontaneous seizures was confirmed by local field potential recordings of the forebrain, measuring epileptiform discharges. Furthermore, we show that these larvae are remarkably sensitive to hyperthermia, similar to what has been described for mouse models of DS, as well as for human DS patients. Pharmacological evaluation revealed that sodium valproate and fenfluramine significantly reduce epileptiform discharges in scn1Lab morphants. Our findings for this zebrafish model of DS are in accordance with clinical data for human DS patients. To our knowledge, this is the first study demonstrating effective seizure inhibition of fenfluramine in an animal model of Dravet syndrome. Moreover, these results provide a basis for identifying novel analogs with improved activity and significantly milder or no side effects.