Short Image Sequences Research Articles

Comic Strip on Students’ Narrative Text Writing Ability

The aim of this research is to prove the effectiveness of comic strips on students' narrative text writing ability. Comic strips are sequential art associated with text with short sequences of images telling short stories. This research applied a pre-experimental method with one group pre-test and post-test design chosen as the research methodology. The eight-grade students in Islamic Middle Schools are taken as the population of this research. Thirty-six students were selected as the samples using the random sampling with the lottery system. The tests were given to measure their narrative writing ability, and the data were analyzed using the t-test formula. From the calculation, the researcher finds the result of the t-test value is 8,49 with the degree of freedom (df) of 35, and the t-table value at the significant level of 0.05 is 2.042. The result shows that the value of the t-test is higher than t-table. It means that the alternative hypothesis (ha) is accepted. Therefore, the researcher concludes that using comic strips positively affects students' ability in writing narrative texts.

Machine-Learning Approach to Non-Destructive Biomass and Relative Growth Rate Estimation in Aeroponic Cultivation

We train and compare the performance of two machine learning methods, a multi-variate regression network and a ResNet-50-based neural network, to learn and forecast plant biomass as well as the relative growth rate based onfrom a short sequence of temporal images from plants in aeroponic cultivation. The training dataset consists of images of 57 plants taken from two different angles every hour during a 5-day period. The results show that images taken from a top-down perspective produce better results for the multi-variate regression network, while images taken from the side are better for the ResNet-50 neural network. In addition, using images from both cameras improves the biomass estimates from the ResNet-50 network, but not those from the multivariatemulti-variatemultivariate regression. However, all relative growth rate estimates were improved by using images from both cameras. We found that the best biomass estimates are produced from the multi-variate regression model trained on top camera images using a moving average filter resulting in a root mean square error of 0.0466 g. The best relative growth rate estimates were produced from the ResNet-50 network training on images from both cameras resulting in a root mean square error of 0.1767 g/(g·day).

Learning to See Through Obstructions With Layered Decomposition

We present a learning-based approach for removing unwanted obstructions, such as window reflections, fence occlusions, or adherent raindrops, from a short sequence of images captured by a moving camera. Our method leverages motion differences between the background and obstructing elements to recover both layers. Specifically, we alternate between estimating dense optical flow fields of the two layers and reconstructing each layer from the flow-warped images via a deep convolutional neural network. This learning-based layer reconstruction module facilitates accommodating potential errors in the flow estimation and brittle assumptions, such as brightness consistency. We show that the proposed approach learned from synthetically generated data performs well to real images. Experimental results on numerous challenging scenarios of reflection and fence removal demonstrate the effectiveness of the proposed method.

A Dataset for Temporal Semantic Segmentation Dedicated to Smart Mobility of Wheelchairs on Sidewalks.

In smart mobility, the semantic segmentation of images is an important task for a good understanding of the environment. In recent years, many studies have been made on this subject, in the field of Autonomous Vehicles on roads. Some image datasets are available for learning semantic segmentation models, leading to very good performance. However, for other types of autonomous mobile systems like Electric Wheelchairs (EW) on sidewalks, there is no specific dataset. Our contribution presented in this article is twofold: (1) the proposal of a new dataset of short sequences of exterior images of street scenes taken from viewpoints located on sidewalks, in a 3D virtual environment (CARLA); (2) a convolutional neural network (CNN) adapted for temporal processing and including additional techniques to improve its accuracy. Our dataset includes a smaller subset, made of image pairs taken from the same places in the maps of the virtual environment, but from different viewpoints: one located on the road and the other located on the sidewalk. This additional set is aimed at showing the importance of the viewpoint in the result of semantic segmentation.

Conflict-Based Search for Explainable Multi-Agent Path Finding

The goal of the Multi-Agent Path Finding (MAPF) problem is to find non-colliding paths for agents in an environment, such that each agent reaches its goal from its initial location. In safety-critical applications, a human supervisor may want to verify that the plan is indeed collision-free. To this end, a recent work introduces a notion of explainability for MAPF based on a visualization of the plan as a short sequence of images representing time segments, where in each time segment the trajectories of the agents are disjoint. Then, the problem of Explainable MAPF via Segmentation asks for a set of non-colliding paths that admit a short-enough explanation. Explainable MAPF adds a new difficulty to MAPF, in that it is NP-hard with respect to the size of the environment, and not just the number of agents. Thus, traditional MAPF algorithms are not equipped to directly handle Explainable MAPF. In this work, we adapt Conflict Based Search (CBS), a well-studied algorithm for MAPF, to handle Explainable MAPF. We show how to add explainability constraints on top of the standard CBS tree and its underlying A* search. We examine the usefulness of this approach and, in particular, the trade-off between planning time and explainability.

Saliency Detection with Moving Camera via Background Model Completion.

Detecting saliency in videos is a fundamental step in many computer vision systems. Saliency is the significant target(s) in the video. The object of interest is further analyzed for high-level applications. The segregation of saliency and the background can be made if they exhibit different visual cues. Therefore, saliency detection is often formulated as background subtraction. However, saliency detection is challenging. For instance, dynamic background can result in false positive errors. In another scenario, camouflage will result in false negative errors. With moving cameras, the captured scenes are even more complicated to handle. We propose a new framework, called saliency detection via background model completion (SD-BMC), that comprises a background modeler and a deep learning background/foreground segmentation network. The background modeler generates an initial clean background image from a short image sequence. Based on the idea of video completion, a good background frame can be synthesized with the co-existence of changing background and moving objects. We adopt the background/foreground segmenter, which was pre-trained with a specific video dataset. It can also detect saliency in unseen videos. The background modeler can adjust the background image dynamically when the background/foreground segmenter output deteriorates during processing a long video. To the best of our knowledge, our framework is the first one to adopt video completion for background modeling and saliency detection in videos captured by moving cameras. The F-measure results, obtained from the pan-tilt-zoom (PTZ) videos, show that our proposed framework outperforms some deep learning-based background subtraction models by 11% or more. With more challenging videos, our framework also outperforms many high-ranking background subtraction methods by more than 3%.

Comparison of lock-in correlation and a novel periodogram method for experimental multi-harmonic thermoelastic analysis

Lock-in correlation is a simple, efficient and widely employed method for thermoelastic stress analysis under cyclic loading. Frequency domain analysis of the thermoelastic signal using the Fourier transform has been also valuable for researchers, even though it presents some drawbacks. In this paper, a new frequency-domain method for thermoelastic analysis through periodograms using Welch’s averaging method has been developed. This method supplies the lack of phase shift information of the simple Fourier transform for thermoelastic signals and considerably reduce the data size. This work evaluates its performance against lock-in correlation through two experiments with opposite features for multi-harmonic analysis. One experiment consisted in the analysis of the thermoelastic signal in a plate with a centred hole, exhibiting a stress concentration and using the second harmonic as an indicator of the non-linearities. The second experiment consisted in the characterisation of mode shapes thermal maps during the simultaneous excitation of multiple specimen resonances. In comparison with the stress concentrator experiment, here the level of the thermal signal is lower and the number of frequencies to analyse is higher. The analysis of the thermal maps by both methods revealed that they yield practically identical results for any experiment. However, the evaluation of the computation efficiency highlights that the periodogram method has fewer memory requirements whereas, in terms of computation time, lock-in correlation is more efficient to analyse fewer frequencies in short image sequence tests, being the periodogram method more efficient for the opposite case.

SeqNet: Learning Descriptors for Sequence-Based Hierarchical Place Recognition

Visual Place Recognition (VPR) is the task of matching current visual imagery from a camera to images stored in a reference map of the environment. While initial VPR systems used simple direct image methods or hand-crafted visual features, recent work has focused on learning more powerful visual features and further improving performance through either some form of sequential matcher / filter or a hierarchical matching process. In both cases the performance of the initial single-image based system is still far from perfect, putting significant pressure on the sequence matching or (in the case of hierarchical systems) pose refinement stages. In this paper we present a novel hybrid system that creates a high performance initial match hypothesis generator using short learnt sequential descriptors, which enable selective control sequential score aggregation using single image learnt descriptors. Sequential descriptors are generated using a temporal convolutional network dubbed SeqNet, encoding short image sequences using 1-D convolutions, which are then matched against the corresponding temporal descriptors from the reference dataset to provide an ordered list of place match hypotheses. We then perform selective sequential score aggregation using shortlisted single image learnt descriptors from a separate pipeline to produce an overall place match hypothesis. Comprehensive experiments on challenging benchmark datasets demonstrate the proposed method outperforming recent state-of-the-art methods using the same amount of sequential information. Source code and supplementary material can be found online. <xref ref-type="fn" rid="fn1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">¹</xref> <fn id="fn1" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><label>¹</label> [Online]. Available: <uri>https://github.com/oravus/seqNet</uri>. </fn>

Transformer guided geometry model for flow-based unsupervised visual odometry

Existing unsupervised visual odometry (VO) methods either match pairwise images or integrate the temporal information using recurrent neural networks over a long sequence of images. They are either not accurate, time-consuming in training or error accumulative. In this paper, we propose a method consisting of two camera pose estimators that deal with the information from pairwise images and a short sequence of images, respectively. For image sequences, a transformer-like structure is adopted to build a geometry model over a local temporal window, referred to as transformer-based auxiliary pose estimator (TAPE). Meanwhile, a flow-to-flow pose estimator (F2FPE) is proposed to exploit the relationship between pairwise images. The two estimators are constrained through a simple yet effective consistency loss in training. Empirical evaluation has shown that the proposed method outperforms the state-of-the-art unsupervised learning-based methods by a large margin and performs comparably to supervised and traditional ones on the KITTI and Malaga dataset.

Drone-Based Optical Measurements of Heterogeneous Surface Velocity Fields around Fish Passages at Hydropower Dams

In Austria, more than a half of all electricity is produced with the help of hydropower plants. To reduce their ecological impact, dams are being equipped with fish passages that support connectivity of habitats of riverine fish species, contributing to hydropower sustainability. The efficiency of fish passages is being constantly monitored and improved. Since the likelihood of fish passages to be discovered by fish depends, inter alia, on flow conditions near their entrances, these conditions have to be monitored as well. In this study, we employ large-scale particle image velocimetry (LSPIV) in seeded flow conditions to analyse images of the area near a fish passage entrance, captured with the help of a ready-to-fly consumer drone. We apply LSPIV to short image sequences and test different LSPIV interrogation area sizes and correlation methods. The study demonstrates that LSPIV based on ensemble correlation yields velocities that are in good agreement with the reference values regarding both magnitude and flow direction. Therefore, this non-intrusive methodology has a potential to be used for flow monitoring near fish passages on a regular basis, enabling timely reaction to undesired changes in flow conditions when possible.

Topological Sweep for Multi-Target Detection of Geostationary Space Objects

Conducting surveillance of the geocenric orbits is a key task towards achieving space situational awareness (SSA). Our work focuses on the optical detection of man-made objects (e.g., satellites, space debris) in Geostationary orbit (GEO), which is home to major space assets such as telecommunications and Earth observing satellites. GEO object detection is challenging due to the distance of the targets, which appear as small dim point-like objects among a background of streak-like objects. In this paper, we propose a novel multi-target detection technique based on topological sweep, to find GEO objects from a short sequence of optical images. Our topological sweep technique exploits the geometric duality that underpins the approximately linear trajectory of target objects across the sequence, to extract the targets from significant clutter and noise. Unlike standard multi-target methods, our algorithm deterministically solves a combinatorial problem to ensure high-recall rates without requiring accurate initializations. The usage of geometric duality also yields an algorithm that is computationally efficient and suitable for online processing.

VISION-BASED INDOOR LOCALIZATION VIA A VISUAL SLAM APPROACH

Abstract. With an increasing interest in indoor location based services, vision-based indoor localization techniques have attracted many attentions from both academia and industry. Inspired by the development of simultaneous localization and mapping technique (SLAM), we present a visual SLAM-based approach to achieve a 6 degrees of freedom (DoF) pose in indoor environment. Firstly, the indoor scene is explored by a keyframe-based global mapping technique, which generates a database from a sequence of images covering the entire scene. After the exploration, a feature vocabulary tree is trained for accelerating feature matching in the image retrieval phase, and the spatial structures obtained from the keyframes are stored. Instead of querying by a single image, a short sequence of images in the query site are used to extract both features and their relative poses, which is a local visual SLAM procedure. The relative poses of query images provide a pose graph-based geometric constraint which is used to assess the validity of image retrieval results. The final positioning result is obtained by selecting the pose of the first correct corresponding image.

Systematic approach to binary classification of images in video streams using shifting time windows

Multiple algorithms classifying frames in video sequences consider them only as separate images. After pointing out the properties of real-life recordings and classifications of their frames, we propose a new shifting time window approach for improving binary classifications. It proceeds in two steps: First, well-known classification algorithms are used separately for each frame to acquire preliminary classifications. Secondly, the results of the previous step are analyzed in relatively short sequences of consecutive images (the shifting time window). Taking into account the continuous nature of analyzed real-life videos, the preliminary binary classification sequences can be corrected. In consequence, the classification quality is improved. Furthermore, we offer a systematic approach where all parameters of the proposed algorithm (such as the window length or vote weight distribution in the window) are considered and their optimal values are determined. Experiments on representative examples confirm the advantages of the proposed approach.

Video-Based Estimation of Surface Currents Using a Low-Cost Quadcopter

Video imagery of surface waves recorded from a small off-the-shelf quadcopter with a self-stabilizing camera gimbal is analyzed to estimate the surface current field. The nadir looking camera acquires a short image sequence, which is geocoded to Universal Transverse Mercator coordinates. The resulting image sequence is used to quantify characteristic parameters (wavelength, period, and direction) of short (0.1–1 m) surface waves in space and time. This opens the opportunity to fit the linear dispersion relation to the data and thus monitor the frequency shift induced by an ambient current. The fitting is performed by applying a spectral energy-based maximization technique in the wavenumber–frequency domain. The current field is compared with measurements acquired by an acoustic Doppler current profiler mounted on a small boat, showing an overall good agreement. The root-mean-square error in current velocity is 0.09 m/s with no bias.

NETWORK DESIGN IN CLOSE-RANGE PHOTOGRAMMETRY WITH SHORT BASELINE IMAGES

Abstract. The avaibility of automated software for image-based 3D modelling has changed the way people acquire images for photogrammetric applications. Short baseline images are required to match image points with SIFT-like algorithms, obtaining more images than those necessary for “old fashioned” photogrammetric projects based on manual measurements. This paper describes some considerations on network design for short baseline image sequences, especially on precision and reliability of bundle adjustment. Simulated results reveal that the large number of 3D points used for image orientation has very limited impact on network precision.

False Discovery Rate Approach to Unsupervised Image Change Detection.

In this paper, we address the problem of unsupervised change detection on two or more coregistered images of the same object or scene at several time instants. We propose a novel empirical-Bayesian approach that is based on a false discovery rate formulation for statistical inference on local patch-based samples. This alternative error metric allows to efficiently adjust the family-wise error rate in case of the considered large-scale testing problem. The designed change detector operates in an unsupervised manner under the assumption of the limited amount of changes in the analyzed imagery. The detection is based on the use of various statistical features, which enable the detector to address application-specific detection problems provided an appropriate ad hoc feature choice. In particular, we demonstrate the use of the rank-based statistics: Wilcoxon and Cramér-von Mises for image pairs, and multisample Levene statistic for short image sequences. The experiments with remotely sensed radar, dermatological, and still camera surveillance imagery demonstrate accurate performance and flexibility of the proposed method.

A computational approach for obstruction-free photography

We present a unified computational approach for taking photos through reflecting or occluding elements such as windows and fences. Rather than capturing a single image, we instruct the user to take a short image sequence while slightly moving the camera. Differences that often exist in the relative position of the background and the obstructing elements from the camera allow us to separate them based on their motions, and to recover the desired background scene as if the visual obstructions were not there. We show results on controlled experiments and many real and practical scenarios, including shooting through reflections, fences, and raindrop-covered windows.

Digital stories as a method for evidence-based practice and knowledge co-creation in technology-enhanced learning for children with autism

Storytelling is a powerful means of expression especially for voices that may be difficult to hear or represent in typical ways. This paper reports and reflects on our experiences of co-creating digital stories with school practitioners in a project focusing on embedding innovative technologies for children on the autism spectrum in classroom practice. The digital stories were short films or narrated sequences of slides and images that conveyed key views about experiences and practices with or around the technologies. The creation of the digital stories aimed to empower schools and individual teachers to construct and share their own authentic narratives and to build case examples of creative technology-enhanced teaching and learning. Through focusing on our experiences with one of the schools, we examine the use of digital stories as a method for enabling knowledge co-creation with practitioners and we discuss the evidential potential of digital stories. We argue that the co-creation of digital stories enabled teachers to find their voice in critiquing the usability, usefulness, efficacy and flexibility of the technologies. Furthermore, the stories, both the process of their creation and the final artefacts, provided a concrete grounding for knowledge co-creation about teaching practices and authentic technology-enhanced learning.

Neural correlates of statistical learning in the inferior-temporal cortex of rhesus monkeys.

Event Abstract Back to Event Neural correlates of statistical learning in the inferior-temporal cortex of rhesus monkeys. Peter Kaposvari1*, Susheel Kumar1 and Rufin Vogels1 1 KU Leuven, Laboratorium Neuro - en Psychofysiologie, Belgium We learn to implicitly extract the statistical regularities in the environment around us, e.g. sequences of stimuli that follow each other (e.g. letters in specific words). In the visual domain, human observers can extract regular sequences of visual stimuli from a continuous stream of visual shapes. The neural correlates of such visual sequence learning are still unclear. Here, we determined whether a neural correlate of short sequences of visual images, presented in a continuous stream, is present in macaque inferior temporal (IT) cortex. Two monkeys passively fixated while a continuous stream of 3 fixed sequences of 5 grayscale images of animals of various species each were presented in a random order. After a passive exposure of two months, we assessed whether the responses of IT neurons showed a correlate of the sequences by introducing deviant stimuli in the fixed standard sets. We compared the multiunit neuronal responses for the deviant stimuli with identical stimuli when presented in the standard, learned sequence. We observed a significant enhancement of the responses to the deviant stimulus and the trailing standard stimuli, when the deviant was either from the same sequence (p <0.05 for each monkey) or from the other two sequences (p <0.05 for each monkey). The neuronal responses to a second deviant stimulus (which followed the first deviant in the standard set) did not differ significantly from the standard. These results demonstrate that monkey IT cortex encodes at least part of a learned temporal sequence in which a stimulus is presented. Acknowledgements This work was supported by Fonds voor Wetenschappelijk Onderzoek Vlaanderen (G.0582.12N), Interuniversitaire Attractiepool, Programma Financiering (PF 10/008), European Community’s Seventh Framework Programme FP7/2007-2013 under grant agreement number PITN-GA-2008-290011 and Odysseus grant (G.007.12N). Keywords: sequence learning, statistical learning, rhesus macaque, Inferotemporal cortex, single-unit Conference: 11th National Congress of the Belgian Society for Neuroscience, Mons, Belgium, 22 May - 22 May, 2015. Presentation Type: Poster presentation Topic: Neuroscience Citation: Kaposvari P, Kumar S and Vogels R (2015). Neural correlates of statistical learning in the inferior-temporal cortex of rhesus monkeys.. Front. Neurosci. Conference Abstract: 11th National Congress of the Belgian Society for Neuroscience. doi: 10.3389/conf.fnins.2015.89.00063 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 30 Apr 2015; Published Online: 05 May 2015. * Correspondence: PhD. Peter Kaposvari, KU Leuven, Laboratorium Neuro - en Psychofysiologie, Leuven, 3000, Belgium, Peter.Kaposvari@med.kuleuven.be Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Peter Kaposvari Susheel Kumar Rufin Vogels Google Peter Kaposvari Susheel Kumar Rufin Vogels Google Scholar Peter Kaposvari Susheel Kumar Rufin Vogels PubMed Peter Kaposvari Susheel Kumar Rufin Vogels Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

An autonomous image based approach for detecting glacial lake outburst floods

Abstract. The potential danger caused by glacier margin lakes and the related risk of glacier lake outburst floods (GLOF) increases constantly due to glaciers retreating in many parts of the world. Reasons for this development are on the one hand the new formation and enlargement of glacier margin lakes due to melt water. On the other hand, retreating and thinning glacier tongues lead to a decrease of the back pressure against the dammed glacier lakes. The paper describes the design of a photogrammetric GLOF monitoring system, based on monoscopic image sequence analysis for automatic detection of water level changes. The presented approach for measuring the water line in an image sequence is based on directional edge detection in LoG-filtered image data. After that, the water level is determined by a transformation of image measurements into object space based on orientation parameters of the camera and a geo-referenced lake basin model. The model can for instance be determined by photogrammetric methods after a GLOF; it may also be determined portion-wise by analysing shore lines at various water levels. Camera orientation parameters are determined by a local GPS-supported photogrammetric network. Comparing the determined water level changes with reference data provided by a water gauge, the precision is estimated in the order of one decimetre. A major challenge is the automatic detection of the water line in image sequences under varying light and visibility conditions. The paper will also discuss promising approaches such as multispectral images as well as a statistical analysis of grey value changes over short image sequences to eliminate disturbing reflections on the rough water surface.