Spatio-temporal Signal Processing Research Articles

Thin-film transistor for temporal self-adaptive reservoir computing with closed-loop architecture.

Reservoir computing is a powerful neural network-based computing paradigm for spatiotemporal signal processing. Recently, physical reservoirs have been explored based on various electronic devices with outstanding efficiency. However, the inflexible temporal dynamics of these reservoirs have posed fundamental restrictions in processing spatiotemporal signals with various timescales. Here, we fabricated thin-film transistors with controllable temporal dynamics, which can be easily tuned with electrical operation signals and showed excellent cycle-to-cycle uniformity. Based on this, we constructed a temporal adaptive reservoir capable of extracting temporal information of multiple timescales, thereby achieving improved accuracy in the human-activity-recognition task. Moreover, by leveraging the former computing output to modify the hyperparameters, we constructed a closed-loop architecture that equips the reservoir computing system with temporal self-adaptability according to the current input. The adaptability is demonstrated by accurate real-time recognition of objects moving at diverse speed levels. This work provides an approach for reservoir computing systems to achieve real-time processing of spatiotemporal signals with compound temporal characteristics.

Spatiotemporal diffractive deep neural networks.

A spatiotemporal diffractive deep neural network (STD2NN) is proposed for spatiotemporal signal processing. The STD2NN is formed by gratings, which convert the signal from the frequency domain to the spatial domain, and multiple layers consisting of spatial lenses and space light modulators (SLMs), which conduct spatiotemporal phase modulation. An all-optical backpropagation (BP) algorithm for SLM phase tuning is proposed, with the gradient of the loss function computed by the inner product of the forward propagating optical field and the backward propagating conjugated error field. As a proof of concept, a spatiotemporal word "OPTICA" is generated by the STD2NN. Afterwards, a spatiotemporal optical vortex (STOV) beam multiplexer based on the STD2NN is demonstrated, which converts the spatially separated Gaussian beams into the STOV wave-packets with different topological charges. Both cases illustrate the capability of the proposed STD2NN to generate and process the spatiotemporal signals.

High consistency VO2 memristor for artificial auditory neuron

With scalability and diverse device behavior, memristors present potential for executing neuromorphic computation with lower hardware cost and power consumption. However, Low consistency currently limited the application of memristors. Herein, we report a VO2-based memristor fabricated through magnetron sputtering and multiple annealing processes, exhibiting extremely low in both cycle-to-cycle (C2C) and device-to-device (D2D) variations. Further, a Hodgkin-Huxley model neuron circuit based on the extremely high consistency of the devices is established, which achieves auditory neuron perception simulation through spatiotemporal processing of spike signals, allowing for clear differentiation of sound source direction and displaying patterns akin to biological behavior. This easily implemented and highly consistent artificial neuron offers a promising approach for the development of next-generation artificial auditory systems.

Reconfigurable Spatiotemporal Optical Signal Processors

AbstractAnalogue optical computing offers a promising approach for developing super‐fast computation techniques, benefiting from its extraordinary parallel processing capability, low energy consumption, and high integrability compared to digital electronic technologies. In this work, a reconfigurable spatiotemporal optical signal processor using the phase‐change material vanadium dioxide (VO2) to further expand the processing bandwidth and enrich the operation functions is demonstrated. As a specific example, a meta‐device is designed that can switch between a first‐order differentiator and a good reflector in the spatiotemporal domain by controlling the operation temperature. It is shown that the spatiotemporal transfer functions can be precisely engineered to meet the requirements for ideal first‐order differentiation and bright‐field imaging. This approach enables high‐accuracy spatial and temporal signal processing, as demonstrated by inspecting Gaussian signals. The theoretical evaluation determines the resolution limits of spatial and temporal edge detection. It also highlights that the device can perform spatiotemporal signal processing for pulses with complex profiles. The presented reconfigurable spatiotemporal optical signal processor holds great potential in applications such as remote sensing and fast imaging processing, among others.

Mathematical model of spatial and temporal processing of broadband signals in satellite radio systems of broadband access and radio navigation

To effectively receive a useful broadband signal, it is necessary to compensate for distortions caused by the complex frequency-dependent coefficient of the receiving and transmitting antenna, multiplicative interference and noise from the heterogeneity of the propagation medium, additive interference and noise. Using only correlation processing under certain conditions will not allow receiving a useful broadband signal with the required quality. The article develops a mathematical model of spatio-temporal processing of broadband signals in satellite broadband access systems, showing that spatio-temporal processing should be carried out in two stages. At the first stage, space-time processing is carried out in the channel adaptively to the antenna array based on the formation of a complex frequency-dependent vector of weighting coefficients, which should change with a change in the signal-interference situation in real time, taking into account the change in the direction of radiation sources. At the second stage, correlation processing of the broadband signal is performed based on the use of long-length code sequences. This model is the basis for the development of methods for the formation and correlation (temporal) processing of broadband signals and methods of spatio-temporal processing of broadband signals under conditions of intentional and unintentional exposure. Based on the presented mathematical model of spatio-temporal processing of broadband signals, the main directions of increasing the noise immunity of radio-electronic systems are determined.

ПРОБЛЕМА ОПТИМАЛЬНОСТІ АДАПТИВНИХ РАДІОТЕХНІЧНИХ СИСТЕМ В СИТУАЦІЇ З НЕКЛАСИФІКОВАНОЮ НАВЧАЛЬНОЮ ВИБІРКОЮ СПОСТЕРЕЖЕНЬ

Statistical synthesis of adaptive systems for spatiotemporal signal processing in most radio engineering problems occurs under the condition of parametric a priori uncertainty. The prospective direction of overcoming parametric a priori uncertainty is affiliated with the prior learning procedure. As a result of the training, sufficient statistics of likelihood ratios are formed, which are used in decision-making. The asymptotic optimality of adaptive systems with alternative benchmarks, one of which is peculiar to the class of radio structures with an adaptive antenna array and the other with an adaptive interference compensator, is studied subjectively with unclassified training sampling. The optimality of the system is determined by the criterion of maximum signal-to-interference ratio. The energy parameter representing the signal-to-interference ratio at the output of adaptive radio systems with a generalized reference was chosen as an indicator of optimality. As a result of the research, analytical expressions for the estimation and comparative analysis of energy losses at the output of adaptive systems with different standards under conditions of classified and unclassified learning were obtained. The invariance of the system with an adaptive antenna array to any type of training with preservation of the asymptotic properties of the radio system in the situation with a finite size of the training sample is proved. This feature is considered as a basis for factorization of the transfer function of the adaptive system. This makes it possible to form an optimal hierarchical algorithm for compensation of combined passive and active interference, as well as active interference with an arbitrary spatial power spectrum. The obligatory classification of a training sample of observations for radio engineering systems with an adaptive interference compensator under conditions of signal-interference a priori uncertainty is substantiated. Keywords: asymptotic optimality, adaptive system, alternative benchmark, unclassified training sample, correlation matrix of observations, energy index.

A memristor-based analogue reservoir computing system for real-time and power-efficient signal processing

Reservoir computing offers a powerful neuromorphic computing architecture for spatiotemporal signal processing. To boost the power efficiency of the hardware implementations of reservoir computing systems, analogue devices and components—including spintronic oscillators, photonic modules, nanowire networks and memristors—have been used to partially replace the elements of fully digital systems. However, the development of fully analogue reservoir computing systems remains limited. Here we report a fully analogue reservoir computing system that uses dynamic memristors for the reservoir layer and non-volatile memristors for the readout layer. The system can efficiently process spatiotemporal signals in real time with three orders of magnitude lower power consumption than digital hardware. We illustrate the capabilities of the system using temporal arrhythmia detection and spatiotemporal dynamic gesture recognition tasks, achieving accuracies of 96.6% and 97.9%, respectively. Our memristor-based fully analogue reservoir computing system could be of use in edge computing applications that require extremely low power and hardware cost.

Statistical synthesis of aerospace radars structure with optimal spatio-temporal signal processing, extended observation area and high spatial resolution

Using the statistical theory of optimization of radio engineering systems the optimal method of coherent radar imaging of surfaces in airborne synthetic aperture radar with planar antenna arrays is developed. This method summarizes several modes of terrain observation and it is fully consistent with current trends in the development of cognitive radars with the possibilities of radiation pattern restructuring in space and adaptive reception of reflected signals. Possible modifications of the obtained optimal method for the operation of high-precision airborne radars with a wide swath are presented. The idea is to create a theoretical basis and lay the foundations for its practical application in solving a wide range of issues of statistical optimization of methods and algorithms for optimal spatiotemporal signal processing in cognitive radar systems for the formation of both high-precision and global radar images. To implement the idea, the article highlights the concept of statistical optimization of spatio-temporal processing of electromagnetic fields in on-board cognitive radar systems, which will be based on the synthesis and analysis of methods, algorithms and structures of radar devices for coherent imaging, the study of limiting errors in restoring the spatial distribution of the complex scattering coefficient, the synthesis of optimal feedback for receiver and transmitter adaptations in accordance with a priori information about the parameters of the objects of study, the area of observation and the existing sources of interference. Objective is to develop the theory and fundamentals of the technical implementation of airborne radar systems for the formation of high-precision radar images in an extended field of view from aerospace carriers. Tasks. To reach the objective it is necessary to solve following tasks:– formalize mathematical models of spatiotemporal stochastic radio signals and develop likelihood functional for observation equations in which the useful signal, receiver internal noise and interference radiation of anthropogenic objects are random processes;– to synthesize algorithms for optimal processing of spatio-temporal stochastic signals in multi-channel radar systems located on aerospace-based mobile platforms;- in accordance with the synthesized methods, to substantiate the block diagrams of their implementation;– obtain analytical expressions for the potential characteristics of the quality of radar imaging and determine the class of probing signals and space scanning methods necessary to perform various tasks of radar surveillance;‒ to confirm some of the theoretical results by simulation methods, in which to reveal the features of the technical implementation of aerospace remote sensing radar systems.

Computational Array Signal Processing via Modulo Non-Linearities

Conventionalliterature on array signal processing (ASP) is based on the “capture first, process later” philosophy and to this end, signal processing algorithms are typically decoupled from the hardware. This poses fundamental limitations because if the sensors result in information loss, the algorithms may no longer be able to achieve their guaranteed performance. In this paper, our goal is to overcome the barrier of information loss via sensor saturation and clipping. This is a significant problem in application areas including physiological monitoring and extra-terrestrial exploration where the amplitudes may be unknown or larger than the dynamic range of the sensor. To overcome this fundamental bottleneck, we propose “computational arrays” which are based on a co-design approach so that a collaboration between the sensor array hardware and algorithms can be harnessed. Our work is inspired by the recently introduced unlimited sensing framework. In this context, our computational arrays encode the high-dynamic-range information by folding the signal amplitudes, thus introducing a new form of information loss in terms of the modulo measurements. On the decoding front, we develop mathematically guaranteed recovery algorithms for spatio-temporal array signal processing tasks that include DoA estimation, beamforming and signal reconstruction. Numerical examples corroborate the applicability of our approach and pave a path for the development of novel computational arrays for ASP.

Freestanding multi-gate IZO-based neuromorphic transistors on composite electrolyte membranes

Brain-inspired neuromorphic computing would bring a breakthrough to the classical computing paradigm through its massive parallelism and potential low power consumption advantages. Introduction of flexibility may bring vitality to this area by expanding its application areas to such as wearable and implantable electronics. At present, the development of flexible neuromorphic devices makes it a choice with wide prospect for next-generation wearable artificial neuromorphic computing. In this study, a freestanding graphene oxide/polyvinyl alcohol composite solid electrolyte membrane is utilized as the gate dielectric and support material, and indium-zinc-oxide (IZO) neuromorphic transistors are fabricated on such membrane. Based on the in-plane gate modulation, many key synaptic plasticity behaviors have been successfully emulated, including excitatory postsynaptic current, paired-pulse facilitation, high-pass filtering, and spatiotemporal signal processing. Moreover, transition of the spiking logic and the superlinear and sublinear dendritic integration function are realized. Our results indicate that these freestanding IZO-based neuromorphic transistors may of great significance for future flexible anthropomorphic robots, wearable bionic perception.

Synthesis of the optimal algorithm and structure of contactless optical device for estimating the parameters of statistically uneven surfaces

The production of parts and (or) finished products in electronics, mechanical engineering and other industries is inextricably linked with the control of the accuracy and cleanliness of the processed surfaces. Currently existing meters of parameters of statistically uneven surfaces, both contact and non-contact have some disadvantages, as well as limitations due to methods and design features of measurement. Speckle interferometric methods for measuring parameters of statistically uneven surfaces make it possible to get away from some disadvantages inherent in existing methods and measurements. The use of methods of statistical radio engineering, methods of optimization of statistical solutions and estimates of parameters of predictive distributions for optimal radio engineering system synthesis is promising for the analysis and processing optical-electronic coherent laser space-time signals (speckle images) form with the laser radiation scattered by statistically uneven surfaces. This work synthesizes the optimal algorithm and structure for analyzing the parameters of statistically-temporal surfaces based on spatio-temporal processing of optical speckle interference signals and images using modern methods of optimal synthesis of radio engineering and coherent optoelectronic systems. In this work, an algorithm for processing optical signals scattered by statistically uneven surfaces is synthesized and investigated for problems of optimal estimation of parameters and statistical characteristics of statistically uneven surfaces. A block diagram of the optical contactless device for evaluating the parameters of statistically uneven surfaces is proposed. The limiting errors of estimation parameters of statistically uneven surfaces and the optimal installation angles of the emitters and the optical receiver are investigated. Equations are obtained for estimating the root-mean-square height of the ridges and the correlation radius of small-scale statistically uneven surfaces in the approximation of small perturbations. The proposed method for evaluating the parameters of statistically uneven surfaces allows to increase the accuracy of measurements, to conduct a non-contact assessment of the parameters - even statistically uneven surfaces that have geometric surface irregularities or located in hard-to-reach places, for example, grooves, holes, as well as products of cylindrical, spherical and other shapes.

Switchable and simultaneous spatiotemporal analog computing with computational graphene-based multilayers

In the past few years, analog computing has experienced rapid development but mostly for a single function. Motivated by parallel space-time computing and miniaturization, we show that reconfigurable graphene-based multilayers offer a promising path towards spatiotemporal computing with integrated functionalities by properly engineering both spatial- and temporal-frequency responses. This paper employs a tunable graphene-based multilayer to enable analog signal and image processing in both space and time by tuning the external bias. In the first part of the paper, we propose a switchable analog computing paradigm in which the proposed multilayer can switch among defined performances by selecting a proper external voltage for graphene monolayers. Spatial isotropic differentiation and edge detection in the spatial channel and first-order temporal differentiation and multilayer-based phaser with linear group-delay response in the temporal channel are demonstrated. In the second section of the paper, simultaneous and parallel spatiotemporal analog computing is demonstrated. The proposed multilayer processor has almost no static power consumption due to its floating-gate configuration. The spatial- and temporal-frequency transfer functions (TFs) are engineered using a transmission line (TL) model, and the obtained results are validated with full-wave simulations. Our proposal will enable real-time parallel spatiotemporal analog signal and image processing.

Energy Efficient In-Memory Hyperdimensional Encoding for Spatio-Temporal Signal Processing

The emerging brain-inspired computing paradigm known as hyperdimensional computing (HDC) has been proven to provide a lightweight learning framework for various cognitive tasks compared to the widely used deep learning-based approaches. Spatio-temporal (ST) signal processing, which encompasses biosignals such as electromyography (EMG) and electroencephalography (EEG), is one family of applications that could benefit from an HDC-based learning framework. At the core of HDC lie manipulations and comparisons of large bit patterns, which are inherently ill-suited to conventional computing platforms based on the von-Neumann architecture. In this work, we propose an architecture for ST signal processing within the HDC framework using predominantly in-memory compute arrays. In particular, we introduce a methodology for the in-memory hyperdimensional encoding of ST data to be used together with an in-memory associative search module. We show that the in-memory HDC encoder for ST signals offers at least 1.80x energy efficiency gains, 3.36x area gains, as well as 9.74x throughput gains compared with a dedicated digital hardware implementation. At the same time it achieves a peak classification accuracy within 0.04% of that of the baseline HDC framework.

Spatiotemporal signal processing and device stability based on bi-layer biomimetic memristor

The neural network under the current computer architecture is difficult to process complex data efficiently, thus becoming one of the bottlenecks restricting the development of artificial intelligence technology. The human brain has the characteristics of high efficiency, low power consumption and integration of memory and computing, and is regarded as a most potential computing system to break the traditional von Neumann computing system. Synaptic biomimetic device is to realize the neural mimicry of human brain from the hardware level. It can simulate the information processing mode of brain nerve, that is, the process of “memory” and “calculation” can be realized on the same device, which is of great significance in building a new computing system. In recent years, the fabrication of memristor materials for bio-mimetic synaptic devices has made progress, but most of them focus on the simulation of synaptic function. The key research of pulse signal perception and information transmission is relatively lacking. In this paper, an bi-layer memristor with structure Al/nc-Al AlN/A<sub>2</sub>O<sub>3</sub>/Ag is fabricated by rf sputtering method to realize the basic functions of bionic synaptic devices. It is found that this bio-mimetic memristor exhibits bipolar switching property which is the basic condition to produce memristor based neural synapse. Both of PPF and PPD process can be observed and there will be no firing signal observed if the pulse interval is as large as 350 ms. The change of device conductance should be related to pulse voltage, frequency and pulse number applied. The larger pulse voltage, frequency and number will cause device conductance to increase sharply in both positive and negative pulse voltage region. The STDP measurement is executed with different sequence pulses from post and previous neuron separately. If the pulse of previous synapse comes in front of pulse from post synapse, the conductance will increase, which is so-called LTP process. If the pulse of previous neuron comes behind of pulse from post neuron, the conductance will be reduced as well. Triplet STDP measurement is executed with at least three pulses from previous and post neuron at the meanwhile. It is concluded that if the interval time of the first two pulses is fixed, the device conductance more depends on the value of the second and third pulse interval. Ebbinghaus forgetting curve can be used to explain the reason why the device conductance declines with time going by. The stability study of this memristor includes endurance and retention properties at both room and high temperature. It is found this biomimetic memristor can maintain its conductance for over 115.7 days at 85 ℃, which is long enough for current neural network design.

Flexible electro-optical neuromorphic transistors with tunable synaptic plasticity and nociceptive behavior

An electro-optical modulation synaptic device is proposed; it can emulate both brain-like processing and nervous perception functions by using a special nanoparticle-based conductive channel for the first time. In an electrical-processing synapse (EPS), ion diffusion behaviors during point contact and plane contact showed short-term plasticity with different gains and various synaptic functions such as paired-pulse facilitation, memory enhancement, spatiotemporal signal processing, and fault-tolerant behavior of logic operations. The EPS array is also mechanically flexible; the postsynaptic current retained 41.8% of the initial value after 10,000 bends, which represents the largest number of bending cycles of a flexible synaptic transistor so far. As a nervous optical sensing-processing synapse (OSPS), it can respond to ultraviolet light of different intensities and durations under sensitive millivolt reading voltage and also act as nociceptors to realize the pain perception. The ability of synaptic device to adjust their responses is important to connect brain-like devices to an artificial nervous system, so this electro-optical synaptic unit is a promising candidate for use in neuromorphic and flexible electronics, to combine brain-like processing and artificial sensory nerves.

METHOD FOR SELECTING A CLASSIFIED TRAINING SAMPLE FOR RADAR’S ELEVATION CHANNELS

Context. Under conditions of simultaneous exposure to active and passive interference, the passive component decorrelates the active component of the combined interference in the receiving channels of the surveillance radar, which significantly reduces the effectiveness of its suppression. Therefore, in order to adapt the weight coefficients of the spatial filter during spatio-temporal processing of radar signals, it is necessary, by one method or another, to form a classified training sample generated only by active noise interference.Objective. The goal is to optimize the choice of a method for generating a classified training sample for various elevation channels of the radar field of view, taking into account the predicted distribution of passive interference in range.Method. An analytical assessment of the distribution of the most powerful Cb type cumulonimbus cloud systems, followed by mathematical modeling of the effectiveness of the active noise suppression using various training samples generation methods.Results. Based on the performed analytical calculations, it was shown that in the upper part of the field of view of surveillance radar in the sector of elevation angles from 6º to 30º, the upper boundary of Cb type clouds does not exceed 75 km, which guarantees the absence of passive interference at the end of the radar range. This makes it possible to form a training sample at a time interval located before the radiation of the next probe pulse. In the lower part of the field of view in the sector of angles 0º to 6º, the upper boundary of the clouds is located at a distance of 75–270 km, which does not allow the formation of a training sample at the end of the radar range. It is proposed to use the correlation method, in which the time interval for the formation of the training sample is determined by the “on the fly” analysis of the magnitude of the inter-channel correlation coefficient. The choice of the interval with the highest value of inter-channel correlation during the repetition period allows us to form a training sample with the least influence of passive interference on the process of suppressing the active component of the combined interference. As a result of modeling, it was found that the use of a training sample significantly reduces the duration of the transient process when forming the weight coefficients of the spatial filter and, accordingly, improves the quality of the suppression of active interference.Conclusions. The scientific novelty of the work lies in the fact that for the first time the problem of individual selection of a classified training sample for the surveillance radar elevation channels has been solved. Practical significance consists in the development of a methodology for analyzing the location of the most powerful cloud systems in range in the radar field of view with varying degrees of influence of passive interference in elevation channels in a real signal-noise environment. To the practical novelty should also include the results of simulation modeling, which confirm the need to choose a method for generating a classified training sample in different elevation channels of the radar viewing area should also be attributed.

Measuring the radar cross section of objects with electrodynamically related elements by measuring system with synthesized aperture antenna

The problem of measuring the effective scattering surface (radar cross section) by systems with a synthesized antenna aperture is considered. Using the methods of the statistical theory of detection and spatio-temporal signal processing, the regularities of the formation of the signal component of the optimal filter characterizing the magnitude of the RCS of the measured object are established. The results of mathematical modeling are consistent with the obtained laws.

Synthesis of interference resistant spatiotemporal filter for navigation parameters high precision measurements using global navigation satellite systems

The article considers problems of modern global navigation satellite systems high-precision user terminals performance in jamming and spoofing environment. The solution to the low interference resistance problem is the use of digital antenna arrays with spatiotemporal signal processing algorithms. The well-known, most studied and brought to practice algorithms for the spatiotemporal signal processing are described, reasons that do not allow the use of these algorithms in global navigation satellite systems high-precision user terminals are given. This article proposes a spatiotemporal signal processing algorithm based on a spatiotemporal filter of finite length with a special, theoretically justified requirement for the Hermitian symmetry of the matrix impulse response, which guarantees the absence of signal distortion in any interference environment. In this case, the spatiotemporal filter impulse response is calculated by the criterion of optimal interference suppression. The proposed spatiotemporal filter characteristics, as well as other spatiotemporal signal processing algorithms characteristics, were studied by mathematical simulation with random enumeration of interference environment parameters (directions to signals, to numerous interferences and their reflections, range of interference reflectors, reflection phases, interference and reflections levels, etc.). Simulation results are presented in the form of distribution functions of signal-to-noise ratios at the output of spatiotemporal signal processing algorithms and distribution functions of phase and signal time biases. The obtained dependencies confirm the absence of phase and signal time biases in the spatiotemporal filter in absolutely any interference environment with interfering multipath, while the spatiotemporal filter provides greater interference resistance than compensating spatiotemporal signal processing algorithms.

Selecting the Pitch of a Double Crystal Transducer in Ultrasonic Testing of Products Made of Complex Structured Materials with a Nonplanar Surface

It is noted that the ultrasonic testing of products made of complex structured materials is based, as a rule, on spatiotemporal signal processing, using ultrasonic antenna arrays to extract useful echo signal from pattern noise. If the product has a nonplanar surface, one should use a synthesized aperture antenna, which is a double crystal transducer (DC PET) moved along the product surface in steps of d. The dependence of the cross-correlation of pattern-noise instances on the distance between two DC PET positions was established and used to determine the optimum (maximizing the signal–to–pattern-noise ratio) DC PET movement step d, depending on transducer’s aperture size.

ПОТЕНЦІЙНІ ХАРАКТЕРИСТИКИ ДВОАНТЕННИХ НАДШИРОКОСМУГОВИХ СИСТЕМ АПЕРТУРНОГО СИНТЕЗУ

For radiometric imaging (RMI) with high spatial resolution are used aperture synthesis systems. The basis cell of such system is the two-antennas radio interferometer. Optimal and quazioptimal algorithms for passive radar imaging by it radio interferometer with ultrawideband spatio-temporal signal processing are known. At the same time questions the limit estimation error of the RMI and the potential fluctuation sensitivity of such systems are not investigated. In the article for the first time the analytical expressions for the limit estimation error of RMI and the potential fluctuation sensitivity of two-antenna UWB aperture synthesis system that provides a high spatial resolution of the angular coordinates are obtained and investigated.