Orientation Detection Research Articles

Endoscopic Image-Based Skill Assessment in Robot-Assisted Minimally Invasive Surgery.

Objective skill assessment-based personal performance feedback is a vital part of surgical training. Either kinematic—acquired through surgical robotic systems, mounted sensors on tooltips or wearable sensors—or visual input data can be employed to perform objective algorithm-driven skill assessment. Kinematic data have been successfully linked with the expertise of surgeons performing Robot-Assisted Minimally Invasive Surgery (RAMIS) procedures, but for traditional, manual Minimally Invasive Surgery (MIS), they are not readily available as a method. 3D visual features-based evaluation methods tend to outperform 2D methods, but their utility is limited and not suited to MIS training, therefore our proposed solution relies on 2D features. The application of additional sensors potentially enhances the performance of either approach. This paper introduces a general 2D image-based solution that enables the creation and application of surgical skill assessment in any training environment. The 2D features were processed using the feature extraction techniques of a previously published benchmark to assess the attainable accuracy. We relied on the JHU–ISI Gesture and Skill Assessment Working Set dataset—co-developed by the Johns Hopkins University and Intuitive Surgical Inc. Using this well-established set gives us the opportunity to comparatively evaluate different feature extraction techniques. The algorithm reached up to 95.74% accuracy in individual trials. The highest mean accuracy—averaged over five cross-validation trials—for the surgical subtask of Knot-Tying was 83.54%, for Needle-Passing 84.23% and for Suturing 81.58%. The proposed method measured well against the state of the art in 2D visual-based skill assessment, with more than 80% accuracy for all three surgical subtasks available in JIGSAWS (Knot-Tying, Suturing and Needle-Passing). By introducing new visual features—such as image-based orientation and image-based collision detection—or, from the evaluation side, utilising other Support Vector Machine kernel methods, tuning the hyperparameters or using other classification methods (e.g., the boosted trees algorithm) instead, classification accuracy can be further improved. We showed the potential use of optical flow as an input for RAMIS skill assessment, highlighting the maximum accuracy achievable with these data by evaluating it with an established skill assessment benchmark, by evaluating its methods independently. The highest performing method, the Residual Neural Network, reached means of 81.89%, 84.23% and 83.54% accuracy for the skills of Suturing, Needle-Passing and Knot-Tying, respectively.

Polarized Raman Spectroscopy for Determining Crystallographic Orientation of Low-Dimensional Materials.

Raman spectroscopy is a fast and nondestructive characterization technique, which has been widely used for the characterization of the composition and structure information of various materials. The symmetry-dependent Raman tensor allows the detection of crystallographic orientation of materials by using polarization information. In this Perspective, we discuss polarized Raman spectroscopy as a powerful tool for determination of the crystallographic orientation of various materials. First, we introduce the basic principles of polarized Raman spectroscopy and the corresponding experimental setups; the determination of crystallographic orientation of two-dimensional (2D) materials with in-plane isotropy and in-plane anisotropy using linearly polarized Raman scattering are then discussed. Furthermore, we discuss that using circularly polarized Raman spectroscopy, the azimuthal angle of materials in three dimensions (3D) can be characterized. In the final section, we show that the orientation distribution of nanomaterial assemblies can be measured using polarized Raman spectroscopy by introducing the orientation distribution function.

Device-Free Orientation Detection Based on CSI and Visibility Graph

Nonintrusive orientation detection is an important yet largely unaddressed area. It can be used in many important applications, e.g., interactive games, medical care, and various industrial scenarios. Moreover, our novel techniques can be readily implemented in other critical areas, such as indoor localization, objective tracking, movement detection, etc. Many factors can limit the performance of detection algorithms in real-word applications, an important one of which is the negligence of subcarrier correlations. Therefore, we propose to build our system based on existing WiFi infrastructure and its channel state information (CSI). To explore the correlations of adjacent subcarriers, we apply the visibility graph (VG)-based network analysis method to process the CSI data. Specifically, in this article we make the following contributions: 1) we design a CSI-based orientation detection system; 2) we model the correlations between subcarriers using complex network and propose a VG-based feature extraction technique; and 3) we demonstrate the performance and effectiveness of our system with commercial products in real-world deployments. The experimental results show that our technique can achieve more than 98% accuracy and at least 26% better than the baseline approaches.

A Robust Hand Silhouette Orientation Detection Method for Hand Gesture Recognition

The computer vision approach is most widely used for research related to hand gesture recognition. The detection of the image orientation has been discovered to be one of the keys to determine its success. The degree of freedom for a hand determines the shape and orientation of a gesture, which further causes a problem in the recognition methods. This article proposed evaluating orientation detection for silhouette static hand gestures with different poses and orientations without considering the forearm. The longest chord and ellipse were the two popular methods compared. The angles formed from two wrist points were selected as ground truth data and calculated from the horizontal axis. The performance was analyzed using the error values obtained from the difference in ground truth data angles compared to the method's results. The method has errors closer to zero that were rated better. Moreover, the method was evaluated using 1187 images, divided into four groups based on the forearm presence, and the results showed its effect on orientation detection. It was also discovered that the ellipse method was better than the longest chord. This study's results are used to select hand gesture orientation detection to increase accuracy in the hand gesture recognition process.

Image orientation detection by ensembles of Stochastic CNNs

In this paper we deal with the problem of accurately and automatically detecting the orientation of general images, for instance, of holiday snapshots. Detecting image orientation is an easy task for a human being but can be a long and tedious activity during processing and management of digital photos. Several attempts have been made in the design of systems for automated displaying images in their correct orientation, however, this is still an open problem. In this work we exploit the power of deep learning proposing a transfer learning approach that adjusts pre-trained convolutional neural networks to this classification task. We create ensembles of different Convolutional Neural Network models designed by randomly changing the activation functions in all the activation layers of a given network. Along with several known activation functions we also include the novel Soft Learnable activation function in the “random set”. Our resulting ensembles have been extensively evaluated on more than 45,000 images taken from four different public datasets, showing a remarkable performance improvement with respect to other state-of-the-art approaches. All the source code used for this work is freely available at https://github.com/LorisNanni/.

Automatic orientation detection of abstract painting

Detecting the correct orientation of an image is an important part of computer vision and the image processing pipeline. To determine the orientation of abstract paintings, as a special image type with ambiguous content, is difficult. There are several problems in the current orientation detection research: one is the use of a great deal of low-level image features for classification; the second is that the input image size is typically required to be consistent when using a deep learning model. To solve these problems, we propose a multi-scale and multi-layer feature fusion Net (MMFF-Net). First, local binary patterns (LBP) were used to generate three LBP-RGB feature maps with different scales in RGB mode, which could express the rotation characteristics of the image well; secondly, a neural network model based on AlexNet and spatial pyramid pooling (SPP) was constructed. Two data sets were selected to compare the training effect of different model parameters and model structures. Compared with the experimental results of several models, the proposed model effectively improved the accuracy of the correct orientation detection of abstract painting images.

Serial dependence revealed in history-dependent perceptual templates

In any given perceptual task, the visual system selectively weighs or filters incoming information. The particular set of weights or filters form a kind of template, which reveals the regions or types of information that are particularly useful for a given perceptual decision.1,2 Unfortunately, sensory input is noisy and ever changing. To compensate for these fluctuations, the visual system could adopt a strategy of biasing the templates such that they reflect a temporal smoothing of input, which would be a form of serial dependence.3-5 Here, we demonstrate that perceptual templates are, in fact, altered by serial dependence. Using a simple orientation detection task and classification-image technique, we found that perceptual templates are systematically biased toward previously seen, task-irrelevant orientations. The results of an orientation discrimination task suggest that this shift in perceptual template derives from a change in the perceptual appearance of orientation. Our study reveals how serial dependence biases internal templates of orientation and suggests that the sensitivity of classification-image techniques in general could be improved by taking into account history-dependent fluctuations in templates.

WeMos IoT Controller-Based Low-Cost Fall Detection System for Elderly Users

Fall detection systems for the elderly are very important to protect this type of users. The early detection of the fall of the elderly has a major impact on saving their lives and avoiding the deterioration of the negative medical effects resulting from the effect of the patient falling on a hard surface. One of the constraints in fall detection systems are false-negative errors (no fall detection) or false-positive errors (sending a false warning without real fall accident). These errors have to be reduced significantly. In this paper, an innovative method to reduce fall detection system errors is proposed. The system consists of two orientation detection sensors to track the body orientation instead of using a single sensor in the previous systems which enhances the system accuracy and reduces the false-negative and false-positive errors. The system uses a small size IoT-based controller to process the sensor's information and make the alarm decision based on specific thresholds. The output alarm of the system includes an email sent to the caregivers via the embedded Wi-Fi ESP8266 module as well as an SMS message to the caregivers’ phones via GSM modules to ensure that the alarm message arrives in the absence of internet coverage for the patient or the caregiver. The system is powered by a small lithium-Ion battery. All sensors and modules of the system are combined in a small rubber box that can be fixed in a waist belt or the chest rejoin of the user body. Several tests have been made in different procedures. The tests revealed that the new approach improves the accuracy of the system and reduces the possibility of triggering wrong alarms.

An Open-Source Platform for Human Pose Estimation and Tracking Using a Heterogeneous Multi-Sensor System

Human pose estimation and tracking in real-time from multi-sensor systems is essential for many applications. Combining multiple heterogeneous sensors increases opportunities to improve human motion tracking. Using only a single sensor type, e.g., inertial sensors, human pose estimation accuracy is affected by sensor drift over longer periods. This paper proposes a human motion tracking system using lidar and inertial sensors to estimate 3D human pose in real-time. Human motion tracking includes human detection and estimation of height, skeletal parameters, position, and orientation by fusing lidar and inertial sensor data. Finally, the estimated data are reconstructed on a virtual 3D avatar. The proposed human pose tracking system was developed using open-source platform APIs. Experimental results verified the proposed human position tracking accuracy in real-time and were in good agreement with current multi-sensor systems.

DBS electrode localization and rotational orientation detection using SQUID-based magnetoencephalography

Objective. The aim of the present study was to investigate the accuracy of localization and rotational orientation detection of a directional deep brain stimulation (DBS) electrode using a state-of-the-art magnetoencephalography (MEG) scanner. Approach. A directional DBS electrode along with its stimulator was integrated into a head phantom and placed inside the MEG sensor array. The electrode was comprised of six directional and two omnidirectional contacts. Measurements were performed while stimulating with different contacts and parameters in the phantom. Finite element modeling and fitting approach were used to compute electrode position and orientation. Main results. The electrode was localized with a mean accuracy of 2.2 mm while orientation was determined with a mean accuracy of 11∘. The limitation in detection accuracy was due to the lower measurement precision of the MEG system. Considering an ideal measurement condition, these values represent the lower bound of accuracy that can be achieved in patients. Significance. However, a future magnetic measuring system with higher precision will potentially detect location and orientation of a DBS electrode with an even greater accuracy.

Variable Statistical Structure of Neuronal Spike Trains in Monkey Superior Colliculus.

Popular models of decision-making propose that noisy sensory evidence accumulates until reaching a bound. Behavioral evidence as well as trial-averaged ramping of neuronal activity in sensorimotor regions of the brain support this idea. However, averaging activity across trials can mask other processes, such as rapid shifts in decision commitment, calling into question the hypothesis that evidence accumulation is encoded by delay period activity of individual neurons. We mined two sets of data from experiments in four monkeys in which we recorded from superior colliculus neurons during two different decision-making tasks and a delayed saccade task. We applied second-order statistical measures and spike train simulations to determine whether spiking statistics were similar or different in the different tasks and monkeys, despite similar trial-averaged activity across tasks and monkeys. During a motion direction discrimination task, single-trial delay period activity behaved statistically consistent with accumulation. During an orientation detection task, the activity behaved superficially like accumulation, but statistically consistent with stepping. Simulations confirmed both findings. Importantly, during a simple saccade task, with similar trial-averaged activity, neither process explained spiking activity, ruling out interpretations based on differences in attention, reward, or motor planning. These results highlight the need for exploring single-trial spiking dynamics to understand cognitive processing and raise the interesting hypothesis that the superior colliculus participates in different aspects of decision-making depending on task differences.SIGNIFICANCE STATEMENT How are decisions based on sensory information transformed into actions? We report that single-trial neuronal activity dynamics in the superior colliculus of monkeys show differences in decision-making tasks depending on task idiosyncrasies and requirements and despite similar trial-averaged ramping activity. These results highlight the importance of exploring single-trial spiking dynamics to understand cognitive processing and raise the interesting hypothesis that the superior colliculus participates in different aspects of decision-making depending on task requirements.

Image processing based detection of the fibre orientation during depth-controlled laser ablation of CFRP monitored by optical coherence tomography

Layer-accurate and precisely stepped removal of damaged areas is required for the repairing process of parts made of carbon fibre-reinforced plastic (CFRP) to obtain a high bonding strength of the repair plies in the rebuilding process. Conventional techniques are characterized by a high degree of manual labour and mechanical load of the processed part. Here, a novel laser-based and controlled approach for automated and layer-accurate removal of damaged areas is presented. In-line depth measurements by optical coherence tomography during nanosecond laser ablation enable depth-controlled homogeneous material removal with minimum damage <10 μm and low surface roughness in the range of 7 μm to 13 μm. The processed layer must be detected for an unknown or varying layer thickness to stop the ablation process at the interface between two layers and not somewhere within one ply. The image processing-based determination of the fibre orientation from the depth measurements allow for a reliable online detection of the processed layer with a maximum error of 4° and root mean square error of 1.1°. As a result, layer-accurate damage removal is demonstrated for a complex repairing geometry. The suitability of the automated laser-based approach as preparation for repairing is proven by cross-sections and scanning-electron-microscopy.

Shape-Based Features for Optimized Hand Gesture Recognition

Gesture recognition is the most intuitive form of human-computer interface. Hand gestures provide a natural way for humans to interact with computers to perform a variety of different applications. However, factors such as complexity of hand gesture structures, differences in hand size, hand posture, and environmental illumination can influence the performance of hand gesture recognition algorithms. Considering the above factors, this paper aims to present a real time system for hand gesture recognition on the basis of detection of some meaningful shape-based features like orientation, center of mass, status of fingers, thumb in terms of raised or folded fingers of hand and their respective location in image. The internet is growing at a very fast pace. The use of web browser is also growing. Everyone has at least two or three most frequently visited website. Thus, in this paper, effectiveness of the gesture recognition and its ability to control the browser via the recognized hand gestures are experimented and the results are analyzed.

A Pulse-chase EdU Method for Detection of Cell Division Orientation in Arabidopsis and Juncus prismatocarpus Leaf Primordia.

In plants, the morphological diversity of leaves is largely determined by cell division, especially cell division orientation. Whereas cell division itself is easily monitored, the detection and quantification of cell division orientation are difficult. The few existing methods for detection and quantification of cell division orientation are either inefficient or laborious. Here, we describe a pulse-chase strategy using a 5-ethynyl-2'-deoxyuridine (EdU) labeling assay. Plant tissues are first incubated with EdU for a short period (pulse), followed by a long incubation without EdU (chase). Using this method, the positions of daughter cells are easily detected and can be used to quantify cell division orientation. Our protocol is rapid and very efficient for quantitative analysis of cell division orientation, and can be applied to both model and non-model plant species. Graphic abstract: Plant cell division pairs clearly visualized by a pulse-chase EdU method.

Crack orientation detection in plate-like structures via Lamb wave specular reflection analysis

The determination of crack orientation plays an important role in the non-destructive evaluation (NDE) of plate-like structures. In this paper, the interaction between Lamb waves and cracks is investigated using the time domain spectral element method (SEM) as the simulation tool. On that basis, we analyse the geometric principle of specular reflections and propose an efficient crack angle detection algorithm. During the process, four sensors take turns to excite the Lamb wave which would result in twelve crack reflection signals. Using the corresponding actuator as the centre and the reflection length as the radius, we can obtain twelve reflection circles in total. When these reflections are specular, the mirror points of the actuators with respect to the crack line are exactly the intersection points of the circles. Thus, we can obtain the crack angle by this geometric relation. Validations on different cracks are made and the result of the algorithm shows a good agreement with the real value.

Research on Vision-based Position Measurement System for Rocket Propellant Loading

In the process of rocket propellant loading, the propellant's own gravity and wind pendulum may cause the connector to exert a large force on the rocket body, which may lead to connector disconnection and fuel leakage, or even damage to the rocket body. To improve the reliability and safety of rocket propellant loading, a position measurement system based on dynamic orientation detection alignment technology is proposed to achieve tracking of the rocket refueling port. In this paper, the two-dimensional position of the target in the camera plane is quickly extracted by identifying the direction of the target image and the size of the marker in the image, while the distance between the target and the camera is accurately obtained by a laser rangefinder. With this measurement solution, the deviation of the connector from the propellant filler port at the current moment can be modified in real time, thus reducing the forces on the rocket and increasing its launch reliability.

Selective age-related changes in orientation perception.

Orientation perception is a fundamental property of the visual system and an important basic processing stage for visual scene perception. Neurophysiological studies have found broader tuning curves and increased noise in orientation-selective neurons of senescent monkeys and cats, results that suggest an age-related decline in orientation perception. However, behavioral studies in humans have found no evidence for such decline, with performance being comparable for younger and older participants in orientation detection and discrimination tasks. Crucially, previous behavioral studies assessed performance for cardinal orientation only, and it is well known that the human visual system prefers cardinal over oblique orientations, a phenomenon called the oblique effect. We hypothesized that age-related changes depend on the orientation tested. In two experiments, we investigated orientation discrimination and reproduction for a large range of cardinal and oblique orientations in younger and older adults. We found substantial age-related decline for oblique but not for cardinal orientations, thus demonstrating that orientation perception selectively declines for oblique orientations. Taken together, our results serve as the missing link between previous neurophysiological and human behavioral studies on orientation perception in healthy aging.

Automatic detection of fibers orientation on composite laminates using convolutional neural networks

The mechanical behaviour and failure of the composite materials are highly influenced by the fibre’s orientation. It is important to decide which type of layers and orientations to use for the layup sequence such that the composite laminate is as light and/or cheap as possible while being capable to carry the load for which it is designed. During the production process it is critical to cut and lay the composite woven according to the optimal layup sequence resulting from the optimization process. Therefore, it is important to develop an automatic system which is capable to accurately detect the fibres orientation from a material picture. In our previous work we proposed such a system based on image processing and computational geometry. The main disadvantage of this system is that it has a lot of parameters which need to be manually tuned for each material. In this paper we propose a simple system which doesn’t need any parameter tuning. It is based on convolutional neural networks and we demonstrate with a lot of examples that it is very accurate, stable and insensitive to image variations. The method was tested on diverse composite laminates and woven with different chromatic and morphological properties.

A Hermaphrodite Electromechanical Connector for Self-Reconfigurable Robot Modules

Connectors that realize connections between modules are an indispensable component of modular self-reconfigurable robots. In this paper, a hermaphrodite connector coupled with electromechanical connection capability is proposed, called HerCel. The specially designed connection mechanism and electrical interface of HerCel can realize both mechanical and electrical connections simultaneously at four orientations to achieve power transmission, communication, and orientation detection between modules, except for the basic functions required for connectors, such as symmetry, high misalignment tolerance, self-locking, and sensing. A detailed design of HerCel is presented, including the connection mechanism, sensor layout, and electrical interface. A series of experiments with HerCel prototypes testing electromechanical connection, misalignment tolerance, and load capability were carried out to illustrate the design feasibility. Furthermore, a self-reconfiguration motion test of a module was conducted to verify the performance of HerCel.

Single-Nanoparticle Orientation Sensing by Deep Learning

This paper describes a computational imaging platform to determine the orientation of anisotropic optical probes under differential interference contrast (DIC) microscopy. We established a deep-learning model based on data sets of DIC images collected from metal nanoparticle optical probes at different orientations. This model predicted the in-plane angle of gold nanorods with an error below 20°, the inherent limit of the DIC method. Using low-symmetry gold nanostars as optical probes, we demonstrated the detection of in-plane particle orientation in the full 0–360° range. We also showed that orientation predictions of the same particle were consistent even with variations in the imaging background. Finally, the deep-learning model was extended to enable simultaneous prediction of in-plane and out-of-plane rotation angles for a multibranched nanostar by concurrent analysis of DIC images measured at multiple wavelengths.