Perceptual Capabilities Research Articles

Collision Risk Assessment and Operation Assistant Strategy for Teleoperation System

Teleoperation robots remain superior to fully automated robots in complicated and unstructured environments (e.g., in-orbit assembly). However, the collision risk is also greatly increased in these environments. Therefore, the teleoperation robot should possess the capability of collision risk perception and be configured with security assistance strategy to improve safety and efficiency. With this objective in mind, this paper proposes a collision risk assessment system based on fuzzy theory, which comprehensively considers the effects of shortest distance, operation speed, and delay time. The introduction of fuzzy theory makes the risk assessment results more accurate. Furthermore, this paper also proposes a creative discrete expandable bounding box method to make the calculation of the nearest distance between complicated obstacles and the robot more efficient and faster. Secondly, to improve the safety and efficiency of teleoperation, this paper proposes a set of teleoperation assistance strategies for teleoperation robots based on collision risk. The strategies include partial view highlighting, variation in the motion mapping ratio, and haptic risk warning, all of which are achieved in the graphical interactive interface. Finally, this paper verifies the proposed collision risk model and the operation assistant strategy through experiments. The results show that the proposed fuzzy collision risk model has a correct trend with risk factors, and the proposed operation assistance strategies can effectively reduce the robot collision risk and improve the safety of teleoperation robots. In conclusion, this research contributes to the collision risk assessment and obstacle avoidance assistance strategy of teleoperation robots.

A self-rotating, single-actuated UAV with extended sensor field of view for autonomous navigation.

Uncrewed aerial vehicles (UAVs) rely heavily on visual sensors to perceive obstacles and explore environments. Current UAVs are limited in both perception capability and task efficiency because of a small sensor field of view (FoV). One solution could be to leverage self-rotation in UAVs to extend the sensor FoV without consuming extra power. This natural mechanism, induced by the counter-torque of the UAV motor, has rarely been exploited by existing autonomous UAVs because of the difficulties in design and control due to highly coupled and nonlinear dynamics and the challenges in navigation brought by the high-rate self-rotation. Here, we present powered-flying ultra-underactuated LiDAR (light detection and ranging) sensing aerial robot (PULSAR), an agile and self-rotating UAV whose three-dimensional position is fully controlled by actuating only one motor to obtain the required thrust and moment. The use of a single actuator effectively reduces the energy loss in powered flights. Consequently, PULSAR consumes 26.7% less power than the benchmarked quadrotor with the same total propeller disk area and avionic payloads while retaining a good level of agility. Augmented by an onboard LiDAR sensor, PULSAR can perform autonomous navigation in unknown environments and detect both static and dynamic obstacles in panoramic views without any external instruments. We report the experiments of PULSAR in environment exploration and multidirectional dynamic obstacle avoidance with the extended FoV via self-rotation, which could lead to increased perception capability, task efficiency, and flight safety.

Generative Adversarial Network for Overcoming Occlusion in Images: A Survey

Although current computer vision systems are closer to the human intelligence when it comes to comprehending the visible world than previously, their performance is hindered when objects are partially occluded. Since we live in a dynamic and complex environment, we encounter more occluded objects than fully visible ones. Therefore, instilling the capability of amodal perception into those vision systems is crucial. However, overcoming occlusion is difficult and comes with its own challenges. The generative adversarial network (GAN), on the other hand, is renowned for its generative power in producing data from a random noise distribution that approaches the samples that come from real data distributions. In this survey, we outline the existing works wherein GAN is utilized in addressing the challenges of overcoming occlusion, namely amodal segmentation, amodal content completion, order recovery, and acquiring training data. We provide a summary of the type of GAN, loss function, the dataset, and the results of each work. We present an overview of the implemented GAN architectures in various applications of amodal completion. We also discuss the common objective functions that are applied in training GAN for occlusion-handling tasks. Lastly, we discuss several open issues and potential future directions.

Visuo-haptic object perception for robots: an overview

The object perception capabilities of humans are impressive, and this becomes even more evident when trying to develop solutions with a similar proficiency in autonomous robots. While there have been notable advancements in the technologies for artificial vision and touch, the effective integration of these two sensory modalities in robotic applications still needs to be improved, and several open challenges exist. Taking inspiration from how humans combine visual and haptic perception to perceive object properties and drive the execution of manual tasks, this article summarises the current state of the art of visuo-haptic object perception in robots. Firstly, the biological basis of human multimodal object perception is outlined. Then, the latest advances in sensing technologies and data collection strategies for robots are discussed. Next, an overview of the main computational techniques is presented, highlighting the main challenges of multimodal machine learning and presenting a few representative articles in the areas of robotic object recognition, peripersonal space representation and manipulation. Finally, informed by the latest advancements and open challenges, this article outlines promising new research directions.

Framework for Network-Constrained Tracking of Cyclists and Pedestrians

The increase in perception capabilities of connected mobile sensor platforms (e.g., self-driving vehicles, drones, and robots) leads to an extensive surge of sensed features at various temporal and spatial scales. Beyond their traditional use for safe operation, available observations could enable to see how and where people move on sidewalks and cycle paths, to eventually obtain a complete microscopic and macroscopic picture of the traffic flows in a larger area. This paper proposes a new method for advanced traffic applications, tracking an unknown and varying number of moving targets (e.g., pedestrians or cyclists) constrained by a road network, using mobile (e.g., vehicles) spatially distributed sensor platforms. The key contribution in this paper is to introduce the concept of network bound targets into the multi-target tracking problem, and hence to derive a network-constrained multi-hypotheses tracker (NC-MHT) to fully utilize the available road information. This is done by introducing a target representation, comprising a traditional target tracking representation and a discrete component placing the target on a given segment in the network. A simulation study shows that the method performs well in comparison to the standard MHT filter in free space. Results particularly highlight network-constraint effects for more efficient target predictions over extended periods of time, and in the simplification of the measurement association process, as compared to not utilizing a network structure. This theoretical work also directs attention to latent privacy concerns for potential applications.

Hierarchical Integration of Model Predictive and Fuzzy Logic Control for Combined Coverage and Target-Oriented Search-and-Rescue via Robots with Imperfect Sensors

Search-and-rescue (SaR) in unknown environments is a crucial task with life-threatening risks. SaR requires precise, optimal, and fast decisions to be made. Robots are promising candidates expected to execute various SaR tasks autonomously. While humans use heuristics to effectively deal with uncertainties of SaR, optimisation of multiple objectives (e.g., the mission time, the area covered, the number of victims detected), in the presence of physical and control constraints, is a mathematical challenge that requires machine computations. Thus including both human-inspired and mathematical capabilities in decision making of SaR robots is highly desired. However, developing control approaches that exhibit both capabilities has been significantly ignored in literature. Moreover, coordinating the decisions of the robots in large-scale SaR missions with affordable computation costs is an open challenge. Finally, in real-life, due to defects (e.g., in the sensors of the robots) or environmental factors (e.g., smoke) data perceived by SaR robots may be prone to uncertainties. We introduce a hierarchical multi-agent control architecture that simultaneously provides the following advantages: exploiting non-homogeneous and imperfect perception capabilities of SaR robots; improving the global performance as it is provided by centralised controllers; computational efficiency and robustness to failure of the central controller as offered by decentralised control methods. The integrated structure of the proposed control framework allows to combine human-inspired and mathematical decision making methods, via respectively fuzzy logic and model predictive control, in a coordinated and computationally efficient way. Our results for various computer-based simulations show that while the area coverage with the proposed control approach is comparable to existing heuristic methods that are particularly developed for coverage-oriented SaR, our approach has a significantly better performance regarding locating the trapped victims. Furthermore, with comparable computation times, the proposed control approach successfully avoids conflicts that may appear in non-cooperative control methods. In summary, the proposed multi-agent control system is capable of combining coverage-oriented and target-oriented SaR in a balanced and coordinated way.

Stochasticity may generate coherent motion in bird flocks

Murmurations along with other forms of flocking have come to epitomize collective animal movements. Most studies into these stunning aerial displays have aimed to understand how coherent motion may emerge from simple behavioral rules and behavioral correlations. These studies may now need revision because recently it has been shown that flocking birds, like swarming insects, behave on the average as if they are trapped in elastic potential wells. Here I show, somewhat paradoxically, how coherent motion can be generated by variations in the intensity of multiplicative noise which causes the shape of a potential well to change, thereby shifting the positions and strengths of centres of attraction. Each bird, irrespective of its position in the flock will respond in a similar way to such changes, giving the impression that the flock behaves as one, and typically resulting in scale-free correlations. I thereby show how correlations can be an emergent property of noisy, confining potential wells. I also show how such wells can lead to high density borders, a characteristic of flocks, and I show how they can account for the complex patterns of collective escape patterns of starling flocks under predation. I suggest swarming and flocking do not constitute two distinctly different kinds of collective behavior but rather that insects are residing in relatively stable potential wells whilst birds are residing in unstable potential wells. It is shown how, dependent upon individual perceptual capabilities, bird flocks can be poised at criticality.

Fine-Grained Facial Expression Recognition in Multiple Smiles

Smiling has often been incorrectly interpreted as “happy” in the popular facial expression datasets (AffectNet, RAF-DB, FERPlus). Smiling is the most complex human expression, with positive, neutral, and negative smiles. We focused on fine-grained facial expression recognition (FER) and built a new smiling face dataset, named Facial Expression Emotions. This dataset categorizes smiles into six classes of smiles, containing a total of 11,000 images labeled with corresponding fine-grained facial expression classes. We propose Smile Transformer, a network architecture for FER based on the Swin Transformer, to enhance the local perception capability of the model and improve the accuracy of fine-grained face recognition. Moreover, a convolutional block attention module (CBAM) was designed, to focus on important features of the face image and suppress unnecessary regional responses. For better classification results, an image quality evaluation module was used to assign different labels to images with different qualities. Additionally, a dynamic weight loss function was designed, to assign different learning strategies according to the labels during training, focusing on hard yet recognizable samples and discarding unidentifiable samples, to achieve better recognition. Overall, we focused on (a) creating a novel dataset of smiling facial images from online annotated images, and (b) developing a method for improved FER in smiling images. Facial Expression Emotions achieved an accuracy of 88.56% and could serve as a new benchmark dataset for future research on fine-grained FER.

Automatic focal EEG identification based on deep reinforcement learning

Electroencephalogram (EEG) signals convey information about the electrical activity of neurons and are commonly used in clinical practice to evaluate the epileptic activity of patients. There is a part of the human brain that is closely linked to epileptic activity, namely the epileptogenic zone. Successful resection of the epileptogenic zone requires high precision classification of focal (F) and non-focal (NF) EEG signals. In this paper, we improve the Deep Q-Network (DQN) by adding Additional Functional Modules (AFM) to propose a novel focal EEG recognition method which is named AFM-DQN. Compared to the traditional Reinforcement Learning (RL), this model incorporates a deep convolutional neural network (CNN), and unifies the decision making capability of Q-learning and the perceptual capability of CNN, which greatly improves the learning of this network. The AFM, including pre-training, the high performance classifier (HPC), reward control mechanism and three RL related techniques, imparts this model a stronger generalization ability. To reduce computational burden of the network while preserving the correlation and interdependence of EEG signals, we introduce different kinds of EEG features and a semi-supervised feature selection algorithm based on joint mutual information (Semi-JMI algorithm). Experiments on the two publicly available EEG databases have achieved classification accuracy of 95.87% and 97.5%, respectively, which demonstrates that the proposed method has the potential for clinical application of accurate localization of epileptogenic zone. Compared with other published methods, this method has better performance and generalization ability.

An ultrasensitive three-dimensional structured multi-mode sensor for out-of-plane forces

This work presents an ultrasensitive three-dimensional (3D) structured sensor with out-of-plane tactility and non-contact perception capabilities. It is composed of flexible silver nanowire-patterned magnetic film and the printed heat-responsive polymer skeleton by means of spin-coating, masking, and printing technologies. Besides the capabilities in detecting tension, compression, shear, and even air vibration/flow, the 3D-structured sensor also exhibits the advantages of prompt response, super flexibility, outstanding sensitivity, high reversibility and stability. The responsive relative resistance variations of the 3D-structured sensor exhibit an excellent linear relationship with the pull-out/press-in forces, and the sensitivities are estimated as 21.8 % N−1 and −19.6 % N−1, respectively. Furthermore, it is demonstrated that the applied direction and strength of shear excitation can be well predicted via applying machine learning methods. Moreover, the sensor is able to clearly detect the loading paths of external excitations, including the load waveform and even the initial phase. Notably, the normalized resistance of the sensor changes with the magnetic flux density which is promising in tuning the sensitivity of the sensor within a magnetic field. The exceptional multi-mode sensing characteristics demonstrate that the 3D-structured sensor possess great potentials for implementation in next-generation smart devices or systems.

Research on the Impact of Big Data Analysis Capability on Enterprise Supply Chain Resilience

Based on the current economic environment and from the perspective of dynamic capability, this paper divides big data analysis capability into big data perception capability, big data capture capability and big data transformation capability, and divides supply chain ambidexterity into supply chain agility and supply chain adaptability. A questionnaire survey was used to quantify the impact of big data analysis capability (BDAC) on supply chain elasticity (SC-RE). A total of 300 questionnaires were distributed to managers of supply chain nodes of enterprises, and 217 valid questionnaires were obtained. The regression results of questionnaire data showed that BDAC was positively correlated with SC-RE. Supply chain duality (SC-AM) is positively correlated with SC-RE. SC-AM plays a partial mediating role in the positive effect of BDAC on SC-RE. Based on the regression results of this paper and the research results of existing scholars, the corresponding conclusions are drawn, and some suggestions are put forward for enterprise managers. BDAC is an effective way to improve the performance level and competitive strength of supply chain enterprises. Enterprises should attach importance to the useful information contained in big data, take certain measures to enhance BDAC, improve the information processing capacity and speed of enterprises, reduce the probability of risk occurrence, so as to improve the supply chain resilience of enterprises and improve the level of supply chain risk management.

Deep Q-Learning based resource allocation and load balancing in a mobile edge system serving different types of user requests

Abstract With the expansion of the communicative and perceptual capabilities of mobile devices in recent years, the number of complex and high computational applications has also increased rendering traditional methods of traffic management and resource allocation quite insufficient. Recently, mobile edge computing (MEC) has emerged as a new viable solution to these problems. It can provide additional computing features at the edge of the network and allow alleviation of the resource limit of mobile devices while increasing the performance for critical applications especially in terms of latency. In this work, we addressed the issue of reducing the service delay by choosing the optimal path in the MEC network, which consists of multiple MEC servers that has different capabilities, applying network load balancing where multiple requests need to be handled simultaneously and routing selection based on a deep- Q network (DQN) algorithm. A novel traffic control and resource allocation method is proposed based on deep Q-learning (DQL) which allows reducing the end-to-end delay in cellular networks and in the mobile edge network. Real life traffic scenarios with various types of user requests are considered and a novel DQL resource allocation scheme which adaptively assigns computing and network resources is proposed. The algorithm optimizes traffic distribution between servers reducing the total service time and balancing the use of available resources under varying environmental conditions.

EventTube: An Artificial Intelligent Edge Computing Based Event Aware System to Collaborate With Individual Devices in Logistics Systems

Artificial intelligence has been adopted to facilitate monitoring, operation, and decision in the logistics field. Logistics robots with environment perception capability have been used to improve warehousing efficiency in logistics systems. However, autonomous mobile robots face computationally intensive and real-time demanding tasks such as navigation, localization, and obstacle avoidance. In this article, we present EventTube, an edge computing based event-aware system that can efficiently discover events from the video data captured by RGB-Monoculars and collaborate with individual devices to make timely decisions. EventTube deploys a semantic context extraction pipeline on edge servers to aggregate video streams from mobile robots and feed a few keyframes, including the start and end of the specific events to the successive perception pods, accelerating logistics robots’ response speed. The event-related model parameters are trained and updated online on a server. The video data collected at the warehouse site for our mobile robots show that EventTube significantly improves parcel delivery efficiency without affecting regular deliveries.

False data injection attack in smart grid: Attack model and reinforcement learning-based detection method

The smart grid, as a cyber-physical system, is vulnerable to attacks due to the diversified and open environment. The false data injection attack (FDIA) can threaten the grid security by constructing and injecting the falsified attack vector to bypass the system detection. Due to the diversity of attacks, it is impractical to detect FDIAs by fixed methods. This paper proposed a false data injection attack model and countering detection methods based on deep reinforcement learning (DRL). First, we studied an attack model under the assumption of unlimited attack resources and information of complete topology. Different types of FDIAs are also enumerated. Then, we formulated the attack detection problem as a Markov decision process (MDP). A deep reinforcement learning-based method is proposed to detect FDIAs with a combined dynamic-static detection mechanism. To address the sparse reward problem, experiences with discrepant rewards are stored in different replay buffers to achieve efficiency. Moreover, the state space is extended by considering the most recent states to improve the perception capability. Simulations were performed on IEEE 9,14,30, and 57-bus systems, proving the validation of attack model and efficiency of detection method. Results proved efficacy of the detection method in different scenarios.

CenterTransFuser: radar point cloud and visual information fusion for 3D object detection

Sensor fusion is an important component of the perception system in autonomous driving, and the fusion of radar point cloud information and camera visual information can improve the perception capability of autonomous vehicles. However, most of the existing studies ignore the extraction of local neighborhood information and only consider shallow fusion between the two modalities based on the extracted global information, which cannot perform a deep fusion of cross-modal contextual information interaction. Meanwhile, in data preprocessing, the noise in radar data is usually only filtered by the depth information derived from image feature prediction, and such methods affect the accuracy of radar branching to generate regions of interest and cannot effectively filter out irrelevant information of radar points. This paper proposes the CenterTransFuser model that makes full use of millimeter-wave radar point cloud information and visual information to enable cross-modal fusion of the two heterogeneous information. Specifically, a new interaction called cross-transformer is explored, which cooperatively exploits cross-modal cross-multiple attention and joint cross-multiple attention to mine radar and image complementary information. Meanwhile, an adaptive depth thresholding filtering method is designed to reduce the noise of radar modality-independent information projected onto the image. The CenterTransFuser model is evaluated on the challenging nuScenes dataset, and it achieves excellent performance. Particularly, the detection accuracy is significantly improved for pedestrians, motorcycles, and bicycles, showing the superiority and effectiveness of the proposed model.

Acoustic Wake-Up Technology for Microsystems: A Review.

Microsystems with capabilities of acoustic signal perception and recognition are widely used in unattended monitoring applications. In order to realize long-term and large-scale monitoring, microsystems with ultra-low power consumption are always required. Acoustic wake-up is one of the solutions to effectively reduce the power consumption of microsystems, especially for monitoring sparse events. This paper presents a review of acoustic wake-up technologies for microsystems. Acoustic sensing, acoustic recognition, and system working mode switching are the basis for constructing acoustic wake-up microsystems. First, state-of-the-art MEMS acoustic transducers suitable for acoustic wake-up microsystems are investigated, including MEMS microphones, MEMS hydrophones, and MEMS acoustic switches. Acoustic transducers with low power consumption, high sensitivity, low noise, and small size are attributes needed by the acoustic wake-up microsystem. Next, acoustic features and acoustic classification algorithms for target and event recognition are studied and summarized. More acoustic features and more computation are generally required to achieve better recognition performance while consuming more power. After that, four different system wake-up architectures are summarized. Acoustic wake-up microsystems with absolutely zero power consumption in sleep mode can be realized in the architecture of zero-power recognition and zero-power sleep. Applications of acoustic wake-up microsystems are then elaborated, which are closely related to scientific research and our daily life. Finally, challenges and future research directions of acoustic wake-up microsystems are elaborated. With breakthroughs in software and hardware technologies, acoustic wake-up microsystems can be deployed for ultra-long-term and ultra-large-scale use in various fields, and play important roles in the Internet of Things.

Review of David Michael Kleinberg-Levin, Critical Studies on Heidegger: The Emerging Body of Understanding

The present paper is a review of Critical Studies on Heidegger: The Emerging Body of Understanding (SUNY, 2023), by David Michael Kleinberg-Levin, who argues we can find a phenomenology of perception in Heidegger ultimately no different than that of Merleau-Ponty. The concept of “the emerging body of understanding” means the growth or “perfection” of human capabilities in perception – touch, vision, and hearing – that are attentive to our interconnectedness with others and nature as presented by the Fourfold. The conclusion of the review offers some evaluations regarding questions of influence and recently available course notes by Merleau-Ponty about Heidegger’s philosophy.

Mpc-based motion planning and control enables smarter and safer autonomous marine vehicles: perspectives and a tutorial survey

Autonomous marine vehicles (AMVs) have received considerable attention in the past few decades, mainly because they play essential roles in broad marine applications such as environmental monitoring and resource exploration. Recent advances in the field of communication technologies, perception capability, computational power and advanced optimization algorithms have stimulated new interest in the development of AMVs. In order to deploy the constrained AMVs in the complex dynamic maritime environment, it is crucial to enhance the guidance and control capabilities through effective and practical planning, and control algorithms. Model predictive control (MPC) has been exceptionally successful in different fields due to its ability to sys-tematically handle constraints while optimizing control performance. This paper aims to provide a review of recent progress in the context of motion planning and control for AMVs from the perceptive of MPC. Finally, future research trends and directions in this substantial research area of AMVs are highlighted.

Вопрос о нарративе и архитектонике — применительно к прозе Даниила Хармса

The present article was written as a response to the formulation of the theme of the conference Musical Composition: Narrative and Architectonics (Gnesin’ Russian Academy of Music, 2022), where in the commonality between music, literature and architecture was stated – i.e., between music and those forms of creativity to whom narrative and architectonics initially pertain. Music possesses the equable capability of perception of compositional principles relevant to both the temporal and the spatial arts, and this circumstance places music in the center of a triple juxtaposition with narrative (and literature presented by it) and architectonics (and, correspondingly, architecture). The author of the article suggests turning the scheme of the triple juxtaposition in such a way, that not music, but architecture would appear at the center. And the focus would be not literature in general, but specifically on Daniil Kharms’ flash fiction — the primary sphere of observation of the peculiarities of the narrative in his works. In the organization of Kharms’ literary works both musical and architectural analogies play a discernible role, the substantiation of which we find in a number of his diary notes, drawings and theoretical texts. The “musical” traits of Kharms’ poetry and prose have been studied to a sufficient degree, which cannot be said about the inherent architectonic qualities intrinsic to them — let us compare them with “the law of symmetry, the law of equilibrium” mentioned in Kharms’ notes (Treatise about the Infinite, 1932). The instrument of the revealing of symmetry in a literary text is the metrotectonic method of Konyus. The article examines three examples of Kharms’ symmetrically aligned prose — the short stories The Optical Illusion, Symphony No. 2 and The Trunk Box.

PSIHOLOŠKE STUDIJE LIČNOSTI UMETNIKA

The paper analyzes different theoretical and empirical approaches to the personality of the artist. It presents a synthesis of various relevant ideas and their perception from a more gen- eral level, to assemble these ideas into a broader picture concerning the issue of the artist’s per- sonality. The analysis shows that all psychological theories were focused on individual aspects of the personality of the artist. Psychoanalytic approaches are based on the idea of discovering personal neurotic roots hidden in the subconsciousness of the creator. Gestalt psychology sees creation as based on talent, and gift and the artist is observed as a person with specially devel- oped capabilities for perception and visual thinking. Humanistic psychology sees creativity as a universal human potential and trait theories are looking for the unique composition of personality traits in the artist. However, each one is individually important, because it con- tributed to the understanding of artistic creation and the personality of the artist as a whole. Throughout history, the artists were observed as great creators, or as madmen. Varieties and differences between the concepts could be understood by the fact that each artist is great and unique in his/her originality.