High-level Information Fusion Research Articles

Two-phase Multi-document Event Summarization on Core Event Graphs

Succinct event description based on multiple documents is critical to news systems as well as search engines. Different from existing summarization or event tasks, Multi-document Event Summarization (MES) aims at the query-level event sequence generation, which has extra constraints on event expression and conciseness. Identifying and summarizing the key event from a set of related articles is a challenging task that has not been sufficiently studied, mainly because online articles exhibit characteristics of redundancy and sparsity, and a perfect event summarization needs high level information fusion among diverse sentences and articles. To address these challenges, we propose a two-phase framework for the MES task, that first performs event semantic graph construction and dominant event detection via graph-sequence matching, then summarizes the extracted key event by an event-aware pointer generator. For experiments in the new task, we construct two large-scale real-world datasets for training and assessment. Extensive evaluations show that the proposed framework significantly outperforms the related baseline methods, with the most dominant event of the articles effectively identified and correctly summarized.

End-to-End Fusion of Hyperspectral and Chlorophyll Fluorescence Imaging to Identify Rice Stresses.

Herbicides and heavy metals are hazardous substances of environmental pollution, resulting in plant stress and harming humans and animals. Identification of stress types can help trace stress sources, manage plant growth, and improve stress-resistant breeding. In this research, hyperspectral imaging (HSI) and chlorophyll fluorescence imaging (Chl-FI) were adopted to identify the rice plants under two types of herbicide stresses (butachlor (DCA) and quinclorac (ELK)) and two types of heavy metal stresses (cadmium (Cd) and copper (Cu)). Visible/near-infrared spectra of leaves (L-VIS/NIR) and stems (S-VIS/NIR) extracted from HSI and chlorophyll fluorescence kinetic curves of leaves (L-Chl-FKC) and stems (S-Chl-FKC) extracted from Chl-FI were fused to establish the models to detect the stress of the hazardous substances. Novel end-to-end deep fusion models were proposed for low-level, middle-level, and high-level information fusion to improve identification accuracy. Results showed that the high-level fusion-based convolutional neural network (CNN) models reached the highest detection accuracy (97.7%), outperforming the models using a single data source (<94.7%). Furthermore, the proposed end-to-end deep fusion models required a much simpler training procedure than the conventional two-stage deep learning fusion. This research provided an efficient alternative for plant stress phenotyping, including identifying plant stresses caused by hazardous substances of environmental pollution.

Inference of Mood State Indices by Using a Multimodal High-Level Information Fusion Technique

Mood state assessment (MSA) is increasingly important for diagnosis and treatment of depression. Recent years, many approaches have been proposed for the process of MSA. When using a single approach for MSA, the user often has to deal with possible noisy data and unacceptable error rates. Novelty: In order to improve the accuracy of MSA, in this paper, we propose a novel high-level information fusion method for determining the MS of users by fusing physiological data, such as heart rate and brainwave information collected through a wearable device, and psychological data collected through a monthly mood chart. The multifaceted information must be collected and analyzed simultaneously. Contribution: In the inference process of proposed framework, we adopted a Bayesian Network (BN) to perform high-level information fusion. We exploited various evaluation approaches to evaluate the performance of the proposed approach. Result: We have conducted experiments using two datasets and evaluated the performance using various factors. The results show that the proposed method (7-M Bayesian Fusion) is superior to other methods averagely 9.48 % improvement in most evaluation factors. It reveals that the proposed approach is efficient in fusing the MS information required for accurate diagnosis of depression compared with those approaches without fusion approach or with few information fusing.

Optimising commercial port operations through high-level information fusion

In order to remain profitable, commercial maritime ports must maintain high throughput of inbound vessels. The gantry cranes that load and unload the vessels are the primary point of interaction be...

Origin identification of Panax notoginseng by multi-sensor information fusion strategy of infrared spectra combined with random forest.

Traditional Chinese medicine Panax notoginseng is a valuable geo-authentic herbal material. The difference of growth environment in different producing areas has significant influence on the quality of traditional Chinese medicine, and origin identification is an important part of the quality assessment of P.notoginseng. In this study, Fourier transform mid-infrared (FT-MIR) and near infrared (NIR) sensor technologies combined with single spectra analysis and multi-sensor information fusion strategy (low-, mid- and high-level) for the origin identification of 210 P.notoginseng samples from five cities in Yunnan Province, China. FT-MIR spectra were considered to play a greater role in data analysis than NIR spectra. Random forest (RF) was used to establish classification models. The result of the random forest Boruta (RF-Bo) model and the random forest variable selection (RF-Vs) model based on high-level multi-sensor information fusion strategy was satisfactory. In addition, the RF-Bo model based on high-level multi-sensor information fusion strategy was faster and simpler in data analysis and the accuracy was 95.6%.

Recent Trends in Context Exploitation for Information Fusion and AI

AI is related to information fusion (IF). Many methods in AI that use perception and reasoning align to the functionalities of high‐level IF (HLIF) operations that estimate situational and impact states. To achieve HLIF sensor, user, and mission management operations, AI elements of planning, control, and knowledge representation are needed. Both AI reasoning and IF inferencing and estimation exploit context as a basis for achieving deeper levels of understanding of complex world conditions. Open challenges for AI researchers include achieving concept generalization, response adaptation, and situation assessment. This article presents a brief survey of recent and current research on the exploitation of context in IF and discusses the interplay and similarities between IF, context exploitation, and AI. In addition, it highlights the role that contextual information can provide in the next generation of adaptive intelligent systems based on explainable AI. The article describes terminology, addresses notional processing concepts, and lists references for readers to follow up and explore ideas offered herein.

Multi-source information fusion strategies of aerial parts in FTIR-ATR spectroscopic characterization and classification of Paris polyphylla var. yunnanensis

The rational utilization of aerial parts (stem and leaf) of Paris polyphylla var. yunnanensis (P. yunnanensis) is increasingly being studied, due to the growing scarcity of resources. In our study, the geographical origin and cultivation year classification of 662 cultivated P. yunnanensis rhizome, stem and leaf samples from Yuxi, Lijiang and Honghe cities were analyzed by Fourier transform infrared-attenuated total reflectance (FTIR-ATR) spectra using principal component analysis (PCA), partial least squares-discriminant analysis (PLS-DA), random forest (RF) methods combined with multi-source information fusion strategies. Firstly, samples collected from Honghe procedure among all geographical origins were showed higher consistency by biomass analysis. Besides, the PLS-DA model of leaves part was obtained the better geographical origin classification ability than that of underground part (rhizomes), which demonstrate that leaves can be applied to identify geographical origins of P. yunnanensis effectively. And the geographical origins have a greater influence on the P. yunnanensis samples than the cultivation years. Thirdly, the PLS-DA model based on aerial parts combined with high-level multi-source information fusion strategy and important variables selection method can reach the best classification effect (100%) as same as the models of individual dataset of rhizomes. Our study provides a reference for the rational utilization of aerial parts (stems and leaves) for P. yunnanensis such as in the field of fast and effectively classification researches.

Collision risk prediction for visually impaired people using high level information fusion

The technologies developed so far to help visually impaired people (VIP) navigate meet only some of their everyday needs. This project allows the visually impaired to improve the comprehension of their context by generating a risk map following an analysis of the position, distance, size and motion of the objects present in their environment. This comprehension is refined by data fusion steps applied to the High Level Information Fusion (HLIF) to predict possible impacts in the near future. A risk map is made up of probabilities generated after executing a set of inferences. These inferences allow the evaluation of future collision risks in different directions by detecting static objects, detecting free passage and analyzing paths followed by dynamic objects in a 3D plane. Different datasets were modeled and a comparative analysis was performed to check the percentage of correct answers and the accuracy of the inferences made using different classifiers. Thus, in order to demonstrate the advantages of the HLIF implementation in a dedicated VIP navigation system, the proposed architecture was tested against three other navigation systems that use different approaches. The generation of specific results made it possible to validate and compare these navigation systems. For this comparative analysis, different environments were used with the goal of indicating a direction for the VIP to move in with fewer collision risks. In addition to providing a risk map giving possible collisions, this project system provided greater reliability for navigation, especially when obstacles were very close and moving objects were detected and tracked.

Hybrid data fusion model for restricted information using Dempster–Shafer and adaptive neuro-fuzzy inference (DSANFI) system

Information fusion is the crux of data fusion which is used to compare large numerical data using normalization and aggregate function. The recent trends in information fusion have the limitations in fusing the important details in order to overcome the security issues in both low-level and high-level information fusion systems. The information from heterogeneous sources is different from one another like conceptual, contextual and graphical. This information is included for the fusion process, but the ancient approach has the limitation, and this research work proposes a new model of information fusion for the restricted content for various types of information. In this research work, an algorithm for information fusion is implemented for decision making based on Dempster–Shafer and adaptive neuro-fuzzy inference (DSANFI) system. Proposed hybrid work is applicable in data fusion process based on the theoretical approach of Dempster–Shafer and then into ANFIS. The proposed data fusion method is employed in a wide range of fields which include robotics, statistics, estimation and control.

A Bayesian Framework for Joint Target Tracking, Classification, and Intent Inference

Intent inference has attracted considerable interest for achieving situation awareness in the high-level information fusion community. Different from traditional tracking-then-inference methods for intent inference, a novel scheme for joint target tracking, classification, and intent inference (JTCI) is developed based on the Bayesian framework. The proposed JTCI scheme exploits the dependence of target state on target class and intent by defining intent and class dependent dynamic model sets. Then, the joint target state, intent, and class density are obtained recursively under the assumption, and the kinematic and attribute measurement processes are conditional independent. Finally, simulations about tracking in the air surveillance system are utilized to demonstrate the superiority of the proposed JTCI to the state-of-the-art JTC.

Road Scene Content Analysis for Driver Assistance and Autonomous Driving

This paper aims to develop a vision-based driver assistance system for scene awareness using video frames obtained from a dashboard camera. A saliency image map is devised with features pertinent to the driving scene. This saliency map mimics the human contour and motion sensitive visual perception by extracting spatial, spectral, and temporal information from the input frames and applying entropy driven image-context-feature data fusion. The resultant fusion output comprises high-level descriptors for still segment boundaries and non-stationary object appearance. Following the segmentation and foreground object detection stage, an adaptive maximum likelihood classifier selects road surface regions. The proposed scene driven vision system improves the driver’s situational awareness by enabling adaptive road surface classification. As experimental results demonstrate, context-aware low-level to high-level information fusion based on human vision model produces superior segmentation, tracking, and classification results that lead to high- level abstraction of driving scene.

Philosophical foundations for information fusion : towards a social-process paradigm

In the past decade the focus of research into artificial intelligence (AI) has moved from individual agents to multi-agent systems (MAS). This shift has been reflected in the increasing use of social metaphors to describe MAS-related design problems and solutions. The social interpretation of MAS has also been embraced by social scientists interested in developing MAS-based simulations of society. The design of some MAS applications, such as high-level information fusion, involves additional social complexities that arise from their embedded role in complex human social organizations. One response to these developments has been the plea for ‘socionic’ collaboration between computer scientists and sociologists akin to the inter-disciplinary collaboration that produced bionics. This suggestion has produced some useful results; however the socionic traffic of ideas has been hindered by the paradigm gulf that separates sociology and computer science. Dawson’s work is a step toward bridging the philosophical component of that gulf. Dawson’s goal is to develop a coherent system of concepts that encompasses both MAS-based and human forms of social interaction and will serve to frame and guide the development of ontology based multi-agent systems. To create this unifying vision Dawson draws upon the philosophy of A. N. Whitehead to develop a socially oriented, Whiteheadian philosophical interpretation of Dale Lambert’s Post-Classical MAS. While Whitehead’s work is relatively well known, Lambert’s unique process philosophy inspired approach to AI has attracted little attention. Drawing upon Lambert’s unpublished doctoral dissertation, Engineering Machines with Commonsense (1996), Dawson lays out the broad epistemic issues, metaphysical presuppositions, and heuristic strategies that have motivated Lambert’s turn from Classical AI to his process inspired alternative: ‘Post-Classical AI’. Lambert’s primary objective in his dissertation was to develop a Post-Classical blue-print for a MAS-based mind. To that end Lambert developed Post-Classical philosophies of process, representation and belief, and showed how each of these elements could be formally implemented to provide a Post-Classical artificial mind with its cognitive capability. In order to reveal the deeper implications of Lambert’s tersely expressed debt to process philosophy, Dawson provides a commentary which explains the key formal elements of Lambert’s work in terms of Whitehead’s metaphysical system. Through this interpretation it is possible to view the computational operations of Lambert’s MAS as a partial model of the complex creative processes that lie at the heart of Whitehead’s metaphysical vision. This interpretation of Lambert's Post-Classical paradigm signals the possibility of a novel 'Whiteheadian AI'. A key aspect of Dawson’s analysis is his identification of the ‘sociological turn’ latent in Lambert’s Post-Classical paradigm. Lambert’s approach to grounding agent belief is analogous in form to Whitehead’s explanation of the relationship between human ideas and the actualities that they denote. In both cases agents shape and are shaped by social processes that selectively fine tune and prioritize the social customs that mediate interaction between agents and their environments. The sociological dimension of Lambert’s Post-Classical paradigm is further illustrated with reference to the ontology-based multi-agent architecture at the core of Lambert’s design of a high-level information fusion system. For nearly two decades a process vision has inspired and guided Lambert’s internationally recognized contribution to the development of high-level information fusion systems. Interpreted in the light of Whitehead’s philosophy, the striking originality of Lambert’s Post-Classical paradigm and subsequent work in information fusion is revealed. Dawson concludes that Lambert’s Post-Classical paradigm represents a promising nexus of philosophical vision, formal method, and practical technology and outlines his own socionic proposal for its further development.

A System-of-Systems perspective for information fusion system design and evaluation

This paper provides a System-of-Systems (SoS) perspective for integrated design and evaluation of an Information Fusion System (IFS). IFS is comprised of distributed and heterogeneous systems that accomplish low-level and high-level information fusion (LLIF and HLIF) functionality. LLIF and HLIF functions are developed independent from one another but require collaboration to achieve the IFS mission objectives. The distribution and heterogeneity of systems, in addition to the multiplicity of LLIF and HLIF functions, creates an extensively large design space for the IFS. We apply a SoS engineering architecting process to obtain integrated architectures of IFS and propose guidelines to constrain an otherwise infinite design space of Information Fusion System-of-Systems (IF-SoS). Furthermore, we elaborate a multi-agent system modeling approach and pair it with Design of Experiments for objective evaluation of the IF-SoS design space. The statistical analysis, based on analysis of variance (ANOVA) and Tukey Honest Significant Difference (HSD) Range Tests, quantifies the impact of interactions between LLIF and HLIF design considerations on the IF-SoS performance. Furthermore, statistical evidence is provided to demonstrate that the interactions among JDL levels, in particular between LLIF and HLIF, are the most significant design considerations for fusion performance which necessitate an integrated design and evaluation of LLIF and HLIF—a manifestation of the SoS perspective for the IFS.

Multi-layer ontology based information fusion for situation awareness

Originated from the military domain, Situation Awareness (SAW) is proposed with the aim to obtain information superiority through information fusion and thus to achieve decision superiority. It requires not only the perception of the environment, but also the reasoning of the implicit or implicated meaning under the explicit phenomenon. The principal goal of this paper is to exploit the semantic web technologies to enhance the situation awareness through autonomous information fusion and inference. Recently, ontology has played a significant role in the representation and integration of domain knowledge for high-level reasoning. The multi-level ontology merging paradigm is followed in this work for the conceptual modeling and knowledge representation. Firstly, Military Scenario Ontology (MSO) and Battle Management Ontology (BMO) are defined according to corresponding reputable standards as the domain ontology. We propose the Situation Awareness Ontology (SAO) as the core ontology to integrate MSO, BMO and even other publicly defined ontology for higher-level information fusion. The SAO is composed of objects representations, relations and events that are necessary to capture the information for further cognition, reasoning and decision-making about the situation evolving over time. Military doctrines and domain knowledge are expressed as Horn clause type rules for reasoning and inference. Multi-layered semantic information fusion that integrates ontologies, semantic web technologies and rule-based reasoning can therefore be conducted. An experimental scenario is presented to demonstrate the feasibility of this architecture.

Attention Assist: A High-Level Information Fusion Framework for Situation and Threat Assessment in Vehicular Ad Hoc Networks

Driver inattentiveness constitutes the main cause of road accidents, which makes it a major factor in road safety. In this paper, we propose a comprehensive framework to address the road safety problem by tackling it from a high-level information fusion standpoint, considering vehicular ad hoc networks (VANETs) as the deployment platform. The proposed framework relies on the multientity Bayesian networks (MEBNs), which exploit the expressiveness of first-order logic for semantic relations, and the strength of the Bayesian networks in handling uncertainty. First, the entities that influence the inattention phenomenon, as well as both their causal and semantic relationships, are identified. Next, an MEBN-based high-level information fusion framework is proposed through which entities, situations, and their relationships in specific contexts are modeled using MEBN fragments. Furthermore, MEBN inference is used to assess the situations of interest by estimating their states. To demonstrate the capabilities of the proposed framework, a collision warning system simulator has been developed, which evaluates the likelihood of a vehicle being in a near-collision situation using a wide variety of local and global information sources available in various VANET environments. If the threat of being in a near-collision situation is determined to be high, then the driver is warned accordingly. Our experimental results for two distinct single-vehicle and multivehicle categories of driving scenarios, as well as a novel hybrid MEBN inference, demonstrate the capability of the proposed framework to efficiently achieve situation and threat assessment on the road.

Human-machine conversations to support multi-agency missions

In domains such as emergency response, environmental monitoring, policing and security, sensor and information networks are deployed to assist human users across multiple agencies to conduct missions at or near the 'front line'. These domains present challenging problems in terms of human-machine collaboration: human users need to task the network to help them achieve mission objectives, while humans (sometimes the same individuals) are also sources of mission-critical information. We propose a natural language-based conversational approach to supporting humanmachine working in mission-oriented sensor networks. We present a model for human-machine and machine-machine interactions in a realistic mission context, and evaluate the model using an existing surveillance mission scenario. The model supports the flow of conversations from full natural language to a form of Controlled Natural Language (CNL) amenable to machine processing and automated reasoning, including high-level information fusion tasks. We introduce a mechanism for presenting the gist of verbose CNL expressions in a more convenient form for human users. We show how the conversational interactions supported by the model include requests for expansions and explanations of machine-processed information.

直觉云推理及其在空中目标意图分析中的应用

针对高层信息融合中的不确定性推理问题，提出了直觉云模型和直觉云推理方法。首先，考虑到问题的直觉模糊性和随机性，将云模型与直觉模糊集理论相结合，提出了直觉云模型，实现定性概念与其定量表示之间的不确定性转换；然后，根据专家知识构建推理规则库，设计直觉云推理算法，实现不确定性推理；最后，将直觉云推理方法应用到空中目标攻击意图分析中，说明了直觉云推理方法的有效性和实用性。 To solve the questions of uncertain reasoning in high level information fusion, intuitionistic cloud model and intuitionistic cloud reasoning are proposed. First, considering the intuitionistic fuzziness and randomness, intuitionistic cloud model is proposed by combining the cloud model and intuitionistic fuzzy set theroy. The intuitionistic cloud model can establish transformation of un-certainty between the qualitative concept and the quantitative description. Then, the rules of rea-soning are constructed according to the expert knowledge, the algorithms of reasoning are designed and uncertain reasoning is carried out. Finally, the approach based on intuitionistic cloud reasoning is used for attacking intention analysis of aerial target, which demonstrates the validity and practicability of the intuitionistic cloud reasoning method.

High-level geospatial information discovery and fusion for geocoded multimedia

Purpose – Improvements and portability of technologies and smart devices have enabled a rapid growth in the amount of user-generated media such as photographs and videos. Whilst various media generation and management systems exist, it still remains a challenge to discover the right information, for the right purpose. This paper aims to propose an approach to reverse geocoding by cross-referencing multiple geospatial data sources to enable the enrichment of media and therefore enable better organisation and searching of the media to create an overall picture about places. Design/methodology/approach – The paper presents a system architecture that incorporates the proposed approach to aggregate several geospatial databases to enrich geo-tagged media with human readable information, which will further enable the goal of creating an overall picture about places. The approach enables the semantic information relating to point of interest. Findings – Implementation of the proposed approach shows that a single geospatial data source does not contain enough information to accurately describe the high-level geospatial information for geocoded multimedia. However, fusing several geospatial data sources together enables richer, more accurate high-level geospatial information to be tagged to the geocoded multimedia. Originality/value – The contribution in this paper shows that high-level geospatial information can be retrieved from many data sources and fused together to enrich geocoded multimedia which can facilitate better searching and retrieval of the multimedia.

High Level Information Fusion (HLIF): Survey of models, issues, and grand challenges

High-level information fusion (situation and threat assessment, process and user refinement) requires novel solutions for the operational transition of information fusion designs. Low-level (signal processing, object state estimation and characterization) is well-vetted in the community as compared to high-level information fusion (control and relationships to the environment). Specific areas of interest include modeling (situations, environments), representations (semantic, knowledge, and complex), information management (ontologies, protocols) systems design (scenario-based, user-based, distributed-agent) and evaluation (measures of performance/effectiveness, and empirical case studies).

High level information fusion for tracking and projection of multistage cyber attacks

The use of computer networks has become a necessity for government, industry, and personal businesses. Protection and defense against cyber attacks on computer networks, however, are becoming inadequate as attackers become more sophisticated and as the networks and systems become more complex. Drawing analogies from other application domains, this paper introduces information fusion to provide situation awareness and threat prediction from massive volumes of sensed data. An in-depth discussion is provided to define fusion tasks for cyber defense. A novel cyber fusion system is proposed to address specifically the tracking and projection of multistage attacks. Critical assessments of the developed attack tracking and threat projection sub-components are provided with simulation results. This pioneering work elaborates the benefits, limitations, and future challenges of high level information fusion for cyber security.