High-level Context Information Research Articles

EACNet: Enhanced Asymmetric Convolution for Real-Time Semantic Segmentation

Although deep neural networks have made significant progress in semantic segmentation, speed and computational cost still can't meet the strict requirements of real-world applications. In this paper, we present an enhanced asymmetric convolution network (EACNet) to seek a balance between accuracy and speed. Specifically, we design a pair of enhancing asymmetric convolution modules constructed by depth-wise asymmetric convolution and dilated convolution to extract short-range and long-range features, which are efficient and powerful. Additionally, we apply a bilateral structure in which the detail branch preserves low-level spatial details while the semantic branch captures high-level context information. The two branches are merged at different stages of the network to strengthen information propagation between different levels. The experiments on the Cityscapes dataset show that our method achieves high accuracy and speed with relatively small parameters. Compared with other real-time semantic segmentation methods, our network attains a good trade-off among parameters, speed, and accuracy.

MSB-FCN: Multi-Scale Bidirectional FCN for Object Skeleton Extraction.

The performance of state-of-the-art object skeleton detection (OSD) methods have been greatly boosted by Convolutional Neural Networks (CNNs). However, the most existing CNN-based OSD methods rely on a 'skip-layer' structure where low-level and high-level features are combined to gather multi-level contextual information. Unfortunately, as shallow features tend to be noisy and lack semantic knowledge, they will cause errors and inaccuracy. Therefore, in order to improve the accuracy of object skeleton detection, we propose a novel network architecture, the Multi-Scale Bidirectional Fully Convolutional Network (MSB-FCN), to better gather and enhance multi-scale high-level contextual information. The advantage is that only deep features are used to construct multi-scale feature representations along with a bidirectional structure for better capturing contextual knowledge. This enables the proposed MSB-FCN to learn semantic-level information from different sub-regions. Moreover, we introduce dense connections into the bidirectional structure to ensure that the learning process at each scale can directly encode information from all other scales. An attention pyramid is also integrated into our MSB-FCN to dynamically control information propagation and reduce unreliable features. Extensive experiments on various benchmarks demonstrate that the proposed MSB-FCN achieves significant improvements over the state-of-the-art algorithms.

EHANet: An Effective Hierarchical Aggregation Network for Face Parsing

In recent years, benefiting from deep convolutional neural networks (DCNNs), face parsing has developed rapidly. However, it still has the following problems: (1) Existing state-of-the-art frameworks usually do not satisfy real-time while pursuing performance; (2) similar appearances cause incorrect pixel label assignments, especially in the boundary; (3) to promote multi-scale prediction, deep features and shallow features are used for fusion without considering the semantic gap between them. To overcome these drawbacks, we propose an effective and efficient hierarchical aggregation network called EHANet for fast and accurate face parsing. More specifically, we first propose a stage contextual attention mechanism (SCAM), which uses higher-level contextual information to re-encode the channel according to its importance. Secondly, a semantic gap compensation block (SGCB) is presented to ensure the effective aggregation of hierarchical information. Thirdly, the advantages of weighted boundary-aware loss effectively make up for the ambiguity of boundary semantics. Without any bells and whistles, combined with a lightweight backbone, we achieve outstanding results on both CelebAMask-HQ (78.19% mIoU) and Helen datasets (90.7% F1-score). Furthermore, our model can achieve 55 FPS on a single GTX 1080Ti card with 640 × 640 input and further reach over 300 FPS with a resolution of 256 × 256, which is suitable for real-world applications.

Progressive Feature Polishing Network for Salient Object Detection

Feature matters for salient object detection. Existing methods mainly focus on designing a sophisticated structure to incorporate multi-level features and filter out cluttered features. We present Progressive Feature Polishing Network (PFPN), a simple yet effective framework to progressively polish the multi-level features to be more accurate and representative. By employing multiple Feature Polishing Modules (FPMs) in a recurrent manner, our approach is able to detect salient objects with fine details without any post-processing. A FPM parallelly updates the features of each level by directly incorporating all higher level context information. Moreover, it can keep the dimensions and hierarchical structures of the feature maps, which makes it flexible to be integrated with any CNN-based models. Empirical experiments show that our results are monotonically getting better with increasing number of FPMs. Without bells and whistles, PFPN outperforms the state-of-the-art methods significantly on five benchmark datasets under various evaluation metrics. Our code is available at: https://github.com/chenquan-cq/PFPN.

Knowledge Integration Networks for Action Recognition

In this work, we propose Knowledge Integration Networks (referred as KINet) for video action recognition. KINet is capable of aggregating meaningful context features which are of great importance to identifying an action, such as human information and scene context. We design a three-branch architecture consisting of a main branch for action recognition, and two auxiliary branches for human parsing and scene recognition which allow the model to encode the knowledge of human and scene for action recognition. We explore two pre-trained models as teacher networks to distill the knowledge of human and scene for training the auxiliary tasks of KINet. Furthermore, we propose a two-level knowledge encoding mechanism which contains a Cross Branch Integration (CBI) module for encoding the auxiliary knowledge into medium-level convolutional features, and an Action Knowledge Graph (AKG) for effectively fusing high-level context information. This results in an end-to-end trainable framework where the three tasks can be trained collaboratively, allowing the model to compute strong context knowledge efficiently. The proposed KINet achieves the state-of-the-art performance on a large-scale action recognition benchmark Kinetics-400, with a top-1 accuracy of 77.8%. We further demonstrate that our KINet has strong capability by transferring the Kinetics-trained model to UCF-101, where it obtains 97.8% top-1 accuracy.

An effective automatic system deployed in agricultural Internet of Things using Multi-Context Fusion Network towards crop disease recognition in the wild

Automatic crop disease recognition in the wild is a challenging topic in modern intelligent agriculture due to the appearance variances and cluttered background among crop diseases. To overcome these obstacles, the popular methods are to design a Convolutional Neural Network (CNN) model that extracts visual features and identifies crop disease images based on these features. These methods work well on laboratory environment under simple background but achieve low accuracy and poor robustness in processing the raw images captured from practical fields that contain inevitable noises. In this case, Internet of Things (IoT) is attracting increasing attention, with many alternatives to collect high-level contextual information that helps modern recognition system to effectively identify crop diseases in the wild. Motivated by the usefulness of agricultural IoT, a deep learning system using a novel approach named Multi-Context Fusion Network (MCFN), is developed to be deployed in agricultural IoT towards practical crop disease recognition in the wild. Our MCFN firstly adopts a standard CNN backbone to extract highly discriminative and robust visual features from over 50,000 in-field crop disease samples. Next, we exploit contextual features collected from image acquisition sensors as prior information to assist crop disease classification and reduce false positives in our presented ContextNet. Finally, a deep fully connected network is designed to fuse visual features as well as contextual features and output the crop disease prediction. Experimental results on 77 common crop diseases captured in our newly built domain specific dataset show that MCFN with the deep fusion model outperforms the state-of-the-art methods in wild crop disease recognition, and achieves a good identification accuracy of 97.5%.

The effect of contextual plausibility on word skipping during reading

Recent eye-movement evidence suggests readers are more likely to skip a high-frequency word than a low-frequency word independently of the semantic or syntactic acceptability of the word in the sentence. This has been interpreted as strong support for a serial processing mechanism in which the decision to skip a word is based on the completion of a preliminary stage of lexical processing prior to any assessment of contextual fit. The present large-scale study was designed to reconcile these findings with the plausibility preview effect: higher skipping and reduced first-pass reading times for words that are previewed by contextually plausible, compared to implausible, sentence continuations that are unrelated to the target word. Participants' eye movements were recorded as they read sentences containing a short (3–4 letters) or long (6 letters) target word. The boundary paradigm was used to present parafoveal previews which were either higher or lower frequency than the target, and either plausible or implausible in the sentence context. The results revealed strong, independent effects of all three factors on target skipping and early measures of target fixation duration, while frequency and plausibility interacted on later measures of target fixation duration. Simulations using the E-Z Reader model of eye-movement control in reading demonstrated that plausibility effects on skipping are potentially consistent with the assumption that higher-level contextual information only affects post-lexical integration processes. However, no current model of eye movements in reading provides an explicit account of the information or processes that allow readers to rapidly detect an integration failure.

Adapting Semantic Segmentation Models for Changes in Illumination and Camera Perspective

Semantic segmentation using deep neural networks has been widely explored to generate high-level contextual information for autonomous vehicles. To acquire a complete $180^\circ$ semantic understanding of the forward surroundings, we propose to stitch semantic images from multiple cameras with varying orientations. However, previously trained semantic segmentation models showed unacceptable performance after significant changes to the camera orientations and the lighting conditions. To avoid time-consuming hand labeling, we explore and evaluate the use of data augmentation techniques, specifically skew and gamma correction, from a practical real-world standpoint to extend the existing model and provide more robust performance. The presented experimental results have shown significant improvements with varying illumination and camera perspective changes.

Intelligent Context-Aware Communication Paradigm Design for IoVs Based on Data Analytics

IoVs have been envisioned to improve road safety and efficiency, and provide Internet access on the move, by providing a myriad of safety and infotainment applications to drivers and passengers. However, with limited spectrum resource, harsh wireless channel, and variable vehicle density, IoV communication faces severe challenges to achieve scalability, efficiency, and reliability. In this article, we propose a context-aware IoV paradigm design to enhance the communication performance, where the high-level contextual information is utilized to bring intelligence in the design. Specifically, through big data analytics on large-scale IoV communication traces collected from an extensive experiment conducted in Shanghai, we investigate the impacts of different contextual information on V2V communication performance. We reveal that among many types of contextual information, the NLoS link condition is a major one that significantly affects V2V link performance. Based on that observation, we discuss three critical but challenging communication paradigm designs with context awareness of V2V link conditions: smart medium resource allocation, efficient routing establishment, and reliable safety message broadcasting. Furthermore, we present a case study of a cooperative beaconing scheme, where machine learning methods are utilized to learn the real-time link contextual information, and vehicles in deep NLoS condition choose helpers to enhance the overall beaconing reliability.

A service-oriented context-awareness reasoning framework and its implementation

Purpose This paper aims to propose an extensible, service-oriented framework for context-aware data acquisition, description, interpretation and reasoning, which facilitates the development of mobile applications that provide a context-awareness service. Design/methodology/approach First, the authors propose the context data reasoning framework (CDRFM) for generating service-oriented contextual information. Then they used this framework to composite mobile sensor data into low-level contextual information. Finally, the authors exploited some high-level contextual information that can be inferred from the formatted low-level contextual information using particular inference rules. Findings The authors take “user behavior patterns” as an exemplary context information generation schema in their experimental study. The results reveal that the optimization of service can be guided by the implicit, high-level context information inside user behavior logs. They also prove the validity of the authors’ framework. Research limitations/implications Further research will add more variety of sensor data. Furthermore, to validate the effectiveness of our framework, more reasoning rules need to be performed. Therefore, the authors may implement more algorithms in the framework to acquire more comprehensive context information. Practical implications CDRFM expands the context-awareness framework of previous research and unifies the procedures of acquiring, describing, modeling, reasoning and discovering implicit context information for mobile service providers. Social implications Support the service-oriented context-awareness function in application design and related development in commercial mobile software industry. Originality/value Extant researches on context awareness rarely considered the generation contextual information for service providers. The CDRFM can be used to generate valuable contextual information by implementing more reasoning rules.

Dense RGB-D Semantic Mapping with Pixel-Voxel Neural Network.

In this paper, a novel Pixel-Voxel network is proposed for dense 3D semantic mapping, which can perform dense 3D mapping while simultaneously recognizing and labelling the semantic category each point in the 3D map. In our approach, we fully leverage the advantages of different modalities. That is, the PixelNet can learn the high-level contextual information from 2D RGB images, and the VoxelNet can learn 3D geometrical shapes from the 3D point cloud. Unlike the existing architecture that fuses score maps from different modalities with equal weights, we propose a softmax weighted fusion stack that adaptively learns the varying contributions of PixelNet and VoxelNet and fuses the score maps according to their respective confidence levels. Our approach achieved competitive results on both the SUN RGB-D and NYU V2 benchmarks, while the runtime of the proposed system is boosted to around 13 Hz, enabling near-real-time performance using an i7 eight-cores PC with a single Titan X GPU.

A Knowledge Fusion Approach for Context Awareness in Vehicular Networks

Vehicular ad-hoc networks (VANETs) are a challenging Internet of Things scenario. While research is proposing increasingly sophisticated hardware and software solutions for on-board context detection, probably high-level context information sharing has not been adequately addressed so far. This paper proposes a novel logic-based framework enabling a contextual data management and mining in VANETs. It grounds on a knowledge fusion algorithm based on nonstandard, nonmonotonic inference services in Description Logics, adopting standard Semantic Web languages. Ontology-referred context annotations produced by individual VANET nodes are merged with automatic reconciliation of inconsistencies. An efficient information dissemination protocol complements the proposal. The approach has been implemented in a vehicular network simulator and early experimental results proved its effectiveness and feasibility.

Automatic Raft Labeling for Remote Sensing Images via Dual-Scale Homogeneous Convolutional Neural Network

Raft-culture is a way of utilizing water for farming aquatic product. Automatic raft-culture monitoring by remote sensing technique is an important way to control the crop’s growth and implement effective management. This paper presents an automatic pixel-wise raft labeling method based on fully convolutional network (FCN). As rafts are always tiny and neatly arranged in images, traditional FCN method fails to extract the clear boundary and other detailed information. Therefore, a homogeneous convolutional neural network (HCN) is designed, which only consists of convolutions and activations to retain all details. We further design a dual-scale structure (DS-HCN) to integrate higher-level contextual information for accomplishing sea–land segmentation and raft labeling at the same time in a uniform framework. A dataset with Gaofen-1 satellite images was collected to verify the effectiveness of our method. DS-HCN shows a satisfactory performance with a better interpretability and a more accurate labeling result.

Multi-Scale Bidirectional FCN for Object Skeleton Extraction

Object skeleton detection is a challenging problem with wide application. Recently, deep Convolutional Neural Networks (CNNs) have substantially improved the performance of the state-of-the-art in this task. However, most of the existing CNN-Based methods are based on a skip-layer structure where low-level and high-level features are combined and learned so as to gather multi-level contextual information. As shallow features are too messy and lack semantic knowledge, they may cause errors and inaccuracy. Therefore, we propose a novel network architecture, Multi-Scale Bidirectional Fully Convolutional Network (MSB-FCN), to better capture and consolidate multi-scale high-level context information for object skeleton detection. Our network uses only deep features to build multi-scale feature representations, and employs a bidirectional structure to collect contextual knowledge. Hence the proposed MSB-FCN has the ability to learn the semantic-level information from different sub-regions. Furthermore, we introduce dense connections into the bidirectional structure of our MSB-FCN to ensure that the learning process at each scale can directly encode information from all other scales. Extensive experiments on various commonly used benchmarks demonstrate that the proposed MSB-FCN has achieved significant improvements over the state-of-the-art algorithms.

Higher Order Support Vector Random Fields for Hyperspectral Image Classification

This paper addresses the problem of contextual hyperspectral image (HSI) classification. A novel conditional random fields (CRFs) model, known as higher order support vector random fields (HSVRFs), is proposed for HSI classification. By incorporating higher order potentials into a support vector random fields with a Mahalanobis distance boundary constraint (SVRFMC) model, the HSVRFs model not only takes advantage of the support vector machine (SVM) classifier and the Mahalanobis distance boundary constraint, but can also capture higher level contextual information to depict complicated details in HSI. The higher order potentials are defined on image segments, which are created by a fast unsupervised over-segmentation algorithm. The higher order potentials consider the spectral vectors of each of the segment’s constituting pixels coherently, and weight these pixels with the output probability of the support vector machine (SVM) classifier in our framework. Therefore, the higher order potentials can model higher-level contextual information, which is useful for the description of challenging complex structures and boundaries in HSI. Experimental results on two publicly available HSI datasets show that the HSVRFs model outperforms traditional and state-of-the art methods in HSI classification, especially for datasets containing complicated details.

A Fuzzy based Diagnostic Agent for Context Aware Patient Monitoring

It is widely known that Remote healthcare monitoring can be a solution to provide an alternative healthcare service that can reduce the amount of strain that the current health care systems experience with the ever increasing demand of health care services world wide. However monitoring a patient remotely requires an accurate interpretation of the data regarding the patients condition. This position paper presents a Fuzzy expert system used to reason medical contexts from a Body Area Network consisting of several sensors monitoring vital healthcare indicators. Fuzzy Logic is used to aggregate the data from the sensors and handle the reasoning required to abstract high-level context information from the low-level context from the sensors and determine the patients condition.

Integrating Social Grouping for Multitarget Tracking Across Cameras in a CRF Model

Tracking multiple targets across nonoverlapping cameras aims at estimating the trajectories of all targets, and maintaining their identity labels consistent while they move from one camera to another. Matching targets from different cameras can be very challenging, as there might be significant appearance variation and the blind area between cameras makes the target’s motion less predictable. Unlike most of the existing methods that only focus on modeling the appearance and spatiotemporal cues for inter-camera tracking, this paper presents a novel online learning approach that considers integrating high-level contextual information into the tracking system. The tracking problem is formulated using an online learned conditional random field (CRF) model that minimizes a global energy cost. Besides low-level information, social grouping behavior is explored in order to maintain targets’ identities as they move across cameras. In the proposed method, pairwise grouping behavior of targets is first learned within each camera. During inter-camera tracking, track associations that maintain single camera grouping consistencies are preferred. In addition, we introduce an iterative algorithm to find a good solution for the CRF model. Comparison experiments on several challenging real-world multicamera video sequences show that the proposed method is effective and outperforms the state-of-the-art approaches.

A Social Approach to High-level Context Generation for Supporting Context-aware M-Learning

In m-learning environments, context-awareness is for wide use where learners’ situations are varied, dynamic and unpredictable. We are facing the challenge of requirements of both generality and depth in generating and processing high-level context. In this paper, we present a social approach which exploits social dynamics and social computing for generating high-level context. It is a novel and generic paradigm where the crowds of learners in m-learning environments directly engage in creating contents about high-level context and interactions by social tagging, and these contents and interactions are further explored to discover more implicit and complex high-level contextual information. We present the concept model, the context representation, the context matrix, and the context retrieval method. We evaluate our approach by a social simulation based experiment. The experimental results demonstrate that the context retrieval performance is improved in both the accuracy and the diversity, and validate that the proposed social approach is effective for generating high-level context.

The Fusion Model of Multidomain Context Information for the Internet of Things

The Internet of Things aims to provide the user with deep adaptive intelligence services according to the user’s personalized characteristics. Most of the characteristics are presented in the form of high-level context. But it often lacks methods to obtain high-level context information directly in the Internet of Things. In this paper, so as to achieve the corresponding high-level context information using the specific low-level multidomain context directly obtained by different sensors in the Internet of Things, we present a machine learning method to construct a context fusion model based on the feature selection algorithm and the multiclassification algorithm. First, we propose a wrapper feature selection method based on the genetic algorithm to obtain a simpler and more important subset of the context features from the low-level multidomain context, by defining a suitable fitness function and a convergence condition. Then, we use the decision tree algorithm which is a multiclassification algorithm, based on the rules obtained by training the subset of context features, to determine which high-level context the record set of the low-level context information belongs to. Experiments confirm that the model can be used to achieve higher classification accuracy without more significant time consumption.

Ontology-Based High-Level Context Inference for Human Behavior Identification

Recent years have witnessed a huge progress in the automatic identification of individual primitives of human behavior, such as activities or locations. However, the complex nature of human behavior demands more abstract contextual information for its analysis. This work presents an ontology-based method that combines low-level primitives of behavior, namely activity, locations and emotions, unprecedented to date, to intelligently derive more meaningful high-level context information. The paper contributes with a new open ontology describing both low-level and high-level context information, as well as their relationships. Furthermore, a framework building on the developed ontology and reasoning models is presented and evaluated. The proposed method proves to be robust while identifying high-level contexts even in the event of erroneously-detected low-level contexts. Despite reasonable inference times being obtained for a relevant set of users and instances, additional work is required to scale to long-term scenarios with a large number of users.