Multi-source Data Integration Research Articles

Research on Intelligent Flight Test Monitoring Technology Based on Expert System

The current manual-based traditional flight test monitoring technology can no longer fulfill the requirements of high integration, system complexity and technological advancement of new aircraft. In addition, the existing flight monitoring systems are mostly single-point, single-system applications, including ground-assisted systems such as meteorology, radar, and response system. It is easy to cause problems such as data islands, which is not conducive to data sharing and comprehensive evaluation and analysis. This paper proposes a new method for analyzing and mining existing test data based on expert system technology to build a flight test expert system library. A new real-time monitoring system is designed that can meet the real-time processing and integration of multi-source data, comprehensive evaluation and analysis, intelligent warning and assistant decision-making. This system expands the function and intelligence of the existing system, and further improves the quality, safety and efficiency of the flight test monitoring system.

Development of Virtual disassembly and assembly platform for marine air compressor

The use of virtual reality technology to assist equipment maintenance and skills training has a broad application prospect, but the existing systems are often lack of sense of reality, over-simplified because it’s difficult to establish the disassembly process model, the lack of 3D interaction design, etc. Taking marine air compressor as an example, this paper analyzes the functional requirements of the virtual disassembly and assembly platform, studies the key technologies of multi-source data integration, 3d modeling with high sense of reality and high complexity disassembly process modeling, uses Maya/Unity3D and other software to realize relevant technologies and functions, and gives the development process of the system. The developed system has been applied in many maritime institutions in China. It has been demonstrated with reference value and application prospect.

Multisource data integration for groundwater prospecting in Precambrian shear zones, Espírito Santo State (Southeastern Brazil)

The Precambrian metamorphic rocks in the southern region of the Espírito Santo State(southeastern Brazil) is intercepted by ductile and brittle/ductile shear zones over which towns like Guaçuí, Marechal Floriano and Domingos Martins were established. In the years of 2014 to 2016 the Espírito Santo State faced the worst drought in the past 80 years. In situations like that, the only source of fresh water available is underground. Therefore, the purpose of this study was to integrate multisource data for groundwater prospecting and management. The study area comprises the central sectors of the Guaçuí and Batatal shear zones. Methodology involved the interpretation of stereo-pairs for the identification of lineaments, electrical resistivity surveys and geological fieldwork. The results were tentatively validated with available groundwater well production data. Six resistivity profiles with a total length of 2,400 m were surveyed and 15 outcrops were described across these structures. Results have shown that the structural framework of the shear zones is predominantly made of NW to NNW and NE to NNE-trending lineaments. The former is related to brittle tectonics and the latter coincides with the ductile Precambrian fabric of the crystalline basement rocks. A near-surface porous aquifer regarded to the regolith and recent unconsolidated sedimentary deposits presenting resistivity values < 1,372 Ohm.m and depths < 30 m have good potential for groundwater storage. However, results arenot conclusive about the hydrogeological significance of tectonic structures for groundwaterprospection in the study area.

Research on the Construction of a Knowledge Graph and Knowledge Reasoning Model in the Field of Urban Traffic

The integration of multi-source transportation data is complex and insufficient in most of the big cities, which made it difficult for researchers to conduct in-depth data mining to improve the policy or the management. In order to solve this problem, a top-down approach is used to construct a knowledge graph of urban traffic system in this paper. First, the model layer of the knowledge graph was used to realize the reuse and sharing of knowledge. Furthermore, the model layer then was stored in the graph database Neo4j. Second, the representation learning based knowledge reasoning model was adopted to implement knowledge completion and improve the knowledge graph. Finally, the proposed method was validated with an urban traffic data set and the results showed that the model could be used to mine the implicit relationship between traffic entities and discover traffic knowledge effectively.

LeapR: An R Package for Multiomic Pathway Analysis.

A generalized goal of many high-throughput data studies is to identify functional mechanisms that underlie observed biological phenomena, whether they be disease outcomes or metabolic output. Increasingly, studies that rely on multiple sources of high-throughput data (genomic, transcriptomic, proteomic, metabolomic) are faced with a challenge of summarizing the data to generate testable hypotheses. However, this requires a time-consuming process to evaluate numerous statistical methods across numerous data sources. Here, we introduce the leapR package, a framework to rapidly assess biological pathway activity using diverse statistical tests and data sources, allowing facile integration of multisource data. The leapR package with a user manual and example workflow is available for download from GitHub (https://github.com/biodataganache/leapR).

Building scientific capacity in disaster risk reduction for sustainable development

As climate warming intensifies, the frequency and intensity of disasters are also increasing, posing challenges to global sustainable development. The concept of disaster risk reduction (DRR) provides strong impetus for reducing disaster risk and vulnerabilities by employing the scientific and technological developments of recent decades. However, there is a need to enhance the capacities of different communities to use emerging digital infrastructure, not only in promoting DRR but also in ensuring sustainable future development. Limited access to and availability of data are restricting comprehensive understanding of these challenges. In many countries, the key areas for capacity development include collecting information from alternative and emerging data sources and meaningfully integrating it with data from traditional sources. Software and data analysis are becoming widely accessible due to open-source initiatives, while cloud computing technologies and programmes such as CASEarth provide valuable resources for multisource data integration, contributing to information-driven policy and decision-support systems for DRR.

Mapping Paddy Rice with Satellite Remote Sensing: A Review

Paddy rice is a staple food of three billion people in the world. Timely and accurate estimation of the paddy rice planting area and paddy rice yield can provide valuable information for the government, planners and decision makers to formulate policies. This article reviews the existing paddy rice mapping methods presented in the literature since 2010, classifies these methods, and analyzes and summarizes the basic principles, advantages and disadvantages of these methods. According to the data sources used, the methods are divided into three categories: (I) Optical mapping methods based on remote sensing; (II) Mapping methods based on microwave remote sensing; and (III) Mapping methods based on the integration of optical and microwave remote sensing. We found that the optical remote sensing data sources are mainly MODIS, Landsat, and Sentinel-2, and the emergence of Sentinel-1 data has promoted research on radar mapping methods for paddy rice. Multisource data integration further enhances the accuracy of paddy rice mapping. The best methods are phenology algorithms, paddy rice mapping combined with machine learning, and multisource data integration. Innovative methods include the time series similarity method, threshold method combined with mathematical models, and object-oriented image classification. With the development of computer technology and the establishment of cloud computing platforms, opportunities are provided for obtaining large-scale high-resolution rice maps. Multisource data integration, paddy rice mapping under different planting systems and the connection with global changes are the focus of future development priorities.

Technical progress of China’s national remote sensing mapping: from mapping western China to national dynamic mapping

ABSTRACT Remote sensing mapping is an important research direction in the development of geographic surveying and mapping. In order to successfully implement the project of Mapping Western China (MWC), a technical mapping system has been established. In this project, many problems have been solved through technological innovation, such as block adjustment with scarce control points, large-scale aerial/satellite image mapping, and intelligent interpretation of multi-source images. Several softwares were developed, e.g. PixelGrid for aerial/satellite image mapping in a large area, FeatureStation for the integration of multi-source data in the complex terrain areas, and an airborne multi-band and multi-polarization interferometric data acquisition system for SAR mapping. For the first time, full coverage of 1: 50,000 topographic data of China’s land territory has been produced, which means the geospatial framework of digital China is basically completed. With the implementation of other key national plans and projects (i.e. national geographic conditions monitoring and national remote sensing mapping), the focus has changed from MWC to national dynamic mapping. Accordingly, a dynamic mapping system is established. The data acquisition capability has developed from a single source to multiple sources and multiple modalities. The mapping capability has developed into dynamic mapping, and the capability for database update shows the characteristics of collaboration. The national geographic condition monitoring creates a multi-scale index system for statistical analysis for various needs. A multi-level and multi-dimensional technical system for statistical computing and decision-making service is developed for the transformation from dynamic monitoring to information service. In this paper, we give a brief introduction about the recent development of remote sensing mapping in China with respect to data acquisition, map production, and information service. The purpose of this paper is to motivate the establishment of theory and method for remote sensing mapping, technical and equipment in the smart mapping era, to improve the capability of perceiving, analyzing, mining, and applying geographic data, and to promote the intelligent development of geographic surveying and mapping.

全球主要生态退化区和研究热点区的空间分布与演变

掌握生态退化区和研究热点区的空间分布、退化区生态系统的演变态势是认识生态问题、开展生态治理的基础，但目前缺乏全球主要生态退化区空间分布图等基础数据和相关知识。应用多源数据集成融合、长时序卫星遥感分析、互联网文献大数据建模分析等方法，对以荒漠化、水土流失、石漠化为代表的全球主要生态退化类型区的空间分布、演变态势、研究关注热度等进行了研究。结果表明：（1）全球荒漠化区面积约15.4×10⁶ km²，水土流失区面积约14.3×10⁶ km²，石漠化区面积约1.1×10⁶ km²；这些生态退化区主要分布在非洲撒哈拉沙漠南北边缘，欧洲西部、地中海沿岸、东欧平原南部，南亚印度河流域，中国西北地区、云贵高原，北美洲落基山脉以及南美洲阿根廷等地区。（2）2000年以来，上述退化区中约有3.9%的面积处于退化加重态势，73.3%的面积处于脆弱平衡状态，22.8%的区域出现好转趋势。（3）全球生态退化研究热点区的分布与全球生态退化区的分布总体呈现一致性。但在沙特阿拉伯中部、哈萨克斯坦北部，巴西大部，安哥拉、南非等生态退化区，存在生态系统继续恶化、缺乏研究界足够关注的情况。研究成果深化了对全球主要生态退化区分布格局的认识，对于防范全球发展和建设中出现加重的生态退化等具有参考价值。;Knowing the spatial distribution of ecological degradation regions and research hotspot regions, as well as the evolution of ecosystems in the degradation regions are the basis for understanding ecological problems and ecological environment management. However, there is a lack of basic data and relevant knowledge research in the world. In this paper, multi-source data integration, long-term satellite remote sensing analysis, and the Internet literature big data modeling analysis are applied to study the spatial distribution, evolution trend and the research hotspots of global major ecological degradation, i.e. land desertification, soil erosion and karst rocky desertification. The results show that:(1) the global desertification area is about 15.4×10⁶ km², the soil erosion area is about 14.3×10⁶ km², and the karst rocky desertification area is about 1.1×10⁶ km². The global major ecological degradation regions are mainly distributed in northern and southern margin of the Sahara in Africa, the Mediterranean coast in Western Europe and southern plain of the Eastern European, the Indus River Basin in South Asia, the Northwestern China and the Yunnan-Guizhou Plateau of China, the Rocky Mountains in North America and Argentina in South America. (2) Since 2000, about 3.9% of the above degraded areas have been deteriorating, 73.3% of them were still in a fragile dynamic balance state, and 22.8% of them have been improved. In terms of spatial distribution, the attention of global researchers is generally consistent with that of typical ecological degradation regions. However, there are still some areas where the ecosystems continue to deteriorate, but at the same time they lack sufficiently scientific research attention, on the central Saudi Arabia and Northern Kazakhstan in Asia, Brazil in South America, Angola and South Africa in southern Africa, etc. The research results deepen people's understanding of the distribution pattern of global major ecological degradation regions, and have reference value for preventing the aggravated ecological degradation in the developments and engineering construction worldwide.

Academic Influence Aware and Multidimensional Network Analysis for Research Collaboration Navigation Based on Scholarly Big Data

Scholarly big data, which is a large-scale collection of academic information, technical data, and collaboration relationships, has attracted increasing attentions, ranging from industries to academic communities. The widespread adoption of social computing paradigm has made it easier for researchers to join collaborative research activities and share academic data more extensively than ever before across the highly interlaced academic networks. In this study, we focus on the academic influence aware and multidimensional network analysis based on the integration of multi-source scholarly big data. Following three basic relations: Researcher-Researcher, Researcher-Article, and Article-Article, a set of measures is introduced and defined to quantify correlations in terms of activity-based collaboration relationship, specialty-aware connection, and topic-aware citation fitness among a series of academic entities (e.g., researchers and articles) within a constructed multidimensional network model. An improved Random Walk with Restart (RWR) based algorithm is developed, in which the time-varying academic influence is newly defined and measured in a certain social context, to provide researchers with research collaboration navigation for their future works. Experiments and evaluations are conducted to demonstrate the practicability and usefulness of our proposed method in scholarly big data analysis using DBLP and ResearchGate data.

Mapping Microscale PM2.5 Distribution on Walkable Roads in a High-Density City

Monitoring pollution of PM2.5 on walkable roads is important for resident health in high-density cities. Due to the spatiotemporal resolution limitations of aerosol optical depth observation, fixed-point monitoring, or traditional mobile measurement instruments, the microscale PM2.5 distribution in the walking environment cannot be fully estimated at the fine scale. In this article, by the integration of mobile measurement data, OpenStreetMap (OSM) data, Landsat images, and other multisource data in land-use regression (LUR) models, a novel framework is proposed to estimate and map PM2.5 distribution in a typical microscale walkable environment of the high-density city Hong Kong. First, the PM2.5 data on the typical walking paths were collected by the handheld mobile measuring instruments, to be selected as the dependent variables. Second, geographic prediction factors calculated by Google Street View, OSM data, Landsat images, and other multisource data were further selected as independent variables. Then, these dependent and independent variables were put into the LUR models to estimate the PM2.5 concentration on sidewalks, footbridges, and footpaths in the microscale walkable environment. The proposed models showed high performance relative to those in similar studies (adj <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</i> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> , 0.593 to 0.615 [sidewalks]; 0.641 to 0.682 [footpaths]; 0.783 to 0.797 [footbridges]). This article is beneficial for mapping PM2.5 concentration in the microscale walking environment and the identification of hot spots of air pollution, thereby helping people avoid the PM2.5 hotspots and indicating a healthier walking path.

Key Technologies of Multi-Source Data Integration System for Large Power Grid Operation

With the continuous expansion of the scale of the power grid and the continuous increase in the proportion of new energy and power electronic equipment, higher requirements are placed on the basic data of the online security analysis of the power grid. This paper proposes a multi-source data integration system for large power grid operation. It studies the rapid interaction and intelligent integration of multi-source heterogeneous information such as offline and online models, breaking the technical application barriers of various types of data, and quickly generating real-time power grid model. The analysis model provides a reliable data basis for the online analysis of the power grid. Finally, based on the PSASP simulation software, the correctness and effectiveness of the multi-source data integration system of large power grid are verified.

Morphological Changes Detection of a Large Earthflow Using Archived Images, LiDAR-Derived DTM, and UAV-Based Remote Sensing

In mountainous landscapes, where strongly deformed pelitic sediments outcrop, earthflows can dominate denudation processes and landscape evolution. This paper investigated geological and geomorphological features and space-time evolution over a 65-year time span (1954–2019) of a large earthflow, representative of wide sectors of the Apennine chain of southern Italy. The landslide, with a maximum length of 1.85 × 103 m, affects an area of 4.21 × 105 m2 and exhibits two source zones: a narrow and elongated transport zone and a lobate accumulation zone. Spatial and temporal morphological changes of the earthflow were assessed, comparing multi-source and multi-temporal data (aerial photographs, Google Earth satellite images, Light Detection and Ranging (LiDAR) and Unmanned Aerial Vehicles (UAV) system data). Geomorphic changes, quantified using Digital Terrain Models (DTMs) of differences, highlighted an extensive lowering of the topographic surface in the source area and a significant uplift at the landslide toe. Moreover, the multi-temporal analysis showed a high increase of landslide surface (more than 66%) during the last 65 years. The volumetric analyses showed that different sectors of the earthflow were active at different times, with different rates of topographic change. Overall, the used approach highlighted the great potentiality of the integration of multi-source and multi-temporal data for the diachronic reconstruction of morphological landslide evolution.

Two-stage adaptive integration of multi-source heterogeneous data based on an improved random subspace and prediction of default risk of microcredit

Some scholars have shown that the machine learning methods based on a single-source data can successfully monitor the risks of formal financial activities, but not those of informal financial activities. This is because the data generated by formal financial activities, whether it is the structured or unstructured data, are of high quality and quantity, while the data generated by informal financial activities are not. Therefore, multi-source data are the key to monitor the risks of informal financial activities through machine learning. Although a few studies attempted to use multi-source data for financial risk prediction, they simply stack the obtained multi-source data, but ignore the original sources, heterogeneity, mutual redundancy and other characteristics of the data, so that the improvement of the prediction effect is not obvious. Therefore, TSAIB_RS method based on the two-stage adaptive integration of multi-source heterogeneous data was constructed in the paper, in which the data with different sources and different distributions were adaptively integrated. In order to test the reliability of TSAIB_RS method, the paper takes the default risk of microcredit in China as the test target and compares the prediction results of various test methods. It concludes that TSAIB_RS method can significantly improve the prediction effects.

Integration of Geospatial Tools and Multi-source Geospatial Data to Evaluate the Tropical Forest Cover Change in Central America and Its Methodological Replicability in Brazil and the DRC

Satellite monitoring of forests plays a relevant role in the agendas of tropical countries, mainly in the framework of international negotiations to implement a mechanism that ensures a reduction in global CO2 emissions from deforestation. An efficient way to approach this monitoring is to avoid duplication of efforts, generating products in a regional context that are subsequently adopted at the national level. In this effort, you should take advantage of the different data sources available by integrating geospatial tools and satellite image classification algorithms. In this research, a methodological framework was developed to generate cost-efficient national maps of forest cover and its dynamics for the countries of Central America, and its scalability and replicability was explored in the Democratic Republic of the Congo (DRC) and the State of Pará in Brazil. The maps were generated from Landsat images from the years 2000, 2012, and 2017. New geoprocessing elements have been incorporated into the digital classification procedures for satellite images, such as the automated extraction of training samples from secondary sources, the use of official national reference maps that respond to nationally adopted forest definitions, and automation of post-classification adjustments incorporating expert criteria. The applied regional approach offers advantages in terms of reducing costs and time, as well as improving the consistency and coherence of reports at different territorial levels (regional and national), reducing duplication of efforts and optimizing technical and financial resources. In Central America, the percentage of forest area decreased from 44% in 2000 to 38% in 2017. Average deforestation in the 2000–2012 period was 197,443 ha/year and that of 2012–2017 was 332,243 ha/year. Average deforestation for the complete period 2000–2017 was 264,843 ha/year. The tropical forests in both the State of Pará, Brazil, and the DRC have decreased over time.

Monitoring and Modelling Abandoned Agricultural Land Based on Multisource Data Integration

Given the sustained growth of the global population, the problem of abandoned agricultural land (AAL) has gradually attracted the attention of governments and scholars. Here, four high-resolution remote sensing data sources (Sentinel-2A satellite, Resources satellite 3, Gaofen-1 satellite, and Gaofen-2 satellite) and two geographic information survey data sources (land use data and returning farmland to forest data) were used in combination. An integrated method of combined annual and interannual detection was used to monitor AAL. The integrated method produced total abandonment information and distinguished the types of AAL (completely AAL, half-AAL, and transitional AAL). By data mining the monitoring results, eight driving factors of abandonment were determined at the plot scale, including village distance, height difference, highway distance, plot area, fractal dimension index, shelter forest, irrigation condition and neighbourhood characteristics. The logistical regression results showed that seven independent variables except village distance had a significant impact on abandonment, and the prediction accuracy of the model was 95.4%. This study can be applied to monitoring AAL and analysing the driving forces of abandonment.

Application of low-cost fine particulate mass monitors to convert satellite aerosol optical depth to surface concentrations in North America and Africa

Abstract. Low-cost particulate mass sensors provide opportunities to assess air quality at unprecedented spatial and temporal resolutions. Established traditional monitoring networks have limited spatial resolution and are simply absent in many major cities across sub-Saharan Africa (SSA). Satellites provide snapshots of regional air pollution but require ground-truthing. Low-cost monitors can supplement and extend data coverage from these sources worldwide, providing a better overall air quality picture. We investigate the utility of such a multi-source data integration approach using two case studies. First, in Pittsburgh, Pennsylvania, both traditional monitoring and dense low-cost sensor networks are compared with satellite aerosol optical depth (AOD) data from NASA's MODIS system, and a linear conversion factor is developed to convert AOD to surface fine particulate matter mass concentration (as PM2.5). With 10 or more ground monitors in Pittsburgh, there is a 2-fold reduction in surface PM2.5 estimation mean absolute error compared to using only a single ground monitor. Second, we assess the ability of combined regional-scale satellite retrievals and local-scale low-cost sensor measurements to improve surface PM2.5 estimation at several urban sites in SSA. In Rwanda, we find that combining local ground monitoring information with satellite data provides a 40 % improvement in surface PM2.5 estimation accuracy with respect to using low-cost ground monitoring data alone. A linear AOD-to-surface-PM2.5 conversion factor developed in Kigali, Rwanda, did not generalize well to other parts of SSA and varied seasonally for the same location, emphasizing the need for ongoing and localized ground-based monitoring, which can be facilitated by low-cost sensors. Overall, we find that combining ground-based low-cost sensor and satellite data, even without including additional meteorological or land use information, can improve and expand spatiotemporal air quality data coverage, especially in data-sparse regions.

Urban function recognition by integrating social media and street-level imagery

Recognizing urban functions is crucial for understanding urban spatial structures and urban planning. Previous work has investigated urban functions based on human activities that were derived from mobile phone positioning data, check-in data, taxi data, etc. However, urban functions can only be comprehensively sensed from both human activities and the physical environment together. To do so, a deep learning method was proposed to predict urban functions by integrating social media data and street-level imagery. The verbs extracted from social media posts were taken as the proxy for human activities, and we identified urban physical environmental information from street-level imagery. Then urban functions were uncovered from both the verbs in terms of human activities and street-level imagery from the perspective of the physical environment. Twelve types of urban function were recognized by verbs in social media posts, which were then improved by integrating street-level imagery within the 5th Ring Road of Beijing, China. The experiment demonstrated that verbs as direct proxies for human activities can avoid noise, and the multi-source data integration eliminated biases caused by a single data source. This work provides a comprehensive understanding of urban structure and dynamics for urban management and planning.

Building a PubMed knowledge graph

PubMed® is an essential resource for the medical domain, but useful concepts are either difficult to extract or are ambiguous, which has significantly hindered knowledge discovery. To address this issue, we constructed a PubMed knowledge graph (PKG) by extracting bio-entities from 29 million PubMed abstracts, disambiguating author names, integrating funding data through the National Institutes of Health (NIH) ExPORTER, collecting affiliation history and educational background of authors from ORCID®, and identifying fine-grained affiliation data from MapAffil. Through the integration of these credible multi-source data, we could create connections among the bio-entities, authors, articles, affiliations, and funding. Data validation revealed that the BioBERT deep learning method of bio-entity extraction significantly outperformed the state-of-the-art models based on the F1 score (by 0.51%), with the author name disambiguation (AND) achieving an F1 score of 98.09%. PKG can trigger broader innovations, not only enabling us to measure scholarly impact, knowledge usage, and knowledge transfer, but also assisting us in profiling authors and organizations based on their connections with bio-entities.

Application of Python web crawler technology in infodemiology

Python web crawler technology, which automatically and massively getting information from the Internet by mimicking net users' browsing behavior, is a basic supporting technique to extract and integrate multi-source heterogeneous data in the field of Infodemiology. There are two types of Python web crawler: simple and massive-scale, both collect information simultaneously from the database establishment. Advantages of this technique are characterized as: being simple syntax, in high flexibility and low cost in learning and maintenance. Contents of the current application scenarios include surveillance, implementation and evaluation of health intervention programs on public health issues, as well as on smart doctor seeking. For the last two years, the Chinese government started to encourage the integration and utilization of multi-source heterogeneous data including internet information. Hence, the number of application scenarios for Python web crawler technology are bound to increase in the foreseeable future. Corresponding matched talent cultivations and technical innovations are suggested to add to the current education and research systems on public health issues.