Sort by
Video Moment Retrieval With Noisy Labels.

Video moment retrieval (VMR) aims to localize the target moment in an untrimmed video according to the given nature language query. The existing algorithms typically rely on clean annotations to train their models. However, making annotations by human labors may introduce much noise. Thus, the video moment retrieval models will not be well trained in practice. In this article, we present a simple yet effective video moment retrieval framework via bottom-up schema, which is in end-to-end manners and robust to noisy label training. Specifically, we extract the multimodal features by syntactic graph convolutional networks and multihead attention layers, which are fused by the cross gates and the bilinear approach. Then, the feature pyramid networks are constructed to encode plentiful scene relationships and capture high semantics. Furthermore, to mitigate the effects of noisy annotations, we devise the multilevel losses characterized by two levels: a frame-level loss that improves noise tolerance and an instance-level loss that reduces adverse effects of negative instances. For the frame level, we adopt the Gaussian smoothing to regard noisy labels as soft labels through the partial fitting. For the instance level, we exploit a pair of structurally identical models to let them teach each other during iterations. This leads to our proposed robust video moment retrieval model, which experimentally and significantly outperforms the state-of-the-art approaches on standard public datasets ActivityCaption and textually annotated cooking scene (TACoS). We also evaluate the proposed approach on the different manual annotation noises to further demonstrate the effectiveness of our model.

Relevant
Aboveground Forest Biomass Estimation Using Tent Mapping Atom Search Optimized Backpropagation Neural Network with Landsat 8 and Sentinel-1A Data

Accurate forest biomass estimation serves as the foundation of forest management and holds critical significance for a comprehensive understanding of forest carbon storage and balance. This study aimed to integrate Landsat 8 OLI and Sentinel-1A SAR satellite image data and selected a portion of the Shanxia Experimental Forest in Jiangxi Province as the study area to establish a biomass estimation model by screening influencing factors. Firstly, we extracted spectral information, vegetation indices, principal component features, and texture features within 3 × 3-pixel neighborhoods from Landsat 8 OLI. Moreover, we incorporated Sentinel-1’s VV (vertical transmit–vertical receive) and VH (vertical transmit–horizontal receive) polarizations. We proposed an ensemble AGB (aboveground biomass) model based on a neural network. In addition to the neural network model, namely the tent mapping atom search optimized BP neural network (Tent_ASO_BP) model, partial least squares regression (PLSR), support vector machine (SVR), and random forest (RF) regression prediction techniques were also employed to establish the relationship between multisource remote sensing data and forest biomass. Optical variables (Landsat 8 OLI), SAR variables (Sentinel-1A), and their combinations were input into the four prediction models. The results indicate that Tent_ ASO_ BP model can better estimate forest biomass. Compared to pure optical or single microwave data, the Tent_ASO_BP model with the optimal combination of optical and microwave input features achieved the highest accuracy. Its R2 was 0.74, root mean square error (RMSE) was 11.54 Mg/ha, and mean absolute error (MAE) was 9.06 Mg/ha. Following this, the RF model (R2 = 0.54, RMSE = 21.33 Mg/ha, MAE = 17.35 Mg/ha), SVR (R2 = 0.52, RMSE = 17.66 Mg/ha, MAE = 15.11 Mg/ha), and PLSR (R2 = 0.50, RMSE = 16.52 Mg/ha, MAE = 12.15 Mg/ha) models were employed. In conclusion, the BP neural network model improved by tent mapping atom search optimization algorithm significantly enhanced the accuracy of AGB estimation in biomass studies. This will provide a new avenue for large-scale forest resource surveys.

Open Access
Relevant
Spatiotemporal disturbances and attribution analysis of mangrove in southern China from 1986 to 2020 based on time-series Landsat imagery

As one of the most productive ecosystems in the world, mangrove has a critical role to play in both the natural ecosystem and the human economic and social society. However, two thirds of the world's mangrove have been irreversibly damaged over the past 100 years, as a result of ongoing human activities and climate change. In this paper, adopting Landsat for the past 36 years as the data source, the detection of spatiotemporal changes of mangrove in southern China was carried out based on the Google Earth Engine (GEE) cloud platform using the LandTrendr algorithm. In addition, the attribution of mangrove disturbances was analyzed by a random forest algorithm. The results indicated the area of mangrove recovery (5174.64 hm2) was much larger than the area of mangrove disturbances (1625.40 hm2) over the 35-year period in the study area. The disturbances of mangrove in southern China were dominated by low and low-to-medium-level disturbances, with an area of 1009.89 hm2, accounting for 57.50 % of the total disturbances. The mangrove recovery was also dominated by low and low-to-medium-level recovery, with an area of 3239.19 hm2, accounting for 62.61 % of the total recovery area. Both human and natural factors interacted and influenced each other, together causing spatiotemporal disturbances of mangrove in southern China during 1986–2020. The mangrove disturbances in the Phase I (1986–2000) and Phase III (2011–2020) were characterized by human-induced (50.74 % and 58.86 %), such as construction of roads and aquaculture ponds. The mangrove disturbances in the Phase II (2001–2010) were dominated by natural factors (55.73 %), such as tides, flooding, and species invasions. It was also observed that the area of mangrove recovery in southern China increased dramatically from 1986 to 2020 due to the promulgation and implementation of the Chinese government's policy on mangrove protection, as well as increased human awareness of mangrove wetland protection.

Relevant
Fine-grained regional economic forecasting for a megacity using vector-based cellular automata

It is important to measure uncoordinated regional urban economic development to help guide government policy. However, previous models have often struggled to capture the fine-grained spatiotemporal characteristics of economic development, thereby failing to provide insight into the fine-scale patterns. Sectoral structure and its evolution are strongly related to economic development and provide finer spatial information. Therefore, this paper proposes a framework for forecasting the spatiotemporal evolution of urban economic development at the cadastral parcel scale based on sectoral land use dynamics modeling and the S-curve economic model. The results of the case study conducted in Shenzhen, China, show good performance (FoM = 0.182, average R2 = 0.748, median R2 = 0.877). The sectoral structure in high-economic-level areas was found to be more balanced, and the economic volume tended to increase more. In contrast, sectoral land use types change more frequently in low-economic-level areas, and the economic growth rates are generally higher due to their lower economic base. Without government intervention, disparities in simulated economic volumes between regions will continue to widen in the short term. Hence, the government is encouraged to consider optimizing the sectoral structure in low-economic-level areas to promote coordinated regional economic development.

Relevant
Forest digital twin: A new tool for forest management practices based on Spatio-Temporal Data, 3D simulation Engine, and intelligent interactive environment

Existing forest digitization studies focus on one-way forest management practice visualization simulation, lacking decision-making feedback and virtual-real interaction synchronization. This paper presents the vision of the forest digital twin paradigm. We construct a forest digital twin to explore a new digital carrier of forest resources using remote sensing data, forest inventory data, the Cesium Digital Earth Engine, forest planning theory and parametric 3D modeling technology. The two-way interaction and thinning experiments showed that the forest digital twin could provide a novel pattern for in-depth analysis of forest spatial structure, individual tree dynamic growth and human-digital twin interaction effects. The successful recognition rate in matching the forest structure seen on real forest structure images with the forest digital scene was 91.3%, indicating that the forest digital twin can characterize the real forest structure significantly. The prediction accuracy of the multi-grade growth model integrating the Bayesian method for DBH, H was more than 90.4%. In addition, ASS-FDT interaction is superior to the assessors (ASS) and forest digital twin (FDT) for stand spatial structure overall optimization. The multi-dimensional stand spatial structure index (F-index) increased by 22.82%. The constructed forest digital twin model shows superior performance in optimizing the stand growth model and enhancing the overall stand spatial structure under the decision-making feedback and real-time interaction strategies. The automatic operation pattern provides a user-friendly forest management practice solution.

Open Access
Relevant
Is Spectral Unmixing Model or Nonlinear Statistical Model More Suitable for Shrub Coverage Estimation in Shrub-Encroached Grasslands Based on Earth Observation Data? A Case Study in Xilingol Grassland, China

Due to the effects of global climate change and altered human land-use patterns, typical shrub encroachment in grasslands has become one of the most prominent ecological problems in grassland ecosystems. Shrub coverage can quantitatively indicate the degree of shrub encroachment in grasslands; therefore, real-time and accurate monitoring of shrub coverage in large areas has important scientific significance for the protection and restoration of grassland ecosystems. As shrub-encroached grasslands (SEGs) are a type of grassland with continuous and alternating growth of shrubs and grasses, estimating shrub coverage is different from estimating vegetation coverage. It is not only necessary to consider the differences in the characteristics of vegetation and non-vegetation variables but also the differences in characteristics of shrubs and herbs, which can be a challenging estimation. There is a scientific need to estimate shrub coverage in SEGs to improve our understanding of the process of shrub encroachment in grasslands. This article discusses the spectral differences between herbs and shrubs and further points out the possibility of distinguishing between herbs and shrubs. We use Sentinel-2 and Gao Fen-6 (GF-6) Wide Field of View (WFV) as data sources to build a linear spectral mixture model and a random forest (RF) model via space–air–ground collaboration and investigate the effectiveness of different data sources, features and methods in estimating shrub coverage in SEGs, which provide promising ways to monitor the dynamics of SEGs. The results showed that (1) the linear spectral mixture model can hardly distinguish between shrubs and herbs from medium-resolution images in the SEG. (2) The RF model showed high estimation accuracy for shrub coverage in the SEG; the estimation accuracy (R2) of the Sentinel-2 image was 0.81, and the root-mean-square error (RMSE) was 0.03. The R2 of the GF6-WFV image was 0.72, and the RMSE was 0.03. (3) Texture feature introduced in RF models are helpful to estimate shrub coverage in SEGs. (4) Regardless of the linear spectral mixture model or the RF model being employed, the Sentinel-2 image presented a better estimation than the GF6-WFV image; thus, this data has great potential to monitor shrub encroachment in grasslands. This research aims to provide a scientific basis and reference for remote sensing-based monitoring of SEGs.

Open Access
Relevant
Exploring the role of canopy triangular units in analysing canopy effects on saplings

On the distribution and growth of saplings, canopy structure has a considerable influence. However, traditional canopy metrics are inadequate in irregular stands due to their inability to consider the heterogeneous horizontal and vertical distributions of canopy structure. To surmount this inherent limitation and achieve an advanced comprehension of the three-dimensional attributes characterizing the canopy, we introduce the innovative framework of canopy triangular units. This conceptual framework encapsulates a comprehensive array of multifaceted information, encompassing intricate forest layer intricacies such as tree height differentials, intricate competitive attributes exemplified by disparities in diameter at breast height and crown width, alongside precise geospatial location information. In this study, we formulated geometrically organized triangular networks based on precise tree coordinates, subsequently delineating ten distinct triangular unit categories predicated upon diameter at breast height measurements: DT (3,0,0), DT (2,1,0), DT (2,0,1), DT (1,2,0), DT (1,1,1), DT (1,0,2), DT (0,3,0), DT (0,2,1), DT (0,1,2), and DT (0,0,3). The values enclosed within parentheses represent the quantities of large trees, medium trees, and small trees contained within each respective triangular unit type. Our analytical framework commenced with an in-depth evaluation of the differences in quantities and areas across these diverse triangular unit types. Subsequent to this, these identified triangular unit categories underwent systematic classification into three distinct groups, thereby enabling meticulous investigation into variations within the spatial dispersion of sapling populations. Furthermore, a comprehensive suite of seven triangular unit variables encapsulating the multifaceted three-dimensional attributes of the canopy was established. This enabled a comprehensive investigation into their nuanced relationships with the growth patterns of saplings. Analyzing the triangular unit types in the three sample plots, a clear pattern emerged: maximum occurrences of DT (0,1,2) units alongside minimal DT (3,0,0) occurrences. Moreover, saplings show a clear preference for triangular unit types with low abundance of large trees. The correlation analysis revealed a significant positive link between the individual basal area increment of saplings and relevant parameters - opening area, triangular perimeter, and triangular area. As a practical implication, during the replanting of saplings, the selective choice of triangular units with fewer large trees can be prioritized to support their growth and development. To optimize sapling growth, selective tree cutting may be deemed necessary to create new triangular units and expand the opening area. These findings emphasize the significance of accounting for the framework of canopy triangular units in the development of management strategies aimed at improving both sapling quantity and growth.

Relevant
Combining Multisource Data and Machine Learning Approaches for Multiscale Estimation of Forest Biomass

Forest biomass is an important indicator of forest ecosystem productivity, and it plays vital roles in the global carbon cycling, global climate change mitigating, and ecosystem researches. Multiscale, rapid, and accurate extraction of forest biomass information is always a research topic. In this study, comprehensive investigation of a larch (Larix olgensis) plantation was performed using remote sensing and field-based monitoring methods, in combination with LiDAR-based multisource data and machine learning methods. On this basis, a universal, multiscale (single tree, stand, management unit, and region), and unit-high-precision continuous monitoring method was proposed for forest biomass components. The results revealed the following. (1) Airborne LiDAR point cloud variables exhibited significant correlation with the aboveground components (except leaves) and the whole-plant biomass (Radj2 > 0.91), suitable for extraction or estimation of forest parameters such as biomass and stock volume. (2) In terms of biomass monitoring at forest stand and management unit scale, a random forest model performed well in fitting accuracy and generalization ability, whereas a multiple linear regression model produced clearer explanation regarding the biomass of each forest component. (3) Using seasonal phenological characteristics in the study area, larch distribution information was extracted effectively. The overall accuracy reached 90.0%, and the kappa coefficient reached 0.88. (4) A regional-scale forest biomass component estimation model was constructed using a long short-term memory model, which effectively reduced the probability of biomass underestimation while ensuring good estimation accuracy, with R2 exceeding 0.6 for the biomass of the aboveground and whole-plant components. This research provides theoretical support for rapid and accurate acquisition of large-scale forest biomass information.

Open Access
Relevant