Overestimation Issue Research Articles

Optimization Effectiveness of Multi-Intelligence Consistent Energy System Based on Digital Advertising and Data Analysis

This paper introduces a novel approach to address the issue of overestimation of state-action values in reinforcement learning algorithms based on the Q-framework. It integrates a dual-layer Q-learning algorithm with a grey wolf intelligent optimization method, enabling rapid search for optimal allocations in unknown search spaces. This integration results in the development of a multi-agent collaborative Area-Grid Coordination (A-GC) strategy, termed the grey wolf double Q (GWDQ) strategy, tailored for multi-area energy interconnection scenarios. The proposed GWDQ strategy is evaluated through simulation experiments on comprehensive energy system models, including mixed gas turbine systems, Combined Cooling, Heating, and Power (CCHP) systems, and a multi-area energy-interconnected Northeast power grid model. A centralized architecture is established to analyze the optimization effects of digital advertising. The performance of the GWDQ strategy is compared with traditional reinforcement learning algorithms through simulation and empirical data validation. Results indicate that the GWDQ strategy exhibits stronger learning capabilities, improved stability, and enhanced control performance compared to traditional methods. It demonstrates superior optimization for digital advertising and enables swift acquisition of optimal coordination in A-GC processes across multiple regions. Additionally, the paper analyzes the environmental and economic impacts of the proposed strategy.

Adaptive urban traffic signal control based on enhanced deep reinforcement learning

One of the focal points in the field of intelligent transportation is the intelligent control of traffic signals (TS), aimed at enhancing the efficiency of urban road networks through specific algorithms. Deep Reinforcement Learning (DRL) algorithms have become mainstream, yet they suffer from inefficient training sample selection, leading to slow convergence. Additionally, enhancing model robustness is crucial for adapting to diverse traffic conditions. Hence, this paper proposes an enhanced method for traffic signal control (TSC) based on DRL. This approach utilizes dueling network and double q-learning to alleviate the overestimation issue of DRL. Additionally, it introduces a priority sampling mechanism to enhance the utilization efficiency of samples in memory. Moreover, noise parameters are integrated into the neural network model during training to bolster its robustness. By representing high-dimensional real-time traffic information as matrices, and employing a phase-cycled action space to guide the decision-making of intelligent agents. Additionally, utilizing a reward function that closely mirrors real-world scenarios to guide model training. Experimental results demonstrate faster convergence and optimal performance in metrics such as queue length and waiting time. Testing experiments further validate the method's robustness across different traffic flow scenarios.

Intensified response of extreme precipitation to rising temperature over the Tibetan Plateau from CMIP6 multi-model ensembles

This study conducted bias correction of the high-performing general circulation model (GCM) data in combination with observed data based on the latest GCM data released by the Coupled Model Intercomparison Project Phase 6 (CMIP6). The ability of GCMs to simulate the characteristics of extreme precipitation over the Tibetan Plateau (TP) was systematically assessed, and the CMIP6 multi-model ensemble was used to accurately project extreme precipitation. Furthermore, the response of extreme precipitation over the TP to global warming was explored. The results demonstrated that daily translation efficiently addressed the issue of apparent overestimation in the model’s raw output, and CMIP6 multi-model ensemble mean exhibited better simulation performance for all extreme precipitation indices over the TP. Additionally, the simulation performance of each index exhibited a divergence across various elevation bands as altitude increased, with the best and worst performance observed at the altitude bands of 1500–2000 m and above 5000 m, respectively. For future changes in extreme precipitation, the indices representing the amount and intensity of extreme precipitation increase significantly, whereas the indices representing the duration of precipitation decrease. Moreover, the plateau will show a spatial pattern of “wet regions becoming dry and dry regions becoming wet”. The relationship between extreme precipitation and temperature follows a positive and peak pattern over most areas of the plateau, while the response of extreme precipitation to temperature will increase by 0.6 %–52.6 % with rising temperature. The present results have important implications for investigating the impacts of climate change on the water cycle in alpine regions.

Archaeal Hydroxylated Isoprenoid GDGTs in Asian Lake Sediments: A New Tool for Terrestrial Paleotemperature Reconstructions

AbstractHydroxylated isoprenoid GDGTs (OH‐GDGTs) have emerged as a novel tool for reconstructing sea surface temperatures. However, when using marine OH‐GDGT calibration in lacustrine settings, it leads to a significant overestimation of temperatures, emphasizing the necessity for a thorough examination of OH‐GDGTs in lakes. Here, we investigated OH‐GDGT distributions in surface sediments from 65 lakes in West China and compiled published Asian lake and global marine OH‐GDGT data sets. Among all GDGT‐based indices, RI‐OH showed the strongest correlation with temperature across Asian lakes. The RI‐OH value was higher in lakes than in marine sediments, likely due to differences in the composition of Group 1.1a thaumarchaeotal species between the two settings. The first RI‐OH temperature calibration for lakes was developed and it addressed the issue of temperature overestimation when applied to both water column and sediment core, highlighting the potential of OH‐GDGTs as a new terrestrial paleothermometer.

TD3-Based Optimization Framework for RSMA-Enhanced UAV-Aided Downlink Communications in Remote Areas

The need for reliable wireless communication in remote areas has led to the adoption of unmanned aerial vehicles (UAVs) as flying base stations (FlyBSs). FlyBSs hover over a designated area to ensure continuous communication coverage for mobile users on the ground. Moreover, rate-splitting multiple access (RSMA) has emerged as a promising interference management scheme in multi-user communication systems. In this paper, we investigate an RSMA-enhanced FlyBS downlink communication system and formulate an optimization problem to maximize the sum-rate of users, taking into account the three-dimensional FlyBS trajectory and RSMA parameters. To address this continuous non-convex optimization problem, we propose a TD3-RFBS optimization framework based on the twin-delayed deep deterministic policy gradient (TD3). This framework overcomes the limitations associated with the overestimation issue encountered in the deep deterministic policy gradient (DDPG), a well-known deep reinforcement learning method. Our simulation results demonstrate that TD3-RFBS outperforms existing solutions for FlyBS downlink communication systems, indicating its potential as a solution for future wireless networks.

A Bayesian Deep Reinforcement Learning-Based Resilient Control for Multi-Energy Micro-Gird

Aiming at a cleaner future power system, many regimes in the world have proposed their ambitious decarboniz-ing plan, with increasing penetration of renewable energy sources (RES) playing an alternative role to conventional energy. As a re-sult, power system tends to have less backup capacity and operate near their designed limit, thus exacerbating system vulnerability against extreme events. Under this reality, resilient control for the multi-energy micro-grid is facing the following challenges, which are: 1) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">the effect from the stochastic uncertainties of</i> RES; 2) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">the need for a model-free and fast-reacting control scheme under extreme events; and</i> 3) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">efficient exploration and robust performance with limited extreme events data</i> . <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">To deal with the aforementioned challenges, this paper pro-poses a novel Bayesian Deep Reinforcement Learning-based resilient control approach for multi-energy micro-grid. In partic-ular, the proposed approach replaces the deterministic network in traditional Reinforcement Learning with a Bayesian probabilistic network in order to obtain an approximation of the value function distribution, which effectively solves the Q-value overestimation issue. Compared with the naive Deep Deterministic Policy Gra-dient (DDPG) method and optimization method, the effectiveness and importance of employing the Bayesian Reinforcement Learn-ing approach are investigated and illustrated across different operating scenarios. Case studies have shown that by using the Monte Carlo posterior mean of the Bayesian value function distribution instead of a deterministic estimation, the proposed Bayesian Deep Deterministic Policy Gradient (BDDPG) method achieves a near-optimum policy in a more stable process, which verifies the robustness and the practicability of the proposed approach.

Assessing the Complementary Role of Surface Flux Equilibrium (SFE) Theory and Maximum Entropy Production (MEP) Principle in the Estimation of Actual Evapotranspiration

AbstractAlthough evapotranspiration (ET) from the land is a key variable in Earth system models, the accurate estimation of ET based on physical principles remains challenging. Parameters used in current ET models are largely empirically based, which could be problematic under rapidly changing climatic conditions. Here, we propose a physically based ET model that estimates ET based on the surface flux equilibrium (SFE) theory and the maximum entropy production (MEP) principle. We derive an expression for aerodynamic resistance based on the MEP principle, then propose a novel ET model that integrates the SFE model and the MEP principle. The proposed model, which is referred to as the SFE‐MEP model, becomes equivalent to the MEP state in non‐equilibrium conditions when turbulent mixing is weak and the land surface is dry. Under conditions meeting land‐atmosphere equilibrium, the SFE‐MEP model is similar to ET estimation based on the SFE model. This blended nature of the SFE‐MEP ET model allows accurate ET estimation for most inland regions by overcoming the ET overestimation issue of the SFE model in dry conditions. As a result, the SFE‐MEP model significantly improves the performance of SFE ET estimation, particularly for arid regions. The proposed model and its high accuracy of ET estimation enable novel insight into various Earth system models as it does not require any empirical parameters and only uses readily obtainable meteorological variables including reference height air temperature, relative humidity, available energy, and radiometric surface temperature.

ChainSketch: An Efficient and Accurate Sketch for Heavy Flow Detection

Identifying heavy flows is essential for network management. However, it is challenging to detect heavy flow quickly and accurately under the highly dynamic traffic and rapid growth of network capacity. Existing heavy flow detection schemes can make a trade-off in efficiency, accuracy and speed. However, these schemes still require memory large enough to obtain acceptable performance. To address this issue, we propose ChainSketch, which has the advantages of good memory efficiency, high accuracy and fast detection. Specifically, ChainSketch uses the selective replacement strategy to mitigate the over-estimation issue. Meanwhile, ChainSketch utilizes the hash chain and compact structure to improve memory efficiency. We implement the ChainSketch on OVS platform, P4-based testbed and large-scale simulations to process heavy hitter and heavy changer detection. The results of trace-driven tests show that, ChainSketch greatly improves the F1-score by up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$3.43\times $ </tex-math></inline-formula> compared with the state-of-the-art solutions especially for small memory.

Improved anthropogenic heat flux model for fine spatiotemporal information in Southeast China

Anthropogenic heat emission (AHE) is an important driver of urban heat islands (UHIs). Further, both urban thermal environment research and sustainable development planning require an efficient estimation of anthropogenic heat flux (AHF). Therefore, this study proposed an improved multi-source AHF model, which was constructed using diverse data sources and small-scale samples, to better represent the spatiotemporal distribution of AHF. The performances of three machine learning algorithms (Cubist, gradient boosting decision tree, and simple linear regression) were quantitatively evaluated, and the impact of spatiotemporal heterogeneity on AHF estimation was considered for the first time. The results showed that multi-source datasets and sophisticated algorithms could more effectively reduce the estimation error and improve the accuracy of the spatiotemporal distribution of AHF than simple linear regression. In practical applications, the Cubist model performed better, with prediction errors being less than 0.9 W⋅m−2. Further, the characteristics of different heat sources from the model outputs varied widely, and the building metabolic heat exhibited significant seasonal spatiotemporal variations, which were largely determined by the regional climate. In contrast, industrial and transportation heat showed marginal monthly fluctuations. Similarly, spatiotemporal heterogeneity significantly affected the estimation of building metabolic heat (0.62 W⋅m−2), but it did not affect other heat sources. The proposed improved AHF model was verified to effectively capture the spatiotemporal variations of building heat and solve the issue of overestimation of industrial heat in urban regions. This study provides new methods and ideas for the accurate spatiotemporal quantification of AHF that can supplement future studies on climate warming, UHI, and air pollution.

Using Double DQN and Sensor Information for Autonomous Driving

Background: In most of the research the autonomous driving problem is either solved with policy gradients or DQN. In this paper, we have tried to eliminate the problem of policy gradients which is high variance and an overestimation of values in DQN. We have used DDQN as it has low variance, and it solves the problem of overestimation in DQN. Aim: The main aim of this paper is to propose a framework for an autonomous driving model that takes in raw sensor information as input data and predicts actions as output from the model which could then be used for simulating the car. Objective: The main objective of this paper is to use DDQN and Discretization technique to solve the autonomous driving problem and get better results even with a continuous action space. Methods: To solve the bridge between self-driving cars and reinforcement learning we have used Double Deep QNetworks as this could help to prevent the overestimation of values by decoupling the selection from the evaluation. Also, to solve the problem of continuous action space we have used the discretization technique in which variables are grouped into bins and each bin is assigned a value in such a way that the relationship between the bins is preserved. Result: The experimental results showed improved performance of the agent. The agent was tested for different conditions like curve roads and traffic, which showed the agent can drive at different conditions as well. We have also illustrated how DDQN performed well over policy gradients just by adding a simple discretization technique to make the action space discrete and overcoming the issue of overestimation of q-values. Conclusion: We design the gym environment and reward function for DDQN to work. We have also used CARLA as a virtual simulator for training purposes. Finally, we have demonstrated that our agent was able to perform well in different cases and conditions. As a reminder note, we can improve our agent to also work for following traffic light rules and other road safety measures.

Effects of cohort size on college premium: Evidence from China's higher education expansion

In this paper, we document the lesser-known heterogeneous trends of college/non-college earnings premium across age groups from 1995 to 2013 in China. Specifically, the college premium in 2013 for the younger group (age 25–34) was about 30 percentage points, similar to the level in 1995, while the college premium in 2013 for the older group (age 45–54) increased to 50 percentage points, nearly double that of 1995. To attribute these divergent trends of the college premium to the changes in the relative size of college workers, we use the model by Card and Lemieux (2001), which incorporates imperfect substitution between similarly educated workers in different age cohorts. Due to the distinctions of these trends in China, our identification is free of the overestimation issue that the existing studies suffer. Our results are similar to those in the U.S., U.K., Canada, and Japan. Holding the age cohort and survey year constant, a one unit increase in log relative size of college workers is associated with about 10 percentage points decrease in college/non-college premium and about 18 percentage points decrease in college/high school premium. We further find that the negative effect is much more substantial among the new entrants (age 25–29) than experienced workers (age 30–54). By this pattern, we demonstrate that the new labor market entrants are more sensitive to their own cohort size and argue that the confounding ability composition effect should not be a serious issue.

Bridging the 12-6-4 Model and the Fluctuating Charge Model.

Metal ions play important roles in various biological systems. Molecular dynamics (MD) using classical force field has become a popular research tool to study biological systems at the atomic level. However, meaningful MD simulations require reliable models and parameters. Previously we showed that the 12-6 Lennard-Jones nonbonded model for ions could not reproduce the experimental hydration free energy (HFE) and ion-oxygen distance (IOD) values simultaneously when ion has a charge of +2 or higher. We discussed that this deficiency arises from the overlook of the ion-induced dipole interaction in the 12-6 model, and this term is proportional to 1/r 4 based on theory. Hence, we developed the 12-6-4 model and showed it could solve this deficiency in a physically meaningful way. However, our previous research also found that the 12-6-4 model overestimated the coordination numbers (CNs) for some highly charged metal ions. And we attributed this artifact to that the current 12-6-4 scheme lacks a correction for the interactions among the first solvation shell water molecules. In the present study, we considered the ion-included dipole interaction by using the 12-6 model with adjusting the atomic charges of the first solvation shell water molecules. This strategy not only considers the ion-induced dipole interaction between ion and the first solvation shell water molecules but also well accounts for the increased repulsion among these water molecules compared to the bulk water molecules. We showed this strategy could well reproduce the experimental HFE and IOD values for Mg2+, Zn2+, Al3+, Fe3+, and In3+ and solve the CN overestimation issue of the 12-6-4 model for Fe3+ and In3+. Moreover, our simulation results showed good agreement with previous ab initio MD simulations. In addition, we derived the physical relationship between the C 4 parameter and induced dipole moment, which agreed well with our simulation results. Finally, we discussed the implications of the present work for simulating metalloproteins. Due to the fluctuating charge model uses a similar concept to the 12-6 model with adjusting atomic charges, we believe the present study builds a bridge between the 12-6-4 model and the fluctuating charge model.

Photodetectors of 2D Materials from Ultraviolet to Terahertz Waves.

2D materials are considered to be the most promising materials for photodetectors due to their unique optical and electrical properties. Since the discovery of graphene, many photodetectors based on 2D materials have been reported. However, the low quantum efficiency, large noise, and slow response caused by the thinness of 2D materials limit their application in photodetectors. Here, recent progress on 2D material photodetectors is reviewed, covering the spectrum from ultraviolet to terahertz waves. First the interaction of 2D materials with light is analyzed in terms of optical physics. Then the present methods to improve the performance of 2D material photodetectors are summarized, such as defect engineering, p-n junctions and hybrid detectors, and the issue of serious overestimation of the performance in reported photodetectors based on 2D materials is discussed. Next, a comparison of 2D material photodetectors with traditional commercially available detectors shows that it is difficult to balance the current 2D material photodetectors with regard to having simultaneously both high sensitivity and fast response. Finally, a possible novel EIW mechanism is suggested to advance the performance of 2D material photodetectors in the future.

BBR-S: A Low-Latency BBR Modification for Fast-Varying Connections

The new possibilities offered by 5G and beyond networks have led to a change in the focus of congestion control from capacity maximization for web browsing and file transfer to latency-sensitive interactive and real-time services, and consequently to a renaissance of research on the subject, whose most well-known result is Google's Bottleneck Bandwidth and Round-trip propagation time (BBR) algorithm. BBR's promise is to operate at the optimal working point of a connection, with the minimum Round Trip Time (RTT) and full capacity utilization, striking the balance between resource use efficiency and latency performance. However, while it provides significant performance improvements over legacy mechanisms such as Cubic, it can significantly overestimate the capacity of fast-varying mobile connections, leading to unreliable service and large potential swings in the RTT. Our BBR-S algorithm replaces the max filter that causes this overestimation issue with an Adaptive Tobit Kalman Filter (ATKF), an innovation on the Kalman filter that can deal with unknown noise statistics and saturated measurements, achieving a 40% reduction in the average RTT over BBR, which increases to 60% when considering worst-case latency, while maintaining over 95% of the throughput in 4G and 5G networks.

Double Deep Q-Network-Based Energy-Efficient Resource Allocation in Cloud Radio Access Network

Cloud radio access network (CRAN) has been shown as an effective means to boost network performance. Such gain stems from the intelligent management of remote radio heads (RRHs) in terms of on/off operation mode and power consumption. Most conventional resource allocation (RA) methods, however, optimize the network utility without considering the switching overhead of RRHs in adjacent time intervals. When the network environment becomes time-correlated, mathematical optimization is not directly applicable. In this paper, we aim to optimize the energy efficiency (EE) subject to the constraints on per-RRH transmission power and user data rates. To this end, we formulate the EE problem as a Markov decision process (MDP) and subsequently adopt deep reinforcement learning (DRL) technique to reap the cumulative EE rewards. Our starting point is the deep Q network (DQN), which is a combination of deep learning and Q-learning. In each time slot, DQN configures the status of RRHs yielding the largest Q-value (known as state-action value) prior to solving a power minimization problem for active RRHs. To overcome the Q-value overestimation issue of DQN, we propose a Double DQN (DDQN) framework that obtains optimal reward better than DQN by separating the selected action from the target Q-value generator. Simulation results validate that the DDQN-based RA method is more energy-efficient than the DQN-based RA algorithm and a baseline solution.

Soil Moisture Mapping Based on Multi-Source Fusion of Optical, Near-Infrared, Thermal Infrared, and Digital Elevation Model Data via the Bayesian Maximum Entropy Framework

This paper proposes a combined approach wherein the optical, near-infrared, and thermal infrared data from the Landsat 8 satellite and the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) global digital elevation model (GDEM) data are fused for soil moisture mapping under sparse sampling conditions, based on the Bayesian maximum entropy (BME) framework. The study was conducted in three stages. First, based on the maximum entropy principle of the information theory, a Lagrange multiplier was introduced to construct general knowledge, representing prior knowledge. Second, a principal component analysis (PCA) was conducted to extract three principal components from the multi-source data mentioned above, and an innovative and operable discrete probability method based on a fuzzy probability matrix was used to approximate the probability relationship. Thereafter, soft data were generated on the basis of the weight coefficients and coordinates of the soft data points. Finally, by combining the general knowledge with the prior information, hard data (HD), and soft data (SD), we completed the soil moisture mapping based on the Bayesian conditioning rule. To verify the feasibility of the combined approach, the ordinary kriging (OK) method was taken as a comparison. The results confirmed the superiority of the soil moisture map obtained using the BME framework. The map revealed more detailed information, and the accuracies of the quantitative indicators were higher compared with that for the OK method (the root mean squared error (RMSE) = 0.0423 cm3/cm3, mean absolute error (MAE) = 0.0399 cm3/cm3, and Pearson correlation coefficient (PCC) = 0.7846), while largely overcoming the overestimation issue in the range of low values and the underestimation issue in the range of high values. The proposed approach effectively fused inexpensive and easily available multi-source data with uncertainties and obtained a satisfactory mapping accuracy, thus demonstrating the potential of the BME framework for soil moisture mapping using multi-source data.

Soil Moisture Retrievals by Combining Passive Microwave and Optical Data

This paper aims to retrieve the temporal dynamics of soil moisture from 2015 to 2019 over an agricultural site in Southeast Australia using the Soil Moisture Active Passive (SMAP) brightness temperature. To meet this objective, two machine learning approaches, Random Forest (RF), Support Vector Machine (SVM), as well as a statistical Ordinary Least Squares (OLS) model were established, with the auxiliary data including the 16-day composite MODIS NDVI (MOD13Q1) and Surface Temperature (ST). The entire data were divided into two parts corresponding to ascending (6:00 p.m. local time) and descending (6:00 a.m. local time) orbits of SMAP overpasses. Thus, the three models were trained using the descending data acquired during the five years (2015 to 2019), and validated using the ascending product of the same period. Consequently, three different temporal variations of the soil moisture were obtained based on the three models. To evaluate their accuracies, the retrieved soil moisture was compared against the SMAP level-2 soil moisture product, as well as to in-situ ground station data. The comparative results show that the soil moisture obtained using the OLS, RF and SVM algorithms are highly correlated to the SMAP level-2 product, with high coefficients of determination (R2OLS = 0.981, R2SVM = 0.943, R2RF = 0.983) and low RMSE (RMSEOLS = 0.016 cm3/cm3, RMSESVM = 0.047 cm3/cm3, RMSERF = 0.016 cm3/cm3). Meanwhile, the estimated soil moistures agree with in-situ station data across different years (R2OLS = 0.376~0.85, R2SVM = 0.376~0.814, R2RF = 0.39~0.854; RMSEOLS = 0.049~0.105 cm3/cm3, RMSESVM = 0.073~0.1 cm3/cm3, RMSERF = 0.047~0.102 cm3/cm3), but an overestimation issue is observed for high vegetation conditions. The RF algorithm outperformed the SVM and OLS, in terms of the agreement with the ground measurements. This study suggests an alternative soil moisture retrieval scheme, in complementary to the SMAP baseline algorithm, for a fast soil moisture retrieval.

Soil moisture retrievals using ALOS2-ScanSAR and MODIS synergy over Tibetan Plateau

This paper investigates the soil moisture retrievals over Tibetan Plateau using multi-temporal ALOS2-PALSAR2 data acquired in ScanSAR mode, in contrary to the commonly used StripMap SAR imaging mode. Considering the dual-polarimetry with limited observables, the surface roughness parameters such as the RMS height and auto-correlation length are first estimated by optimizing the semi-empirical Oh2004 model applied to the ALOS2 data acquired under bare soil condition. The vegetation water content corresponding to each ALOS2 acquisition time is derived from an empirical model applied to the temporally interpolated MODIS NDVI data. Then, the obtained roughness and vegetation optical depth are substituted into the Water Cloud Model which we modified by adding a double-bounce component according to the L-band scattering processes to retrieve the effective scattering albedo and soil moisture. The results show that the optimized surface RMS height and the associated slope are negatively related to a dual-angular radar index ΔHH, indicating the feasibility to optimize the relatively stable surface roughness parameters before retrieving the soil moisture dynamics. The obtained vegetation optical depth which is cross-validated against a normalized cross-polarized radar descriptor indicates a significant increase from May to August, in response to the vegetation phenological growth. Furthermore, the time-variable scattering albedo is less than 0.08, and slightly increases with the vegetation development. By accounting for the double-bounce component, the retrieved soil moisture better agrees with the ground measurements with R = 0.89 and RMSE = 0.058 m3/m3, but exhibits an overestimation issue for the soil moisture higher than 0.35 m3/m3 due to the saturation of SAR signal and the relatively strong vegetation cover. A positive correspondence (R = 0.81) between the retrieved soil moisture and the interpolated NDVI was found, verifying their close coupling in the soil-vegetation system. This study deepens the insights in the potential integration between the optical and L-band microwave observations to retrieve soil moisture over vegetated area.

Evaluation of Ocean Color Remote Sensing Algorithms for Diffuse Attenuation Coefficients and Optical Depths with Data Collected on BGC-Argo Floats

The vertical distribution of irradiance in the ocean is a key input to quantify processes spanning from radiative warming, photosynthesis to photo-oxidation. Here we use a novel dataset of thousands local-noon downwelling irradiance at 490 nm (Ed(490)) and photosynthetically available radiation (PAR) profiles captured by 103 BGC-Argo floats spanning three years (from October 2012 to January 2016) in the world’s ocean, to evaluate several published algorithms and satellite products related to diffuse attenuation coefficient (Kd). Our results show: (1) MODIS-Aqua Kd(490) products derived from a blue-to-green algorithm and two semi-analytical algorithms show good consistency with the float-observed values, but the Chla-based one has overestimation in oligotrophic waters; (2) The Kd(PAR) model based on the Inherent Optical Properties (IOPs) performs well not only at sea-surface but also at depth, except for the oligotrophic waters where Kd(PAR) is underestimated below two penetration depth (2zpd), due to the model’s assumption of a homogeneous distribution of IOPs in the water column which is not true in most oligotrophic waters with deep chlorophyll-a maxima; (3) In addition, published algorithms for the 1% euphotic-layer depth and the depth of 0.415 mol photons m−2 d−1 isolume are evaluated. Algorithms based on Chla generally work well while IOPs-based ones exhibit an overestimation issue in stratified and oligotrophic waters, due to the underestimation of Kd(PAR) at depth.

A TD3-based multi-agent deep reinforcement learning method in mixed cooperation-competition environment

We explored the problem about function approximation error and complex mission adaptability in multi-agent deep reinforcement learning. This paper proposes a new multi-agent deep reinforcement learning algorithm framework named multi-agent time delayed deep deterministic policy gradient. Our work reduces the overestimation error of neural network approximation and variance of estimation result using dual-centered critic, group target network smoothing and delayed policy updating. According to experiment results, it improves the ability to adapt complex missions eventually. Then, we discuss that there is an inevitable overestimation issue about existing multi-agent algorithms about approximating real action-value equations with neural network. We also explain the approximate error of equations in the multi-agent deep deterministic policy gradient algorithm mathematically and experimentally. Finally, the application of our algorithm in the mixed cooperative competition experimental environment further demonstrates the effectiveness and generalization of our algorithm, especially improving the group’s ability of adapting complex missions and completing more difficult missions.