TD Methods Research Articles

Experimental investigation and energy-exergy-environmental-economic analysis of modified indirect solar dual collector dryer while drying myrobalan slices.

The paper presents the performance evaluation of a modified indirect solar dual collector dryer (MIS2CD) integrated with a thermal storage system for drying myrobalan slices. The design of the solar collector and solar collector with thermal storage was to supply uninterrupted thermal energy to the drying chamber during sunny and sunset hours. To evaluate the dryer performances, one lot (20kg) of myrobalan was dried in the MIS2CD, and as a result, the thermal efficiency and energy supply period of MIS2CD increased by 12 ± 02% and 41 ± 1.2%, respectively. Drying characteristics of myrobalan slices in MIS2CD, TD, and OSD were studied and compared. A two-term exponential model best explains the drying kinetics of myrobalan slices dried in MIS2CD. The dried sample in MIS2CD results in lesser ΔE* values than TD and OSD methods. The highest exergy efficiency of 78.2% and lower exergy losses were recorded. The energy payback period of the MIS2CD was evaluated as 1.42years. The CO2 emitted and CO2 reduced reduction are calculated for drying myrobalan in MIS2CD for a lifetime (20years) of 67.85kg and 20.65 tons, respectively. The capital cost of the solar dryer design was estimated depending on the economic considerations of the state. The drying hours were increased in MIS2CD against OSD by 59% on the annual sunny days (210days). The sample drying period MSD and TD to reach the final moisture level of 7% was 9h and 5h, respectively. The total economic benefit is 22,622 INR (annually), and the 2.08 benefit-cost ratio for myrobalan dried in MIS2CD compared to TD. The MIS2CD's payback period is nearly 2.18years, much less than the dryer's lifetime.

Estimation of R0 for COVID-19 in India through different mathematical model and their comparison

Background: The cases of novel coronavirus (COVID- 2019)-infected pneumonia started since the 19th of December, 2019, in Wuhan (Central China). A large scale outbreak of the disease resulted in a pandemic. This outbreak of the COVID -19 disease has spread on a wide scale. World health organization (WHO) has identified the ongoing outbreak of corona virus disease (COVID 2019) as pandemic on 11 March 2020. Basic reproduction number (R0)- is one of the most important predictors of epidemic severity. It can help to understand the path of the epidemic and to assess the effectiveness of the various interventions to control the epidemic. The purpose of this study is to estimate R0 by using five methods based on the Indian COVID-19 dataset and compare them. Methods: We obtained data on daily confirmed, recovered and deaths cases from official site of ministry of health and family welfare. We implemented 5 mathematical methods to calculate R0. We estimated the number of active cases till 14th of April. We also compare these methods to find out the best method to predict R0.Results: The estimated R0 for the AR, EG, ML, TD, and gamma-distributed methods were 1.0004, 2.102, 1.895, 1.872 and 1.46 respectively. The computed R0 in the TD method is closer to the actual R0 and have a good fit on data as confirmed with MSE criterion.Conclusions: Awareness of the basic reproduction number of COVID-19 is useful for controlling the spread of disease and for planning. It is therefore necessary to know the best method that has better performance.

Neural networks and Monte-Carlo method usage in multi-agent systems for sudoku problem solving

The object of research is multi-agent systems based on Deep Reinforcement Learning algorithms and analysis of ways to establish interaction within the system, based on intelligent agents. Also, part of the material in this paper covers ways to organize the management and administration of agents at the meta-level: external controllers and tools to optimize their work, describing architectural solutions that should accelerate agents’ training. The studied full-fledged multi-agent system would be flexible to expansion and would give effective acceleration in agent training and problem-solving quality.In this paper, the following neural network models were considered: DQN, DDQN, PPO, TD (methods based on Q-Learning), an approach using a neural network with Monte-Carlo tree search. The presented models were tested on a Sudoku problem with a dataset of 5039 combinations, dimensions 2x2, 4x4, and 9x9. Several sets of agent rewards were used. The presentation of data during the learning and problem-solving process was described. Also was built a multi-agent system based on the model using a Monte-Carlo tree search.According to the study results, it was revealed that for tasks in a complex environment, the models based on Q-Learning are practically ineffective (plots support the statement). The training process for these models is quite demanding on the characteristics of the workstation hardware. It was also determined that the Monte-Carlo tree search method does a good job. Even with a small number of iterations, it shows results better than other Deep Learning methods (45–50 % accuracy for 9x9). However, a significant drawback is a complexity of training the model, and the hardware requirements are too large for this kind of research.

Thermodilution vs indirect fick cardiac output measurement in clinical practice: insights from a tertiary centre

Abstract Introduction Thermodilution (TD) and indirect Fick (IF) methods are widely used to measure cardiac output (CO). They are often used interchangeably to make critical clinical decisions, yet few studies have compared these approaches concerning agreement and comparative prognostic value as applied in medical practice. Purpose To assess agreement between TD and IF methods and to compare how well these methods predict mortality. Methods Retrospective cohort study including all consecutive right heart catheterizations performed in a referral pulmonary hypertension (PH) centre from 2010 to 2018. Cardiac index (CI) was calculated by indexed CO to body surface area. PH was classified according to the new definition of the 6st World Symposium on Pulmonary Hypertension 2018 [mean pulmonary arterial pressure (mPAP) >20 mmHg]. Patients with cardiac or extra-cardiac shunts or significant (moderate to severe or severe) tricuspid regurgitation were excluded. All-cause mortality over 1 year after right heart catheterization was recorded. Logistic regression was used to identify predictors of the adverse event. Results From a total of 569 procedures, 424 fulfilled the inclusion criteria: mean age 56.7±15.4 years, 67.3% female. Haemodynamic parameters were diagnosed of PH in 86.2% of cases: mPAP 35.3±15.3 mmHg, 83.6% pre-capillary subtype, 42.9% belonging to group 4 (chronic thromboembolic pulmonary hypertension) and 26.6% to group 1 (pulmonary arterial hypertension). Mean values of CO and CI were, respectively, 4.5±2.8 L/min and 2.5±0.8 L/min/m2 measured by TD and 4.6±2.4 L/min and 2.6±1.3 L/min/m2 measured by IF method. There was a median difference (IF minus TD) of - 0.03 / min to CO and - 0.05 L/min/m2 to CI but both meausres correlated only modestly (r=0.6 to TD and r=0.5 to IF). One-year all-cause mortality rate was 5.4% (median time to death was 50.5 days). Lower values of CO and CI assessed by TD were significantly associated with all-cause mortality occurrence (CO TD: 4.5±1.3 L/min versus 3.6±1.0 L/min, p<0.01; CI TD: 2.6±0.7 L/min/m2 versus 2.1±0.4 L/min/m2, p<0.01). No association was observed between CO (p=0.31) and CI (p=0.42) measured by IF method and the adverse event. Logistic regression identified 2 independent predictors of all-cause mortality: TD CO (OR 0.55, 95% CI 0.38–0.79, p<0.01) and TD CI (OR 0.34, 95% CI 0.17–0.67, p<0.01). Similar results were obtained when patients diagnosed with PH were independently analyzed. Conclusions There is only modest agreement between TD and IF CO and CI estimates. Despite being more time-consuming, TD measurements were predictors of all-cause mortality and present a highest prognostic value. These findings favored their used over IF in clinical practice. Funding Acknowledgement Type of funding source: None

Synthesis, spectral characterization, and theoretical investigations of a new azo-stilbene dye for acrylic resins

This paper presents the synthesis of a new symmetrical disazo direct dye with good coloring properties for acrylic resins and several theoretical approaches to choose the best simulation method of the dye structure and spectra. The synthesized dye contains 4,4′-diaminostilbene-2,2′-disulfonic acid, as middle component and 2,7-dihydroxynaphthalene, as coupling component. It was analyzed by thin-layer chromatography (TLC), FT/IR, UV/VIS, 1H NMR, 13C NMR spectroscopy, and HRMS spectrometry. The molecular dye structure was investigated using molecular mechanics and the Density Functional Theory (DFT) approach at the B3LYP/6-31G(d,p), B3LYP/6-31+G(d,p) and B3PW91/6-311G(d,p) levels. Theoretical vibrational frequencies and structural parameters were calculated and the dye UV/VIS spectrum was simulated using the CIS, TD and ZINDO methods and compared with the experimental data. The ZINDO approach gave better maximum absorption predicted values compared to the CIS and TD methods. A generally good agreement between the experimental and theoretical computed absorption maxima and excellent correspondences between the experimental and calculated vibrational frequencies and chemical shifts were found. The experimental and simulated vibrational, UV/VIS and 13C NMR spectra confirmed the trans-configuration form of the π-bonds of the investigated dye. The CIEL∗a∗b∗ color space was used in all color measurements for the dye in powder and its mixtures with a water-based acrylic resin and confirmed the color properties of the newly synthesized dye.

WBPAT for FDTD simulation of microwave devices with homogeneous waveport

Accurate power-to-amplitude transformation (PAT) has been proposed for the continuous wave (CW, i.e. sinusoidal wave or multi-carrier) to stimulate the quantitative electromagnetic fields under specified average input power with the finite-difference time-domain (FDTD) method for simulation of microwave devices. In this study, the authors propose wideband PAT (WBPAT), by which quantitative amplitude and phase of electromagnetic fields at desired frequencies can be accurately extracted for microwave devices with homogeneous waveport under specified average input power when the excitation waveform of specified waveguide mode is wideband impulse signal. In the proposed WBPAT, the information of incident wave for normalisation is needed, which is investigated and discussed. Furthermore, a post-processing procedure called as the mode-impedance-modified normalisation is proposed to consider the frequency dependence of mode impedance, in which the mode impedance in wideband is obtained by FDTD simulation, per se. The average input power is defined for each frequency within the frequency range. As a result, accurate amplitude and phase, which is equivalent to that by frequency-domain methods or TD methods with the excitation of CW after steady state, can be extracted. Several numerical examples show the validity of the proposed method.

Comparison of methods to Estimate Basic Reproduction Number (R 0) of influenza, Using Canada 2009 and 2017-18 A (H1N1) Data.

Background:The basic reproduction number (R0) has a key role in epidemics and can be utilized for preventing epidemics. In this study, different methods are used for estimating R0's and their vaccination coverage to find the formula with the best performance.Materials and Methods:We estimated R0 for cumulative cases count data from April 18 to July 6, 2009 and 35-2017 to 34-2018 weeks in Canada: maximum likelihood (ML), exponential growth rate (EG), time-dependent reproduction numbers (TD), attack rate (AR), gamma-distributed generation time (GT), and the final size of the epidemic. Gamma distribution with mean and standard deviation 3.6 ± 1.4 is used as GT.Results:The AR method obtained a R0 (95% confidence interval [CI]) value of 1.116 (1.1163, 1.1165) and an EG (95%CI) value of 1.46 (1.41, 1.52). The R0 (95%CI) estimate was 1.42 (1.27, 1.57) for the obtained ML, 1.71 (1.12, 2.03) for the obtained TD, 1.49 (1.0, 1.97) for the gamma-distributed GT, and 1.00 (0.91, 1.09) for the final size of the epidemic. The minimum and maximum vaccination coverage were related to AR and TD methods, respectively, where the TD method has minimum mean squared error (MSE). Finally, the R0 (95%CI) for 2018 data was 1.52 (1.11, 1.94) by TD method, and vaccination coverage was estimated as 34.2%.Conclusion:For the purposes of our study, the estimation of TD was the most useful tool for computing the R0, because it has the minimum MSE. The estimation R0 > 1 indicating that the epidemic has occurred. Thus, it is required to vaccinate at least 41.5% to prevent and control the next epidemic.

Proximal Gradient Temporal Difference Learning: Stable Reinforcement Learning with Polynomial Sample Complexity


 
 
 In this paper, we introduce proximal gradient temporal difference learning, which provides a principled way of designing and analyzing true stochastic gradient temporal difference learning algorithms. We show how gradient TD (GTD) reinforcement learning methods can be formally derived, not by starting from their original objective functions, as previously attempted, but rather from a primal-dual saddle-point objective function. We also conduct a saddle-point error analysis to obtain finite-sample bounds on their performance. Previous analyses of this class of algorithms use stochastic approximation techniques to prove asymptotic convergence, and do not provide any finite-sample analysis. We also propose an accelerated algorithm, called GTD2-MP, that uses proximal "mirror maps" to yield an improved convergence rate. The results of our theoretical analysis imply that the GTD family of algorithms are comparable and may indeed be preferred over existing least squares TD methods for off-policy learning, due to their linear complexity. We provide experimental results showing the improved performance of our accelerated gradient TD methods.
 
 

Thermodilution vs Estimated Fick Cardiac Output Measurement in Clinical Practice

Thermodilution (Td) and estimated oxygen uptake Fick (eFick) methods are widely used to measure cardiac output (CO). They are often used interchangeably to make critical clinical decisions, yet few studies have compared these approaches as applied in medical practice. To assess agreement between Td and eFick CO and to compare how well these methods predict mortality. This investigation was a retrospective cohort study with up to 1 year of follow-up. The study used data from the Veterans Affairs Clinical Assessment, Reporting, and Tracking (VA CART) program. The findings were corroborated in a cohort of patients cared for at Vanderbilt University, an academic referral center. Participants were more than 15 000 adults who underwent right heart catheterization, including 12 232 in the Veterans Affairs cohort between October 1, 2007, and September 30, 2013, and 3391 in the Vanderbilt cohort between January 1, 1998, and December 31, 2014. A single cardiac catheterization was performed on each patient with CO estimated by both Td and eFick methods. Cardiac output was indexed to body surface area (cardiac index [CI]) for all analyses. All-cause mortality over 90 days and 1 year after catheterization. Among 12 232 VA patients (mean [SD] age, 66.4 [9.9] years; 3.3% female) who underwent right heart catheterization in this cohort study, Td and eFick CI estimates correlated modestly (r = 0.65). There was minimal mean difference (eFick minus Td = -0.02 L/min/m2, or -0.4%) but wide 95% limits of agreement between methods (-1.3 to 1.3 L/min/m2, or -50.1% to 49.4%). Estimates differed by greater than 20% for 38.1% of patients. Low Td CI (<2.2 L/min/m2 compared with normal CI of 2.2-4.0 L/min/m2) more strongly predicted mortality than low eFick CI at 90 days (Td hazard ratio [HR], 1.71; 95% CI, 1.47-1.99; χ2 = 49.5 vs eFick HR, 1.42; 95% CI, 1.22-1.64; χ2 = 20.7) and 1 year (Td HR, 1.53; 95% CI, 1.39-1.69; χ2 = 71.5 vs eFick HR, 1.35; 1.22-1.49; χ2 = 35.2). Patients with a normal CI by both methods had 12.3% 1-year mortality. There was no significant additional risk for patients with a normal Td CI but a low eFick CI (12.9%, P = .51), whereas a low Td CI but normal eFick CI was associated with higher mortality (15.4%, P = .001). The results from the Vanderbilt cohort were similar in the context of a more balanced sex distribution (46.6% female). There is only modest agreement between Td and eFick CI estimates. Thermodilution CI better predicts mortality and should be favored over eFick in clinical practice.

Acquisition of earthworm-like movement patterns of many-segmented peristaltic crawling robots

In recent years, attention has been increasingly devoted to the development of rescue robots that can protect humans from the inherent risks of rescue work. Particularly, anticipated is the development of a robot that can move deeply through small spaces. We have devoted our attention to peristalsis, the movement mechanism used by earthworms. A reinforcement learning technique used for the derivation of the robot movement pattern, Q-learning, was used to develop a three-segmented peristaltic crawling robot with a motor drive. Characteristically, peristalsis can provide movement capability if at least three segments work, even if a segmented part does not function. Therefore, we had intended to derive the movement pattern of many-segmented peristaltic crawling robots using Q-learning. However, because of the necessary increase in calculations, in the case of many segments, Q-learning cannot be used because of insufficient memory. Therefore, we devoted our attention to a learning method called Actor–Critic, which can be implemented with low memory. Because Actor-Critic methods are TD methods that have a separate memory structure to explicitly represent the policy independent of the value function. Using it, we examined the movement patterns of six-segmented peristaltic crawling robots.

Theoretical study on the electronic spectra in cyclic 1,2-diketones

The structural and electronic properties of some α-diketones have been investigated theoretically by performing both Hartree–Fock and density functional theory calculations at HF/6-31G(d,p) and B3LYP/6-31G(d,p) levels of theory.The electronic spectra were calculated by ZINDO and TD methods at each level of theory. The wavelength of the n→π∗ electronic transitions was correlated with the torsion angle between the two carbonyl groups in these compounds. The study revealed that the n→π∗ electronic transitions in the studied compounds are functions of the torsion angles between the two carbonyl groups within the linkage CO–CO.

Toponym Disambiguation by Arborescent Relationships

Problem statement: The way of referring to a place in the geographical space can be formal, based on the spatial coordinates, or informal, which we use in natural language by using toponyms (place names). A toponym can represent several geographical places. This ambiguity made problematic its conversion towards a unique formal representation. Toponym disambiguation in text is the task of assigning a unique location to an ambiguous place name in a given textual context. Approach: Several toponym disambiguation heuristics assumed a geographical proximity between the toponyms of the same context. This proximity can be in terms of spatial distance or in terms of arborsecent relationships, i.e., proximity in the hierarchical tree of the world places. This study presented a new toponym disambiguation heuristic in text based on the quantification of the arborescent proximity between toponyms. This quantification was done by a new measure of geographical correlation that we call the Geographical Density. Results: Our method was compared to the state of the art methods using GeoSemCor corpus and it has outperformed them in term of recall (87.4%) and coverage (99.0%). The results showed that the toponyms of the same context are much closer in terms of arborescent relationships than in terms of spatial relationships. Conclusion: We believe that the quantification of arborescent relationships between toponyms of the same textual context is a good way to improve the recall of TD task. However, all the arborescent relationships’ types must be considered and not only the meronymy, which is the relation the most exploited in the existing TD methods.

Self-Organizing Neural Architectures and Cooperative Learning in a Multiagent Environment

Temporal-Difference-Fusion Architecture for Learning, Cognition, and Navigation (TD-FALCON) is a generalization of adaptive resonance theory (a class of self-organizing neural networks) that incorporates TD methods for real-time reinforcement learning. In this paper, we investigate how a team of TD-FALCON networks may cooperate to learn and function in a dynamic multiagent environment based on minefield navigation and a predator/prey pursuit tasks. Experiments on the navigation task demonstrate that TD-FALCON agent teams are able to adapt and function well in a multiagent environment without an explicit mechanism of collaboration. In comparison, traditional Q-learning agents using gradient-descent-based feedforward neural networks, trained with the standard backpropagation and the resilient-propagation (RPROP) algorithms, produce a significantly poorer level of performance. For the predator/prey pursuit task, we experiment with various cooperative strategies and find that a combination of a high-level compressed state representation and a hybrid reward function produces the best results. Using the same cooperative strategy, the TD-FALCON team also outperforms the RPROP-based reinforcement learners in terms of both task completion rate and learning efficiency.

Evolutionary Function Approximation for Reinforcement Learning

Temporal difference methods are theoretically grounded and empirically effective methods for addressing reinforcement learning problems. In most real-world reinforcement learning tasks, TD methods ...

An experimental and theoretical study of the electronic spectra of tetraethylammonium [bis(1,3-dithiole-2-thione-4,5-dithiolato)zincate(II)], [NEt 4] 2[Zn(dmit) 2], and tetraethylammonium [bis(1,3-dithiole-2-one-4,5-dithiolato)zincate(II)], [NEt 4] 2[Zn(dmio) 2

Metal-dmit complexes and related compounds have been the object of intense study in the last decade. Despite such efforts on the study of its structural properties, very few attempts have been made to the spectroscopic study of these metal complexes. Experimental reports of its infrared, Raman and UV–Vis spectra present the main spectroscopic features, however, many details of the electronic structure have still to be fully investigated and inconsistent assignments are found in the literature. This work presents a detailed analysis of the UV–Vis spectra of the zinc-dmit, [Zn(dmit) 2] −2, and the zinc-dmio complex, [Zn(dmio) 2] −2. The experimental spectrum was deconvoluted and analysed with several theoretical methodologies including ab initio CI calculations, ab initio TD and zindo semi-empirical methods. The results confirm the multi-configuration nature of several excited states and the calculated results were concordant for several transitions. The results lead to a new assignment of the 457 nm band in the [Zn(dmit) 2] −2 as π(pS m) → π*C S band. In the metal-dmio, the sulfur substitution by oxygen results in a larger HOMO–LUMO gap and a change in the nature of the frontier orbitals. As the first transition we found, for the dmit compound, a high-intensity π → π*C S while for the zinc-dmio, a low-intensity π → σ*C–S transition.

Simulation of hysteresis effects with the classical Preisach model in FI ^2 TD methods

The paper presents a numerical method for transient magneto-quasistatic field analysis including hysteretic fer- romagnetic materials. The procedure is based on a Finite Integration Implicit Time Domain (FI 2 TD) formulation using the classical Preisach model. An hybrid scheme using the magnetic polarization update scheme for hysteretic materials and the successive approximation scheme for non-hysteretic nonlinear materials is given. Relaxation parameters are introduced to avoid convergence problems of the iterative scheme around the critical zones of the hysteresis curve. In numerical experiments with a test configuration the simulated hysteresis loops are compared with those achieved by measurements and the corresponding hysteretic losses are presented.

F+ laser performance and interaction of Cu, Ag and Au at the reduced oxygen coordination of MgO surface: first principles calculations

F+ laser performance and interaction of the title group IB transition metals at the reduced oxygen coordination of MgO surface were investigated using the TD and DFT methods of ab initio molecular electronic structure calculations. The considered ion clusters were embedded in simulated Coulomb fields that closely approximate the Madelung fields of the host surfaces and the nearest neighbor ions to the F+ site were allowed to relax to equilibrium in each case. The F+ laser performance fades quickly as the reduced oxygen coordination decreases from 5 to 4 to 3. The relaxed excited states (RESs) of the defect containing surfaces are compact and deep below the conduction bands of the perfect MgO surface. The probability of orientational destruction of the center in laser experiment is expected to follow the order flat>corner>edge. The excited state at the edge has higher energy than that at the flat or at the corner. F+ is easily formed at the lower oxygen coordination and the disappearance of anisotropy and 2p splitting observed in absorption of F+ at the surface follow the order corner>flat>edge. The Glasner–Tompkins relation is generalized to include the F+ bands at the reduced oxygen coordination of a metal oxide surface. As far as the adsorbate–substrate interactions are concerned, the F+ center enhances the adsorptivity of Ag, Cu and Au by ca. 1.91–3.33 eV and changes the nature of adsorption from physical adsorption to chemical adsorption. The adsorption energies follow the order Cu>Au>Ag and are explainable in terms of electrostatic potential curves, energy gaps and spin pairing. Cu and Ag act as electron donors while Au acts as electron acceptor and the MgO surface cannot be made semiconducting by F+ imperfection.

TD(λ) algorithms for average reward problems

In this paper, some on-line TD(λ) learning algorithms for average reward stochastic dynamic programming problems are presented. Instead of the cost-to-go function in previous algorithms, the relative function is the principal object to be learned in the algorithms proposed. This work is an extension and generalization of the work on previous TD(λ) methods and R-leammg algorithm. A robot experiment called Wall-Following is presented to test and illustrate the new algorithms with variant parameters. The results show that the new methods work well in approximating the relative value function.

AN ANALYSIS OF EXPERIENCE REPLAY IN TEMPORAL DIFFERENCE LEARNING

Temporal difference TD methods are used by reinforcement learning algorithms for predicting future rewards. This article analyzes theoretically and illustrates experimentally the effects of performing TD lambda prediction udpates backwards for a number of past experiences. More exactly, two related techniques described in the literature are examined, referred to as replayed TD and backwards TD. The former is essentially an online learning method which performs at each time step a regular TD 0 update, and then replays updates backwards for a number of previous states. The latter operates in offline mode, after the end of a trial updating backwards the predictions for all visited states. They are both shown to be approximately equivalent to TD lambda with variable lambda values selected in a particular way. This is true even if they perform only TD 0 updates. The experimental results show that replayed TD 0 is competitive to TD lambda with regard to learning speed and quality.

Practical issues in temporal difference learning

This paper examines whether temporal difference methods for training connectionist networks, such as Sutton's TD(λ) algorithm, can be successfully applied to complex real-world problems. A number of important practical issues are identified and discussed from a general theoretical perspective. These practical issues are then examined in the context of a case study in which TD(λ) is applied to learning the game of backgammon from the outcome of self-play. This is apparently the first application of this algorithm to a complex non-trivial task. It is found that, with zero knowledge built in, the network is able to learn from scratch to play the entire game at a fairly strong intermediate level of performance, which is clearly better than conventional commercial programs, and which in fact surpasses comparable networks trained on a massive human expert data set. This indicates that TD learning may work better in practice than one would expect based on current theory, and it suggests that further analysis of TD methods, as well as applications in other complex domains, may be worth investigating.