Sparse Feedback Research Articles

Integrating Sparse Reward Handling, Ethical Considerations, And Domain-Specific Adaptation In RlBased Machine Translation For Low-Resource Languages

Effective communication across languages remains a critical challenge, particularly in low-resource settings where conventional machine translation approaches falter due to sparse data and limited quality feedback. This paper presents a holistic framework to enhance reinforcement learning (RL) based machine translation systems tailored for such environments. We address the trifecta of challenges: sparse feedback on translation quality, ethical implications in algorithmic decision-making, and the imperative to adapt models to nuanced linguistic domains. This approach integrates advanced techniques in sparse reward handling, ensuring RL models learn efficiently despite limited feedback. Ethical considerations drive our methodology, emphasizing fairness, bias mitigation, and cultural sensitivity to uphold ethical standards in AI-driven translations. Additionally, domain-specific adaptation strategies are explored to tailor models to diverse linguistic contexts, from technical jargon to colloquialisms, enhancing translation accuracy and relevance. Through a rigorous experimental framework, including evaluation metrics like BLEU score and user feedback, we demonstrate substantial improvements in translation quality and ethical compliance compared to traditional methods. This research contributes to the evolution of robust, inclusive translation technologies pivotal for fostering global understanding and equitable access to information. This paper not only addresses current challenges but also sets a precedent for future research in AI ethics and machine learning applications, advocating for responsible innovation in crosscultural communication technologies

Model-based sparse optimal control of the hydrogen sulfide synthesis process for acidic wastewater sulfidation

Hydrogen sulfide is one of the commonly used materials of the gas-liquid sulfidation process for treating nonferrous metallurgical acidic wastewater. Using hydrogen and sulfur to synthesize hydrogen sulfide can obtain hydrogen sulfide with higher purity and at a lower cost. Due to the complex multiphase reaction mechanism of the hydrogen sulfide synthesis (HSS) process and production safety conditions, the manual operation requires a long time to ensure the continued stability of the system under high temperatures and high pressure. In this paper, a dynamic model and a novel sparse control strategy of the HSS process for acidic wastewater sulfidation are proposed to enable the HSS process operates continuously and safely. Firstly, a multivariate and multiphase mechanism model with multiple inputs and outputs of the HSS process is established. Subsequently, the novel sparse feedback matrix is designed under the constraint of the fixed l0-norm and the solvability of the eigenvalue determinant equation. The sparse feedback matrix solutions are optimized by pole placement and dynamic programming for continuous and discrete systems, respectively. Finally, the effectiveness of the HSS model is validated with industrial data. The superiority of the proposed model-based control strategy is demonstrated in the HSS system compared with conventional control methods.

Unveiling the Significance of Toddler-Inspired Reward Transition in Goal-Oriented Reinforcement Learning

Toddlers evolve from free exploration with sparse feedback to exploiting prior experiences for goal-directed learning with denser rewards. Drawing inspiration from this Toddler-Inspired Reward Transition, we set out to explore the implications of varying reward transitions when incorporated into Reinforcement Learning (RL) tasks. Central to our inquiry is the transition from sparse to potential-based dense rewards, which share optimal strategies regardless of reward changes. Through various experiments, including those in egocentric navigation and robotic arm manipulation tasks, we found that proper reward transitions significantly influence sample efficiency and success rates. Of particular note is the efficacy of the toddler-inspired Sparse-to-Dense (S2D) transition. Beyond these performance metrics, using Cross-Density Visualizer technique, we observed that transitions, especially the S2D, smooth the policy loss landscape, promoting wide minima that enhance generalization in RL models.

Pole placement with minimum gain and sparse static feedback

The work addresses the issue of developing polar positioning algorithms for linear control systems with despotic thinning constraints on a static feedback matrix. The design of sparse-limiting feedback control systems is considered to be one of the more complex issues, but the issue of defining the sparse feedback matrix, which allows the selection of eigenvalues or, in other words, the placement of poles, has not been fully studied. Algorithms for determining the eigenvalues, i.e., solving the problem of pole placement and finding the optimal solution of the technological process control system are considered in the work. The results of numerical analysis confirmed the effectiveness of these algorithms, which allows them to be used in solving practical problems of synthesizing adaptive control systems for technological objects.

A survey on compressed sensing approach to systems and control

AbstractIn this survey paper, we review recent advances of compressed sensing applied to systems and control. Compressed sensing has been actively researched in the field of signal processing and machine learning. More recently, the method has been applied to systems and control problems, such as sparse feedback gain design, reduced-order control, and maximum hands-off control. This paper introduces these important applications of compressed sensing to systems and control. MATLAB programs for the numerical examples shown in this survey paper are available as supplementary materials.

Sparse feedback controller: from open-loop solution to closed-loop realization

Sparse feedback controller: from open-loop solution to closed-loop realization

UAV Trajectory Optimization for Spectrum Cartography: A PPO Approach

Benefiting from the flexibility and low operational cost, dispatching unmanned aerial vehicles (UAVs) to collect measurements is promising in spectrum cartography (SC). The main goal is to optimize the trajectory of an UAV to seek the most informative measurement under the environment with dynamic emitters. In this letter, we formulate a Markov Decision Process to find the optimal flight trajectory of an UAV that maximizes the accuracy of SC and minimizes energy consumption. However, due to the unavailable instantaneous feedback about the accuracy of SC, the existing methods are unable to work efficiently with sparse feedback. To tackle those issues, a Proximal Policy Optimization (PPO)-based algorithm is proposed to approach the optimal navigation policy for UAV by training with the delay interactive information at the base station. Moreover, A backtracking advantage function is further constructed to cope with sparse feedback in real-world scenario, which can avoid converging to local solutions. Extensive simulation results demonstrate that our proposed algorithm can significantly increase the accuracy of SC while reducing energy consumption.

A Hierarchical Reinforcement Learning Framework for UAV Path Planning in Tactical Environments

Tactical UAV path planning under radar threat using reinforcement learning involves particular challenges ranging from modeling related difficulties to sparse feedback problem. Learning goal-directed behavior with sparse feedback from complex environments is a fundamental challenge for reinforcement learning algorithms. In this paper we extend our previous work in this area to provide a solution to the problem setting stated above, using Hierarchical Reinforcement Learning (HRL) in a novel way that involves a meta controller for higher level goal assignment and a controller that determines the lower-level actions of the agent. Our meta controller is based on a regression model trained using a state transition scheme that defines the evolution of goal designation, whereas our lower-level controller is based on a Deep Q Network (DQN) and is trained via reinforcement learning iterations. This two-layer framework ensures that an optimal plan for a complex path, organized as multiple goals, is achieved gradually, through piecewise assignment of sub-goals, and thus as a result of a staged, efficient and rigorous procedure.

Internal control of brain networks via sparse feedback

AbstractThe human brain is a complex system whose function depends on interactions between neurons and their ensembles across scales of organization. These interactions are restricted by anatomical and energetic constraints, and facilitate information processing and integration in response to cognitive demands. In this work, we considered the brain as a closed loop dynamic system under sparse feedback control. This controller design considered simultaneously control performance and feedback (communication) cost. As proof of principle, we applied this framework to structural and functional brain networks. Under high feedback cost only a small number of highly connected network nodes were controlled, which suggests that a small subset of brain regions may play a central role in the control of neural circuits, through a trade‐off between performance and communication cost.

Sparse feedback stabilization in high‐order linear systems

AbstractWe consider the column sparsity of the feedback stabilization gain matrix in high‐order linear systems. By means of a special matrix norm and the state transition matrix quadratic cost function (SQF) of the systems, the sparse feedback stabilization controller design problem is formulated as a regularized SQF optimization problem. We further derive the proximal mapping of the special matrix norm, and then based on the gradient descent of the SQF part of the objective function, the proximal gradient method is introduced to develop an algorithm for solving the non‐smooth optimization problem. Numerical examples are given to illustrate the effectiveness of the proposed method.

Optimization of Invariant Ellipsoid Technique for Sparse Controllers Design

The paper deals with the method for the design of linear controllers with sparse state feedback matrices for control the plants under conditions of unknown and bounded disturbances. The importance of the sparsity property in feedback can be explained by two factors. First, by minimizing the columnar norm of the feedback matrix in the control it becomes pos- sible to use a minimum number of measuring devices. Secondly, by minimizing the row norm of the feedback matrix, the required number of executive (control) devices is minimized. Both properties, if they are achievable in the synthesis of the controller, reduce the cost of the system and improve the fault tolerance and quality of regulation by reducing the structural complexity. The search algorithm for sparse matrices is based on the method of invariant ellipsoids and is formulated as a solution to a system of linear matrix inequalities with additional constraints. A special set of optimization conditions is proposed which for a disturbed system minimizes overshoot and overshoots in transient processes of the disturbed closed- loop system simultaneously with minimizing errors in the steady state. The proposed method also assumes the possibility of minimizing both the row norm of the feedback matrix and the column one, while preserving the robustness properties, which makes it possible to solve the sparse control problem (a sparse control is understood as a linear controller with a sparse feedback matrix). The efficiency of the proposed control scheme is confirmed by the results of computer modeling and comparison with some existing ones.

Obstacle avoidance strategy for an autonomous surface vessel based on modified deep deterministic policy gradient

In the present paper, a decision-making agent based on reinforcement learning is designed for establishing an obstacle avoidance strategy of an autonomous surface vessel (ASV). To solve the sparse feedback in obstacle avoidance issue, the Modified Deep Deterministic Policy Gradient (MDDPG) method is proposed in the present study, using an actor-network to generate actions from the state of the ASV and a critic-network to evaluate the behavior of the actor-network. The gradient descend method is applied to improve the networks. Memory pool modification, success pool modification and target network modification are developed to smooth the training process of the ASV. A comparative analysis based on numerical simulations are carried out using Deep Q Network (DQN) method, Modified Deep Q Network (MDQN) method and MDDPG method. The results demonstrate that the MDDPG method smooths and accelerates the learning process. Compared with the other two methods, the effectiveness and viability of the proposed method is significantly increased. Random obstacles tests are also conducted to confirm the applicability and generalization of the method.

Curriculum learning with Hindsight Experience Replay for sequential object manipulation tasks

Learning complex tasks from scratch is challenging and often impossible for humans as well as for artificial agents. Instead, a curriculum can be used, which decomposes a complex task – the target task – into a sequence of source tasks. Each source task is a simplified version of the next source task with increasing complexity. Learning then occurs gradually by training on each source task while using knowledge from the curriculum’s prior source tasks. In this study, we present a new algorithm that combines curriculum learning with Hindsight Experience Replay (HER), to learn sequential object manipulation tasks for multiple goals and sparse feedback. The algorithm exploits the recurrent structure inherent in many object manipulation tasks and implements the entire learning process in the original simulation without adjusting it to each source task. We test our algorithm on three challenging throwing tasks in simulation and show significant improvements compared to vanilla-HER.

Sparse feedback stabilisation of linear delay systems by proximal gradient method

ABSTRACT In this paper, the sparse feedback stabilisation in which the gain matrix has as many zero components as possible in linear delay systems is investigated. The necessary and sufficient condition for the asymptotic stability of the delay systems under sparse state feedback is given. By means of a special matrix norm and the transition matrix (fundamental matrix) of delay systems, the sparse gain matrix design problem is transformed into an optimisation problem. We further derive the proximal mapping of the special matrix norm, and then based on the gradient descent of the smooth part of the objective function, the proximal gradient method is introduced to develop an algorithm for solving the non-smooth optimisation problem. Numerical examples are given to illustrate the effectiveness of the proposed method.

MSE minimized joint transmission in coordinated multipoint systems with sparse feedback and constrained backhaul requirements

In a joint transmission coordinated multipoint (JT-CoMP) system, a shared spectrum is utilized by all neighbor cells. In the downlink, a group of base stations (BSs) coordinately transmit the users’ data to avoid serious interference at the users in the boundary of the cells, thus substantially improving area fairness. However, this comes at the cost of high feedback and backhaul load; In a frequency division duplex system, all users at the cell boundaries have to collect and send feedback of the downlink channel state information (CSI). In centralized JT-CoMP, although with capabilities for perfect coordination, a central coordination node have to send the computed precoding weights and corresponding data to all cells which can overwhelm the backhaul resources. In this paper, we design a JT-CoMP scheme, by which the sum of the mean square error (MSE) at the boundary users is minimized, while feedback and backhaul loads are constrained and the load is balanced between BSs. Our design is based on the singular value decomposition of CSI matrix and optimization of a binary link selection matrix to provide sparse feedback—constrained backhaul link. For comparison, we adopt the previously presented schemes for feedback and backhaul reduction in the physical layer. Extensive numerical evaluations show that the proposed scheme can reduce the MSE with at least 25%, compared to the adopted and existing schemes.

Filtration network: A frame sampling strategy via deep reinforcement learning for video captioning

Recently many methods use encoder-decoder framework for video captioning, aiming to translate short videos into natural language. These methods usually use equal interval frame sampling. However, lacking a good efficiency in sampling, it has a high temporal and spatial redundancy, resulting in unnecessary computation cost. In addition, the existing approaches simply splice different visual features on the fully connection layer. Therefore, features cannot be effectively utilized. In order to solve the defects, we proposed filtration network (FN) to select key frames, which is trained by deep reinforcement learning algorithm actor-double-critic. According to behavior psychology, the core idea of actor-double-critic is that the behavior of agent is determined by both the external environment and the internal personality. It avoids the phenomenon of unclear reward and sparse feedback in training because it gives steady feedback after each action. The key frames are sent to combine codec network (CCN) to generate sentences. The operation of feature combination in CCN make fusion of visual features by complex number representation to make good semantic modeling. Experiments and comparisons with other methods on two datasets (MSVD/MSR-VTT) show that our approach achieves better performance in terms of four metrics, BLEU-4, METEOR, ROUGE-L and CIDEr.

Analysis of Aircraft Maintenance Related Accidents and Serious Incidents in Nigeria

The maintenance of aircraft presents considerable challenges to the personnel that maintain them. Challenges such as time pressure, system complexity, sparse feedback, cramped workspaces, etc., are being faced by these personnel on a daily basis. Some of these challenges cause aircraft-maintenance-related accidents and serious incidents. However, there is little formal empirical work that describes the influence of aircraft maintenance to aircraft accidents and incidents in Nigeria. This study, therefore, sets out to explore the contributory factors to aircraft-maintenance-related incidents from 2006 to 2019 and accidents from 2009 to 2019 in Nigeria, to achieve a deeper understanding of this safety critical aspect of the aviation industry, create awareness amongst the relevant stakeholders and seek possible mitigating factors. To attain this, a content analysis of accident reports and mandatory occurrence reports, which occurred in Nigeria, was carried out using the Maintenance Factors and Analysis Classification System (MxFACS) and Hieminga’s maintenance incidents taxonomy. An inter-rater concordance value was used to ascertain research accuracy after evaluation of the data output by subject matter experts. The highest occurring maintenance-related incidents and accidents were attributed to “removal/installation”, working practices such as “accumulation of dirt and contamination”, “inspection/testing”, “inadequate oversight from operator and regulator”, “failure to follow procedures” and “incorrect maintenance”. To identify the root cause of these results, maintenance engineers were consulted via a survey to understand the root causes of these contributory factors. The results of the study revealed that the most common maintenance-related accidents and serious incidents in the last decade are “collision with terrain” and “landing gear events’’. The most frequent failures at systems level resulting in accidents are the “engines” and “airframe structure”. The maintenance factors with the highest contribution to these accidents are “operator and regulatory oversight”, “inadequate inspection” and “failure to follow procedures”. The research also highlights that the highest causal and contributory factors to aviation incidents in Nigeria from 2006 to 2019 are “installation/removal issues”, “inspection/testing issues”, “working practices”, “job close up”, “lubrication and servicing”, all of which corresponds to studies by other researchers in other countries.

Sparsity Learning-Based CSI Feedback for FDD Massive MIMO Systems

In frequency-division duplexing mode, accurate channel state information (CSI) needs to be acquired at the base station by feedback from the terminals. However, the costly feedback overhead generated by large-scale arrays poses an arduous challenge to the traditional feedback scheme. In this letter, we propose a sparse learning-based CSI feedback method in a compressive sensing framework for massive MIMO systems. The key insights are to learn the sparse structure of CSI through the recursive least squares algorithm and process a continuous update of the sparse basis. Accordingly, a CSI sparse characteristic adaptive dictionary is constructed. The simulation results reveal that the proposed scheme achieves excellent performance in terms of both compression efficacy and recovery accuracy.

Adversarial Learning for Product Recommendation

Product recommendation can be considered as a problem in data fusion—estimation of the joint distribution between individuals, their behaviors, and goods or services of interest. This work proposes a conditional, coupled generative adversarial network (RecommenderGAN) that learns to produce samples from a joint distribution between (view, buy) behaviors found in extremely sparse implicit feedback training data. User interaction is represented by two matrices having binary-valued elements. In each matrix, nonzero values indicate whether a user viewed or bought a specific item in a given product category, respectively. By encoding actions in this manner, the model is able to represent entire, large scale product catalogs. Conversion rate statistics computed on trained GAN output samples ranged from 1.323% to 1.763%. These statistics are found to be significant in comparison to null hypothesis testing results. The results are shown comparable to published conversion rates aggregated across many industries and product types. Our results are preliminary, however they suggest that the recommendations produced by the model may provide utility for consumers and digital retailers.

Design of PI Controllers for Irrigation Canals Based on Linear Matrix Inequalities

A new Proportional-Integral (PI) tuning method based on Linear Matrix Inequalities (LMIs) is presented. In particular, an LMI-based optimal control problem is solved to obtain a sparse feedback that provides the PI tuning. The ASCE Test Canal 1 is used as a case study. Using a linearised model of the canal, different tunings for the design of the PI controller are developed and tested using the software Sobek. Furthermore, the proposed method is also compared with other tunings proposed for the same canal available in the literature. Our results show that the proposed method reduces by half the maximum errors with respect to other assessed alternatives and minimizes undesired mutual interactions between canal pools. Also, our method improves the optimality degree of the PI tuning by 30%. Therefore, it is concluded that the LMI based PI controllers lead to satisfactory performance in regulating water levels and canal flows/structure outflows, outperforming other tested alternatives, thus becoming a useful tool for irrigation canal control.