Reduced State Space Research Articles

Rare events and first passage time statistics from the energy landscape.

We analyze the probability distribution of rare first passage times corresponding to transitions between product and reactant states in a kinetic transition network. The mean first passage times and the corresponding rate constants are analyzed in detail for two model landscapes and the double funnel landscape corresponding to an atomic cluster. Evaluation schemes based on eigendecomposition and kinetic path sampling, which both allow access to the first passage time distribution, are benchmarked against mean first passage times calculated using graph transformation. Numerical precision issues severely limit the useful temperature range for eigendecomposition, but kinetic path sampling is capable of extending the first passage time analysis to lower temperatures, where the kinetics of interest constitute rare events. We then investigate the influence of free energy based state regrouping schemes for the underlying network. Alternative formulations of the effective transition rates for a given regrouping are compared in detail to determine their numerical stability and capability to reproduce the true kinetics, including recent coarse-graining approaches that preserve occupancy cross correlation functions. We find that appropriate regrouping of states under the simplest local equilibrium approximation can provide reduced transition networks with useful accuracy at somewhat lower temperatures. Finally, a method is provided to systematically interpolate between the local equilibrium approximation and exact intergroup dynamics. Spectral analysis is applied to each grouping of states, employing a moment-based mode selection criterion to produce a reduced state space, which does not require any spectral gap to exist, but reduces to gap-based coarse graining as a special case. Implementations of the developed methods are freely available online.

Consensus, cooperative learning, and flocking for multiagent predator avoidance.

Multiagent coordination is highly desirable with many uses in a variety of tasks. In nature, the phenomenon of coordinated flocking is highly common with applications related to defending or escaping from predators. In this article, a hybrid multiagent system that integrates consensus, cooperative learning, and flocking control to determine the direction of attacking predators and learns to flock away from them in a coordinated manner is proposed. This system is entirely distributed requiring only communication between neighboring agents. The fusion of consensus and collaborative reinforcement learning allows agents to cooperatively learn in a variety of multiagent coordination tasks, but this article focuses on flocking away from attacking predators. The results of the flocking show that the agents are able to effectively flock to a target without collision with each other or obstacles. Multiple reinforcement learning methods are evaluated for the task with cooperative learning utilizing function approximation for state-space reduction performing the best. The results of the proposed consensus algorithm show that it provides quick and accurate transmission of information between agents in the flock. Simulations are conducted to show and validate the proposed hybrid system in both one and two predator environments, resulting in an efficient cooperative learning behavior. In the future, the system of using consensus to determine the state and reinforcement learning to learn the states can be applied to additional multiagent tasks.

Model checking safety and liveness via k-induction and witness refinement with constraint generation

• Automatic SAT-based model checking of concurrent software systems. • Incremental and complete bounded model checking based on k -induction. • Multi-model abstraction refinement technique that constructs global and local models. • Refinement based on path constraints derived from unsatisfiable cores. • Support of model checking liveness properties based on a reduction to safety. In this article, we revise our constraint-based abstraction refinement technique for checking temporal logic properties of concurrent software systems. Our technique employs predicate abstraction and SAT-based three-valued bounded model checking. In contrast to classical refinement techniques where a single state space model is iteratively explored and refined with predicates, our approach is as follows: We use a coarsely-abstracted global state space model where we check for abstract witness paths for the property of interest. For each detected abstract witness we construct a local model whose state space is restricted to refinements of the witness only. On the local models we check whether the witness is real or spurious. We eliminate spurious witnesses in the global model via spurious segment constraints , which do not increase the state space complexity. Our technique is complete and terminates when a real witness in a local model can be detected, or no more witnesses in the global model exist. While our technique was originally restricted to the verification of safety properties, we extend it here to the verification of liveness properties. For this, we make use of the state recording translation of the input system, which reduces liveness model checking to safety checking. Another restriction of our original approach was its incompleteness due to the nature of bounded model checking. Here we show how abstraction refinement-based bounded model checking can be combined with the k -induction principle, which enables unbounded model checking. Our approach is iterative with regard to the bound. The extended approach also allows us to define enhanced concepts for strengthening the constraints that we use to rule out spurious behaviour and for reusing constraints between bound iterations. We demonstrate that our approach enables the complete verification of safety and liveness properties with a reduced state space complexity and a better solving time in comparison to classical abstraction refinement techniques.

Uncertainty quantification of bladed disc systems using data driven stochastic reduced order models

This study focusses on the development of stochastic reduced order model for probabilistic characterisation of bladed disc systems with random spatial inhomogeneities. High fidelity finite element modelling is used to mathematically model the system. A two step reduction strategy is applied involving reduction in the state space dimension and reduction in the stochastic dimensions. Information of the spatial inhomogeneities are assumed to be available from limited in situ measurements across the spatial extent and are modelled as non-Gaussian random fields. The stochastic version of the finite element matrices are developed using a polynomial chaos based framework, which optimizes the stochastic dimensionality of the problem. The uncertainties in the input propagates through the system into the response, which are also random. Surrogate models for these response quantities are obtained as PCE and are constructed using the method of stochastic collocations. Challenges involved in application of PCE on complex geometrically irregular spatial domains are addressed. The efficacy of the proposed framework is demonstrated through two numerical examples -an academic bladed disc system and an industrial turbine rotor blade.

Solving K-MDPs

Markov Decision Processes (MDPs) are employed to model sequential decision-making problems under uncertainty. Traditionally, algorithms to solve MDPs have focused on solving large state or action spaces. With increasing applications of MDPs to human-operated domains such as conservation of biodiversity and health, developing easy-to-interpret solutions is of paramount importance to increase uptake of MDP policies. Here, we define the problem of solving K-MDPs, i.e., given an original MDP and a constraint on the number of states (K), generate a reduced state space MDP that minimizes the difference between the original optimal MDP value function and the reduced optimal K-MDP value function. Building on existing non-transitive and transitive approximate state abstraction functions, we propose a family of three algorithms based on binary search with sub-optimality bounded polynomially in a precision parameter: ϕQ*εK-MDP-ILP, ϕQ*dK-MDP and ϕa*dK-MDP. We compare these algorithms to a greedy algorithm (ϕQ*ε Greedy K-MDP) and clustering approach (k-means++ K-MDP). On randomly generated MDPs and two computational sustainability MDPs, ϕa*dK-MDP outperformed all algorithms when it could find a feasible solution. While numerous state abstraction problems have been proposed in the literature, this is the first time that the general problem of solving K-MDPs is suggested. We hope that our work will generate future research aiming at increasing the interpretability of MDP policies in human-operated domains.

Solving high-level Petri games

The manual implementation of local controllers for autonomous agents in a distributed and concurrent setting is an ambitious and error-prune task. Synthesis algorithms, however, allow for the automatic generation of such controllers given a formal specification of the system’s goal. Recently, high-level Petri games were introduced to allow for a concise modeling technique of distributed systems with a safety objective. One way of solving these games is by a translation to low-level Petri games and applying an existing solving algorithm. In this paper we present a new solving technique for a subclass of high-level Petri games with a single uncontrollable player, a bounded number of controllable players, and a local safety objective. The technique exploits symmetries in the high-level Petri game. We report on encouraging experimental results of a prototype implementation generating the reduced state space. The results for four existing and one new benchmark family show a state space reduction by up to three orders of magnitude.

Dynamics of quantum systems driven by time-varying Hamiltonians: Solution for the Bloch-Siegert Hamiltonian and applications to NMR

Comprehending the dynamical behaviour of quantum systems driven by time-varying Hamiltonians is particularly difficult. Systems with as little as two energy levels are not yet fully understood as the usual methods including diagonalisation of the Hamiltonian do not work in this setting. In fact, since the inception of Magnus' expansion in 1954, no fundamentally novel mathematical approach capable of solving the quantum equations of motion with a time-varying Hamiltonian has been devised. We report here of an entirely different non-perturbative approach, termed path-sum, which is always guaranteed to converge, yields the exact analytical solution in a finite number of steps for finite systems and is invariant under scale transformations of the quantum state space. Path-sum can be combined with any state-space reduction technique and can exactly reconstruct the dynamics of a many-body quantum system from the separate, isolated, evolutions of any chosen collection of its sub-systems. As examples of application, we solve analytically for the dynamics of all two-level systems as well as of a many-body Hamiltonian with a particular emphasis on NMR (Nuclear Magnetic Resonance) applications: Bloch-Siegert effect, coherent destruction of tunneling and $N$-spin systems involving the dipolar Hamiltonian and spin diffusion.

Ruelle–Pollicott Resonances of Stochastic Systems in Reduced State Space. Part I: Theory

A theory of Ruelle-Pollicott (RP) resonances for stochastic differential systems is presented. These resonances are defined as the eigenvalues of the generator (Kolmogorov operator) of a given stochastic system. By relying on the theory of Markov semigroups, decomposition formulas of correlation functions and power spectral densities (PSDs) in terms of RP resonances are then derived. These formulas describe, for a broad class of stochastic differential equations (SDEs), how the RP resonances characterize the decay of correlations as well as the signal's oscillatory components manifested by peaks in the PSD.It is then shown that a notion reduced RP resonances can be rigorously defined, as soon as the dynamics is partially observed within a reduced state space V . These reduced resonances are obtained from the spectral elements of reduced Markov operators acting on functions of the state space V , and can be estimated from series. They inform us about the spectral elements of some coarse-grained version of the SDE generator. When the time-lag at which the transitions are collected from partial observations in V , is either sufficiently small or large, it is shown that the reduced RP resonances approximate the (weak) RP resonances of the generator of the conditional expectation in V , i.e. the optimal reduced system in V obtained by averaging out the contribution of the unobserved variables. The approach is illustrated on a stochastic slow-fast system for which it is shown that the reduced RP resonances allow for a good reconstruction of the correlation functions and PSDs, even when the time-scale separation is weak.The companions articles, Part II and Part III, deal with further practical aspects of the theory presented in this contribution. One important byproduct consists of the diagnosis usefulness of stochastic dynamics that RP resonances provide. This is illustrated in the case of a stochastic Hopf bifurcation in Part II. There, it is shown that such a bifurcation has a clear manifestation in terms of a geometric organization of the RP resonances along discrete parabolas in the left half plane. Such geometric features formed by (reduced) RP resonances are extractable from time series and allow thus for providing an unambiguous "signature" of nonlinear oscillations embedded within a stochastic background. By relying then on the theory of reduced RP resonances presented in this contribution, Part III addresses the question of detection and characterization of such oscillations in a high-dimensional stochastic system, namely the Cane-Zebiak model of El Ni{\~n}o-Southern Oscillation subject to noise modeling fast atmospheric fluctuations.

Learning When to Trust a Dynamics Model for Planning in Reduced State Spaces

When the dynamics of a system are difficult to model and/or time-consuming to evaluate, such as in deformable object manipulation tasks, motion planning algorithms struggle to find feasible plans efficiently. Such problems are often reduced to state spaces where the dynamics are straightforward to model and evaluate. However, such reductions usually discard information about the system for the benefit of computational efficiency, leading to cases where the true and reduced dynamics disagree on the result of an action. This paper presents a formulation for planning in reduced state spaces that uses a classifier to bias the planner away from state-action pairs that are not reliably feasible under the true dynamics. We present a method to generate and label data to train such a classifier, as well as an application of our framework to rope manipulation, where we use a Virtual Elastic Band (VEB) approximation to the true dynamics. Our experiments with rope manipulation demonstrate that the classifier significantly improves the success rate of our RRT-based planner in several difficult scenarios which are designed to cause the VEB to produce incorrect predictions in key parts of the environment.

Ruelle-Pollicott Resonances of Stochastic Systems in Reduced State Space. Part II: Stochastic Hopf Bifurcation

The spectrum of the generator (Kolmogorov operator) of a diffusion process, referred to as the Ruelle-Pollicott (RP) spectrum, provides a detailed characterization of correlation functions and power spectra of stochastic systems via decomposition formulas in terms of RP resonances; see Part I of this contribution (Chekroun et al. in Theory J Stat. https://doi.org/10.1007/s10955-020-02535-x , 2020). Stochastic analysis techniques relying on the theory of Markov semigroups for the study of the RP spectrum and a rigorous reduction method is presented in Part I Chekroun et al. (2020). This framework is here applied to study a stochastic Hopf bifurcation in view of characterizing the statistical properties of nonlinear oscillators perturbed by noise, depending on their stability. In light of the Hormander theorem, it is first shown that the geometry of the unperturbed limit cycle, in particular its isochrons, i.e., the leaves of the stable manifold of the limit cycle generalizing the notion of phase, is essential to understand the effect of the noise and the phenomenon of phase diffusion. In addition, it is shown that the RP spectrum has a spectral gap, even at the bifurcation point, and that correlations decay exponentially fast. Explicit small-noise expansions of the RP eigenvalues and eigenfunctions are then obtained, away from the bifurcation point, based on the knowledge of the linearized deterministic dynamics and the characteristics of the noise. These formulas allow one to understand how the interaction of the noise with the deterministic dynamics affect the decay of correlations. Numerical results complement the study of the RP spectrum at the bifurcation point, revealing useful scaling laws. The analysis of the Markov semigroup for stochastic bifurcations is thus promising in providing a complementary approach to the more geometric random dynamical system (RDS) approach. This approach is not limited to low-dimensional systems and the reduction method presented in Chekroun et al. (2020) is applied to a stochastic model relevant to climate dynamics in the third part of this contribution (Tantet et al. in J Stat Phys. https://doi.org/10.1007/s10955-019-02444-8 , 2019).

Quantum state reduction: Generalized bipartitions from algebras of observables

Reduced density matrices are a powerful tool in the analysis of entanglement structure, approximate or coarse-grained dynamics, decoherence, and the emergence of classicality. It is straightforward to produce a reduced density matrix with the partial-trace map by ``tracing out'' part of the quantum state, but in many natural situations this reduction may not be achievable. We investigate the general problem of identifying how the quantum state reduces given a restriction on the observables. For example, in an experimental setting, the set of observables that can actually be measured is usually modest (compared to the set of all possible observables) and their resolution is limited. In such situations, the appropriate state-reduction map can be defined via a generalized bipartition, which is associated with the structure of irreducible representations of the algebra generated by the restricted set of observables. One of our main technical results is a general, not inherently numeric, algorithm for finding irreducible representations of matrix algebras. We demonstrate the viability of this approach with two examples of limited--resolution observables. The definition of quantum state reductions can also be extended beyond algebras of observables. To accomplish this task we introduce a more flexible notion of bipartition, the partial bipartition, which describes coarse-grainings preserving information about a limited set (not necessarily algebra) of observables. We describe a variational method to choose the coarse-grainings most compatible with a specified Hamiltonian, which exhibit emergent classicality in the reduced state space. We apply this construction to the concrete example of the 1-D Ising model. Our results have relevance for quantum information, bulk reconstruction in holography, and quantum gravity.

AUV 3D Path Planning Based on the Improved Hierarchical Deep Q Network

This study proposed the 3D path planning of an autonomous underwater vehicle (AUV) by using the hierarchical deep Q network (HDQN) combined with the prioritized experience replay. The path planning task was divided into three layers, which realized the dimensionality reduction of state space and solved the problem of dimension disaster. An artificial potential field was used to design the positive rewards of the algorithm to shorten the training time. According to the different requirements of the task, this study modified the rewards in the training process to obtain different paths. The path planning simulation and field tests were carried out. The results of the tests corroborated that the training time of the proposed method was shorter than that of the traditional method. The path obtained by simulation training was proved to be safe and effective.

A Lyapunov Approach for Time-Bounded Reachability of CTMCs and CTMDPs

Time-bounded reachability is a fundamental problem in model checking continuous-time Markov chains (CTMCs) and Markov decision processes (CTMDPs) for specifications in continuous stochastic logics. It can be computed by numerically solving a characteristic linear dynamical system, but the procedure is computationally expensive. We take a control-theoretic approach and propose a reduction technique that finds another dynamical system of lower dimension (number of variables), such that numerically solving the reduced dynamical system provides an approximation to the solution of the original system with guaranteed error bounds. Our technique generalizes lumpability (or probabilistic bisimulation) to a quantitative setting. Our main result is a Lyapunov function characterization of the difference in the trajectories of the two dynamics that depends on the initial mismatch and exponentially decreases over time. In particular, the Lyapunov function enables us to compute an error bound between the two dynamics as well as a convergence rate. Finally, we show that the search for the reduced dynamics can be computed in polynomial time using a Schur decomposition of the transition matrix. This enables us to efficiently solve the reduced dynamical system by computing the exponential of an upper-triangular matrix characterizing the reduced dynamics. For CTMDPs, we generalize our approach using piecewise quadratic Lyapunov functions for switched affine dynamical systems. We synthesize a policy for the CTMDP via its reduced-order switched system that guarantees that the time-bounded reachability probability lies above a threshold. We provide error bounds that depend on the minimum dwell time of the policy. We demonstrate the technique on examples from queueing networks, for which lumpability does not produce any state space reduction, but our technique synthesizes policies using a reduced version of the model.<?vsp -2pt?>

Time Causal Processes in Time Petri Nets with Weak Semantics

In this paper, we present a method for state space reduction of dense-time Petri nets (TPNs) – an extension of Petri nets by adding a time interval to every transition for its firing. The time elapsing and memory operating policies define different semantics for TPNs. The decidability of many standard problems in the context of TPNs depends on the choice of their semantics. The state space of the TPN is infinite and non-discrete, in general, and, therefore, the analysis of its behavior is rather complicated. To cope with the problem, we elaborate a state space discretization technique and develop a partial order semantics for TPNs equipped with weak time elapsing and intermediate memory policies.

Variational Approach to Closure of Nonlinear Dynamical Systems: Autonomous Case

A general, variational approach to derive low-order reduced systems for nonlinear systems subject to an autonomous forcing, is introduced. The approach is based on the concept of optimal parameterizing manifold (PM) that substitutes the more classical notion of slow manifold or invariant manifold when breakdown of slaving occurs. An optimal PM provides the manifold that describes the average motion of the neglected scales as a function of the resolved scales and allows, in principle, for determining the best vector field of the reduced state space that describes e.g. the dynamics' slow motion. The underlying optimal parameterizations are approximated by dynamically-based formulas derived analytically from the original equations. These formulas are contingent upon the determination of only a few (scalar) parameters obtained from minimization of cost functionals, depending on training dataset collected from direct numerical simulation. In practice, a training period of length comparable to a characteristic recurrence or decorrelation time of the dynamics, is sufficient for the efficient derivation of optimized parameterizations. Applications to the closure of low-order models of Atmospheric Primitive Equations and Rayleigh-B\'enard convection are then discussed. The approach is finally illustrated --- in the context of the Kuramoto-Sivashinsky turbulence --- as providing efficient closures without slaving for a cutoff scale $k_c$ placed within the inertial range and the reduced state space is just spanned by the unstable modes, without inclusion of any stable modes whatsoever. The underlying optimal PMs obtained by our variational approach are far from slaving and allow for remedying the excessive backscatter transfer of energy to the low modes encountered by classical invariant manifold approximations in their standard forms when the latter are used at this cutoff wavelength.

Ruelle–Pollicott Resonances of Stochastic Systems in Reduced State Space. Part III: Application to the Cane–Zebiak Model of the El Niño–Southern Oscillation

The response of a low-frequency mode of climate variability, El Nino–Southern Oscillation, to stochastic forcing is studied in a high-dimensional model of intermediate complexity, the fully-coupled Cane–Zebiak model (Zebiak and Cane 1987), from the spectral analysis of Markov operators governing the decay of correlations and resonances in the power spectrum. Noise-induced oscillations excited before a supercritical Hopf bifurcation are examined by means of complex resonances, the reduced Ruelle–Pollicott (RP) resonances, via a numerical application of the reduction approach of the first part of this contribution (Chekroun et al. 2019) to model simulations. The oscillations manifest themselves as peaks in the power spectrum which are associated with RP resonances organized along parabolas, as the bifurcation is neared. These resonances and the associated eigenvectors are furthermore well described by the small-noise expansion formulas obtained by Gaspard (2002) and made explicit in the second part of this contribution (Tantet et al. 2019). Beyond the bifurcation, the spectral gap between the imaginary axis and the real part of the leading resonances quantifies the diffusion of phase of the noise-induced oscillations and can be computed from the linearization of the model and from the diffusion matrix of the noise. In this model, the phase diffusion coefficient thus gives a measure of the predictability of oscillatory events representing ENSO. ENSO events being known to be locked to the seasonal cycle, these results should be extended to the non-autonomous case. More generally, the reduction approach theorized in Chekroun et al. (2019), complemented by our understanding of the spectral properties of reference systems such as the stochastic Hopf bifurcation, provides a promising methodology for the analysis of low-frequency variability in high-dimensional stochastic systems.

Infrastructure maintenance and replacement optimization under multiple uncertainties and managerial flexibility

Infrastructure maintenance and replacement decisions are subject to uncertainties such as regular asset degradation, structural failure, and price uncertainty. In the engineering domain, Markov Decision Processes (MDPs) typically focus on uncertainties regarding asset degradation and structural failure. While the literature in the engineering domain stresses the importance of addressing price uncertainties, it does not substantiate the observations of such uncertainties through optimization modeling. By contrast, real option analyses (ROAs) that originate from the financial domain address price uncertainties but generally disregard asset degradation and structural failure. Accordingly, this piece of current research brings both domains closer together and proposes an optimization approach that incorporates the flexibility to choose between multiple successive intervention strategies, regular asset degradation, structural failure and multiple price uncertainties. A practical result of the current research is a realistic approach to optimization modeling in which state space reduction is achieved by combining prices into portfolios. The current research obtains transition probabilities from existing price data. This approach is demonstrated using a case study of a water authority in the Netherlands and confirms the premise that price fluctuations may influence short-term maintenance and replacement decisions.

Trace aware random testing for distributed systems

Distributed and concurrent applications often have subtle bugs that only get exposed under specific schedules. While these schedules may be found by systematic model checking techniques, in practice, model checkers do not scale to large systems. On the other hand, naive random exploration techniques often require a very large number of runs to find the specific interactions needed to expose a bug. In recent years, several random testing algorithms have been proposed that, on the one hand, exploit state-space reduction strategies from model checking and, on the other, provide guarantees on the probability of hitting bugs of certain kinds. These existing techniques exploit two orthogonal strategies to reduce the state space: partial-order reduction and bug depth. Testing algorithms based on partial order techniques, such as RAPOS or POS, ensure non-redundant exploration of independent interleavings among system events by imposing an equivalence relation on schedules and ideally exploring only one schedule from each equivalence class. Techniques based on bug depth, such as PCT, exploit the empirical observation that many bugs are exposed by the clever scheduling of a small number of key events. They bias the sample space of schedules to only cover all executions of small depth, rather than the much larger space of all schedules. At this point, there is no random testing algorithm that combines the power of both approaches. In this paper, we provide such an algorithm. Our algorithm, trace-aware PCT (taPCTCP), extends and unifies several different algorithms in the random testing literature. It samples the space of low-depth executions by constructing a schedule online, while taking dependencies among events into account. Moreover, the algorithm comes with a theoretical guarantee on the probability of sampling a trace of low depth---the probability grows exponentially with the depth but only polynomially with the number of racy events explored. We further show that the guarantee is optimal among a large class of techniques. We empirically compare our algorithm with several state-of-the-art random testing approaches for concurrent software on two large-scale distributed systems, Zookeeper and Cassandra, and show that our approach is effective in uncovering subtle bugs and usually outperforms related random testing algorithms.

Hierarchical Stochastic Models for Performance, Availability, and Power Consumption Analysis of IaaS Clouds

Infrastructure as a Service (IaaS) is one of the most significant and fastest growing fields in cloud computing. To efficiently use the resources of an IaaS cloud, several important factors such as performance, availability, and power consumption need to be considered and evaluated carefully. Evaluation of these metrics is essential for cost-benefit prediction and quantification of different strategies which can be applied to cloud management. In this paper, analytical models based on Stochastic Reward Nets (SRNs) are proposed to model and evaluate an IaaS cloud system at different levels. To achieve this, an SRN is initially presented to model a group of physical machines which are controlled by a management layer. Afterwards, the SRN models presented for the groups of physical machines in the first stage are combined to capture a monolithic model representing an entire IaaS cloud. Since the monolithic model does not scale well for large cloud systems, two approximate SRN models using folding and fixed-point iteration techniques are proposed to evaluate the performance, availability, and power consumption of the IaaS cloud. The existence of a solution for the fixed-point approximate model is proved using Brouwer's fixed-point theorem. A validation of the proposed monolithic and approximate models against both an ad-hoc discrete-event simulator developed in Java and the CloudSim framework is presented. The analytic-numeric results obtained from applying the proposed models to sample cloud systems show that the errors introduced by approximate models are insignificant while an improvement of several orders of magnitude in the state space reduction of the monolithic model is obtained.

Finite-Horizon Optimal Investment with Transaction Costs: Construction of the Optimal Strategies

We consider the problem of utility maximization in a Black-Scholes market in the presence of proportional transaction costs. While it is well-known that the value function of this problem is the unique viscosity solution of the corresponding Hamilton-Jacobi-Bellman (HJB) equation (cf. Belak et al. (2013)), and that the HJB equation admits a classical solution on a reduced state space (cf. Dai and Yi (2009)), it has been an open problem to construct the optimal strategies. We provide a simple method to prove the existence of the candidate optimal strategies, verify their optimality, study the regularity in detail and show that value function coincides with the classical solution of the HJB on the reduced state space.