Controlling mixed-mode fatigue crack growth using deep reinforcement learning

Abstract

Mechanical discontinuity embedded in a material determines the bulk mechanical, physical, and chemical properties. Under external forces, mechanical discontinuity undergo spatiotemporal propagation; thereby altering various properties of the material. This paper is a proof-of-concept development and deployment of a reinforcement learning framework, based on deep deterministic policy gradient, to precisely control both the direction and rate of the fatigue crack growth. The ability to control mechanical discontinuity in essence determines the key material properties. The desired control is relatively hard to achieve considering the large, continuous state and action spaces along with the exponential relationship between crack growth and stress cycle. The reinforcement-learning scheme is capable of learning an optimal and computational tractable control strategy. In the proposed approach, the reinforcement learning framework is integrated into an OpenAI-Gym-based environment that implements the mechanistic equations governing the fatigue crack growth. The learning agent does not explicitly know about the underlying physics, nonetheless, the learning agent can infer the control strategy by continuously interacting the numerical environment. The paper formulates an adaptive reward function involving reward shaping that can be generalized to similar control problems to improve the training efficiency. The reinforcement learning framework can successfully control the fatigue crack growth in a material despite the complexity of the propagation/growth pathway determined by multiple goal points. The paper provides the mathematical/physical basis of the reward function and the effect of neural network size and architecture and the state and action space that boosts the training speed while preserving the stability of the RL agents for the desired control problem.