Motion planning and optimization are the key and challenging problems for redundant manipulators operating in cluttered environments. This paper proposes a motion planning framework based on the heuristic solution that explores the optimal solutions for path planning and kinematic solutions by estimating the cost of target configurations via dynamic programming methods. A heuristic function model based on artificial neural networks (ANN) is constructed in the path planning structure and rapidly trained through the RRT* algorithm, leveraging value iteration concepts to search the state space. This structure can utilize previous experience to guide future exploration behavior with significant improvements in path quality and algorithm efficiency. The kinematic solving structure is unified with path planning by building a global energy optimal heuristic function. K-means is employed to determine the initial policy, avoid ineffective searches in non-critical spaces, and introduce gradient concepts to explore the optimal policy rapidly. The proposed method can obtain better energy optimization results while ensuring solving efficiency. The optimal joint angles of the manipulator are determined through collision detection and posture adjustment methods. Finally, the performance of the proposed framework is simulated and experimentally verified.