Underwater acoustic communication (UWAC) has a wide range of applications, including marine environment monitoring, disaster warning, seabed terrain exploration, and oil extraction. It plays an indispensable and increasingly important role in marine resource exploration and marine economic development. In current UWAC systems, the terminal nodes are usually powered by energy-limited batteries. Due to the harshness of the underwater environment, especially in the ocean environment, it is very costly and difficult, even impossible, to replace the batteries for the terminal nodes in UWACs, which results in the short lifetime and unreliability of the terminal nodes and the systems. In this paper, we present the application of a wireless powered communication network (WPCN) to the UWAC systems to provide an auxiliary and convenient energy supplement for solving the energy-limited problem of the terminal nodes, where the hybrid access point (H-AP) transfers energy to the terminal nodes in the downlink. In contrast, the terminal nodes use the harvested energy to transmit the information to the H-AP in the uplink. To evaluate the proposed WPCN-based UWAC systems, we investigate the performance of the average bit error rate (BER), outage probability, and achievable information rate for the systems in a frequency-selective sparse channel and non-white noise environment. We derive the closed-form expression for the probability density function (PDF) of the received signal-to-noise ratio (SNR). Based on this, we further derive novel closed-form expressions for the average BER and the outage probability of the systems. Numerical results confirm the validity of the proposed analytical results. It is shown that there exists an optimal signal frequency and time allocation factor for the systems to achieve optimal performance, and a larger optimal time allocation factor is preferred for a smaller hybrid access point (H-AP) transmit power or a larger transmission distance, while a smaller optimal signal frequency is required for a larger transmission distance.