In manual welding process, skilled welders can ensure the weld quality through compensating for deviation observed from the weld pool surface. In this paper a three dimensional vision sensing system was used to mimic the human vision system to observe the three-dimensional weld pool surface in pipe GTAW process. Novel characteristic parameters containing information about the penetration state specified by its back-side weld pool width and height were proposed based on the reconstructed three dimensional weld pool surfaces. Then, variation in characteristic parameters and their relationships with the back-side parameters were studied through experiments under different welding conditions. Direct measurement of penetration is not preferred in a manufacturing site, soft-sensing method was thus proposed as an alternative to obtain it in real time due to established soft-sensing model and auxiliary variables which can be sensed in real time. In order to obtain the penetration status in real time conveniently, back-propagation neural network, principle component analysis based back-propagation neural network and global best adaptive mutation particle swarm optimization based back-propagation neural network models were established to estimate the penetration based on the proposed characteristic parameters. It was found that the top-side characteristic parameters proposed can reflect the back-side weld pool parameters accurately and the models are capable of predicting the penetration status in real time by observing the three-dimensional weld pool surface.