Automated and efficient algorithms to perform segmentation of terrestrial LiDAR data is critical for exploitation of 3D point clouds, where the ultimate goal is CAD modeling of the segmented data. In this work, a novel segmentation technique is proposed, starting with octree decomposition to recursively divide the scene into octants or voxels, followed by a novel split and merge framework that uses graph theory and a series of connectivity analyses to intelligently merge components into larger connected components. The connectivity analysis, based on a combination of proximity, orientation, and curvature connectivity criteria, is designed for the segmentation of pipes, vessels, and walls from terrestrial LiDAR data of piping systems at industrial sites, such as oil refineries, chemical plants, and steel mills. The proposed segmentation method is exercised on two terrestrial LiDAR datasets of a steel mill and a chemical plant, demonstrating its ability to correctly reassemble and segregate features of interest.