Wireless-optical broadband access network (WOBAN) or fiber-wireless (FiWi) is a robust and cost-effective solution to provide high bandwidth network connectivity to mobile subscribers. Optical network unit (ONU) placement is a topological optimization problem that plays a vital role in effective deployment of FiWi network. Recently, some artificial intelligence-based approaches for positioning ONUs were proposed and generated promising results. However, the reported performance in the literature is far from the ideal state. This issue emphasizes the necessity of introducing new placement methods or improving the performance of existing methods. This paper presents a hybrid clustering algorithm for finding the optimal position of multiple ONUs. The proposed backtracking K-harmonic means (BKHM) is an integration of K-harmonic means (KHM) and a backtracking search algorithm (BSA). BKHM not only efficiently specifies the optimal positions of ONUs but also significantly decreases the implementation cost. To demonstrate the performance of the BKHM, we evaluate it on well-known placement benchmarks compared with BSA, KHM, whale optimization algorithm (WOA), enhanced backtracking search (EBS), harris hawks optimization (HHO), and chaotic local search-based levy flight distribution (CLF) algorithms. The experimental results and statistical analysis proved that the proposed BKHM algorithm outperformed its counterparts on most benchmarks in terms of cost optimization, convergence accuracy, and solution quality.