A study on the detection of probable radionuclides and their activity concentrations in the raw material (phosphate rock), final product (fertilizer) and waste samples collected from all the phosphate fertilizer factories (total two factories—A Diammonium Phosphate, DAP and a Triple Super Phosphate, TSP factory) in Bangladesh were carried out. A total of seven types of samples (grossly divided into solid and liquid types); liquid waste, waste-mixed river water, normal river water, phosphate rock, phosphate fertilizer, solid waste and normal soil; were analyzed under this study. Raw material samples were collected only from the TSP fertilizer factory. Fertilizer, solid and liquid waste samples were collected from both the factories under study. Moreover, normal soil and natural surface water samples from the suitable areas adjacent to the factories were also collected for the comparison purpose. The samples were analyzed by gamma ray spectrometry technique using a Hyper-Pure Germanium (HPGe) detector of 40% relative efficiency. The analysis of the samples showed that only natural radionuclides such as 226Ra, 232Th and 40K were present in the samples and no traces of artificial radioactivity were found in any of the samples. 40K was found below the detection limit in some samples. The analysis of the samples in the case of TSP fertilizer factory showed that the average activity concentrations of 226Ra and 232Th and 40K in raw materials were 851.27 ± 7.10, 19.63 ± 6.57 and 54.06 ± 5.93 Bq/kg, respectively, in final product 211.90 ± 4.74, 42.48 ± 10.56 Bq/kg and ND (Not Detected), respectively, in solid waste 187.49 ± 4.88, 70.06 ± 11.76 and 289.27 ± 40.24 Bq/kg, respectively and in liquid waste 6.26 ± 0.63, 10.01 ± 1.39 Bq/L and ND, respectively. The measured results in the case of samples collected from DAP fertilizer factory showed that the average activity concentrations of 226Ra and 232Th and 40K in the final product were 17.31 ± 3.92, 69.74 ± 9.88 and 48.46 ± 17.22 Bq/kg, respectively, in solid waste 24.47 ± 4.15, 164.62 ± 11.08, 191.52 ± 33.74 Bq/kg, respectively and in liquid waste 3.59 ± 1.05, 37.08 ± 3.30 Bq/L and ND, respectively. Considering the stored raw materials and wastes the part of the ambient environment of the factory, radium equivalent activity, radiation hazard index and external annual effective dose to the workers and public due to these materials were also calculated and compared with world average values.