The performance of an underwater wireless optical communication channel (UWOC) is affected by the ambient noise in the water medium. However, no major studies have estimated the spectral bioluminescence upwelling radiance (BLu) numerically with the inputs of measured bioluminescence potential (BP) in oceanic conditions. This study presents a numerical model to estimate the BLu in open ocean and coastal waters and to investigate the UWOC channel performance in terms of the bit error rate (BER) in presence of bioluminescence using Monte Carlo numerical technique. The simulation results suggested that the BER performance degrades in the presence of bioluminescence and can be improved with a narrower receiver FOV. Further, the impact of bioluminescence on the BER performance of the UWOC system at different operational depths is investigated under typical system specifications (APD/PMT, FOV, and receiver aperture). The simulation results showed that BER at different operational depths is more sensitive in open ocean waters than in coastal turbid water with the bioluminescence signal. The results demonstrate that the PMT improves the UWOC channel performance better than the APD. Besides, these results revealed that the BP stimulation at multiple depths deteriorates the BER performance heavily in comparison to the BP stimulation at single depth in the real sea environment. The results presented in this study provide valuable insights and information for accurate design and development of the UWOC system for the enhanced bioluminescence background noise conditions.