A new in situ radioactivity analysis method was developed to determine the fractional contributions of gamma-ray emitters in seawater and sediment from total measured counts. A semi-empirical formula to determine the fractional count contributions was derived using the variation characteristics of the gamma-ray attenuation rate and geometrical efficiency with the measurement points. The proposed method was employed to make in situ gamma-ray measurements using a CeBr3 detector for radioactivity analysis of seawater and sediment at a coastal area with a relatively high tidal range. The full energy peak efficiency of the detector at measurement site was obtained using the GEANT4 simulation code. The radioactivity concentration of 40K in seawater and sediment was determined using the proposed method and laboratory analysis with sampling. The MDA of the in situ measurement for 137Cs and 131I was also estimated assuming they were deposited on the sediment surface-layer. The validation of this method was demonstrated by comparing with the sampling analysis results.