A parallel direct-forcing fictitious domain method is applied in fully-resolved numerical simulations of particle-laden turbulent flows in a square duct. The effects of finite-size heavy particles on the mean secondary flow, the mean streamwise velocity, the root-mean-square velocity fluctuation, and the particle concentration distribution are investigated at the friction Reynolds number of 150, the particle volume fraction of 2.36%, the particle diameter of 0.1 duct width, and the Shields number ranging from 1.0 to 0.2. Our results show that the particle sedimentation breaks the up-down symmetry of the mean secondary vortices, and results in a stronger secondary-flow circulation which transports the fluids downward in the bulk center region and upward along the side walls at a low Shields number. This circulation has a significant impact on the distribution of the mean streamwise velocity, whose maximum value occurs in the lower half duct, unlike in the plane channel case. The flow resistance is increased and the turbulence intensity is reduced, as the Shields number is decreased. The particles accumulate preferentially at the face center of the bottom wall, due to the effect of the mean secondary flow. It is observed that the collision model has an important effect on the results, but does not change the results qualitatively.