Within the QCD factorization approach, we employ the new results of form factors of Bs→ϕ and calculate the branching fractions, CP asymmetries (CPAs) and polarization fractions of the decay modes Bs→ϕ(ρ0,ω) in both the standard model (SM) and the family nonuniversal Z′ model. We find that in SM the above observables are relate to two parameters, ρH and ϕH, which characterize the end-point divergence in the hard spectator-scattering amplitudes. When setting ρH=0.5 and ϕH∈[−180,180]°, the theoretical uncertainties are large. By combining the experimental data, our results can be used to constrain these two parameters. Supposing ρH=0, we study the effects of the Z′ boson on the concerned observables. With the available branching fraction of Bs→ϕρ0, the possible ranges of parameter spaces are obtained. Within the allowed parameter spaces, the branching fraction of Bs→ϕω can be enlarged remarkably. Furthermore, the new introduced weak phase plays important roles in significantly affecting the CPAs and polarization fractions, which are important observables for probing the effects of NP. If these decay modes were measured in the on-going LHC-b and Belle-II experiments in the future, the peculiar deviation from SM could provide a signal of the family nonuniversal Z′ model, which can also be used to constrain the mass of Z′ boson in turn. Published by the American Physical Society 2024