Unmanned aerial vehicle (UAV)-based communication system design has already attracted substantial interest due to UAV flexibility in deployment, cost effectiveness, and in-built line-of-sight air-to-ground channels. However, there is a persistent issue of security threats associated with the broadcast nature of UAV: physical layer security (PLS) can be introduced to enhance the secrecy performance in that regard. In the system proposed here, the combined effect of 3D fractional-order Liu chaotic system and 3D fractional-order Li chaotic system is introduced to enhance the PLS of the UAV-to-ground communication network. In addition, to reduce multi-user interference (MUI), efficient orthogonal variable spreading factor (OVSF) codes are integrated with a zero forcing (ZF) scheme. Moreover, to improve bit error rate (BER), a combined MUI signal discrimination and ZF signal detection scheme is introduced, strengthened by various channel coding techniques with multi-user beamforming weighting. The numerical outcomes establish the efficacy of the proposed system in terms of PLS and improvement of data rate with signal-to-interference-plus-noise ratio. This approach achieved an out-of-band emission reduction of 119 dB. Furthermore, an improved BER of <inline-formula> <tex-math notation="LaTeX">$1 \times 10^{-4}$ </tex-math></inline-formula> is achieved in 16-QAM for a signal-to-noise ratio of 6 dB.