The ultimate boundedness and stability of highly nonlinear neutral stochastic delay differential equations (NSDDEs) are investigated in this paper. Different from many previous works, the highly nonlinear NSDDEs with semi-Markov switching signals are considered for the first time in this paper. Meanwhile, the time delay function in this paper is only required to meet much more relaxed restrictions, which can invalidate plentiful methods with requirements on its derivatives for the stability of NSDDEs. To overcome this difficulty, several novel techniques to tackle NSDDEs with such a delay are developed. Furthermore, a crucial property on the ergodic semi-Markov process is established, which plays a key role in the proof on the existence and boundedness of global solution. The generalized Khasminskii-type theorems are established for the existence and uniqueness of global solution by applying new methods and this property, and the criteria for boundedness and stability are also provided. In particular, feasible measures are provided to deal with the neutral term in our stability criteria. Finally, the effectiveness is verified by a numerical example.