In this work, a new band yarn woven (BYW) fabric has been developed to improve the sound absorption of kapok fibrous materials. BYW fabric materials were prepared by the interweaving of the normal hollow polyester yarn with new band yarn produced by wrapping the super thin nonwoven around a twisted fiber web. This soft, thin, lightweight fabric is promising to get the door of a wide variety of agro-sourced kapok fiber for sound absorption applications. The sound absorption of BYW fabric and the normal woven fabric have been compared, and the effects of fiber type, kapok fiber content, and airflow resistivity on sound absorption of BYW fabric materials have also been investigated. The results show that BYW fabric exhibited better sound absorption than the normal fabric, and it can be further enhanced when the band yarn was prepared from kapok fiber instead of hollow polyester or cotton fiber. The sound absorption can be improved with the increasing content of kapok fiber. Furthermore, a new sound absorption empirical model based on the Delany-Bazley model was developed. Calculated sound absorption coefficients obtained from the new model agreed well with measured results. Kapok fiber, exhibits higher hollowness and smaller fiber diameter than common natural fibers, and it presents promising prospect to reduce environmental noise pollution.