This paper describes an approach to improve the performance of the distributed audio-processing functions for audio surveillance systems. In order to increase portability, current distributed audio-processing uses the default capacities offered by the underlying scheduling facilities of the operating system. In this approach, a set of capacities are added to the distribution software that enable the reduction of the distributed processing time of audio frames at the server side by adding functions that utilize the underlying hardware resources including exclusive core reservation. By loosing some generality in the design of the distribution software, it is possible to increase performance and provide better isolation to selected audio tasks in the presence of other competing software tasks. The approach is designed and implemented as well as analyzed on general purpose computers with a server-client architecture using serial scheduling of the audio tasks and parallelizing the digital signal processing computations. The proposed solution is implemented and analyzed showing benefits in performance and robustness over single threaded audio processing. The resulting system is significantly more robust in the presence of other competing software tasks (noise). These results directly yield the possibility to manage more concurrent audio streams at the server side.