As chronic diseases and demographic changes alter the trends in world population, more pressure is put on health management to deal efficiently with a new scenario of longer life expectancy and chronic disabilities. Advances in telecommunication technology and miniaturization of sensors in combination with medical information technology provide a viable solution to reduce healthcare costs and deliver remote medical services through connected devices. Although remote consultation via video-conferencing has been well established, many of the chronic or long-term musculoskeletal conditions require pro-active management and therapy. There is a need to develop more advanced interactive telerehabilitation systems that support real-time remote delivery of physical therapy sessions into patients’ homes. In this paper, we introduce KinectRTC, an innovative framework that can be used for Kinect-based telerehabilitation with efficient real-time transmission of video, audio, depth, and skeletal data. By taking advantage of the Web Real-Time Communication (WebRTC) technology, the proposed framework is able to manage video and audio streams based on the state of the network and the available bandwidth to guarantee the real-time performance of the communication.