This paper describes a prototype virtual reality (VR) system that has been developed to assist the rehabilitation of elderly with mild-to-moderate memory deficits. The use of virtual reality in rehabilitation is reviewed along with the clinical requirements to establish the framework for the proposed system. This framework is presented together with the use of the prototype system to perform a simple cooking task in a virtual kitchen. The evaluation results show that the completion time for the virtual task is dependent on the person's mobility and knowledge of computers. It was found that the mean completion time decreases significantly with more practices. Although the time taken for the completion of the virtual cooking task is longer than the average time needed to actually cook a package of instant noodle, the overall results give strong indication of the usability of the VR application for rehabilitation. The results are encouraging and show the potential of immersive virtual reality for memory rehabilitation.