Derailment of trains is not unusual all around the world, especially in developing countries, due to unidentified track or rolling stock faults that cause massive casualties each year. For this purpose, a proper condition monitoring system is essential to avoid accidents and heavy losses. Generally, the detection and classification of railway track surface faults in real-time requires massive computational processing and memory resources and is prone to a noisy environment. Therefore, in this paper, we present the development of a novel embedded system prototype for condition monitoring of railway track. The proposed prototype system works in real-time by acquiring railway track surface images and performing two tasks a) detect deformation (i.e., faults) like squats, shelling, and spalling using the contour feature algorithm and b) the vibration signature on that faulty spot by synchronizing acceleration and image data. A new illumination scheme is also proposed to avoid the sunlight reflection that badly affects the image acquisition process. The contour detection algorithm is applied here to detect the uneven shapes and discontinuities in the geometrical structure of the railway track surface, which ultimately detects unhealthy regions. It works by converting Red, Green, and Blue (RGB) images into binary images, which distinguishes the unhealthy regions by making them white color while the healthy regions in black color. We have used the multiprocessing technique to overcome the massive processing and memory issues. This embedded system is developed on Raspberry Pi by interfacing a vision camera, an accelerometer, a proximity sensor, and a Global Positioning System (GPS) sensors (i.e., multi-sensors). The developed embedded system prototype is tested in real-time onsite by installing it on a Railway Inspection Trolley (RIT), which runs at an average speed of 15 km/h. The functional verification of the proposed system is done successfully by detecting and recording the various railway track surface faults. An unhealthy frame’s onsite detection processing time was recorded at approximately 25.6 ms. The proposed system can synchronize the acceleration data on specific railway track deformation. The proposed novel embedded system may be beneficial for detecting faults to overcome the conventional manual railway track condition monitoring, which is still being practiced in various developing or underdeveloped countries.