This work presents the in-house design, characterization, and development of an integrated PZT-based piezoelectric MEMS vibration sensor module and its usage in surface roughness prediction of fused silica substrate in a CNC polishing machine. The differential electrical configuration of the MEMS device ensures enhanced acceleration sensitivity to out-of-plane motion, two orders larger than in-plane sensitivity. The Thin-film Piezo on Silicon (TPoS) accelerometer with a PZT layer was fabricated using photolithography, frontal etching, and backside release. A fully integrated module comprising MEMS, amplifiers, and a microcontroller was designed and characterized in a shaker system. The designed module provides a sensitivity of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$8.12~mV/{g}$ </tex-math></inline-formula> and a resolution of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$5.8~m{g}/\sqrt {Hz}$ </tex-math></inline-formula> . The feed rate, compared to the other polishing parameters, shows the highest correlation with surface roughness. The module’s acceleration-time data were processed by Linear Discriminant Analysis (LDA) and given as input to the neural network together with the feed rate of the polishing spindle. Back Propagation Neural Network (BPNN) algorithm was used to train the machine learning model. The designed module is cost-effective compared to the commercial sensor. The key performance indicators of roughness prediction for the module and the commercial sensor were calculated, showing a Mean Absolute Percentage Error (MAPE) of 6.45% and 5.32%, respectively.