A robotic grinding system requires a force-controlled grinding module to provide a consistent surface roughness and a robot arm to position the grinding module to reach a wide range of surface area on a workpiece. Existing pneumatic grinding modules are heavy and bulky and cannot provide very accurate force control. Articulated 6-axis robot arms are often used for positioning the grinding module, but they require a large accommodation space and have limited access to the surface of a spherical workpiece. This paper proposes a compact 3-axis grinding robot with no grinding surface limitations on spherical workpieces. The robot employs torque-controlled actuators so that a human operator can easily teach grinding paths to the robot. The proposed grinding module uses series elasticity to generate very low reflected inertia and friction. Hence, accurate grinding force control can be achieved. The grinding module also has a small size and low noise. Experimental results verify the high accuracy of grinding force control when compared with existing counterparts. Through an illustration of removing the parting line of a helmet hardshell, the grinding robot can effectively reduce the surface roughness of workpieces that are sensitive to the grinding force. It is expected that the proposed robot can be easily reconfigured to grind workpieces of different geometries.