The variation of chemical and optical properties of sound and effected caries lesion areas produce changes of light reflected and emitted energy as function in wavelength after human teeth interact with laser beam. In this study, custom non-invasive laser-induced fluorescence technique with optical sensor, associated with correlation algorithm was employed for caries detection and ablation control. Tunable argon laser source of 488‐nm and 5 mw was employed for tooth excitation. While sound tooth demonstrated high fluorescence emission at 528 nm; decayed tooth demonstrated low fluoresce emission intensity. Novel digital signal processing algorithm using correlation technique was employed to recognize and localize the caries position. Sound tooth response was employed as a reference. Cross-correlation of emitted signal amplitude from scanned sound position to reference normal sound of tooth will produce high correlation intensity values (maximum values). In the mean time, caries area will develop low intensity values; consequently low correlation to reference will be obtained (low intensity, minimum vales). Threshold for correlation value was developed to convert the analog correlation signal to logic signal (1, 0). This novel approach can offer automated precise drilling using the same laser source after external frequency modulation at 10 Hz and power up to 500 mw to achieve precise caries removal.