BackgroundAutism Spectrum Disorder (ASD), characterized by impaired sensory processing, has a wide range of clinical heterogeneity, which handicaps effective therapeutic interventions. Therefore, it is imperative to develop potential mechanisms for delineating clinically meaningful subgroups, so as to provide individualised medical treatment. In this study, an attempt is being made to differentiate the hyper-responsive subgroup from ASD by analysing the complexity pattern of Visual Evoked Potentials (VEPs), recorded from a group of 30 ASD participants, in the presence of vertical achromatic sinewave gratings at varying contrast conditions of low (5%), medium (50%) and high (90%). MethodThis study proposes a new diagnostic framework incorporating a novel signal decomposition method termed as Modified Variational Mode Extraction (MVME) and a multiscale entropy approach. MVME segments the signal into five constituent modes with less spectral overlap in lower frequencies. Refined Composite Multiscale Fluctuation-based Dispersion entropy (RCMFDE) is extracted from these constituent modes, thereby facilitating the identification of hyper-responsive subgroup in ASD. ResultsWhen tested on both simulated and real VEPs, MVME displays appreciable performance in terms of root mean square error and minimal spectral overlap in the lower frequencies, in comparison with the other state-of-the-art techniques. Relative Complexity analysis with RCMFDE exhibits a rising trend in 43%–50% of ASD in modes 1, 2, 3 and 4. ConclusionThe proposed MVME-RCMFDE approach is efficient in discriminating the hyper-responsive subgroup in ASD in multiple modes namely mode 1, 2, 3 and 4, which correspond to delta, theta, alpha and beta frequency bands of brain signals.