A number of works have claimed detections of a turn-over in the spectral lag data for individual Gamma-Ray Bursts (GRBs), caused by an energy-dependent speed of light, which could be a possible manifestation of Lorentz invariance violation (LIV). Here, we stack the spectral lag data from a total of 37 GRBs (with a total of 91 measurements), to verify if the combined data is consistent with a unified model consisting of intrinsic astrophysical emission, along with another contribution due to LIV. We then carry out Bayesian model comparison to ascertain if this combined spectral lag data shows a preference for an energy-dependent speed of light, as compared to only an intrinsic astrophysical emission mechanism. We do not find a decisive evidence for such an energy-dependent speed of light for two different models of LIV. When we assume a constant intrinsic lag coupled with an unknown intrinsic scatter, we do not find any evidence for LIV. However, when we use GRB-dependent parameters to model the intrinsic emission, we get decisive evidence for LIV violation. We then carry out a search for LIV Standard Model Extension using this dataset as well as an independent search using a separate dataset consisting of rest-frame spectral lags. Finally, none of the models considered here with any of the aforementioned assumptions provide a good fit to the stacked spectral lag data, indicating that there is still missing Physics in the model for intrinsic spectral lags.