In the development of quantum technologies, a reliable means for characterizing quantum devices, be it a measurement device, a state-preparation device, or a transformation device, is crucial. However, the conventional approach based on, for example, quantum state tomography or process tomography relies on assumptions that are often not necessarily justifiable in a realistic experimental setting. Although the device-independent (DI) approach to this problem bypasses the shortcomings above by making only minimal, justifiable assumptions, most of the theoretical proposals to date only work in the idealized setting where independent and identically distributed (i.i.d.) trials are assumed. Here, we provide a versatile solution for rigorous device-independent certification that does not rely on the i.i.d. assumption. Specifically, we describe how the prediction-based ratio (PBR) protocol and martingale-based protocol developed for hypothesis testing can be applied in the present context to achieve a device-independent certification of desirable properties with confidence interval (CI). To illustrate the versatility of these methods, we demonstrate how we can use them to certify—with finite data—the underlying negativity, Hilbert space dimension, entanglement depth, and fidelity to some target pure state. In particular, we provide examples showing how the amount of certifiable negativity and fidelity scales with the number of trials and how many experimental trials one needs to certify a qutrit state space or the presence of genuine tripartite entanglement. Overall, we have found that the PBR protocol and the martingale-based protocol often offer similar performance, even though the latter does have to presuppose any witness (Bell-like inequality). In contrast, our findings also show that the performance of the martingale-based protocol may be severely affected by one’s choice of Bell-like inequality. Intriguingly, a Bell function useful for self-testing does not necessarily give the optimal confidence-gain rate for certifying the fidelity to the corresponding target state.