In this paper, we present a new architecture for a voltage-controlled oscillator (VCO)-based electron spin resonance (ESR) detection for a future use in portable, point-of-care ESR spectrometers. The proposed architecture is centered around an application-specified integrated circuit (ASIC) containing a VCO-based ESR detector with two distinct tuning ports with largely different VCO gains to enable wide frequency sweeps and small-signal frequency modulations while keeping the requirements on the digital-to-analog converter driving the ports manageable. In addition, the proposed ASIC features a second VCO for an on-chip frequency downconversion via mixing. To allow for a precise derivation of the operating frequency from an external reference as it is required for quantitative ESR experiments, the two on-chip VCOs are embedded into an offset phase-locked loop. The proposed architecture is verified with ESR experiments on commonly used ESR standard samples (DPPH and BDPA). In these experiments, a spin sensitivity of $1.7\times 10^{9}~{\text {spins}/(\text {G}\sqrt {\text {Hz}})}$ has been achieved at $B_{0} = {450} {\text {mT}}$ , which is comparable to the state of the art, using a permanent magnet and low-cost signal processing on an field-programmable gate array. The presented proof-of-concept experiments clearly demonstrate the potential of the proposed VCO-based ESR detection system for future point-of-care applications.