A detector system for neutron spectroscopy and imaging was modeled in Monte Carlo N-Particle transport code (MCNP) to examine the possibility of incident neutron energy measurement (spectroscopy) from an incident direction (imaging) in a hand-held detector system. Experiments using a Pu–Be (plutonium beryllium) neutron source were also conducted. The system is composed of three separate scintillation materials; a hydrogenous neutron scintillation material (BC-501A) and Cs2LiYCl6:Ce (CLYC) separated by Lu2(1-x)Y2xSiO5(LYSO). Light production location and magnitude would be collected using multiple silicon-based photo-multiplier arrays, allowing for spectroscopy and imaging of incident neutrons. Modeled neutron interaction criteria are collected and cataloged through the particle tracking supplemental code (PTRAC) of MCNP 6.2 for analog interactions. Implementing classical neutron kinematics and capture-gated time-of-flight, simulations show that neutron energy is measurable in experiment and that there is position sensitivity to be realized in experiment with the proposed detection architecture. The modeled design was experimentally evaluated with a Pu–Be source, confirming that the spectroscopic capabilities of the detector are functional and closely match the modeled expectation. In this work, we direct our attention to neutron spectroscopy, but the imaging capabilities will be addressed in further work.