A comprehensively comparative study of thiol-functionalized mesoporous silica and organic-inorganic hybrid monolithic column was reported, aiming at the separation and enrichment of inorganic arsenic. Thiol-functionalized mesoporous silica was synthesized based on the one-step co-condensation method. At the same time, a novel thiol-functionalized organic-inorganic hybrid monolithic column was synthesized using an unconventional ternary weak basic solvent system via one-step sol-gel process. The approach to prepare monolithic column through mild condition remarkably improved delamination phenomenon of thiol-functionalized hybrid monolithic column easily caused by conventional methods. As(III) can be selectively uptaken by the hybrid monolithic column and homemade syringe-based solid phase extraction device containing mesoporous silica through chelation in a wide range of pH, while As(V) cannot, and then the captured As(III) was eluted by 3% HNO3 (v/v) with 0.01 mol L−1 KIO3. Under the optimized conditions, the extraction recoveries were 91–102% and 93–103% for mesoporous silica and monolithic column, respectively. Although both two materials were ideal solid matrices for the removal and speciation analysis of As(III) in environmental waters, the monolith-based solid phase microextration protocol was more fast and reagent-saving than the other, which endued monolithic columns with more application prospects for trace elemental analysis and speciation. Even so, the exploration of mesoporous silica could efficiently pilot the synthesis of monolithic columns.