Recent studies have demonstrated that the band structure of a carbon nanotube (CNT) depends not only on its geometry but also on various factors such as atmosphere chemical composition and dielectric environment. Systematic studies of these effects require an efficient tool for an in situ investigation of a CNT band structure. In this work, we fabricate tunneling contacts to individual semiconducting carbon nanotubes through a thin layer of alumina and perform tunneling spectroscopy measurements. We use field-effect transistor configuration with four probe contacts (two tunnel and two ohmic) and bottom gates. Bandgap values extracted from tunneling measurements match the values estimated from the diameter value within the zone-folding approximation. We also observe the splitting of Van-Hove singularities of the density of states under an axial magnetic field.