We report on the formation of tubular structures and microneedles on the surface of monocrystalline silicon using ultrashort laser pulses. Highly deterministic surface processing is ensured by single-shot ablative modification of the sample surface using radially polarized doughnut-shaped laser pulses with duration of 70 fs. Under such conditions, well reproducible tubular structures are formed whose height is rising with increasing fluence, culminating by closing the structure on the top with formation of a microneedle with a cavity in its base. Upon multi-pulsed irradiation, the height of the needle structures can further increase as compared to those produced by single pulses and top part of the structure is flattened. However, at a certain number of pulses, melting and ablation cause collapsing the entire structure. The mechanisms responsible for creating the tubular and needle structures are discussed based on the careful analysis of the experimental observations. The generated silicon microtubes and needles can find applications in various fields, such as intracellular delivery of micromolecules, micro/electromechanical systems, photovoltaic devices and silicon-based photonics.