The understanding of roads and lanes incorporates identifying the level of the road, the position and count of lanes, and ending, splitting, and merging roads and lanes in highway, rural, and urban scenarios. Even though a large amount of progress has been made recently, this kind of understanding is ahead of the accomplishments of the present perceptual methods. Nowadays, 3D lane detection has become the trending research in autonomous vehicles, which shows an exact estimation of the 3D position of the drivable lanes. This work mainly aims at proposing a new technique with Phase I (road or non-road classification) and Phase II (lane or non-lane classification) with 3D images. Phase I: Initially, the features, such as the proposed local texton XOR pattern (LTXOR), local Gabor binary pattern histogram sequence (LGBPHS), and median ternary pattern (MTP), are derived. These features are subjected to the bidirectional gated recurrent unit (BI-GRU) that detects whether the object is road or non-road. Phase II: Similar features in Phase I are further classified using the optimized BI-GRU, where the weights are chosen optimally via self-improved honey badger optimization (SI-HBO). As a result, the system can be identified, and whether it is lane-related or not. Particularly, the proposed BI-GRU + SI-HBO obtained a higher precision of 0.946 for db 1. Furthermore, the best-case accuracy for the BI-GRU + SI-HBO was 0.928, which was better compared with honey badger optimization. Finally, the development of SI-HBO was proven to be better than the others.