In the current era, sensing the surroundings is salient development area in the field of autonomous self- driving cars. Aptness of sensing the surrounding helps to determine the travelable area on the road. Various types of road round boundary extraction techniques were available. However, these are not enough for making road boundaries in the unstructured format. In this study, a robust algorithm to detect the road boundaries for both structured and unstructured in different situations is proposed. The objects on the road are also detected in this method that can assist the driver to ensure safe driving. This method is capable of detecting the road and segment the travelable part of the road along with generation of bounding boxes around the detected road objects. The algorithm proposed in this paper is divided into three modules namely Canny module, YOLO v5 module and Hough Transform (HT) module performing three different tasks. The model is robust in boundary detection and road segmentation because the novel architecture that is used exhibits more efficient and reliable results for driver assistance with high accuracy. The NVIDIA RTX 5500 hardware has been used to build the road boundary detecting system. The proposed technique is trained and validated through different sets of images from a data set. The dataset used in making this model is in high-complex non-linear form contain the road images of various situations at different weather conditions. The experimental mean accuracy achieved in this model is 93.45% on both structured and unstructured road. The experiments show that the proposed algorithm can accurately detect the road boundary and segment the travelable road. Key Words: Road Boundary Detection, Road Segmentation, YOLO5, Hough Transform, Autonomous vehicle