The growth of digital and social media has led to an exponential increase in the availability of music and its lyrics. It has been observed that objectionable content is becoming a huge problem in the age of the digital era. This research aims to develop a system for automatically detecting lyrics that may contain offensive or controversial themes, such as hate speech, sexism, racism, and violence, by generating labeled data using weak supervision. In this study, we generated labels using heuristic rules using the Snorkel framework, and the performance of various machine learning classifiers, namely, Logistic Regression, Random Forest, and Support Vector Machine, were evaluated on label-generated data. The results show that DistilBert performs better than frequency-based methods (TF-IDF and CountVectorizer) by 3-5% in F1 score, and the results are even more impressive on SMOTE augmented dataset, where Logistic Regression and SVM on Distilbert vectorized data performed similarly and has exhibited 6-8% of F1 score gain over frequency based methods. In addition, it asserts that the proposed method can be used to detect offensive lyrics in other languages, given the same type of data used in this study. Our work will help music industry professionals and digital content monitoring systems for hate and offensive content to adopt the methodology and approaches used in our work.