This paper introduces hybrid models designed to analyze daily and weekly bitcoin return spanning the periods from 18 July 2010 to 28 December 2023 for daily data, and from 18 July 2010 to 24 December 2023 for weekly data. Firstly, the fractal and chaotic structure of the selected variables was explored. Asymmetric Cantor set, Boundary of the Dragon curve, Julia set z2 −1, Boundary of the Lévy C curve, von Koch curve, and Brownian function (Wiener process) tests were applied. The R/S and Mandelbrot–Wallis tests confirmed long-term dependence and fractionality. The largest Lyapunov test, the Rosenstein, Collins and DeLuca, and Kantz methods of Lyapunov exponents, and the HCT and Shannon entropy tests tracked by the Kolmogorov–Sinai (KS) complexity test determined the evidence of chaos, entropy, and complexity. The BDS test of independence test approved nonlinearity, and the TeraesvirtaNW and WhiteNW tests, the Tsay test for nonlinearity, the LR test for threshold nonlinearity, and White’s test and Engle test confirmed nonlinearity and heteroskedasticity, in addition to fractionality and chaos. In the second stage, the standard ARFIMA method was applied, and its results were compared to the LieNLS and LieOLS methods. The results showed that, under conditions of chaos, entropy, and complexity, the ARFIMA method did not yield successful results. Both baseline models, LieNLS and LieOLS, are enhanced by integrating them with deep learning methods. The models, LieLSTMOLS and LieLSTMNLS, leverage manifold-based approaches, opting for matrix representations over traditional differential operator representations of Lie algebras were employed. The parameters and coefficients obtained from LieNLS and LieOLS, and the LieLSTMOLS and LieLSTMNLS methods were compared. And the forecasting capabilities of these hybrid models, particularly LieLSTMOLS and LieLSTMNLS, were compared with those of the main models. The in-sample and out-of-sample analyses demonstrated that the LieLSTMOLS and LieLSTMNLS methods outperform the others in terms of MAE and RMSE, thereby offering a more reliable means of assessing the selected data. Our study underscores the importance of employing the LieLSTM method for analyzing the dynamics of bitcoin. Our findings have significant implications for investors, traders, and policymakers.