Bananas are a worldwide cultivated crop and one of the main agricultural activities in Brazil. The banana orchards cultivated in the region of São Paulo State are under native areas of the Atlantic Forest biome. The Atlantic Forest has suffered agricultural and urban pressure for many years. Banana crops require soil management and superficial vegetation removal in the first cycles. We conducted a study aiming to understand the impact of long-standing banana cultivation in the Atlantic forest region. Soil samples in banana plantations (EBP) and forest remnants (FR) were collected from trenches with 0- to 100-cm layers. The soil bulk density in EBP until 30-cm depth was 12.76% higher than that in FR. Quantifications of macropores and micropores in FR reached values higher than those in EBP. The results showed that carbon stocks decreased from the top to the deeper layers. Thirty years after the conversion, the FR treatment accumulated 28.23% more carbon than EBP. Considering our results, it was evident that changes in physical and chemical properties reflected the negative impacts of the banana plantations, cropped through conventional management, when converted from forest even in regard to a remnant one. These findings, showed for the first time, lead us to understand the soil management of banana plantations, following conventional agriculture systems, as a potential carbon stock reducer and a factor resulting in the loss of soil quality in the region. Additionally, our data can be used by environmentalists and government policymakers to promote environmental sustainability.