A causative agent of Alzheimer's disease (AD) is a short amphipathic peptide called amyloid beta (Aβ). Aβ monomers undergo structural changes leading to their oligomerization or fibrillization. The monomers as well as all aggregated forms of Aβ, i.e., oligomers, and fibrils, can bind to biological membranes, thereby modulating membrane mechanical properties. It is also known that some isoforms of the large-conductance calcium-activated potassium (BKCa) channel, including the mitochondrial BKCa (mitoBKCa) channel, respond to mechanical changes in the membrane. Here, using the patch-clamp technique, we investigated the impact of full-length Aβ (Aβ1–42) and its fragment, Aβ25–35, on the activity of mitoBKCa channels. We found that all forms of Aβ inhibited the activity of the mitoBKCa channel in a concentration-dependent manner. Since monomers, oligomers, and fibrils of Aβ exhibit different molecular characteristics and structures, we hypothesized that the inhibition was not due to direct peptide-protein interactions but rather to membrane-binding of the Aβ peptides. Our findings supported this hypothesis by showing that Aβ peptides block mitoBKCa channels irrespective of the side of the membrane to which they are applied. In addition, we found that the enantiomeric peptide, D-Aβ1–42, demonstrated similar inhibitory activity towards mitoBKCa channels. As a result, we proposed a general model in which all Aβ forms i.e., monomers, oligomers, and amyloid fibrils, contribute to the progression of AD by exerting a modulatory effect on mechanosensitive membrane components.