The accurate modeling of boundary conditions is important in simulations of the discrete element method (DEM) and can affect the numerical results significantly. In conventional triaxial compression (CTC) tests, the specimens are wrapped by flexible membranes allowing to deform freely. To accurately model the boundary conditions of CTC, new flexible boundary algorithms for 2D and 3D DEM simulations are proposed. The new algorithms are computationally efficient and easy to implement. Moreover, both horizontal and vertical component of confining pressure are considered in the 2D and 3D algorithms, which can ensure that the directions of confining pressure are always perpendicular to the specimen surfaces. Furthermore, the boundaries are continuous and closed in the new algorithms, which can prevent the escape of particles from the specimens. The effectiveness of the proposed algorithms is validated by biaxial and triaxial simulations of granular materials. The results show that the algorithms allow the boundaries to deform non-uniformly on the premise of maintaining high control accuracy of confining pressure. Meanwhile, the influences of different lateral boundary conditions on the numerical results are discussed. It is indicated that the flexible boundary is more appropriate for the models with large strain or significant localization than rigid boundary.