For a massive scalar field with general curvature coupling we evaluate the Wightman function in the geometry of two parallel branes perpendicular to the AdS boundary. On the separate branes, the field operator is constrained by Robin boundary conditions, in general, with different coefficients. In the region between the branes their contribution to the Wightman function is explicitly separated. By using this decomposition, the brane-induced effects on the vacuum expectation values (VEVs) for the field squared and energy-momentum tensor are investigated. The behavior of those expectation values is studied in various asymptotic regions of the parameters. The vacuum energy-momentum tensor in addition to the diagonal components has a nonzero off-diagonal stress. Depending on the boundary conditions and also on the distance from the branes, the vacuum energy density can be either positive or negative. The Casimir forces acting on the branes have two components. The first one corresponds to the standard normal force and the second one is parallel to the branes and presents the vacuum shear force. Unlike to the problem of parallel plates in the Minkowski bulk, the normal Casimir forces acting on separate branes differ if the boundary conditions on the branes are different. They can be either repulsive or attractive. In a similar way, depending on the coefficients in the boundary conditions, the shear force is directed toward or from the AdS boundary. The separate components may also change their signs as functions of the interbrane separation. At large proper separations between the branes, compared to the AdS curvature radius, both the components of the Casimir forces exhibit a power-law decay. For a massive scalar field this behavior is in contrast to that for the Minkowski bulk, where the decrease is exponential.