A new efficient and high-order accurate sharp-interface method for solving the Poisson equation on irregular domains and non-uniform meshes is presented. The approach is based on a combination of a fourth-order compact finite difference scheme and a multiscale multigrid (MSMG) method. The key aspect of the new method is that the regular compact finite difference stencil is modified at the irregular grid points near the immersed boundary to obtain a sharp interface solution while maintaining the formal fourth-order accuracy. The MSMG method is designed based on the standard multigrid V-cycle technique to solve the system of equations derived from the fourth-order compact discretization, while the corresponding multigrid relaxation, restriction and prolongation operators are properly constructed for non-uniform grids with immersed boundaries. The contribution of the present work is the design of a fourth-order-accurate Poisson solver whose accuracy, efficiency and computational cost are independent of the complexity of the geometry and the presence or not of an immersed boundary. The new method is demonstrated and validated for a number of problems including smooth and jagged boundaries. The test cases confirm that the new method is fourth-order accurate in the maximum norm whether an immersed boundary is present or not and on uniform or non-uniform meshes. Furthermore, the computational efficiency of the new method is demonstrated with regard to convergence rate and run time, which shows that the MSMG method is equally efficient for domains with immersed boundaries as for simple domains. The new compact difference method is evaluated by comparison with the standard fourth-order (non-compact) finite difference approximation in terms of both accuracy and computational efficiency. The new compact difference scheme yields indeed more accurate numerical solutions. The striking difference between the two schemes is the much higher computational efficiency: The number of V-cycles needed to reach the discretization error is significantly lower for the new compact method compared to the standard difference scheme. As a result, the new compact scheme requires only a fraction of the computer time for convergence in comparison to the standard fourth-order difference scheme.