A pump coupled to a conserved density generates long-range modulations, resulting from the non-equilibrium nature of the dynamics. We study how these modulations are modified at the critical point where the system exhibits intrinsic long-range correlations. To do so, we consider a pump in a diffusive fluid, which is known to generate a density profile in the form of an electric dipole potential and a current in the form of a dipolar field above the critical point. We demonstrate that while the current retains its form at the critical point, the density profile changes drastically. At criticality, in d<4 dimensions, the deviation of the density from the average is given by sgn[cos(θ)]|cos(θ)/r^{(d-1)}|^{1/δ} at large distance r from the pump and angle θ with respect to the pump's orientation. At short distances, there is a crossover to a cos(θ)/r^{d-3+η} profile. Here δ and η are Ising critical exponents. The effect of the local pump on the domain wall structure below the critical point is also considered.