In this paper, we study the problem of whether the best approximation to any x in a real Hilbert space X from the closed convex set K≔C∩D can be characterized by the best approximation to a perturbation x−l of x from the set C for some l in a certain cone in X. The set C is a closed convex subset of X and D≔{x∈X:gj(x)≤0,∀j=1,2,…,m}, where the functions gj:X⟶R(j=1,2,…,m) are continuously Fréchet differentiable that are not necessarily convex. We show under suitable conditions that this “perturbation property” is characterized by the strong conical hull intersection property of C and D at the point x0∈K. We prove this by first establishing a dual cone characterization of a nearly convex set. Our result shows that the convex geometry of K is critical for the characterization rather than the representation of D by convex inequalities, which is commonly assumed for the problems of best approximation from a convex set. In the special case where the set D is convex, we show that the Lagrange multiplier characterization of best approximation holds under the standard Slater’s constraint qualification together with a non-degeneracy condition. The lack of representation of D by convex inequalities is supplemented by the non-degeneracy condition, but the characterization, even in this special case, allows applications to problems with quasi-convex functions gj,j=1,2,…,m, as they guarantee the convexity of D. Simple numerical examples illustrate the nature of our assumptions.