Photons carry momentum, and thus their scattering not only modifies light propagation but at the same time induces forces on particles. Confining mobile scatterers and light in a closed volume thus generates a complex coupled nonlinear dynamics. As a striking example, one finds a phase transition from random order to a crystalline structure if particles within a resonator are illuminated by a sufficiently strong laser. This phase transition can be simply understood as a minimization of the optical potential energy of the particles in concurrence with a maximization of light scattering into the resonator. Here, we generalize the self-ordering dynamics to several illumination colors and cavity modes. In this enlarged model, particles adapt dynamically to current illumination conditions to ensure maximal simultaneous scattering of all frequencies into the resonator as a sort of self-optimizing light collection system with built-in memory. Such adaptive self-ordering dynamics in optical resonators could be implemented in a wide range of systems from cold atoms and molecules to mobile nanoparticles in solution. In the quantum regime, it enables exploration of uncharted regions of multiparticle phases, allowing simulation of Hopfield networks, associative memories, or generalized Hamiltonian mean field models.