Ensemble control deals with the problem of using a finite number of control inputs to simultaneously steer a large population (in the limit, a continuum) of control systems. Dual to the ensemble control problem, ensemble estimation deals with the problem of using a finite number of measurement outputs to estimate the initial state of every individual system in the ensemble. We introduce in the paper a novel class of ensemble systems, termed distinguished ensemble systems, and establish sufficient conditions for controllability and observability of such systems. Every distinguished ensemble system has two key components, namely a set of distinguished control vector fields and a set of codistinguished observation functions. Roughly speaking, a set of vector fields is distinguished if it is closed (up to scaling) under Lie bracket, and moreover, every vector field in the set can be obtained by a Lie bracket of two vector fields in the same set. Similarly, a set of functions is codistinguished to a set of vector fields if the Lie derivatives of the functions along the given vector fields yield (up to scaling) the same set of functions. We demonstrate in the paper that the structure of a distinguished ensemble system can significantly simplify the analysis of ensemble controllability and observability. Moreover, such a structure can be used as a guiding principle for ensemble system design. We further address in the paper the problem about existence of a distinguished ensemble system for a given manifold. We provide an affirmative answer for the case where the manifold is a connected semi-simple Lie group. Specifically, we show that every such Lie group admits a set of distinguished vector fields, together with a set of codistinguished functions. The proof is constructive, leveraging the structure theory of semi-simple real Lie algebras and representation theory. Examples will be provided along the presentation of the paper illustrating key definitions and main results.