Segregation of the solidification structure of an immiscible alloy is related to the convection induced by the density differences of the components under gravity during solidification. Here, the solidification of the Al–Bi alloy in both opposing and parallel directions to the gravity force was investigated in situ using radiography and optical flow techniques. A quantitative analysis was performed on the flow, solute, and constitutional undercooling fields at the interface. During upward solidification, a flat solute-enriched layer was observed, with the melt flowing toward the center from both sides of the interface. During downward solidification, the solute-enriched layer was deformed, and solute plumes were observed, with the melt flowing toward the sides from the center of the interface. A high solute concentration gradient enhances the local flow and solute transport. The elongation of the plumes was attributed not only to the convective diffusion of the solute but also to the dispersion of droplet clusters. The forces acting on the droplets were calculated, and it was found that the resultant force on the droplets increased the area of the solute-enriched layer. Convection within the bulk melt and the motion of the droplets have synergistic effects on the flow pattern and solute distribution. Distinct microstructures were observed in opposite directions of solidification owing to the differences in constitutional undercooling and flow instability. This work offers insights into the quantitative measurement and understanding of the multi-physics fields during the solidification process.