Ball milling was used on synthesized Cr2AlC powders, and dense Cr2AlC bulk ceramics with almost pure phase were fabricated by hot pressing these ball milled Cr2AlC powders at 1300 °C for 2 h under 30 MPa. The phase compositions and the microstructures of powders and bulk ceramics were characterized. The mechanical properties and dislocation analysis of bulk ceramics were also investigated. Our results indicated that the grain size of powders became uniform and smaller after ball milling, which was also inherited in the bulk ceramics. Moreover, the mechanical properties of hot pressed Cr2AlC ceramics, including flexural strength, fracture toughness, and Vickers hardness, significantly increased by using ball milled powders treated at speed 200 rpm. The organization of hexagonal dislocation networks during hot pressing and reduction of grain size both had a positive effect on improving mechanical properties.