Increasing pressures surrounding efficiency and sustainability are key global drivers in dairy farm management strategies. However, for numerous resource-based, social, and economic reasons sustainable intensification strategies are herd-size dependent. In this study, we investigated the environmental impacts of Latvia’s dairy farms with different management practices. The herd size-dependent management groups varied from extensively managed small herds with 1–9 cows, extending to stepwise more intensively managed herds with 10–50, 51–100, 100–200, and over 200 milking cows. The aim is to compare the environmental impacts of different size-based production strategies on Latvia’s dairy farms. The results show that the gross greenhouse gas emissions differ by 29%: from 1.09 kg CO2 equivalents (CO2e) per kg of raw milk for the farms with 51–100 cows, down to 0.84 kg CO2e/kg milk for farms with more than 200 cows. However, the land use differs even more—the largest farms use 2.25 times less land per kg of milk than the smallest farms. Global warming potential, marine eutrophication, terrestrial acidification, and ecotoxicity were highest for the mid-sized farms. If current domestic, farm-based protein feeds were to be substituted with imported soy feed (one of the most popular high-protein feeds) the environmental impacts of Latvian dairy production would significantly increase, e.g., land use would increase by 18% and the global warming potential by 43%. Environmental policy approaches for steering the farms should consider the overall effects of operation size on environmental quality, in order to support the best practices for each farm type and steer systematic change in the country. The limitations of this study are linked to national data availability (e.g., national data on feed production, heifer breeding, differences among farms regards soil type, manure management, the proximity to marine or aquatic habitats) and methodological shortcomings (e.g., excluding emissions of carbon sequestration, the use of proxy allocation, and excluding social and biodiversity impacts in life-cycle assessment). Further research is needed to improve the data quality, the allocation method, and provide farm-size-specific information on outputs, heifer breeding, manure storage, and handling.