Context: Cloud-based services are increasingly central in financial technology, enabling scalable and efficient transactions. However, they also heighten vulnerability to fraud, challenging the security of online financial activities. Traditional fraud detection struggles against sophisticated tactics, highlighting the need for advanced, cloud-compatible solutions. Objectives: This study assesses machine learning (ML) algorithms' ability to detect fraud in cloud environments, focusing on Logistic Regression (LR), Decision Trees (DT), Random Forest (RF), Support Vector Machines (SVM), and XGBoost (XGB). It uses a comprehensive dataset to determine which ML model best identifies fraudulent transactions, aiming to optimize these models for accuracy, precision, and efficiency in real-time detection. Results: The XGBoost model outperformed others in fraud detection accuracy, with Random Forest and Decision Trees also showing high effectiveness. These models were particularly good at balancing precision and recall, minimizing false positives, and accurately identifying fraud in complex transaction patterns. Conclusion: ML, especially ensemble and boosting models like XGBoost and Random Forest, offers a strong approach to detecting transaction fraud in cloud-based financial systems. Their capacity to handle vast data volumes and adapt to new fraud patterns enhances cloud transaction security. Implication: The study provides a guide for financial and cloud services to implement sophisticated fraud detection systems. It emphasizes the importance of continual ML innovation to tackle digital finance fraud, suggesting that adopting these advanced ML models can significantly reduce transaction fraud risks, ensuring a secure, efficient, and trustworthy platform for users.