In order to maintain serviceability and reliability of concrete structures, it is essential to assess their condition as concrete structures deteriorate in time. Cracks develop in concrete due to several reasons such as severe loading, environmental effects, chemical effects etc. and cause durability loss in the structure which may also lead to loss of stability. In this research, crack detection is realized by machine learning and an infrared image. The effects of infrared images on crack detection are confirmed by random forest algorithm to select useful explanatory features. Selected features are applied to random forest algorithm and neural network algorithm. Effective filters are selected as a feature selection technique to improve the accuracy. Crack detection is also conducted by U-Net with RGB and infrared images, and the detection characteristics are compared to conventional methods. The performance of two conventional machine learning methods, random forest and neural network, are evaluated based on F1 score and false positives. Applying selected features improves the accuracy of the crack detection from an infrared image. False positives decreased due to monitoring conditions and camera specifications in the infrared image. The most effective image processing filter is the blur filter for each algorithm. Comparing algorithms for crack detection using selected features, different accuracy values are obtained. U-Net enables more accurate crack detection compared to conventional methods. The number of false positives is reduced compare to conventional method. In the detection results by three algorithms, infrared image affects the balance of false negatives and false positives.