ABSTRACT Beacon towers are an importantinfrastructure responsible for transmitting military information in ancienttimes. However, many of the beacons have disappeared due to natural erosion andman-made vandalism. Historical U2 aerial images provide heritage andgeographical information over the last century to research the beacon towersystem. However, the fragmented distribution of beacons and the greyscalecolouring of the aerial images make it difficult to manually identifysmall-sized beacons in a wide range of aerials. This study introduced deeplearning to automatically detect beacons in U2 images. Three improvements wereadded to the standard Fully ConvolutionalOne-Stage Object Detection (FCOS) network: 1) Thestructure of the Feature Pyramid Network (FPN) was adjusted to enhance thesmall objects feature at lower layers; 2) The standard convolutional kernel inbackbone network was replaced with DCNv2 to account for irregular towers; 3)NMS was replaced with Soft-NMS to improve the accuracy of the detection boxprediction. Our results demonstrate that more than 60% average precision (AP)can be obtained using our improved FCOS. After testing, the results showed thatthe three-part methodology can automatically detect most beacons in historicalU2 aerial images, reduce the manual miss rate, and improve efficiency. Theresults of the test were the first to successfully identify destroyed beacons,recreate the beacon route, and retrace the beacon siting strategy. Our methodhelps to speed up the efficiency of heritage excavation in historical aerialimages, and it may provide a convenient means of processing in otherarchitectural heritage restoration studies.