After total laryngectomy, the placement of a tracheoesophageal (TE) prosthesis offers the possibility to recover a new voice. However, the quality of the resulting TE speech is known to be degraded. To assess a patient's voice, current approaches rely either on quality-of-life questionnaires or on a perceptual evaluation carried out by speech therapists. These two methods exhibit the disadvantage of being both subjective and time-consuming. In this paper, we propose a dedicated scale, called A4S, for the objective Automatic Acoustic Assessment of Alaryngeal Speech. For this purpose, we first identify the artefacts existing in TE speech. These are linked to the periodicity, regularity, high-frequency noise and gargling noise/creakiness of the signal, as well as to the speaking rate. Specific acoustic features are proposed for the characterization of each artefact. A statistical study shows that TE speakers have a significantly worse voice compared to the control group, except for the speaking rate. Based on these advances, the A4S scale is proposed. This scale is made of five normalized dimensions, related to the five identified artefacts. A given patient's phonation can then be represented by a pentagon in a radar chart, which allows a fast and intuitive visualization of the strengths and flaws of the voice. A4S can then be seen as a useful tool for speech therapists to design tailored exercises specific to the patient's voice. In addition, we show the applicability of A4S for the follow-up of patients, as well as to study the impact of the type of surgery (open neck, robot and flap reconstruction) used for total laryngectomy and of a pre-surgical radiotherapy on various aspects of the TE voice.