The assessment of wear anomalies using vision-based techniques is crucial for automated inspections of machine health conditions. Given the diversity of worn anomalies and the time-consuming nature of labeling, recent advancements in unsupervised anomaly detection have focused on comparing extracted features with those of normal images. However, this approach faces significant challenges in constructing a comprehensive comparison sample database and extracting distinctive features from worn surfaces with varying degrees of severity. To overcome these limitations, we introduce a fully unsupervised anomaly assessment method that detects anomalies solely based on single-bearing images, without the need for unworn samples. This approach eliminates the requirement for a comparison sample database by constructing an implicit feature database from the single-bearing surfaces using pre-trained convolutional neural networks and dimensionality reduction strategies. Additionally, to address the limited differentiability of wear characteristics, we develop a feature refinement strategy that incorporates multi-representation learning and topographical distribution characteristics of worn surfaces. By integrating the statistics of the refined feature database with uncertainty distribution-based normal feature weights, we construct an anomaly distribution map. This map enables us to assess the anomaly degree of a single-bearing effectively. To validate our method, we conducted experiments using real aero-engine bearings. The results demonstrate that our proposed approach can accurately assess anomaly degrees without pre-prepared comparison samples, achieving a detection performance of 90%, which fulfills the requirements for wear anomaly assessment.