Efficiency of the t-distribution stochastic neighbor embedding technique for detailed visualization and modeling interactions between agricultural soil quality indicators

Abstract

Dimensionality reduction is important for revealing important details that may be useful in decision-making. Although different dimensionality reduction methods have been applied in several soil-based studies, Kohonen self-organizing map neural network (KSOM-NN) has attracted significant attention from researchers because of the quality of data visualization and interpretation. However, there is a dearth of studies that compare KSOM-NN and other robust data reduction techniques such as the t -distribution stochastic neighbor embedding ( t -SNE) method to improve visualization and interpretation of the relationships between soil quality indicators in agricultural soil. This study compares the above-mentioned methods for characterizing soil quality indicators including particle size distribution, soil organic matter (SOM), cation exchange capacity (CEC), soil reaction (pH), electrical conductivity (EC), zinc (Zn), iron (Fe), manganese (Mn), potassium (K) and phosphorus (P) in agricultural dryland. There were strongly positive associations identified between some of the variables studied for example, clay/Fe (r = 0.95), clay/SOM (r = 0.79) and Mn/Zn (r = 0.90) based on the correlation matrix output. According to the KSOM-NN, the best map size was a 4 by 7 with Quantization error (QE) = 0.108, Topographic error (TE) = 0.875 and Kaski-Lagus error (K-LE) = 9.104. This map only yielded 2 main clusters. As for the t -SNE, applying various perplexity values (i.e. 5, 6, 7, 8, 9 and 10) enabled better visualization of the soil quality indicators as observed from the cluster formations than the KSOM-NN. Ultimately, the t -SNE was considered a better and promising method for assessing the interactions of soil quality indicators and has the potential to appropriately visualise as well as improve the interpretability of soil results by identifying essential features and similarities. The results of this study have good implications for agricultural soil management and decision-making. Moreover, the results are preliminary findings that may be used in the future to verify detailed aspects of soil biogeochemistry within the study area. • t -SNE outperformed KSOM-NN in soil properties dimensionality reduction and visualization. • The t -SNE method is rarely used for data reduction and visualization in soil-based research. • More detailed clusters of the soil quality indicators were observed while using the t -SNE compared to KSOM-NN. • The t -SNE is recommended for future applications in soil quality characterization research.