Market intelligence, which collects and analyzes market trends and competitive landscape, is crucial for business success in the market. In particular, defining market scope is important because the market analysis outcomes vary depending on where the market boundaries are. However, this has been challenging due to the difficulty in scoping a number of similar products into the target market appropriately. To address this challenge, this study devised a novel method for acquiring reliable market intelligence utilizing the Sentence-RoBERTa (SRoBERTa) model, a pre-trained language model (PLM) specialized in computing sentence similarity. The SRoBERTa model was fine-tuned with 26,248,771 data on product relations and built a product-specific SRoBERTa model (named Pro-SRoBERTa) for effectively identifying the products that fall within the market scope. It outperformed the untrained model by up to 82.8 % on the hierarchical product categorization task. Finally, the Pro-SRoBERTa model was applied to determine the scope of the market followed by obtaining diverse market intelligence. The analysis module was successfully implemented in the market information system using a large-scale, real-world dataset.
Read full abstract