Automated Google Scholar Crawling with a Web-Based Tool for Publication Data Management

Abstract

Here’s the revised abstract in English: The rapid growth of academic publications requires efficient tools for publication data extraction and management, especially from widely used platforms like Google Scholar. To address this need, an automated web-based tool was developed, designed to simplify the processes of data crawling, extraction, and publication data management, allowing researchers to handle large volumes of academic publications more effectively. The tool supports both simple and detailed crawling modes, enabling users to input multiple Google Scholar URLs and neatly organize the extracted data into CSV files. For multiple URLs, the data is compiled into a ZIP file containing separate CSV files for each source, ensuring organized and accessible publication data management. The tool was tested with various dataset sizes. When processing 41 entries, the simple mode completed extraction in 9.054 seconds, while the detailed mode took 71.898 seconds. For smaller datasets of 5 entries, the simple mode executed in 3.283 seconds, while the detailed mode required 11.908 seconds. These results indicate that the tool is efficient and performs well with both small and large datasets. The differences in execution time between the simple and detailed modes offer users flexibility in balancing speed and depth of data extraction according to their research needs. This web-based tool not only automates the data extraction process from Google Scholar but also enhances the organization and accessibility of publication data, making it an asset for researchers and institutions in managing publication data.