Online digital library sampling based on query related graph

Abstract

Purpose Digital library sampling is used to obtain a collection of random literature records from the backend database, which is a crucial issue for a variety of important purposes in many online digital library applications. Digital libraries can only be accessed through their query interfaces. The challenge is how to ensure the randomness of the sample via the autonomous query interface. Design/methodology/approach This paper presents an iterative and incremental approach to obtain samples through the query interface of a digital library. In the approach, a novel graph model, query-related graph, is proposed to transform the flat literature records into a graph structure, and samples are obtained iteratively by traveling the query-related graph. Besides query-related graph, the key components, query generation, termination condition and amending deviation, are also discussed in detail. Findings The extensive experiments over two real digital libraries, ISTIC and IEEE Xplore, show the proposed approach results in a better performance. First, the approach is very effective to obtain high-quality samples which are evaluated by the measure “sample deviation.” Second, the sampling process is very efficient by only submitting fewer random queries. Third, the approach is robust. Research limitations/implications This sampling approach is limited by the query interfaces on a web page. In rare cases (<3 per cent), this approach cannot access query interfaces by sophisticated techniques. Practical implications Digital library sampling is very useful for a variety of important purposes: subject distribution analysis, literature quality evaluation, digital library size estimation, source selection in digital library integration and content freshness evaluation. Social implications Myriads of online digital libraries can be accessed online. Digital library sampling is a useful way to understand digital libraries for many important applications. Originality/value Most of the attributes of a digital library query interface have infinite values, such as keyword attributes, which cannot be handled effectively by the existing sampling approaches.