General-purpose Search Engines Research Articles

A Study of Publicly Available Resources Addressing Legal Data-Sharing Barriers: Systematic Assessment.

BackgroundUnited States data protection laws vary depending on the data type and its context. Data projects involving social determinants of health often concern different data protection laws, making them difficult to navigate.ObjectiveWe systematically aggregated and assessed useful online resources to help navigate the data-sharing landscape.MethodsWe included publicly available resources that discussed legal data-sharing issues with some health relevance and published between 2010 and 2019. We conducted an iterative search with a common string pattern using a general-purpose search engine that targeted 24 different sectors identified by Data Across Sectors for Health. We scored each online resource for its depth of legal and data-sharing discussions and value for addressing legal barriers.ResultsOut of 3710 total search hits, 2721 unique URLs were reviewed for scope, 322 received full-text review, and 154 were selected for final coding. Legal agreements, consent, and agency guidance were the most widely covered legal topics, with HIPAA (The Health Insurance Portability and Accountability Act), Family Educational Rights and Privacy Act (FERPA), Title 42 of the Code of Federal Regulations Part 2 being the top 3 federal laws discussed. Clinical health care was the most prominent sector with a mention in 73 resources.ConclusionsThis is the first systematic study of publicly available resources on legal data-sharing issues. We found existing gaps where resources covering certain laws or applications may be needed. The volume of resources we found is an indicator that real and perceived legal issues are a substantial barrier to efforts in leveraging data from different sectors to promote health.

Q4EDA: A Novel Strategy for Textual Information Retrieval Based on User Interactions with Visual Representations of Time Series

Knowing how to construct text-based Search Queries (SQs) for use in Search Engines (SEs) such as Google or Wikipedia has become a fundamental skill. Though much data are available through such SEs, most structured datasets live outside their scope. Visualization tools aid in this limitation, but no such tools come close to the sheer amount of information available through general-purpose SEs. To fill this gap, this paper presents Q4EDA, a novel framework that converts users’ visual selection queries executed on top of time series visual representations, providing valid and stable SQs to be used in general-purpose SEs and suggestions of related information. The usefulness of Q4EDA is presented and validated by users through an application linking a Gapminder’s line-chart replica with a SE populated with Wikipedia documents, showing how Q4EDA supports and enhances exploratory analysis of United Nations world indicators. Despite some limitations, Q4EDA is unique in its proposal and represents a real advance towards providing solutions for querying textual information based on user interactions with visual representations.

Artificial selection methods from evolutionary computing show promise for directed evolution of microbes.

Directed microbial evolution harnesses evolutionary processes in the laboratory to construct microorganisms with enhanced or novel functional traits. Attempting to direct evolutionary processes for applied goals is fundamental to evolutionary computation, which harnesses the principles of Darwinian evolution as a general-purpose search engine for solutions to challenging computational problems. Despite their overlapping approaches, artificial selection methods from evolutionary computing are not commonly applied to living systems in the laboratory. In this work, we ask whether parent selection algorithms-procedures for choosing promising progenitors-from evolutionary computation might be useful for directing the evolution of microbial populations when selecting for multiple functional traits. To do so, we introduce an agent-based model of directed microbial evolution, which we used to evaluate how well three selection algorithms from evolutionary computing (tournament selection, lexicase selection, and non-dominated elite selection) performed relative to methods commonly used in the laboratory (elite and top 10% selection). We found that multiobjective selection techniques from evolutionary computing (lexicase and non-dominated elite) generally outperformed the commonly used directed evolution approaches when selecting for multiple traits of interest. Our results motivate ongoing work transferring these multiobjective selection procedures into the laboratory and a continued evaluation of more sophisticated artificial selection methods.

Crawling the Deep Web Using Asynchronous Advantage Actor Critic Technique

In the digital world, World Wide Web magnitude is expanding very promptly. Now a day, a rising number of data-centric websites require a mechanism to crawl the information. The information accessible through hyperlinks can easily be retrieved with general-purpose search engines. A massive chunk of the structured information is invisible behind the search forms. Such immense information is recognized as the deep web and has structured information as compared to the surface web. Crawling the content of deep web is very challenging and requires filling the search forms with suitable queries. This paper proposes an innovative technique using an Asynchronous Advantage Actor-Critic (A3C) to explore the unidentified deep web pages. It is based on the policy gradient deep reinforcement learning technique that parameterizes the policy and value function based on the reward system. A3C has one coordinator and various agents. These agents learn from different environments, update the local gradients to a coordinator, and produce a more stable system. The proposed technique has been validated with Open Directory Project (ODP). The experimental outcome shows that the proposed technique outperforms most of the prevailing techniques based on various metrics such as average precision-recall, average harvest rate, and coverage ratio.

Tool to Retrieve Less-Filtered Information from the Internet

While users benefit greatly from the latest communication technology, with popular platforms such as social networking services including Facebook or search engines such as Google, scientists warn of the effects of a filter bubble at this time. A solution to escape from filtered information is urgently needed. We implement an approach based on the mechanism of a metasearch engine to present less-filtered information to users. We develop a practical application named MosaicSearch to select search results from diversified categories of sources collected from multiple search engines. To determine the power of MosaicSearch, we conduct an evaluation to assess retrieval quality. According to the results, MosaicSearch is more intelligent compared to other general-purpose search engines: it generates a smaller number of links while providing users with almost the same amount of objective information. Our approach contributes to transparent information retrieval. This application helps users play a main role in choosing the information they consume.

TOOLS AND TECHNIQUES FOR RETRIEVING THE HIDDEN INFORMATION OF INVISIBLE WEB

Search engines have become an important part of our daily routine. We are so much addicted to search engines, such as Google, that we cannot think of spending even a single day without them. It is a common misconception that googling a search term will reveal each and every site of the web. The fact, however, is that the typical search engines like Google, Yahoo, or Bing actually access only a tiny fraction; estimated at 0.03%; of the Internet. The information resources found by such typical general-purpose search engines are referred to as the “visible web” or “surface web”. The term “invisible” represents those resources that, because of their exclusion by general-purpose search engines, are not so easily found. Invisible web is a vast hidden storehouse of information that cannot be overlooked. The paper explores the importance of unfathomable information buried inside invisible web in our everyday lives. The paper also elucidates the tools and techniques used to fetch information from invisible web in different fields. Links of various web pages, that form invisible web, are mentioned along with their purpose.

Generic-distributed framework for cloud services marketplace based on unified ontology

Cloud computing is a pattern for delivering ubiquitous and on demand computing resources based on pay-as-you-use financial model. Typically, cloud providers advertise cloud service descriptions in various formats on the Internet. On the other hand, cloud consumers use available search engines (Google and Yahoo) to explore cloud service descriptions and find the adequate service. Unfortunately, general purpose search engines are not designed to provide a small and complete set of results, which makes the process a big challenge. This paper presents a generic-distrusted framework for cloud services marketplace to automate cloud services discovery and selection process, and remove the barriers between service providers and consumers. Additionally, this work implements two instances of generic framework by adopting two different matching algorithms; namely dominant and recessive attributes algorithm borrowed from gene science and semantic similarity algorithm based on unified cloud service ontology. Finally, this paper presents unified cloud services ontology and models the real-life cloud services according to the proposed ontology. To the best of the authors’ knowledge, this is the first attempt to build a cloud services marketplace where cloud providers and cloud consumers can trend cloud services as utilities. In comparison with existing work, semantic approach reduced the execution time by 20% and maintained the same values for all other parameters. On the other hand, dominant and recessive attributes approach reduced the execution time by 57% but showed lower value for recall.

ENGLISH

There are many undefined scaling challenges for general purpose crawler and search engines due to the rapid growth of the deep web. Now a days there are increasing numbers of data sources which become available on the web, but often their contents are only accessible through query interface. For harvesting deep web interface problem proposed framework is used and the Parsing process takes place. To achieve more accurate result this proposed crawler calculate binary vector and page rank of pages and Count the given keywords from the URL which is mined from the crawler to accomplish more precise result for a focused crawler give relevant links with ranking. Here experimental result on a set of representative domain show the accuracy of this proposed crawler framework which can efficiently retrieves web interface from large scale sites.

Crawling ranked deep Web data sources

In the era of big data, the vast majority of the data are not from the surface Web, the Web that is interconnected by hyperlinks and indexed by most general purpose search engines. Instead, the trove of valuable data often reside in the deep Web, the Web that is hidden behind query interfaces. Since numerous applications, like data integration and vertical portals, require deep Web data, various crawling methods were developed for exhaustively harvesting a deep Web data source with the minimal (or near-minimal) cost. Most existing crawling methods assume that all the documents matched by queries are returned. In practice, data sources often return the top k matches. This makes exhaustive data harvesting difficult: highly ranked documents will be returned multiple times, while documents ranked low have small chance being returned. In this paper, we decompose this problem into two orthogonal sub-problems, i.e., query and ranking bias problems, and propose a document frequency based crawling method to overcome the ranking bias problem. The rational of our method is to use the queries whose document frequencies are within the specified range to avoid the effect of search ranking plus return limit and significantly reduce the difficulty of crawling ranked data source. The method is extensively tested on a variety of datasets and compared with two existing methods. The experimental result demonstrates that our method outperforms the two algorithms by 58 % and 90 % on average respectively.

Searching and managing references for medical research and results' publishing

The current online services enable detection and access to a significant number of specialised databases, documents and articles in the field of medical and related sciences. However, the amount of provided information emphasizes the problem of classification, relevance and efforts to make a selection, study and logically organize the collected references and documents. Since already faced with such experiences, we have explored: which search engines will provide rational and relevant lists of documents on our entries; which software tools are available for managing, organising and reviewing the collected documents, as well as for coordinating team activities, and finally we checked the effectiveness and efficiency of the selected search engines and tools for reference managing in practice. Materials and methods: In order to find documents online, we used: a general-purpose search engine - Google; a specialised search engine - Google Scholar; a specialised Web service - PubMed and users' library that is integrated into reference manager software tool Mendeley. Our analysis of reference management tools included some of the frequently used, such as: Docear, Endnote, Mendeley and Zotero. The assessment of the search engines and tools for reference managing was based on our comparative analysis by applying pre-defined criteria and desired functionalities. Results: At first, the comparative analysis showed that there was not 'the best' way for searching and finding documents and literature on the Internet. At the very start we eliminated the general-purpose search engine - Google due to insufficient selectivity and relevance. Further on, analyzing the search results for medical-related queries, we concluded that Google Scholar, PubMed and Mendeley offered different, but sufficiently selective and relevant lists of documents and articles. The analysis of tools for reference managing, based on comparative analysis of functionality and our experience, singled out the desktop software tool Mendeley as the most applicable. Conclusions: Writing professional articles faces two completely opposite problems: the lack of information and excessive information. Our research was focused on the problem of information overload, that is: how to select only relevant sources and which software tools are appropriate for managing the collected references. We found out that the combined use of specialized search engines and Web services provide meaningful and relevant sources of information, and that the reference manager tool significantly facilitates reviewing and citation of references, while supporting and affirming teamwork as well.

Improving Results Aggregation Strategies in Distributed Information Retrieval

The top-ranked documents from various information sources that are merged together into a unified ranked list may cover the same piece of relevant information, and cannot satisfy different user needs. Result diversification(RD) solves this problem by diversifying results to cover more information needs. In recent times, RD has attracted much attention as a means of increasing user satisfaction in general purpose search engines. A myriad of approaches have been proposed in the related works for the diversification problem. However, no concrete study of search result diversification has been done in a Distributed Information Retrieval(DIR) setting. In this paper, we survey, classify and propose a theoretical framework that aims at improving diversification at the result merging phase of a DIR environment.

Supporting information retrieval from electronic health records: A report of University of Michigan’s nine-year experience in developing and using the Electronic Medical Record Search Engine (EMERSE)

ObjectiveThis paper describes the University of Michigan’s nine-year experience in developing and using a full-text search engine designed to facilitate information retrieval (IR) from narrative documents stored in electronic health records (EHRs). The system, called the Electronic Medical Record Search Engine (EMERSE), functions similar to Google but is equipped with special functionalities for handling challenges unique to retrieving information from medical text. Materials and methodsKey features that distinguish EMERSE from general-purpose search engines are discussed, with an emphasis on functions crucial to (1) improving medical IR performance and (2) assuring search quality and results consistency regardless of users’ medical background, stage of training, or level of technical expertise. ResultsSince its initial deployment, EMERSE has been enthusiastically embraced by clinicians, administrators, and clinical and translational researchers. To date, the system has been used in supporting more than 750 research projects yielding 80 peer-reviewed publications. In several evaluation studies, EMERSE demonstrated very high levels of sensitivity and specificity in addition to greatly improved chart review efficiency. DiscussionIncreased availability of electronic data in healthcare does not automatically warrant increased availability of information. The success of EMERSE at our institution illustrates that free-text EHR search engines can be a valuable tool to help practitioners and researchers retrieve information from EHRs more effectively and efficiently, enabling critical tasks such as patient case synthesis and research data abstraction. ConclusionEMERSE, available free of charge for academic use, represents a state-of-the-art medical IR tool with proven effectiveness and user acceptance.

Patient-Centered Tools for Medication Information Search.

Recent research focused on online health information seeking highlights a heavy reliance on general-purpose search engines. However, current general-purpose search interfaces do not necessarily provide adequate support for non-experts in identifying suitable sources of health information. Popular search engines have recently introduced search tools in their user interfaces for a range of topics. In this work, we explore how such tools can support non-expert, patient-centered health information search. Scoping the current work to medication-related search, we report on findings from a formative study focused on the design of patient-centered, medication-information search tools. Our study included qualitative interviews with patients, family members, and domain experts, as well as observations of their use of Remedy, a technology probe embodying a set of search tools. Post-operative cardiothoracic surgery patients and their visiting family members used the tools to find information about their hospital medications and were interviewed before and after their use. Domain experts conducted similar search tasks and provided qualitative feedback on their preferences and recommendations for designing these tools. Findings from our study suggest the importance of four valuation principles underlying our tools: credibility, readability, consumer perspective, and topical relevance.

A Retrieval Sorting Approach for Online Forums Based on Domain Topics

Topical search engine is an extension of general-purpose search engines, which has become an important research subject in Web information retrieval recently. Focusing on the development of Web 2.0 applications, a result ranking approach is proposed on the basis of LDA model to rank the search results from Web forums. Compared with traditional methods, this approach takes up less storage space, and can more quickly and accurately respond to user inquiries. This work has important significance for the research of improving the performance of retrieval results of web forums.

BioinQA: metadata-based multi-document QA system for addressing the issues in biomedical domain

Despite the availability of large amount of biomedical literature; extracting relevant information catering to the exact need of the user has been difficult in the absence of efficient domain specific information retrieval tools. Biomedical question answering (QA) systems require special techniques to address domain-specific issues, since a wide variety of user-groups having different information needs; terminology and level of understanding, etc., may access the information. While specialised information retrieval tools are not suitable for beginners, general purpose search engines are not intelligent enough to respond to domain specific questions. This paper presents an intelligent QA system that answers natural language questions while adapting itself to the level of user. The system constructs answers from multiple documents for complex comparison seeking questions. The system utilises metadata knowledge for addressing specific biomedical domain concerns like heterogeneity, acronyms, etc. Experiments ...

Automatic discovery of Web Query Interfaces using machine learning techniques

The amount of information contained in databases available on the Web has grown explosively in the last years. This information, known as the Deep Web, is heterogeneous and dynamically generated by querying these back-end (relational) databases through Web Query Interfaces (WQIs) that are a special type of HTML forms. The problem of accessing to the information of Deep Web is a great challenge because the information existing usually is not indexed by general-purpose search engines. Therefore, it is necessary to create efficient mechanisms to access, extract and integrate information contained in the Deep Web. Since WQIs are the only means to access to the Deep Web, the automatic identification of WQIs plays an important role. It facilitates traditional search engines to increase the coverage and the access to interesting information not available on the indexable Web. The accurate identification of Deep Web data sources are key issues in the information retrieval process. In this paper we propose a new strategy for automatic discovery of WQIs. This novel proposal makes an adequate selection of HTML elements extracted from HTML forms, which are used in a set of heuristic rules that help to identify WQIs. The proposed strategy uses machine learning algorithms for classification of searchable (WQIs) and non-searchable (non-WQI) HTML forms using a prototypes selection algorithm that allows to remove irrelevant or redundant data in the training set. The internal content of Web Query Interfaces was analyzed with the objective of identifying only those HTML elements that are frequently appearing provide relevant information for the WQIs identification. For testing, we use three groups of datasets, two available at the UIUC repository and a new dataset that we created using a generic crawler supported by human experts that includes advanced and simple query interfaces. The experimental results show that the proposed strategy outperforms others previously reported works.

IntegromeDB: an integrated system and biological search engine

BackgroundWith the growth of biological data in volume and heterogeneity, web search engines become key tools for researchers. However, general-purpose search engines are not specialized for the search of biological data.DescriptionHere, we present an approach at developing a biological web search engine based on the Semantic Web technologies and demonstrate its implementation for retrieving gene- and protein-centered knowledge. The engine is available at http://www.integromedb.org.ConclusionsThe IntegromeDB search engine allows scanning data on gene regulation, gene expression, protein-protein interactions, pathways, metagenomics, mutations, diseases, and other gene- and protein-related data that are automatically retrieved from publicly available databases and web pages using biological ontologies. To perfect the resource design and usability, we welcome and encourage community feedback.

Towards Improving Meta-Search through Exploiting an Integrated Search Model

Meta-search engines are created to reduce the burden on the user by dispatching queries to multiple search engines in parallel. Decisions on how to rank the returned results are made based on the query's keywords. Although keyword-based search model produces good results, better results can be obtained by integrating semantic and statistical based relatedness measures into this model. Such integration allows the meta-search engine to search by meanings rather than only by literal strings. In this article, we present Multi-Search+, the next generation of Multi-Search general-purpose meta-search engine. The extended version of the system employs additional knowledge represented by multiple domain-specific ontologies to enhance both the query processing and the returned results merging. In addition, new general-purpose search engines are plugged-in to its architecture. Experimental results demonstrate that our integrated search model obtained significant improvement in the quality of the produced search results.

LESSON: A system for lecture notes searching and sharing over Internet

In this paper, we present a system LESSON for lecture notes searching and sharing, which is dedicated to both instructors and students for effectively supporting their Web-based teaching and learning activities. The LESSON system employs a metasearch engine for lecture notes searching from Web and a peer-to-peer (P2P) overlay network for lecture notes sharing among the users. A metasearch engine provides an unified access to multiple existing component search engines and has better performance than general-purpose search engines. With the help of a P2P overlay network, all computers used by instructors and students can be connected into a virtual society over the Internet and communicate directly with each other for lecture notes sharing, without any centralized server and manipulation. In order to merge results from multiple component search engines into a single ranked list, we design the RSF strategy that takes rank, similarity and features of lecture notes into account. To implement query routing decision for effectively supporting lecture notes sharing, we propose a novel query routing mechanism. Experimental results indicate that the LESSON system has better performance in lecture notes searching from Web than some popular general-purpose search engines and some existing metasearch schemes; while processing queries within the system, it outperforms some typical routing methods. Concretely, it can achieve relatively high query hit rate with low bandwidth consumption in different types of network topologies.

Finding what is missing from a digital library: A case study in the Computer Science field

This article proposes a process to retrieve the URL of a document for which metadata records exist in a digital library catalog but a pointer to the full text of the document is not available. The process uses results from queries submitted to Web search engines for finding the URL of the corresponding full text or any related material. We present a comprehensive study of this process in different situations by investigating different query strategies applied to three general purpose search engines (Google, Yahoo!, MSN) and two specialized ones (Scholar and CiteSeer), considering five user scenarios. Specifically, we have conducted experiments with metadata records taken from the Brazilian Digital Library of Computing (BDBComp) and The DBLP Computer Science Bibliography (DBLP). We found that Scholar was the most effective search engine for this task in all considered scenarios and that simple strategies for combining and re-ranking results from Scholar and Google significantly improve the retrieval quality. Moreover, we study the influence of the number of query results on the effectiveness of finding missing information as well as the coverage of the proposed scenarios.