Vertical Search Research Articles

Market or Markets? Investigating Google Search’s Market Shares under Vertical Segmentation

Is Google Search a monopoly with gatekeeping power? Regulators from the US, UK, and Europe have argued that it is based on the assumption that Google Search dominates the market for horizontal (a.k.a. “general”) web search. Google disputes this, claiming that competition extends to all vertical (a.k.a. “specialized”) search engines, and that under this market definition it does not have monopoly power. In this study we present the first analysis of Google Search’s market share under vertical segmentation of online search. We leverage observational trace data collected from a panel of US residents that includes their web browsing history and copies of the Google Search Engine Result Pages they were shown.We observe that participants’ search sessions begin at Google greater than 50% of the time in 24 out of 30 vertical market segments (which comprise almost all of our participants’ searches). Our results inform the consequential and ongoing debates about the market power of Google Search and the conceptualization of online markets in general.

Impact and development of an Open Web Index for open web search

AbstractWeb search is a crucial technology for the digital economy. Dominated by a few gatekeepers focused on commercial success, however, web publishers have to optimize their content for these gatekeepers, resulting in a closed ecosystem of search engines as well as the risk of publishers sacrificing quality. To encourage an open search ecosystem and offer users genuine choice among alternative search engines, we propose the development of an Open Web Index (OWI). We outline six core principles for developing and maintaining an open index, based on open data principles, legal compliance, and collaborative technology development. The combination of an open index with what we call declarative search engines will facilitate the development of vertical search engines and innovative web data products (including, e.g., large language models), enabling a fair and open information space. This framework underpins the EU‐funded project OpenWebSearch.EU, marking the first step towards realizing an Open Web Index.

A Comprehensive Survey of Facet Ranking Approaches Used in Faceted Search Systems

Faceted Search Systems (FSSs) have gained prominence as one of the dominant search approaches in vertical search systems. They provide facets to educate users about the information space and allow them to refine their search query and navigate back and forth between resources on a single results page. Despite the importance of this problem, it is rare to find studies dedicated solely to the investigation of facet ranking methods, nor to how this step, aside from other aspects of faceted search, affects the user’s search experience. The objective of this survey paper is to review the state of the art in research related to faceted search systems, with a focus on existing facet ranking approaches and the key challenges posed by this problem. In addition to that, this survey also investigates state-of-the-art FSS evaluation frameworks and the most commonly used techniques and metrics to evaluate facet ranking approaches. It also lays out criteria for dataset appropriateness and its needed structure to be used in evaluating facet ranking methods aside from other FSS aspects. This paper concludes by highlighting gaps in the current research and future research directions related to this area.

A Schematic Review of Knowledge Reasoning Approaches Based on the Knowledge Graph

In the contemporary world, the Internet technology and its implementation mode are advancing at a swift pace, leading to an exponential growth in the scale of Internet data. This data contains a significant amount of valuable knowledge. The effective organization and articulation of knowledge, as well as the ability to conduct thorough calculations and analyses, have garnered significant attention and developments within a particular environmental context. The utilization of knowledge graphs for knowledge reasoning has emerged as a prominent area of focus within the realm of knowledge graph research. It holds substantial significance in the realm of vertical search, intelligent answering, and various other applications. This article will be centered on fundamental principles of reasoning. The approach of knowledge reasoning oriented towards knowledge graphs is focused on the derivation of novel knowledge or the detection of erroneous knowledge through the utilization of pre-existing knowledge. In contrast to conventional knowledge reasoning approaches, the knowledge reasoning technique employed in knowledge graphs is characterized by greater diversity, owing to the succinct, adaptable, and flexible representation of knowledge.

Exploring the carry-over of top-down attentional settings in dynamic conditions.

A top-down attentional set can persist from a relevant task to an irrelevant task, influencing allocation of attentional resources, visual search, and performance. While this "carry-over" effect has been found across numerous experiments, past studies have utilised paradigms that present similar tasks to the same spatial location. The present research explored whether attentional settings persist in more dynamic situations. In Experiment 1, participants played a computer game that encouraged a horizontal, vertical, or random spread of search. After 10 or 30 s, they moved 90° to their right and monitored a driving video for hazards. Eye movements to the videos were not affected by the characteristics of the preceding game, revealing no carry-over of attentional settings. One possible explanation for this was the visuospatial shift between the tasks. To explore this further, Experiment 2 adopted a similar paradigm to previous research; participants searched horizontal, vertical, or random letter strings before completing an image search. In one block the tasks were presented to the same screen, and in one block the tasks were presented to different screens (incorporating a 90° visuospatial shift mid-trial). Carry-over was found in the one-screen block, with a significantly wider horizontal search and a narrower vertical search in the pictures after a horizontal letter search. However, there was no carry-over from the letter to the picture task in the two-screen block. This indicates the flexibility of attentional control in dynamic situations, and it is suggested that persistence of attentional settings will be most costly under stable conditions.

Focused Crawler Based on Reinforcement Learning and Decaying Epsilon-Greedy Exploration Policy

In order to serve a diversified user base with a range of purposes, general search engines offer search results for a wide variety of topics and material categories on the internet. While Focused Crawlers (FC) deliver more specialized and targeted results inside particular domains or verticals, general search engines give a wider coverage of the web. For a vertical search engine, the performance of a focused crawler is extremely important, and several ways of improvement are applied. We propose an intelligent, focused crawler which uses Reinforcement Learning (RL) to prioritize the hyperlinks for long-term profit. Our implementation differs from other RL based works by encouraging learning at an early stage using a decaying ϵ-greedy policy to select the next link and hence enables the crawler to use the experience gained to improve its performance with more relevant pages. With an increase in the infertility rate all over the world, searching for information regarding the issues and details about artificial reproduction treatments available is in need by many people. Hence, we have considered infertility domain as a case study and collected web pages from scratch. We compare the performance of crawling tasks following ϵ-greedy and decaying ϵ-greedy policies. Experimental results show that crawlers following a decaying ϵ-greedy policy demonstrate better performance

Multilevel threshold image segmentation for COVID-19 chest radiography: A framework using horizontal and vertical multiverse optimization

COVID-19 is currently raging worldwide, with more patients being diagnosed every day. It usually is diagnosed by examining pathological photographs of the patient's lungs. There is a lot of detailed and essential information on chest radiographs, but manual processing is not as efficient or accurate. As a result, how efficiently analyzing and processing chest radiography of COVID-19 patients is an important research direction to promote COVID-19 diagnosis. To improve the processing efficiency of COVID-19 chest films, a multilevel thresholding image segmentation (MTIS) method based on an enhanced multiverse optimizer (CCMVO) is proposed. CCMVO is improved from the original Multi-Verse Optimizer by introducing horizontal and vertical search mechanisms. It has a more assertive global search ability and can jump out of the local optimum in optimization. The CCMVO-based MTIS method can obtain higher quality segmentation results than HHO, SCA, and other forms and is less prone to stagnation during the segmentation process. To verify the performance of the proposed CCMVO algorithm, CCMVO is first compared with DE, MVO, and other algorithms by 30 benchmark functions; then, the proposed CCMVO is applied to image segmentation of COVID-19 chest radiography; finally, this paper verifies that the combination of MTIS and CCMVO is very successful with good segmentation results by using the Feature Similarity Index (FSIM), the Peak Signal to Noise Ratio (PSNR), and the Structural Similarity Index (SSIM). Therefore, this research can provide an effective segmentation method for a medical organization to process COVID-19 chest radiography and then help doctors diagnose coronavirus pneumonia (COVID-19).

버티컬 검색에서의 확률 그래픽 모델 기반 클릭모형

Modern search engines provide the result including not only the traditional web documents, but also the segments aggregated with specialized information such as images, videos, and news, called vertical, to the users. Most click models associated with vertical search were designed to reduce various biases caused by verticals in search engine result pages to evaluate user behavior of web documents. However, evaluation of verticals themselves has not been conducted. In this paper, we propose vertical click models to analyze user behavior associated with both verticals and documents. The proposed click models based on probabilistic graphical model framework directly reflect events occurred in verticals and documents, so that they allow to elaborately calculate and evaluate the preference of verticals felt by the user. The proposed models were evaluated with a click log dataset from the popular commercial search engine in Korea. The results demonstrated superiority of our model over existing models in terms of normalized discounted cumulative gain.

A novel focused crawler combining Web space evolution and domain ontology

In many fields, how to catch the related-topic Web resources is crucial. As a vertical search method, focused crawler has received great attention in recent years. Currently, most focused crawlers consider multiple evaluating factors of the hyperlinks and use the weighted sum approach to compute the priorities of unvisited hyperlinks. However, the proper weighted coefficients are hard to determine, and their unsuitable values may even cause the direction of crawlers to deviate seriously from the topic. To overcome this issue, this article builds a multi-objective optimization model based on Web text and link structure and designs a crawler framework called the Web space evolution (WSE), where a hyperlink bank whose radius is gradually increased is introduced to extend the search scape of crawlers in Web space. To improve the uniformity and diversity of hyperlinks, a nearest and farthest candidate solution method is combined with the fast non-dominated sorting to choose Pareto-optimal solutions (hyperlinks). A domain ontology based on the formal concept analysis is applied to establish the topic model. By incorporating the WSE and the domain ontology into the focused crawling, a novel focused crawler called FCWSEO is proposed to collect topic-relevant webpages. The experimental results on the rainstorm disaster domain show that the FCWSEO outperforms other focused crawler strategies in terms of the quantity and quality of retrieved relevant webpages.

Horizontal and vertical search artificial bee colony for image segmentation of COVID-19 X-ray images

The artificial bee colony algorithm (ABC) has been successfully applied to various optimization problems, but the algorithm still suffers from slow convergence and poor quality of optimal solutions in the optimization process. Therefore, in this paper, an improved ABC (CCABC) based on a horizontal search mechanism and a vertical search mechanism is proposed to improve the algorithm's performance. In addition, this paper also presents a multilevel thresholding image segmentation (MTIS) method based on CCABC to enhance the effectiveness of the multilevel thresholding image segmentation method. To verify the performance of the proposed CCABC algorithm and the performance of the improved image segmentation method. First, this paper demonstrates the performance of the CCABC algorithm itself by comparing CCABC with 15 algorithms of the same type using 30 benchmark functions. Then, this paper uses the improved multi-threshold segmentation method for the segmentation of COVID-19 X-ray images and compares it with other similar plans in detail. Finally, this paper confirms that the incorporation of CCABC in MTIS is very effective by analyzing appropriate evaluation criteria and affirms that the new MTIS method has a strong segmentation performance.

Brand Bidding Restraints Revisited: What Is the Appropriate Economic and Legal Framework for the Antitrust Analysis of Vertical Online Search Advertising Restraints?

This article explores the law and economics of brand bidding restraints which constitute the most novel type of vertical restraints imposed by brand owners on their distributors in digital markets. The article tests and critically reflects on the restrictive approach European competition watchdogs have recently adopted towards these brand bidding restraints. It contends that this harsh antitrust treatment of brand bidding restraints is not sufficiently grounded in the economic analysis of vertical restraints. In proposing a comprehensive framework for the legal and economic analysis of brand bidding restraints, the article makes three principal contributions. First, it asserts that brand bidding restraints can have a number of procompetitive effects by internalizing advertising-related externalities, addressing free-riding on display and traditional advertising and facilitating fixed cost recovery through price discrimination. Second, the paper considers different ways through which brand bidding restraints may harm competition and consumer welfare when they disproportionately affect infra-marginal consumers, prevent meaningful intra- and inter-brand comparisons or result in price discrimination on the basis of search costs rather than brand preferences. Moreover, brand bidding restraints are of particular concern when adopted in the context of dual distribution systems where vertically integrated brand owners have an incentive to raise their retailers’ costs to prevent them from cannibalizing on their own sales channel. Third, the article explore various filters that may inform an effects-based analysis of brand bidding restraints. In this respect, the article makes a number of policy recommendations for the future antitrust analysis of brand bidding restraints. These proposals could also inform the ongoing revision of the Vertical Block Exemption Regulation (VBER) and Vertical Guidelines in the EU and in the UK. brand bidding restraints, vertical online search advertising restraints, digital vertical restraints, e-commerce sector inquiry, B2-98/11 Asics, Case COMP/AT.40428 Guess, vertical restraints, Vertical Guidelines, Vertical Block Exemption Regulation

Research on Vertical Search Method of Multidimensional Resources in English Discipline Based on Edge Computing

The traditional vertical search method only considers the content of the webpage, and the global master node is not enough, which will lead to premature convergence and fall into the local optimum, resulting in insufficient multi-dimensional search of resources. Therefore, this paper proposes a multidimensional resource vertical edge based on the calculation of English subject search method. This paper analyzes the architecture of search engine firstly and then introduces the multiaccess edge computing architecture. At last, it constructs the vertical search task computing model of multidimensional resources in English discipline. By associating and traversing the attributes of multidimensional resources of English discipline, the vertical search of attribute information is realized offline, and the vertical search method of multidimensional resources of English discipline based on edge calculation is designed. In order to verify the effectiveness of the proposed method, a comparative experiment is designed. Experimental results show that the method can improve the resource search ratio and recall ratio, and it can also effectively improve the search efficiency. For an English subject resource data of 50 MB, the calculation methods of edge multidimensional resource data search recall rate can reach 97% and multidimensional resource data search time consumption is only 39 ms. The experimental results show that the performance of English subject multidimensional resources vertical search is much better.

Protein deep profile and model predictions for identifying the causal genes of male infertility based on deep learning

A principal task in dissecting the genetics of complex traits is to identify causal genes for disease phenotypes. Millions of genes have been sequenced in data-driven genomics era, but their causal relationships with disease phenotypes remain limited, due to the difficulty of elucidating underlying causal genes by laboratory-based strategies. Here, we proposed an innovative deep learning computational modeling alternative (DPPCG framework) for identifying causal (coding) genes for a specific disease phenotype. In terms of male infertility, we introduced proteins as intermediate cell variables, leveraging integrated deep knowledge representations (Word2vec, ProtVec, Node2vec, and Space2vec) quantitatively represented as ‘protein deep profiles’. We adopted deep convolutional neural network (CNN) classifier to model protein deep profiles relationships with male infertility, creatively training deep CNN models of single-label binary classification and multi-label eight classification. We demonstrate the capabilities of DPPCG framework by integrating and fully harnessing the utility of heterogeneous biomedical big data, including literature, protein sequences, protein–protein interactions, gene expressions, and gene–phenotype relationships, and effective indirect prediction of 794 causal genes of male infertility and associated pathological processes. We present this research in an interactive ‘Smart Protein’ intelligent (demo) system (http://www.smartprotein.cloud/public/home). Researchers can benefit from our intelligent system by (i) accessing a shallow gene/protein-radar service involving research status and a knowledge graph-based vertical search; (ii) querying and downloading protein deep profile matrices; (iii) accessing intelligent recommendations for causal genes of male infertility and associated pathological processes, and references for model architectures, parameter settings, and training outputs; and (iv) carrying out personalized analysis such as online K-Means clustering.

Design of topic Web crawler based on improved PageRank algorithm

With the continuous development of network information technology, the network is filled with a large number of all kinds of unstructured data called big data. However, this data is not easily stored in a local database. People realize that it is essential to get useful information from the Internet efficiently. The effort to gather information by human hands has led to the emergence of web crawler technology. However, the existing search engines still have shortcomings in topic similarity judgment and web page sorting algorithm. Therefore, this paper applies PageRank algorithm to topic crawler, constructs a vertical search engine, and introduces topic relevance factor to suppress "topic drift" according to the shortcomings of PageRank algorithm.

Improvement of Information Retrieval Systems by Using Hidden Vertical Search

The exponential growth of the number of documents in digital libraries and on the Web calls for very intensive development of retrieval systems. One possible architectural approach to IRS, an architecture with hidden verticals, is proposed in this paper. In IRS with hidden verticals, documents from the searched corpus are stored into a predefined set of classes. The user's query is classified before the search, and searching is done only within the corresponding class. The performance of the proposed system is compared to the performance of standard IRS (that contains a unique inverted index) and IRS with cluster pruning (in which searching corpus is clustered and query is compared to the clusters' centroids first, then search is done only in the most similar cluster). Search time in the proposed system is 7.9 times shorter than in the standard IRS and 1.7 times shorter than in the system with cluster pruning. The precision of the proposed system is 2.59 times higher than the precision of the standard IRS, and 1.68 times better compared to the IRS with cluster pruning. The recall of the proposed system is 1.09 times smaller than the recall of the standard IRS, but it is 1.28 times better than the recall of the IRS with cluster pruning. Based on the above results, we can say that proposed approach reduces search time and increases search precision with a minimal reduction in recall.

Research on Data Collection and Analysis of Second Hand House in China Based on Python

Abstract With the rapid development of domestic Internet, more and more people choose housing leasing activities through the Internet, however, the existing housing rental information website the quantity many, but because of various advertising, information is scattered, so to HOME LINK sites, according to the position provided by the customer, accurate demand such as rent or house type, vertical search related data from the rental information website and in accordance with the provisions, the structure of the storage, rent information data to provide data for subsequent analysis. Bring users a faster experience. At the same time, we should add the reminding of the houses we are interested in and the reminding of price reduction. For the users who cannot look at their mobile phones all the time, timely reminding can help them avoid missing many desirable houses. This system is committed to solving the current people to rent a request for the search of detailed provide the keywords needed. To help make it easier for everyone who needs to rent. The system is mainly composed of data cleaning, data access, algorithm design and implementation, Python implementation of the front and back end of the system, data formatting and auxiliary decision.

Quantifying Cyber Attacks on Industrial MMC-HVDC Control System Using Structured Pseudospectrum

This letter assesses the impact of cyber-attacks on the control system of the modular multilevel converter (MMC)-based high-voltage dc (MMC-HVdc) transmission technology. Specifically, the small-signal model of the MMC-HVdc is characterized by a closed-loop matrix and the distance to the small-signal instability is then quantified by structured pseudospectrum. Furthermore, a vertical search method is proposed to quantify the boundary of the structured pseudospectrum. The proposed quantification model with the vertical search method can be extended to the control of other power electronics-based systems. Case studies on a two-terminal MMC-HVdc system verify the effectiveness of the qualification method.

A sub-pixel circle detection algorithm combined with improved RHT and fitting

Circle extraction is usually a pre-completed task used in different applications related to medical, robotics, biometrics image analysis among others. Randomized Hough Transform (RHT) determines the parameters of the circle by randomly obtaining three edge pixels, if they are not precisely located on the circumference. The detected circle will not perfectly match the ideal circle. At the same time, three random points are largely not on a circle, which leads to some invalid sampling and parameter accumulation. In this paper, an improved RHT combined with fitting subpixel circle detection algorithm is proposed. The improved RHT algorithm calculates and accumulates parameters by using 1 point obtained from random sampling and another two points obtained from horizontal and vertical search respectively. The algorithm introduces the edge map of the de-soliton point and small region, and improves the probability that three points belong to the same circle. Then, the set of edge pixels corresponding to the identified circle is fitted to reduce the bias effect caused by only using three edge pixels to calculate the circle parameters. In this way, the reliability of the fitting and the precision of the parameters are improved while removing the noise. Experimental tests were conducted for detection performance, accuracy of parameter estimation and noise robustness. Compared with other methods, the proposed method has strong anti-interference ability and high calculation accuracy.

Research on the Focused Crawler of Mineral Intelligence Service Based on Semantic Similarity

Large-scale general search engines have been unable to meet the needs of “specialized, sophisticated and deep” information in the field of mineral intelligence services. Vertical search engines have emerged at the historic moment, and the focused crawler is the key technology to achieve vertical search engines. This paper proposes a hybrid topic strategy based on text content and web link structure for the characteristics of mineral information field. In order to improve the topic relevance determination ability of the focused crawler, this article introduces HowNet and the word embedding technology in the field of natural language processing, and combines them to carry out the text-based topic relevance determination; at the same time, it also introduces the HITS algorithm based on the link structure of web pages. The topic strategy based on text content is organically combined with the topic strategy based on the webpage link structure to realize the recognition and prediction of the webpage topic. Simulation experiment results show that the method proposed in this paper can achieve a high recall rate and precision rate for the acquisition of mineral intelligence information on the Internet.

ADRL: An attention-based deep reinforcement learning framework for knowledge graph reasoning

Knowledge graph reasoning is one of the key technologies for knowledge graph construction, which plays an important part in application scenarios such as vertical search and intelligent question answering. It is intended to infer the desired entity from the entities and relations that already exist in the knowledge graph. Most current methods for reasoning, such as embedding-based methods, globally embed all entities and relations, and then use the similarity of vectors to infer relations between entities or whether given triples are true. However, in real application scenarios, we require a clear and interpretable target entity as the output answer. In this paper, we propose a novel attention-based deep reinforcement learning framework (ADRL) for learning multi-hop relational paths, which improves the efficiency, generalization capacity, and interpretability of conventional approaches through the structured perception of deep learning and relational reasoning of reinforcement learning. We define the entire process of reasoning as a Markov decision process. First, we employ CNN to map the knowledge graph to a low-dimensional space, and a message-passing mechanism to sense neighbor entities at each level, and then employ LSTM to memorize and generate a sequence of historical trajectories to form a policy and value functions. We design a relational module that includes a self-attention mechanism that can infer and share the weights of neighborhood entity vectors and relation vectors. Finally, we employ the actor–critic algorithm to optimize the entire framework. Experiments confirm the effectiveness and efficiency of our method on several benchmark data sets.