NeuralSearchX Serving a Multi-billion-parameter Reranker for Multilingual Metasearch at a Low Cost

2025-05-02 0 0 820.17KB 13 页 10玖币

侵权投诉

NeuralSearchX: Serving a Multi-billion-parameter

Reranker for Multilingual Metasearch at a Low Cost

Thales Sales Almeida*, Thiago Laitz*, João Seródio*, Luiz Henrique Bonifacio,

Roberto Lotufo and Rodrigo Nogueira

NeuralMind, Brazil

Abstract

The widespread availability of search API’s (both free and commercial) brings the promise of increased

coverage and quality of search results for metasearch engines, while decreasing the maintenance costs

of the crawling and indexing infrastructures. However, merging strategies frequently comprise complex

pipelines that require careful tuning, which is often overlooked in the literature. In this work, we describe

NeuralSearchX, a metasearch engine based on a multi-purpose large reranking model to merge results

and highlight sentences. Due to the homogeneity of our architecture, we could focus our optimization

eorts on a single component. We compare our system with Microsoft’s Biomedical Search and show

that our design choices led to a much cost-eective system with competitive QPS while having close to

state-of-the-art results on a wide range of public benchmarks. Human evaluation on two domain-specic

tasks shows that our retrieval system outperformed Google API by a large margin in terms of nDCG@10

scores. By describing our architecture and implementation in detail, we hope that the community will

build on our design choices. The system is available at https://neuralsearchx.neuralmind.ai.

Keywords

Metasearch, Merging strategies, Transformers

1. Introduction

Metasearch engines provide a unied interface for searching and aggregating results from

dierent sources. They take advantage of existing search engines to increase the diversity of

results while decreasing maintenance costs for crawling and indexing infrastructures.

This approach, however, comes with a unique set of challenges: many times metasearch

engines choose to consult only sources that might be relevant to a given query, since searching

all available collections may be unfeasible due to resource constraints, such as limited bandwidth.

For example, given a medical-related query, a metasearch engine could choose to consult only

search engines that are capable of returning medical-related content. Therefore, a metasearch

engine must represent the capabilities and type of data that each source search engine provides,

i.e., the representation problem. When receiving a query, the metasearch engine must be able

to select the appropriate sources, i.e., the selection problem. Furthermore, a metasearch engine

is also responsible for generating a unied list of results from the multiples lists that it retrieves

DESIRES 2022 – 3rd International Conference on Design of Experimental Search & Information REtrieval Systems, 30-31

August 2022, San Jose, CA, USA

*Equal contribution.

"thalesrogerio@gmail.com (T. S. Almeida*)

CEUR

Workshop

Proceedings

http://ceur-ws.org

ISSN 1613-0073

CEUR Workshop Proceedings (CEUR-WS.org)

arXiv:2210.14837v1 [cs.IR] 26 Oct 2022

from, i.e., the merging problem.

In this work, we introduce NeuralSearchX, a multi-stage metasearch engine that takes a

dierent approach to metasearch: to improve the quality of results we retrieve mainly from

well-established general-purpose engines, such as Google and Bing, as well as curated sparse

and dense indexes. Therefore, we avoid the representation and selection problem as we can

simply search all our bases for any given query. To address the merging problem, NeuralSearchX

leverages state-of-the-art Transformer [

] models as a merging strategy. Such models recently

showed promising zero-shot capabilities [

], i.e., they can perform well in unseen domains and

therefore are suitable for general web search.

Our approach does not require any specic knowledge from the source collections since

it ranks documents purely by content. We show in the experimental section that our system

outperforms well-established search engines even without extra information from the document

or its source.

NeuralSearchX also takes advantage of low-reliability cloud infrastructure in its deployment

in order to use high computing power while keeping a low overall cost.

Our contributions are the following:

•

We propose zero-shot Transformer rerankers as an eective merging strategy for

metasearch engines. We demonstrate the eectiveness of our reranking pipeline on

various public datasets as well as human evaluations;

•

By using low-reliability infrastructure, we show that it is possible to use a model with

billions of parameters in a production environment under a reasonable budget, thus

allowing us to deploy a highly eective search engine;

2. Related Work

In this section, we rst discuss metasearch engines and their main challenges. Then we provide

an overview of multistage retrieval pipelines.

2.1. Metasearch Engines

Metasearch engines typically support hundreds or even thousands of search engines and

therefore demand highly elaborated solutions for the representation, selection and merging

problems [3,4,5].

Metasearch engines traditionally handle the collection representation problem by maintaining

representational sets that contain pertinent information for each searchable collection. Such

sets can be created manually [

] but are usually generated automatically due to scaling reasons.

Various methods were proposed to generate such sets [

]. For example, Query-Based

Sampling (QBS) [

] generates the representation set by sampling documents from the collection

using short queries to infer the collection content.

For the selection problem, existing solutions often calculate similarity metrics between the

query and the representational sets of each collection [

], and then select the collections

with the best scores. Other approaches include ranking collections based on the contents of the

top returned documents [

], or estimating the probability that a given collection has at least

one relevant document [16].

Traditional merging strategies are based on scoring functions that take into account textual

content and metadata retrieved from the source search engines. For example, CORI [

]

assigns a belief score for each collection based on a Bayesian inference network. The belief score

is then used in the scoring function to determine the score of each retrieved document in that

collection. SAFE [

] aggregates all retrieved documents as well as samples available from the

source collections, and ts a statistical model to predict the scores. More recently, Vijaya et al.

[20]

proposed the use of a neural network as a merging strategy by providing the model with

numerous corpus and document-level statistics as input in order to compute the relevancy of

the document. Note that this approach is dierent from the one implemented by NeuralSearchX

since our models score relevance based on the document content and do not require any extra

information. Finally, Lu et al.

[21]

make a comparative analysis of a wide variety of merging

strategies and demonstrates that a well-performing merging strategy is fundamental to achieve

adequate results.

2.2. Multistage Ranking

A multistage ranking retrieval pipeline is a common retrieval pipeline that uses multiple

consecutive renement steps to return the nal results. This method was rst referenced

back in 2006 at Matveeva et al.

[22]

. In 2010 it was revealed that the Bing search engine used

multistage ranking to deliver its results, in the next few years, a handful of notorious search

systems also related to multistage ranking [

] but with dierent numbers and types of

stages.

In the last decade, we have seen an increasing dedication to improving the quality of the

results by proposing innovations on dierent stages of the retrieval pipeline [

]. We also

saw a special focus on optimizing search systems for more specic and challenging domains. In

this context, Covidex [

] and Biomedsearch [

] have recently been proposed search engines

that focus on providing scientic information in the medical domain. Both use a multistage

ranking similar to that of NeuralSearchX.

3. Our Solution

NeuralSearchX is a multi-stage metasearch engine: the rst stage consists of a candidate

document retrieval step in which documents are retrieved using federated search over web

search engines along with sparse and dense retrieval from private collections. The second stage

consists of a reranking step of the previously selected documents with a neural model. The

model scores each document by how relevant it is for a given query. The best scored documents

are then sent to a highlighter model that estimates and selects the most relevant sentences from

each document. Finally the documents are formatted and returned to the end user. Figure 1

illustrates the described pipeline.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

NeuralSearchX:ServingaMulti-billion-parameterRerankerforMultilingualMetasearchataLowCostThalesSalesAlmeida*,ThiagoLaitz*,JoãoSeródio*,LuizHenriqueBonifacio,RobertoLotufoandRodrigoNogueiraNeuralMind,BrazilAbstractThewidespreadavailabilityofsearchAPI’s(bothfreeandcommercial)bringsthepromiseofincreased...

展开>> 收起<<

NeuralSearchX Serving a Multi-billion-parameter Reranker for Multilingual Metasearch at a Low Cost.pdf

共13页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

NeuralSearchX Serving a Multi-billion-parameter Reranker for Multilingual Metasearch at a Low Cost

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: