Evaluation of Argo Scholar with Observational Study

2025-05-06 0 0 1.38MB 2 页 10玖币

侵权投诉

Evaluation of ARGO SCHOLAR with Observational Study

Kevin Li1Haoyang Yang1Evan Montoya1Anish Upadhayay1

Zhiyan Zhou1Jon Saad-Falcon1Duen Horng (Polo) Chau1

ABSTRACT

Discovering and making sense of relevant literature is fundamental in

any scientiﬁc ﬁeld. Node-link diagram-based visualization tools can

aid this process; however, existing tools have been evaluated only on

small scales. This paper evaluates ARGO SCHOLAR, an open-source

visualization tool designed for interactive exploration of literature

and easy sharing of exploration results. A large-scale user study of

122 participants from diverse backgrounds and experiences showed

that ARGO SCHOLAR is effective at helping users ﬁnd related work

and understand paper connections, and incremental graph-based

exploration is effective across diverse disciplines. Based on the

user study and user feedback, we provide design considerations and

feature suggestions for future work.

Index Terms:

Human-centered computing—Visualization—

Visualization design and evaluation methods

1 INTRODUCTION

Discovering and making sense of relevant literature is fundamental

in any scientiﬁc ﬁeld. Academic search engines are common starting

point; however, keyword searching does not provide users with all

relevant papers, so incremental exploration, a common and effec-

tive method where users iterate from familiar (identiﬁed via search

engines) to novel works while ﬁltering based on paper attributions,

is also employed [4]. Visualization can aid this process, such as

node-link diagram-based tools that support the visual exploration

of citation networks with a bottom-up approach that allows users to

grow their own networks and understanding of its papers [1]. How-

ever, the evaluations of existing tools often only consist of a dozen

or so participants for certain selected domains [1, 5]. A large-scale

evaluation of such node-link based literature exploration tools across

diverse domains could generate different discoveries and insights.

This paper presents a follow-up, large-scale evaluation of ARGO

SCHOLAR [2], open-sourced, web-based visualization tool that fa-

cilitates the incremental exploration of literature to gain insight and

uncover relevant work. We were interested to see how receptive

participants from diverse backgrounds were to node-link diagram

representations of citation networks and what features of literature

exploration tools did participants ﬁnd the most helpful. To answer

these questions, we recruited 122 participants from a wide range of

backgrounds and research experiences.

2 BACKGROUND: VISUAL EXPLORATION OF LITERATURE

WITH ARGO SCHOLAR

ARGO SCHOLAR [2] is an open-source, web-based visualization

tool for interactive exploration of literature and easy sharing of ex-

ploration results developed by the authors of this poster. ARGO

SCHOLAR supports the method of incremental exploration, an ef-

fective technique to discover relevant and related work [5, 6], on

Semantic Scholar’s live database of 200+ million papers [3].

Georgia Institute of Technology.

{

kevin.li

alexanderyang

emontoya30

aupadhayay3|zzhou406|jonsaadfalcon|polo}@gatech.edu

Figure 1: ARGO SCHOLAR visualizing a literature network of deep

learning papers; nodes are papers and edges are citations.

ARGO SCHOLAR (Fig. 1) enables users to generate personalized

literature exploration results in real-time. ARGO SCHOLAR’s main

view displays the user’s collection of papers as a node-link diagram

where papers are represented by nodes and are connected by citation

edges. Users can easily expand their collection by querying for

papers through keywords or adding the citations or references of

existing papers based on relevance, citation count, inﬂuence, or

recency [3], and nodes can be dragged to be rearranged and display

attributes adjusted to help the sensemaking process. Users can save

or share a snapshot of their results as a URL on the cloud for free.

ARGO SCHOLAR was developed as an open-sourced web-based

tool to help increase the availability and impact. Built with mod-

ern web technologies, such as Three.js WebGL rendering, ARGO

SCHOLAR can smoothly render hundreds of papers and their

relationships and is hosted at

https://poloclub.github.io/

argo-scholar

. The open-sourced nature of ARGO SCHOLAR also

differentiates itself from other existing tools.

3 EVALUATION

To investigate how users would interact with ARGO SCHOLAR’s

features to explore literature networks and ﬁnd relevant work and

evaluate ARGO SCHOLAR’s effectiveness and usability, we con-

ducted a large-scale user study of 122 participants. This section

describes the study’s design and ﬁndings.

3.1 Experimental Design

Participants.

122 U.S.-based participants were recruited from

Proliﬁc

, an online crowdsourcing platform designed speciﬁcally for

academic research. Participants were screened to ensure they had

at least one year of research experience, so they would be able to

accurately gauge the effectiveness of ARGO SCHOLAR for ﬁnding

related work. The resulting sample encompassed diverse knowledge

backgrounds and research experiences (Table 1). 76 participants

were current students, while 46 were not currently enrolled. 21

listed a Doctorate degree as the high level of education completed

or currently completing, 42 Master’s, 44 Bachelor’s, 12 GED or

equivalent, and 3 others (JD, MD, and DDS). Notably, three were

professors and two were postdoctoral researchers.

1https://www.proliﬁc.co/

arXiv:2210.13510v1 [cs.HC] 24 Oct 2022

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

EvaluationofARGOSCHOLARwithObservationalStudyKevinLi1HaoyangYang1EvanMontoya1AnishUpadhayay1ZhiyanZhou1JonSaad-Falcon1DuenHorng(Polo)Chau1ABSTRACTDiscoveringandmakingsenseofrelevantliteratureisfundamentalinanyscienticeld.Node-linkdiagram-basedvisualizationtoolscanaidthisprocess;however,existingtoo...

展开>> 收起<<

Evaluation of Argo Scholar with Observational Study.pdf

共2页,预览1页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Evaluation of Argo Scholar with Observational Study

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: