SLURP Spectroscopy of Liquids Using Robot Pre-Touch Sensing

2025-04-15 0 0 9.29MB 7 页 10玖币

侵权投诉

SLURP! Spectroscopy of Liquids Using Robot Pre-Touch Sensing

Nathaniel Hanson1∗, Wesley Lewis2∗, Kavya Puthuveetil2∗, Donelle Furline1, Akhil Padmanabha2,

Tas¸kın Padır1, Zackory Erickson2

Abstract— Liquids and granular media are pervasive

throughout human environments. Their free-ﬂowing nature

causes people to constrain them into containers. We do so

with thousands of different types of containers made out

of different materials with varying sizes, shapes, and colors.

In this work, we present a state-of-the-art sensing technique

for robots to perceive what liquid is inside of an unknown

container. We do so by integrating Visible to Near Infrared

(VNIR) reﬂectance spectroscopy into a robot’s end effector.

We introduce a hierarchical model for inferring the material

classes of both containers and internal contents given spectral

measurements from two integrated spectrometers. To train

these inference models, we capture and open source a dataset

of spectral measurements from over 180 different combinations

of containers and liquids. Our technique demonstrates over

85% accuracy in identifying 13 different liquids and granular

media contained within 13 different containers. The sensitivity

of our spectral readings allow our model to also identify the

material composition of the containers themselves with 96%

accuracy. Overall, VNIR spectroscopy presents a promising

method to give household robots a general-purpose ability to

infer the liquids inside of containers, without needing to open

or manipulate the containers.

I. INTRODUCTION

Knowing liquid properties is a critical ﬁrst step in properly

handling both open and closed containers. Manipulating

these containers is a general-purpose skill robots must master

to function in a domestic environment, as everything from

medicine to food is stored inside containers. Containers

give deﬁned shape to their otherwise formless contents, but

liquids’ variable viscosities and movable centers of gravity

motivates a need to understand what a container holds before

it is manipulated. Visual sensors (cameras and LIDARs)

are challenged by these scenarios as container opacity lim-

its vision of the internal contents. Tactile sensing is also

constrained by the diversity of liquid densities. Opening a

container with a manipulator to directly observe the contents

is a challenging dexterous manipulation task, and would still

require sensing the internal liquid. The core question of this

research is: Can robots uniquely and reliably identify liquids

without opening or manipulating their containers?

∗These authors contributed equally.

This research is supported by the National Science Foundation un-

der Award Number 1928654 and National Science Foundation Gradu-

ate Research Fellowship Program under Grant No. DGE1745016 and

DGE2140739.

1Institute for Experiential Robotics, Northeastern University, Boston,

Massachusetts, USA.

2Robotics Institute, Carnegie Mellon University, Pittsburgh, Pennsylva-

nia, USA.

Corresponding author: hanson.n@northeastern.edu

Project Code, Materials List, and Assembly Instructions: https://

github.com/RIVeR-Lab/slurp_grasping

Fig. 1: SLURP gripper mounted on a mobile manipulator (Stretch

RE1), surrounded by a variety of containers and contents that the

novel system can identify without touch.

Our solution to this problem leverages spectroscopy to see

into containers and understand their contents. Spectroscopy is

a technique adapted from the domain of analytical chemistry

used to determine the composition of objects through light

reﬂected off their surface. The proportions of light reﬂected,

transmitted, and absorbed at different wavelengths can be

used to ﬁngerprint a material’s properties and composition.

This paper introduces a new sensing platform for inferring

the unknown liquid contents inside an unknown container. To

accomplish this, we integrate micro near-infrared spectrom-

eters into a robot manipulator, enabling it to optically sense

liquids inside visually opaque containers without opening or

otherwise manipulating the container.

In this research, we focus on the design, development,

and validation of VNIR spectroscopy to perceive liquids

and granular solids in containers, shown in Fig 1. Our

work addresses the fundamental question of whether a liquid

can be identiﬁed in its container through spectroscopic

measurements. Our results demonstrate spectroscopy is a

powerful technique that can be used to recognize a variety

of containers and their contents with minimal training data.

The contributions of this paper are:

•Development of a gripper system to obtain VNIR spec-

tral signatures from solids and liquids inside containers.

•Dissemination of a dataset of complex container-content

spectral signatures.

•Demonstration of signal processing and a hierarchical

neural network to generate container and content pre-

dictions at grasp time.

arXiv:2210.04941v2 [cs.RO] 4 May 2023

II. RELATED WORK

A. Robot Liquid Perception

Liquid identiﬁcation remains an open and challenging

problem in robotics. Prior work has demonstrated good

results in identifying containers by their shape as a proxy to

identifying the internal substance [1], [2]. [3] used an RGB-

D camera to segment liquid level and measured refraction

to predict ﬁll contents. Others have presented methods of

identifying liquid or granular media in containers by mea-

suring the response of the contents when the container is

perturbed by an applied force [4], [5], [6], [7]. The utility of

these tactile estimation methods, however, is dependent on

the substrates having distinct viscosity (e.g. water vs. yogurt)

and classiﬁcation results for substances with similar viscosity

remains unknown.

The other core research thrust in liquid perception is the

modeling of poured liquids. Segmenting semi-transparent

liquid ﬂows [8], [9] has been studied to enable better con-

trol over liquids streams between containers. However, this

scenario presupposes that the liquid is in an open container,

and that pouring the liquid from one container to another is

desired and will not create an undue mess.

B. Liquid Spectroscopy

In traditional imaging, red, blue, and green channels sense

reﬂected light at three central wavelengths. Spectroscopy

increases both the number and speciﬁcity of the sensed wave-

lengths of light. The near infrared region (800 - 2500 nm) is

noted for the particular presence of anharmonic vibrational

modes where NIR light is absorbed and reﬂected in distinct

quantities as a function of the abundance of chemical bonds

[10]. Within the domain of NIR spectroscopy, transmis-

sion spectroscopy is typically used to analyze the chemical

composition of samples by measuring the attenuation of

light as it passes through a small liquid sample [11]. The

efﬁcacy of spectroscopic techniques has been demonstrated

in predicting fat and lactose contents in dairy milk [12], as

well as in estimating the freshness [13], composition [14],

and authenticity [15] of various foods. [16] provides an

excellent review of the breadth of liquid foods successfully

analyzed by NIR spectroscopy.

C. Robot Spectroscopy

The use of spectroscopy in robotics is an emerging ﬁeld

and has been demonstrated, often in combination with addi-

tional sensors, in a number of contexts including: classiﬁca-

tion of household objects [17], [18], ripeness evaluation of

mango fruit [19], in-hand object recognition [20], and ground

terrain characterization [21].

These research methods have all yielded highly accurate

classiﬁcation results on substances with distinct material

compositions. This current research is distinct from prior

work since it proposes spectroscopy as a useful method

to classify visually transparent materials. Previous research

focused on sensor design for opaque surfaces with no internal

contents. Our work adds emphasis on recognizing the inter-

nal contents of the container. To the authors’ best knowledge,

Fiber Optic Cable to !

Hamamatsu Spectrometer

QTH Bulb

Arduino Nano

Time of Flight

Sensor

Mantispectra

Board

Fig. 2: (Left) Real-world assembly of the SLURP gripper. (Right)

Exploded CAD rendering of SLURP gripper paddle showing inte-

grated visible to near infrared spectrometers and active illumination

associated physical gripper assembly.

this research is the ﬁrst to propose a non-contact method for

robots to estimate the internal contents of containers.

III. MATERIALS & METHODS

Unlike surface reﬂectance spectroscopy, which assumes

an opaque object surface, spectroscopy of liquids must also

work with a range of container transparencies. To this end,

we created a specialized gripper to provide active illumi-

nation in the VNIR range, under which containers exhibit

transparency [22]. We provide parts listings, models, and

assembly documentation in the project repository.

A. Optical System Design

We developed a parallel jaw gripper with a linear driver

(building on the ViperX 300 gripper from Trossen Robotics)

with integrated spectroscopy sensing. Fig. 2 depicts this

gripper and the key sensing components. We 3D printed

the paddles in Onyx [23] for added rigidity and thermal

stability. The dimensions of the modiﬁed sensor paddle are

106 mm ×69 mm ×51 mm with a contact surface area of

≈48 cm2. The large surface area provided requisite space to

mount sensors. Our system makes use of both paddles in our

signal acquisition procedure. First, light from a Quartz Tung-

sten Halogen (QTH) bulb provides active illumination onto

the nearby container surface. The speciﬁc bulb (ThorLabs

QTH10B) was selected since it provides even illumination

in the 400-2500 nm range. The bulb was operated at 6v and

exhibited an average surface temperature of 36◦C — well

within the safe working temperature of common containers.

The bulb is recessed from the surface plane of the paddle to

avoid direct contact with grasped items. A coated reﬂector

helps focus the illumination on the target item.

In contrast to our previous research which used sin-

gle sensors with primary sensitivity in the visible range,

this work incorporates two spectrometers into a gripper to

extend the sensed wavelength information. This redesign

is necessary to account for spectral variability caused by

the container. To capture the visible range, we utilize a

silicon-detector, micro-electro-mechanical system (MEMS)

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

SLURP!SpectroscopyofLiquidsUsingRobotPre-TouchSensingNathanielHanson1,WesleyLewis2,KavyaPuthuveetil2,DonelleFurline1,AkhilPadmanabha2,Tas¸knPadr1,ZackoryErickson2AbstractLiquidsandgranularmediaarepervasivethroughouthumanenvironments.Theirfree-owingnaturecausespeopletoconstrainthemintocontaine...

展开>> 收起<<

SLURP Spectroscopy of Liquids Using Robot Pre-Touch Sensing.pdf

共7页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

SLURP Spectroscopy of Liquids Using Robot Pre-Touch Sensing

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: