Throwing Objects into A Moving Basket While Avoiding Obstacles Hamidreza Kasaei1and Mohammadreza Kasaei2 Abstract The capabilities of a robot will be increased signif-

2025-05-06 0 0 5.79MB 7 页 10玖币

侵权投诉

Throwing Objects into A Moving Basket While Avoiding Obstacles

Hamidreza Kasaei1and Mohammadreza Kasaei2

Abstract— The capabilities of a robot will be increased signif-

icantly by exploiting throwing behavior. In particular, throwing

will enable robots to rapidly place the object into the target

basket, located outside its feasible kinematic space, without

traveling to the desired location. In previous approaches,

the robot often learned a parameterized throwing kernel

through analytical approaches, imitation learning, or hand-

coding. There are many situations in which such approaches

do not work/generalize well due to various object shapes,

heterogeneous mass distribution, and also obstacles that might

be presented in the environment. It is obvious that a method

is needed to modulate the throwing kernel through its meta-

parameters. In this paper, we tackle object throwing problem

through a deep reinforcement learning approach that enables

robots to precisely throw objects into moving baskets while

there are obstacles obstructing the path. To the best of our

knowledge, we are the ﬁrst group that addresses throwing

objects with obstacle avoidance. Such a throwing skill not only

increases the physical reachability of a robot arm but also

improves the execution time. In particular, the robot detects

the pose of the target object, basket, and obstacle at each time

step, predicts the proper grasp conﬁguration for the target

object, and then infers appropriate parameters to throw the

object into the basket. Due to safety constraints, we develop

a simulation environment in Gazebo to train the robot and

then use the learned policy in real-robot directly. To assess the

performers of the proposed approach, we perform extensive

sets of experiments in both simulation and real robots in

three scenarios. Experimental results showed that the robot

could precisely throw a target object into the basket outside

its kinematic range and generalize well to new locations and

objects without colliding with obstacles. The video of our

experiments can be found at https://youtu.be/VmIFF c 84

I. INTRODUCTION

Almost all humans are familiar with the ability to throw

objects, as we learn how to throw a ball during a game (e.g.,

basketball) or an object into a bin (e.g., tossing dirty clothes

into the laundry basket). We throw objects either to speed

up tasks by reducing the time of pick-and-place or to place

them in an unreachable place [1]. Therefore, adding such

a throwing motion to a robotic manipulator would enhance

its functionality too. In particular, throwing object is a great

way to use dynamics and increase the power of a robot by

enabling it to quickly place objects into the target locations

outside of the robot’s kinematic range. However, the act

of precisely throwing is actually far more complex than

it appears and requires a lot of practice since it depends

on many factors, ranging from pre-throw conditions (e.g.

initial pose of the object inside the gripper) to the physical

1Hamidreza Kasaei is with the Department of Artiﬁcial Intelligence,

Bernoulli Institute, Faculty of Science and Engineering, University of

Groningen, The Netherlands. Email: hamidreza.kasaei@rug.nl

2Mohammadreza Kasaei is with the School of Informatics, University

of Edinburgh, UK. Email: m.kasaei@ed.ac.uk

Fig. 1: An example scenario of throwing an object into a

moving basket located outside of the robot’s workspace while

an obstacle obstructing the path. To accomplish this task suc-

cessfully, the robot should perceive the environment through

its RGB-D camera, and then infer the proper parameters to

throw the object into the basket.

properties of the object (e.g. shape, size, softness, mass,

material, etc.). Many of these elements are challenging to

describe or measure analytically, hence, earlier research has

frequently been limited to assuming predeﬁned objects (e.g.,

ball) and initial conditions (e.g., manually placing objects

in a per-deﬁned location). When obstacles are present in

the environment and the target basket is moving, throwing

becomes even more difﬁcult. To the best of our knowledge,

we are the ﬁrst group to address such a challenging object

throwing problem.

To accomplish the throwing task successfully, a robot must

process the visual information to realize which objects exist

in the scene (i.e., target object, basket, and obstacles), what

are the state of the objects (i.e., pose, speed, etc.), and how to

grasp the target object (grasp synthesis). The robot then ﬁnds

an obstacle-free trajectory to grasp the object. Afterward,

given the obstacles that exist in the scene and the state of

the target basket, it needs to predict throwing parameters to

throw the object to the desired location precisely (e.g., the

velocity of executing the throw trajectory, time of release,

etc.). Lastly, the robot executes the throwing motion using

those parameters.

In this paper, we formulate object throwing as a RL

problem to enable the robot to generalize well across a

variety of objects and react quickly to dynamic environments

(i.e., moving basket). For RL, the exploration phase is often

arXiv:2210.00609v1 [cs.RO] 2 Oct 2022

unsafe in the real-world. It takes a while to build up enough

experience to train the policy to function successfully in

a dynamic environment with moving targets and obstacles.

Therefore, we develop a simulation in Gazebo, very similar

to our real-robot setup, and train the robot in Gazebo initially.

Afterwards, the learned policy is used in real-robot settings

directly. We extensively evaluate the performance of our

approach in both simulation and real-robot using three differ-

ent tasks with ascending levels of difﬁculties. Experimental

results show that the proposed method produces throws that

are more accurate than baseline alternatives. In summary, our

key contributions are threefold:

•To the best of our knowledge, we are the ﬁrst group that

addresses object tossing while obstacles are present in

the environment and the target basket is moving.

•Despite only trained using simulation data, the proposed

approach can be directly applied to real-robot. Further-

more, it shows impressive generalization capability to

new target locations and unseen objects.

•Our experiments show that the trained policy could

achieve above 80% object throwing accuracy for the

most difﬁcult task (i.e., throwing object into the basket

while there is an obstacle obstructing the path) in both

simulation and real robot environments.

II. RELATED WORK

The robotics community has long been interested in giving

service robots the ability to throw objects [2], [3], [4], [5],

[6]. Throwing formulae were mostly inﬂuenced by analyt-

ical models in the late 1990s and early 2000s [7], while

such formulations are increasingly moving toward learning

approaches today [8], [4]. In the following subsections, we

brieﬂy review these approaches.

A. Analytical Approaches

Earlier throwing systems relied on handcrafting or me-

chanical analysis and then optimizing control parameters to

execute a throw such that the projectile (typically a ball)

lands at a target location. As we previously highlighted, pre-

cisely modeling of dynamics is difﬁcult because it calls for

knowledge about the physical characteristics of the object,

gripper and environment, which are hard to quantify [7]. For

instance, Y. Gai et al., derived an analytical approach for

throwing a ball using a manipulator with a single ﬂexible

link through Hamilton’s principle [3]. This is an example

of tuning for a single object, a ball in this case. In another

work, Jwu-Sheng Hu et al., [2] discussed a stereo vision

system for throwing a ball into a basket. They calculated the

ball-throwing transformation for a speciﬁc ball object based

on cubic polynomial. In [9], an analytical approach is used

to predict the end-effector velocity (magnitude and direction)

as well as a duration movement for underhand throwing task

by a humanoid robot. Such approaches to some extend work

for speciﬁc scenario but have difﬁculties generalizing over

changing dynamics and various objects.

B. Learning Approaches

Unlike analytical approaches for throwing, learning-based

methods enable robots to learn/optimize the main task di-

rectly through success or failure signals. In general, learning-

based throwing approaches demonstrate better performance

than analytical methods[10], [11]. In [10], a deep predictive

policy training architecture (DPPT) is presented to teach

a PR2 robot object-grasping and ball-throwing tasks. They

showed DPPT is successful in both simulated and real robots.

In another work, Kober et al. [11] introduced an RL-based

method for dart throwing task based on a kernelized version

of the reward-weighted regression. In both of these works,

the properties of the object (ball and dart) are known a-

priori. In contrast to both of these approaches, we do not

make assumptions about the physical properties of objects

that are thrown.

In some other works, researchers tried to combine the

potential of analytical and learning approaches for robotic

throwing tasks. In particular, analytical models are used to

approximate the initial control parameters, and a learning-

based model is used to estimate residual parameters to adjust

the initial parameters. Such approaches are called residual

physics. For instance, [4] proposed TossingBot, an end-to-

end self-supervised learning method for learning to throw

arbitrary objects with residual physics. Similar to our work,

their approach was able to throw an object into a basket.

Unlike our approach, they used an analytical approach for

estimating initial control parameters, and then used an end-

to-end formulation for learning residual velocity for throwing

motion primitives. We formulate the throwing task as an RL

problem that modulates the parameters of a kernel motion

generator. In contrast to all reviewed works, our formulation

allows the robot to throw the object into a moving basket

while avoiding present obstacles, whereas, in all reviewed

works, the throwing task is considered in an obstacle-free

environment where the target is static and known in advance.

III. METHOD

In this section, the preliminaries are brieﬂy reviewed, fol-

lowed by a discussion of how we formulate object throwing

as an RL problem. The perception that represents the world

model at each time step is the subject of the last subsection.

A. Preliminaries

Markov Decision Process (MDP): An MDP can be

described as a tuple containing four basic elements:

(st, at, p(st+1|st, at), r(st+1|st, at)), where the stand atare

the continuous state and action at time step t, respectively.

p(st+1|st, at)shows the transition probability function to

reach to the next state st+1 given the current state stand

action at. The r(st+1|st, at)denotes the immediate reward

received from the environment after the state transition.

Off-policy RL: In online RL, an agent continuously

interacts with the environment to accumulate experiences for

learning the optimal policyπ∗. The agent seeks to maximize

the expected future return Rt=E[P∞

i=tγi−tri+1]with a

discounted factor γ∈[0,1] weighting the future importance.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

ThrowingObjectsintoAMovingBasketWhileAvoidingObstaclesHamidrezaKasaei1andMohammadrezaKasaei2AbstractThecapabilitiesofarobotwillbeincreasedsignif-icantlybyexploitingthrowingbehavior.Inparticular,throwingwillenablerobotstorapidlyplacetheobjectintothetargetbasket,locatedoutsideitsfeasiblekinematicspac...

展开>> 收起<<

Throwing Objects into A Moving Basket While Avoiding Obstacles Hamidreza Kasaei1and Mohammadreza Kasaei2 Abstract The capabilities of a robot will be increased signif-.pdf

共7页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Throwing Objects into A Moving Basket While Avoiding Obstacles Hamidreza Kasaei1and Mohammadreza Kasaei2 Abstract The capabilities of a robot will be increased signif-

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: