RecipeMind Guiding Ingredient Choices from Food Pairing to Recipe Completion using Cascaded Set Transformer

2025-04-29 0 0 1010.03KB 11 页 10玖币
侵权投诉
RecipeMind: Guiding Ingredient Choices
from Food Pairing to Recipe Completion
using Cascaded Set Transformer
Mogan Gim ∗
Korea University
Seoul, Korea
akim@korea.ac.kr
Donghee Choi ∗
Korea University
Seoul, Korea
choidonghee@korea.ac.kr
Kana Maruyama
Sony AI
Tokyo, Japan
Kana.Maruyama@sony.com
Jihun Choi
Sony AI
Tokyo, Japan
Jihun.A.Choi@sony.com
Hajung Kim
Korea University
Seoul, Korea
hajungk@korea.ac.kr
Donghyeon Park †
Sejong University
Seoul, Korea
parkdh@sejong.ac.kr
Jaewoo Kang †
Korea University
Seoul, Korea
kangj@korea.ac.kr
ABSTRACT
We propose a computational approach for recipe ideation, a down-
stream task that helps users select and gather ingredients for cre-
ating dishes. To perform this task, we developed RecipeMind, a
food anity score prediction model that quanties the suitability
of adding an ingredient to set of other ingredients. We constructed
a large-scale dataset containing ingredient co-occurrence based
scores to train and evaluate RecipeMind on food anity score pre-
diction. Deployed in recipe ideation, RecipeMind helps the user
expand an initial set of ingredients by suggesting additional ingre-
dients. Experiments and qualitative analysis show RecipeMind’s
potential in fullling its assistive role in cuisine domain.
CCS CONCEPTS
•Computing methodologies
→
Knowledge representation
and reasoning;•Applied computing;
KEYWORDS
Recipe Ideation, Computational Cooking, Ingredient Set Expansion,
Food Anity Score, Cascaded Set Transformer, Recipe Context
ACM Reference Format:
Mogan Gim
∗
, Donghee Choi
∗
, Kana Maruyama , Jihun Choi , Hajung
Kim , Donghyeon Park
†
, and Jaewoo Kang
†
. 2022. RecipeMind: Guiding
Ingredient Choices from Food Pairing to Recipe Completion using Cascaded
Set Transformer. In Proceedings of the 31st ACM International Conference on
Information and Knowledge Management (CIKM ’22), October 17–21, 2022,
Atlanta, GA, USA. ACM, New York, NY, USA, 11 pages. https://doi.org/10.
1145/3511808.3557092
∗Equal Contributors.
†Corresponding authors.
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specic permission and/or a
fee. Request permissions from permissions@acm.org.
CIKM ’22, October 17–21, 2022, Atlanta, GA, USA
©2022 Association for Computing Machinery.
ACM ISBN 978-1-4503-9236-5/22/10. . . $15.00
https://doi.org/10.1145/3511808.3557092
Figure 1: Overview of Recipe Ideation. RecipeMind guides
the user’s choices from food pairing (Step 1) shown on left
side in Recipe Ideation) to recipe completion (Step N). In the
middle of ideation (Step 3), RecipeMind recommends adding
baking soda to current set based on its score predictions.
1 INTRODUCTION
Professional chefs and home cooks have pursued to create new
dishes and formulate novel recipe ideas which are important tasks
in culinary domain. Some recipe ideas derive from existing dishes
while others are created from novel ingredient combinations. As
a recipe comprises a set of ingredients and sequence of cooking
instructions, one may want to brainstorm a recipe starting with a set
of few ingredients and consecutively expanding it with additional
ones. As illustrated in Figure 1 starting with buttermilk, we dene
these consecutive steps of selecting ingredients (our,chocolate
chips,baking soda) as recipe ideation, gradually leading to recipe
completion (Chocolate Chip Cookies).
CIKM ’22, October 17–21, 2022, Atlanta, GA, USA Mogan Gim et al
Recipe ideation is challenging due to vast space of cooking possi-
bilities and complexity for avor yet important for creative cooking
in culinary domain [44]. As multiple ingredients used for cooking
a dish form recipe context [
38
], choosing the right ingredient re-
quires comprehensive understanding in culinary aspects such as
aroma or avor [
13
]. A systematic approach towards recipe ideation
would involve initiating with the most basic recipe idea consisting
few ingredients and iteratively updating it with new additional
ingredients that goes well with its overall recipe context.
Computational approaches for assisting recipe ideation pro-
cess have recently become necessary to solve these issues in culi-
nary domain [
21
]. For instance, Kitchenette predicts food pairing
scores [
40
] and RecipeBowl retrieves the best ingredient given a
nearly completed recipe [
24
] which are deemed as earliest and
latest stage of recipe ideation respectively. A more versatile compu-
tational approach deployable in any stages of recipe ideation may
be desirable.
In this work, RecipeMind performs the ideation task by quantify-
ing the suitability of adding an ingredient to set of other ingredients.
Henceforth, we make the following denitions and formulate the
objective of recipe ideation task prior to introducing our proposed
model RecipeMind.
•
Ingredient Space
U
: A union space containing available in-
gredients for recipe ideation.
•
Ingredient Set
S
:Annite subset
S∈U
containing ingredi-
ents. In addition,
S
is a
-sized ingredient set where
|S|=
and ≥1.
•
Additional Ingredient
: A single ingredient to be added to
current ingredient set where ∈U.
•
Food Anity Score
: A score that quanties the suitability
of adding to Sresulting S+1=S∪{}where ∈R.
Problem 1 (Recipe Ideation Task). We dene the objective of
Recipe Ideation Task as nite steps of sequential ingredient set
expansion. Each step involves expanding the ingredient set
S
by
adding another ingredient which results in S+1=S∪{}.
To solve the above problem, we introduce the following two
tasks.
Task 1 (Food Affinity Score Prediction). Given an ingredient
set
S
and additional ingredient
, RecipeMind
predicts the food
anity score between Sand {}through modeling =(S,).
Task 2 (Additional Ingredient Recommendation). Given an
ingredient set
S
, all possible ingredients
∈U−S
and RecipeMind
, the recommended ingredient
to be added to
S
is based on the
top-ranked anity score predictions made by RecipeMind.
=arg max
(S,)(1)
To train our RecipeMind model, we constructed a large-scale
dataset where each data instance is dened as
(S,,)
. The data
instances were built from the ingredient subset co-occurrences in
the Reciptor dataset containing 507,834 recipes and 2,391 ingredi-
ents deemed as Ingredient Space
U
[
27
]. The food anity scores
were calculated based on Signicant PMI based on Document Count
and were applied to our recipe ideation task [15, 16].
We adopted the Set Transformer framework when developing
the model architecture of RecipeMind [
26
]. To help RecipeMind
jointly learn cross-relational features between ingredients in
S
and
, we developed Cascaded Set Transformer using Pooling by
Multihead Cross-Attention (PMX).
We evaluated RecipeMind’s food anity score prediction through
baseline and ablation experiments with expanding ingredient sub-
sets including unseen sizes in training set. We further analyzed
the recommendation results and attention heatmaps after deploy-
ing RecipeMind in example recipe ideation scenarios to explore its
understanding in recipe contexts.
As shown in Figure 1, RecipeMind encompasses from food pair-
ing to recipe completion as it chooses the most suitable ingredient
choices given any number of ingredients in current set. To the
best of our knowledge, this work is the rst attempt to introduce a
data-driven approach that assists ingredient choices at any stage
in recipe ideation. The major contributions of our work can be
summarized as follows,
•
We formulated a downstream task called recipe ideation
which features and food anity score prediction and addi-
tional ingredient recommendation.
•
We created a large-scale dataset that contains anity scores
for each pair of
-sized ingredient set and additional ingre-
dient.
•
We developed RecipeMind utilizing Cascaded Set Transform-
ers using Pooling by Multihead Cross-Attention.
•
We empirically demonstrated RecipeMind’s robustness in
expanding set sizes through experiments and analyzed its
understanding in recipe contexts throughout recipe ideation
scenarios 1.
2 RELATED WORKS
2.1 Representation Learning for Recipes
Previous works that have introduced various deep learning ap-
proaches for improved representation learning of cooking recipes.
Cross-modal or multimodal approaches incorporating recipe texts
with images have been introduced where some focused on improv-
ing representation learning on recipes [
6
,
8
,
32
,
33
,
37
,
48
,
50
,
53
,
54
].
Others utilized these features to improve recipe retrieval tasks [
7
,
9
,
29
,
58
]. Few works have attempted to apply cross-modal feature
learning to recipe numeracy tasks such as predicting calories [
28
]
or food ingredient amounts [18].
Meanwhile, as recipes can be expressed as sets of ingredient,
recent works have proposed set representation learning methods
to eectively learn recipe-related contextual features [
24
,
27
]. Our
work also proposes to apply set representation learning to recipe
ideation since it is crucial to understand various recipe context and
determine the optimal ingredient to be added for the next step.
2.2 Previous Approaches related to Recipe
Ideation
One of the downstream tasks related to our work is food pairing
which can be deemed as the fundamental form of recipe ideation.
Computational methods for food pairing have been introduced in-
cluding Kitchenette. These methods utilize statistical co-occurrences
1
The source code and demo web page for RecipeMind are open for public access.
(https://github.com/dmis-lab/RecipeMind , https://recipemind.korea.ac.kr)
RecipeMind: Guiding Ingredient Choices using Cascaded Set Transformer CIKM ’22, October 17–21, 2022, Atlanta, GA, USA
Figure 2: Distribution of sPMIr scores for each size of
S=
S−1∪{}
where
≥
2. The mean values of sPMIr-based
anity scores are shown in the x-axis for each size of ingre-
dient subset. As
increases, the statistic mean of food anity
scores shifts toward negative while the overall distribution
becomes non-normal.
or chemical similarities of ingredient pairs [
1
,
39
,
40
]. Another
downstream task related to our work is recipe completion [
14
,
17
,
24
]. These methods optimized data-driven models to predict
ingredients given partial or nearly completed recipes. Our work en-
compasses both tasks as recipe ideation aims to guide users’ choices
on ingredient additions given dierent ingredient sets.
Other works have suggested approaches for generating recipes
containing cooking instructions and/or ingredients. Various gen-
erative models using dierent modalities of queries such as food
images, cooking videos and texts have been developed [
19
,
22
,
43
,
52
,
57
]. Recently, a system for recipe editing considering dietary
constraints was introduced [
10
]. While recipe ideation and genera-
tion have common traits such as creativeness, RecipeMind guides
users’ ingredient choices step by step based on its predicted food
anity scores for all expanding sizes of ingredient sets.
3 DATASET
3.1 Obtaining n-sized Ingredient Subsets from
Reciptor Dataset
Table 1 shows the dataset statistics involving
-sized ingredient
subsets obtained from the Reciptor dataset. We extracted
-sized
ingredient subsets from the Reciptor dataset containing 507,834
recipes which was originally used by Reciptor [
27
]. We adopted
Kitchenette’s approach by removing ingredients whose occurrence
count does not exceed 20 [
40
]. As a result, we obtained 2,391 unique
ingredients and used them to generate
-sized subsets based on
their co-occurrence statistics in the Reciptor dataset [27].
While the number of possible 2-sized subsets (doublets) in our
dataset is
(2391
2)=
2
,
857
,
245, we adopted Kitchenette’s approach
by selecting doublets exceeding 5 occurrences in the dataset and
obtained 236,297. The same criteria was applied to obtaining 3
and 4-sized subsets which resulted in a total of 1,226,767 among
(2391
3)
and 1,952,345 among
(2391
4)
possible subsets. Furthermore,
we obtained all available 5,6 and 7-sized ingredient subsets for only
testing purposes. Right side of Table 1 shows the obtained numbers
of -sized ingredient subsets.
3.2 Constructing Data Instances
For each
-sized ingredient subset obtained from the Reciptor
dataset, we rstly split each
-sized ingredient set
S
into a pair of
S−1
and its additional ingredient
where the total number of com-
binations is
. Therefore, the total number of data instances built
from ingredient doublets, triplets and quadruplets is 236
,
297
×
2,
1
,
226
,
767
×
3and 1
,
952
,
345
×
4respectively. The doublet-based data
instances are identical to ingredient pairings as the input ingredient
set is a singleton. Left side of Table 1 shows the actual numbers of
data instances contained in our dataset.
3.3 Generating sPMIr-based Food Anity
Scores
We generated the anity scores for each data instances consisting
a ingredient set and its additional ingredient. Kitchenette used Nor-
malized Point-wise Mutual Information (NPMI) [
5
,
40
,
49
] to formu-
lated food pairing scores. The scores represent the co-occurrence be-
tween two dierent ingredients and intuitively determine whether
each ingredient pair is suitable or not.
In this work, we adopted Damani’s Signicant PMI based on Docu-
ment Count (sPMId) score, an improved version of PMI considering
statistical signicance [
15
,
16
]. The formulation of the sPMId score
between words x,yis mathematically expressed as,
sPMId(x,y)=log (x,y)
(x)(y)
N+max((x),(y)) ∗√ln
−2.0
(2)
where
(
x
)
is the number of documents that contain xat least once,
is the total number of documents and
is the parameter varying
between 0 and 1. Prior to applying the sPMId-based score formula-
tion to our task, we substituted words with ingredient subsets of
varying sizes.
We propose a modied approach compatible with ingredient
subsets used in the Reciptor dataset [
27
]. The modications are the
following,
•
The documents used to calculate occurrences are the recipes
which contain a full list of ingredients used for cooking.
•
Given two disjoint ingredient subsets Xand Y(
|
X
∩
Y
|=
0),
a co-occurrence measure is dened based on their union’s
occurrence (X∪Y).
•
As our task involves adding one ingredient to current to a
-
sized ingredient set, we introduced Signicant PMI based on
Recipe Count (sPMIr) to formulate anity scores for training
RecipeMind using the above modications.
The calculation of a sPMIr-based anity score when adding an
ingredient to -sized ingredient set Sis expressed as,
sPMIr(S,)=log (S⋉∪{})
(S)({} )
R+max((S),({})) ∗√ln
−2.0
(3)
where
(
X
)
is the number of recipes that used ingredient set X
at least once,
is the total number of recipes,
is set as 0.2 and
|S∩{}| =
0since our task involves adding new ingredients.
Unlike the pairing scores originally used in Kitchenette [
40
], the
food anity scores are not bounded since they are not normalized.
RecipeMind Guiding Ingredient Choices from Food Pairing to Recipe Completion using Cascaded Set Transformer.pdf

共11页,预览3页

还剩页未读， 继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

相关推荐

更多
立即下载
分类：图书资源 价格：10玖币 属性：11 页 大小：1010.03KB 格式：PDF 时间：2025-04-29

开通VIP享超值会员特权

  • 多端同步记录
  • 高速下载文档
  • 免费文档工具
  • 分享文档赚钱
  • 每日登录抽奖
  • 优质衍生服务

作者详情

相关内容

更多

热门标签

人际关系 动力学连接体 力的合成 配电装置 高考理综 全宋诗作者索引 公务员考试
/ 11
评分 收藏
分享
立即下载
客服
关注