Retrieving Users Opinions on Social Media with Multimodal Aspect-Based Sentiment Analysis Miriam Ansch utz

2025-05-01 0 0 4.71MB 8 页 10玖币

侵权投诉

Retrieving Users’ Opinions on Social Media with

Multimodal Aspect-Based Sentiment Analysis

Miriam Ansch¨

utz

Faculty of Informatics

Technical University of Munich

Munich, Germany

miriam.anschuetz@tum.de

Tobias Eder

Faculty of Informatics

Technical University of Munich

Munich, Germany

tobias.eder@in.tum.de

Georg Groh

Faculty of Informatics

Technical University of Munich

Munich, Germany

grohg@in.tum.de

Abstract—People post their opinions and experiences on social

media, yielding rich databases of end-users’ sentiments. This

paper shows to what extent machine learning can analyze and

structure these databases. An automated data analysis pipeline

is deployed to provide insights into user-generated content for

researchers in other domains. First, the domain expert can select

an image and a term of interest. Then, the pipeline uses image

retrieval to ﬁnd all images showing similar content and applies

aspect-based sentiment analysis to outline users’ opinions about

the selected term. As part of an interdisciplinary project between

architecture and computer science researchers, an empirical

study of Hamburg’s Elbphilharmonie was conveyed. Therefore,

we selected 300 thousand posts with the hashtag ‘hamburg’

from the platform Flickr. Image retrieval methods generated

a subset of slightly more than 1.5 thousand images displaying

the Elbphilharmonie. We found that these posts mainly convey

a neutral or positive sentiment towards it. With this pipeline,

we suggest a new semantic computing method that offers novel

insights into end-users opinions, e.g., for architecture domain

experts.

Index Terms—Image retrieval, Flickr, multimodal, Opinion

mining, Social media analysis

I. INTRODUCTION

Exceptional architecture or star architecture are buildings

commissioned for their high recognition value and iconicity.

These buildings have a unique design, were designed by a

famous architect, or are visually contrasted with their sur-

roundings. They often cause a shift in the scale, spatiality,

or content attention about the city they are located in [1].

The propagation of images is a central part of the generation

and increase of iconicity. In times of social media, one

image can travel around the world within minutes. Therefore,

analyzing social media data has become crucial for scientists,

e.g., architects who want to investigate the viral effects of

a new building [2], [3]. On social media platforms, any

user can post images of such buildings and express their

opinion towards them. This offers the opportunity to retrieve

unconstrained opinions by any person, i.e., ones that are not

restricted by a questionnaire nor inﬂuenced by a biased study

design. In addition, multiple user groups can be observed on

social media, such as tourists that post about their vacation or

local citizens sharing spots in their home city. Another beneﬁt

of using social media data for analyzing user opinions is the

amount of available data.

Figure 1: Elbphilharmonie press image © Maxim Schulz [4].

However, without the help of automated tools, domain

experts have to review and interpret the data manually. When

conducting large-scale studies, this results in an infeasible

amount of work. Therefore, automated approaches are in-

dispensable for handling large volumes of data. In addition,

automated approaches can be re-applied to other datasets or

domains, making studies more comparable and reducing the

effort even further. A popular automation method is the use

of machine learning algorithms. Mainly due to the diversity

of textual data, handwritten rules for automation fail to cover

the full scope of information in the data. In contrast, machine

learning can capture advanced concepts, e.g., the semantics

of texts, or retrieve latent information such as the underlying

topic distribution. Therefore, machine learning is proposed

for an automated survey on social media data to provide

architects access to an amount of data that would otherwise

be inaccessible to them.

We attempt to show how machine learning can structure

big data and yield interpretations and possible conclusions

based on the data. Therefore, we conveyed an empirical

study on social media data to investigate different opinions

towards the Elbphilharmonie in Hamburg (see Figure 1).

The posts about the city of Hamburg were obtained from

the image-sharing platform Flickr. The Elbphilharmonie is a

philharmonic concert hall by the architecture ﬁrm Herzog &

de Meuron, inaugurated in 2017. Hamburg is a Hanseatic

city in the north of Germany and has the largest port in

its country. Over many years, the city developed northwards,

away from the Elbe river and the harbor. In the 1990s, the

arXiv:2210.15377v2 [cs.IR] 9 Jan 2023

Hamburg Senate decided to revitalize the former warehouse

district at the Elbe riverbank that was abandoned due to the

containerization of goods, making the storage capacities of the

warehouses superﬂuous. This revitalization project was called

HafenCity and yielded a mixed-use urban district with the

Elbphilharmonie, the international maritime museum, or the

HafenCity university being part of it. The Elbphilharmonie

was designed to serve as an icon for this cultural upgrade

and a new landmark for Hamburg. The bottom of the building

is an old brick warehouse. On top of that is a modern glass

construction that imitates a hoisted sail or the sea’s waves.

This unique architecture stands out. However, many voices

were raised that this modern architecture disturbs the view on

the historic part of the city. Moreover, the construction cost of

the Elbphilharmonie was e866 million, several times more

expensive than the initially estimated price. Consequently,

the Elbphilharmonie is a controversial building, admired and

criticized simultaneously [5].

This paper aims to provide an overview to the different

opinions communicated on social media. The proposed data

pipeline processes domain-speciﬁc social media data and

yields a structured data analysis. As part of this pipeline,

the domain expert can select an image of a building as the

aspect of interest. Then, all images in the dataset depicting

the same building are retrieved, and a message-level and

aspect-based sentiment analysis is conducted on these posts.

Therefore, this contribution is two-fold: On the one hand, a

new approach for multimodal aspect-based sentiment analysis

on social media data is proposed. On the other hand, this

approach was proven effective in an interdisciplinary project

between domain experts and computer scientists to conduct an

empirical study about the Elbphilharmonie in Hamburg. The

proposed pipeline can be applied to any building by selecting

other query images and aspects of interest. In addition, it can

be transferred to unseen data by extracting image features from

the respective data and updating the queries accordingly. Our

code and dataset are published on Github1.

The remaining paper is structured as follows: Section II

discusses previous approaches towards image retrieval on

landmark images, sentiment analysis on social media, and the

combination of both. Section III showcases the study design

and the resulting dataset from the platform Flickr. Finally, in

sections IV and V, different image retrieval and sentiment

analysis methods are compared on test datasets, and the best-

performing ones are applied to ﬁlter the Flickr dataset.

II. RELATED WORK

Social media and online review data have been used to mine

users’ opinions in different domains, for example, opinions

towards a speciﬁc brand [6], [7]. The methods used in these

studies include topic modeling [8] or aspect-based sentiment

analysis [6] and focus on textual data. To account for the

multimodal nature of social media posts, the authors in [9]

included the posts’ images in their study by clustering them

1https://github.com/MiriUll/multimodal ABSA Elbphilharmonie

based on their depicted content. As a result, they could identify

different brands and products in the images and applied

sentiment analysis on the accompanying texts. Similarly, Fang

et al. [10] extracted aspects from images, such as buildings,

and performed aspect-based sentiment analysis on them. These

approaches focus on retrieving different targets addressed

in the data. In contrast, the authors in [11] conducted a

case study about a speciﬁc, pre-deﬁned target, the Rinjani

mountain, a popular tourist place in Indonesia. They selected

images portraying the mountain in question from social media

platforms and performed a dictionary-based sentiment analysis

on the image descriptions to retrieve the tourists’ opinions

towards this destination. This paper reports on a similar study.

However, our advanced image-based aspect selection and

sentiment analysis approaches yield a more in-depth analysis.

A. Image retrieval on landmark images

Image retrieval is the task of ﬁnding images showing similar

objects in an extensive database of images. The images are

transformed into a feature space, and the features are compared

to ﬁnd similar contents. In contrast to image classiﬁcation,

where models must classify all images of a class regardless

of the intra-class diversity, the image retrieval features must

account for precisely these differences [12]. Traditional tech-

niques, such as scale-invariant features transform (SIFT) [13]

or KAZE [14], describe images based on distinctive locations

and interest points in them [15]. To build a global feature

vector based on these local properties, the descriptors are

aggregated, e.g., by clustering them into visual words as in the

vectors of locally aggregated descriptors (VLAD) [16]. Other

approaches learn image representations with neural networks

by ﬁne-tuning pre-trained classiﬁcation models for the retrieval

task. To ﬁne-tune for retrieval on landmark images, the authors

in [17] published the Google landmark dataset and trained

their deep local features (DELF) model on it. Other landmark

retrieval models are the average precision model [18] or the

deep local and global features (DELG) model [19].

B. (Aspect-based) sentiment analysis

In this paper, two different types of sentiments are analyzed.

The message-level sentiment describes the overall sentiment

of a post. In contrast, aspect-based models investigate the

sentiment about a speciﬁc word or phrase in the post. With

this, the models can retrieve opinions about a speciﬁc topic,

independent of the overall sentiment of a message [20]. Neural

networks are a popular method to train classiﬁers for sentiment

prediction because their nested structure can perform an in-

depth analysis of the input data and therefore gain a good

understanding of complex text features [21]. The Interna-

tional Workshop on Semantic Evaluation 2017 (SemEval-

2017) featured a task about sentiment analysis in Twitter

posts [22]. At this task, ensemble models, i.e., models that

combine different layer types, were among the most popular

and successful competitors. Among them are combinations of

convolutional neural networks (CNNs) and long short-term

memory networks (LSTMs) [23], [24] and a combination of

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

RetrievingUsers'OpinionsonSocialMediawithMultimodalAspect-BasedSentimentAnalysisMiriamAnsch¨utzFacultyofInformaticsTechnicalUniversityofMunichMunich,Germanymiriam.anschuetz@tum.deTobiasEderFacultyofInformaticsTechnicalUniversityofMunichMunich,Germanytobias.eder@in.tum.deGeorgGrohFacultyofInformatics...

展开>> 收起<<

Retrieving Users Opinions on Social Media with Multimodal Aspect-Based Sentiment Analysis Miriam Ansch utz.pdf

共8页,预览2页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

Retrieving Users Opinions on Social Media with Multimodal Aspect-Based Sentiment Analysis Miriam Ansch utz

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: