Opinion Mining on the Web 2.0 Characteristics of User Generated Content and Their Impacts - PDF

Please download to get full document.

View again

of 12
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Information Report
Category:

Internet & Technology

Published:

Views: 13 | Pages: 12

Extension: PDF | Download: 0

Share
Description
Opinion Mining on the Web 2.0 Characteristics of User Generated Content and Their Impacts Gerald Petz 1, Michał Karpowicz 1, Harald Fürschuß 1, Andreas Auinger 1, Václav Stříteský 2, and Andreas Holzinger
Transcript
Opinion Mining on the Web 2.0 Characteristics of User Generated Content and Their Impacts Gerald Petz 1, Michał Karpowicz 1, Harald Fürschuß 1, Andreas Auinger 1, Václav Stříteský 2, and Andreas Holzinger 3 1 University of Applied Sciences Upper Austria, Campus Steyr, Austria {gerald.petz,michal.karpowicz,harald.fuerschuss, 2 University of Economics, Prague 3 Medical University Graz, Medical Informatics, Statistics and Documentation, Austria Abstract. The field of opinion mining provides a multitude of methods and techniques to be utilized to find, extract and analyze subjective information, such as the one found on social media channels. Because of the differences between these channels as well as their unique characteristics, not all approaches are suitable for each source; there is no one-size-fits-all approach. This paper aims at identifying and determining these differences and characteristics by performing an empirical analysis as a basis for a discussion which opinion mining approach seems to be applicable to which social media channel. Keywords: opinion mining, user generated content, sentiment analysis, text mining, content extraction, language detection, Internet slang, text mining. 1 Introduction and Motivation for Research Opinion mining (some authors use sentiment analysis synonymously), deals with analyzing people s opinions, sentiments, attitudes and emotions towards different brands, companies, products and even individuals [1], [2]. The rise of the Web 2.0 and its user generated content led to many changes of the Internet and its usage, as well as a change in the communication processes. The user created content on the Web 2.0 can contain a variety of important market research information and opinions, through which economic opportunities as well as risks can be recognized at an early stage. Some of the challenges for qualitative market research on the Web 2.0 are on the one hand the variety of information and on the other hand the huge amount of rapidly growing and changing data. Besides the typical challenges known from natural language processing and text processing, many challenges for opinion mining in social media sources make the detection and processing of opinions a complicated task: A. Holzinger and G. Pasi (Eds.): HCI-KDD 2013, LNCS 7947, pp , Springer-Verlag Berlin Heidelberg 2013 36 G. Petz et al. Noisy texts: User generated contents in social media tend to be less grammatically correct, they are informally written and have spelling mistakes. These texts often make use of emoticons and abbreviations or unorthodox capitalisation [3], [4]. Language variations: Texts in user generated content typically contain irony and sarcasm; texts lack contextual information but have implicit knowledge about a specific topic [5]. Relevance and boilerplate: Relevant content on webpages is usually surrounded by irrelevant elements like advertisements, navigational components or previews of other articles; discussions and comment threads can divert to non-relevant topics [5 7]. Target identification: Search-based approaches to opinion mining often face the problem that the topic of the retrieved document does not necessarily match the mentioned object [5]. In the field of opinion mining, where language-specific tools, algorithms and models are frequently utilized, these challenges have quite an important impact on the properness of results, since the application of improper methods leads to incorrect or worse sentiment analysis results. 1.1 Objective and Methodology The objective of this paper is to investigate the differences between social media channels and to discuss the impacts of their characteristics to opinion mining approaches. To attain this objective, we set up a methodology as follows: (i) (ii) (iii) (iv) In the first step, we identify the most popular approaches for opinion mining in the scientific field and their underlying principles of detecting and analyzing text. As a second step we identify and deduce criteria from literature to exhibit differences between the different kinds of social media sources regarding possible impacts on the quality of opinion mining. Subsequently, we carry out an empirical analysis based on the deduced criteria in order to determine the differences between several social media channels. The social media channels taken into consideration in the third step are: social network services (Facebook), microblogs (Twitter), comments on weblogs and product reviews (Amazon and other product review sites). In the last step, the social media source types need to be correlated with applicable opinion mining approaches based on their respective characteristics. The next section gives a short overview about related work and approaches of opinion mining; section 3 describes the empirical analysis and discusses impacts of the characteristics of user generated content to opinion mining. Opinion Mining on the Web 2.0 Characteristics of User Generated Content 37 2 Background, Related Work Opinion mining deals with different methods and algorithms from computational linguistics and natural language processing in order to find, extract and analyze people s opinions about certain topics. 2.1 Opinion Definition Liu defines an opinion as a quintuple (e i, a ij, s ijkl, h k, t l ), where e i is the name of an entity, a ij is an aspect of e i, s ijkl is the sentiment on aspect a ij of entity e i, h k is the opinion holder and t l is the time, when the opinion is expressed. An entity is the target object of an opinion; it is a product, service, topic, person, or event. The aspects represent parts or attributes of an entity (part-of-relation). The sentiment is positive, negative or neutral or can be expressed with intensity levels. The indices i, j, k, l indicate that the items in the definition must correspond to one another [1]. 2.2 Main Research Directions and Technical Approaches Several main research directions can be identified [2], [8]: (1) Sentiment classification: The main focus of this research direction is the classification of content according to its sentiment about opinion targets; (2) Feature-based opinion mining (or aspect-based opinion mining) is about analysis of sentiment regarding certain properties of objects (e.g. [9], [10]) (3) Comparison-based opinion mining deals with texts in which comparisons of similar objects are made (e.g. [11]). Opinion mining has been investigated mainly at three different levels: document level, sentence level and entity/aspect-level. Most classification methods are based on the identification of opinion words or phrases. The underlying algorithms can be categorized as follows: (1) Supervised learning (e.g. [12], [13]), (2) Unsupervised learning (e.g. [14]), (3) Partially supervised learning (e.g. [15]), (4) Other approaches / algorithms like latent variable models (hidden Markov model HMM [16]), conditional random fields CRF [17]), latent semantic association [18], pointwise mutual information (PMI) [19]. Due to the amount of different techniques, several researchers experimented with different algorithms and drew comparisions between them: [20 22]. 2.3 Opinion Mining and Web 2.0 A couple of research papers focus explicitly on Web 2.0: A considerably amount of research work covers weblogs, e.g. [23 26], but most of them investigate the correlation between blog posts and real life -situations. Only a few papers evaluate techniques for opinion mining in the context of weblogs; there is no main direction of used techniques. Liu et al. [27] compare different linguistic features for blog sentiment classification, [28] experimented with lexical and sentiment features and different learning algorithms for identifying opinionated blogs. Surprisingly, little research work can be found about opinion mining in the area of discussion forums 38 G. Petz et al. (e.g. [29], [30]). However, microblogs in particular Twitter seem to be quite attractive to researchers and a variety of papers focussing on microblogs have been published, e.g. [31 35]. The researchers mainly use supervised learning or semisupervised learning as the dominant approach to mine opinions on microblogs. Despite the popularity of social network services like Facebook, relatively little research work about opinion mining in social networks can be found (e.g. [36], [37]). There are numerous research papers that deal with product reviews, and there is not one specific approach that seems to perform best. Many authors use text classification algorithms like SVM or Naïve Bayes and combine different techniques to increase the quality of opinion mining results. A promising technique could be LDA (e.g. [38], [39]). [40] proposed an LDA-based model that jointly identifies aspects and sentiments. This model (also e.g. the approach of [41], [42]) assumes that all of the words in a sentence cover one single topic. 3 Research Work and Results We conducted an empirical analysis in order to find differences between social media channels. The following section describes the empirical analysis as well as the impacts of user generated content on opinion mining. 3.1 Empirical Analysis Methodology of Survey. When starting the empirical analysis, it lends itself to asking the question of how an appropriate sample should to be drawn in order to conduct a representative survey. Basically, a random sample is reasonable, but it is actually a challenge to draw a random sample. Therefore, we have decided to draw a sample of self-selected sources and to make a kind of quota sampling. In order to avoid confounders, systematic errors and bias we define the following constraints: we focus on one specific brand / company (in our case: Samsung) and on a specific time period (in our case: between June, 15 th 2011 and Jan, 28 th 2013) for all sources in social media. Within this time period we conduct a comprehensive survey; if there are too many entries to perform a comprehensive survey, we draw a random sample of the entries. As we do not want to analyze the official postings of the company, we exclude these postings from the analysis. The data sets were labeled manually by four different human labelers. Before the labeling started, we discussed and defined rules for labeling in order to make the labeling consistent among the labelers [11]. The statistical calculations were carried out using SPSS. The following sources have been surveyed in four different languages: social network service (Facebook; 410 postings), microblog (Twitter; 287 tweets), blog (387 blog posts), discussion forum (417 posts from 4 different forums) and product reviews (433 reviews from Amazon, and two product review pages). The collection of the data was performed manually for the discussion forums and automated using the API (Twitter, Facebook) and a Web-crawler for the other sources (Amazon). Opinion Mining on the Web 2.0 Characteristics of User Generated Content 39 Evaluation Criteria. In order to compare different social media channels, we need to determine indicators. These indicators shown in table 1 are derived from two sources: (i) criteria based on simple frequencies from content analysis, and (ii) criteria derived from the definition of opinions (see section 2.1): Table 1. Evaluation criteria Criteria Description Scale type [43] Language Describes the language used, e.g. English, Nominal German, etc. Number of How many words does a posting (e.g. blog Metric words posting, Facebook-post, product review, comment, etc.) contain? Number of sentences How many sentences does a posting contain? Metric Number of Internet slang abbreviations Number of emoticons Number of incorrect sentences Subjectivity Opinion holder Opinion expression Topic-related Aspect How many typical Internet slang abbreviations (e.g. LOL, IMO, IMHO ) does the posting contain? How many emoticons (e.g. ;-) :-) :-o ) does the posting contain? How many sentences contain grammatical and orthographical mistakes or typos per posting? Does the posting contain an opinion? Is the posting subjective or objective? Is the opinion holder the author of the posting? Is the opinion implicitly or explicitly formulated? Does the posting refer to the headline / overall topic? Does the opinion refer to one or more aspects of the entity? Metric Metric Metric Nominal Nominal Nominal Nominal Nominal Results of Survey. All in all we analyzed 1934 postings; in the following section we give a short overview on some key findings: Length of postings: As expected, the length of the postings differs between the social media channels. The average amount of words per posting is highest in product reviews (approx. 119 words), lowest in microblogs (approx. 14 words). Interestingly, the average amount of words per Facebook posting is only 19 words. Emoticons and Internet slang: Emoticons are widely used across all analyzed social media channels, with approximately every third (Facebook: 27.8%, Twitter: 24.4%, blogs: 27.6%) to fifth (discussion forums: 20.1%, product reviews: 15.5%) 40 G. Petz et al. posting containing them. Internet slang is not prominently featured in the analyzed channels, whereby no significant difference between them was detected. While Tweets contain the highest amount of typical abbreviations (20.2% of posting), they only occur in about 12.8% of all discussion forum posts, product reviews and blog comments. Surprisingly, only 8.3% of the analyzed Facebook comments feature Internet slang. Grammatical and orthographical correctness: Postings across all social media channels contain many grammatical as well as orthographical errors. The error ratio (number of incorrect sentences divided by number of sentences) is highest in Twitter (48.8%), Facebook (42.7%) and discussion forums (42.3%), and lowest in product reviews (37.2%) and blogs (35.4%). The detailed correlations between the variables were tested with Post-Hoc-tests / Bonferroni: product review / Twitter (p=0.002), Twitter / blog (p=0.0). Subjectivity: Across all analyzed channels 67.8% of the postings were classified as being subjective, as opposed to 18.1% objective ones. The remaining 14.1% of the postings contain both subjective and objective information. While the highest subjectivity can be detected on Twitter (82.9% of all analyzed Tweets), discussion forums not only features the fewest subjective posts (50.2%) but also the majority of objective ones (35.5%). Many of the postings in discussion forums do not contain an opinion, but questions, solution suggestions and hints how to solve a specific issue. An interesting discovery is the lack of exclusively objective product reviews nearly two thirds (71.7%) of the analyzed reviews are solely subjective, while one quarter (25.4%) is based on both subjective and objective information. 2.9% of the reviews are rated as being objective. The detailed correlations between the variables were tested with Post-Hoc-tests / Bonferroni: Facebook / discussion forum (p=0.001), Twitter / product review (p=0.0), Twitter / blog (p=0.033), Twitter / discussion forum (p=0.0). Table 2. Subjectivity in postings Social media channel Subjective Objective Subjective & objective Microblog (Twitter) 82,9% 12,8% 4,3% Product Review 71,7% 2,9% 25,4% Blog 69,3% 19,6% 11,1% Social Network 67,3% 26,1% 6,6% (Facebook) Discussion forum 50,2% 35,5% 14,3% Aspects and details: As expected, the social media channels that tend to feature longer postings contain more details on certain aspects of entities. The detailed figures are exhibited in Table 4. While product review postings go into detail (39.6%) and contain aspects as well as opinions on entity-level (27.0%), Twitter and Facebook-postings mainly contain postings on entity-level (56.6%, 65.4%). Opinion Mining on the Web 2.0 Characteristics of User Generated Content 41 Social media channel Table 3. Opinions about entites and aspects Contains one or more aspects Does not contain aspects Contains opinion about entity and aspect Discussion forum 60,6% 33,1% 6,3% Blog 55,3% 39,1% 5,6% Microblog (Twitter) 43,4% 56,6% 0% Product Review 39,6% 33,4% 27,0% Social Network 33,0% 65,4% 1,6% (Facebook) Opinion holder: The survey exhibited that in most cases the opinion holder is equal to the author of the posting; in Facebook, Twitter, product reviews and blogs between 95% and 97.6% of the postings reveal the author as the opinion holder. Only the postings in the discussion forums have a lower percentage (90.7%). 6.2% of the entries in discussion forums have several opinion holders, and 3.1% depict the opinion of another person. Topic relatedness: At the beginning of our survey we were curious about the users discipline regarding the topic relatedness of their postings. Surprisingly, the postings in all the social media channels are highly related to the overall discussion topic. As shown in the following table, the highest relatedness can be found in discussion forums, which may be related to the presence of moderators and forum rules. Table 4. Topic relatedness Social media channel Topic related Not topic related Topic and non-topic related content Discussion forum 95.6% 3.4% 1.0% Microblog (Twitter) 95.3% 4.7% 0% Product review 93.1% 1.2% 5.8% Blog 92.6% 6.3% 1.1% Social Network 82.3% 16.6% 1.1% (Facebook) Discussion of Survey. The criteria we used for the survey are often criticized in research papers for their ambiguity, e.g. subjective vs. objective. The team that conducted the survey exchanged their experiences and carried out multiple evaluations on the same sample set. There remains the question of how to conduct a survey that is both representative and accomplishable with manageable efforts. In our survey we used one brand from the electronic consumer market, but the results may vary depending on other market segments or genres. 42 G. Petz et al. 3.2 Impact on Opinion Mining Based on the empirical analysis the following impacts can be derived for the opinion mining process: Impacts on Opinion Mining Process. Many research papers in the field of opinion mining assume grammatically correct texts [4], but as shown in the empirical analysis, user generated texts contain many mistakes, emoticons and Internet slang words. Therefore it is reasonable and necessary to preprocess texts from Web 2.0- sources. In some cases the text languages changed on the same channel, e.g. some Facebook postings on the German Facebook site are written in English, Turkish and other languages. In these cases the application of language detection methods is reasonable. In general, because of the grammatical mistakes, grammar-based approaches (e.g. [44], [45]) are not appropriate. The above figures showed, that user generated texts contain Internet slang as well as emoticons. These text parts could be considered as input for feature generation to improve sentiment classification. Furthermore, people often use different names for the same object, e.g. Samsung Galaxy S3 is also being called Galaxy S3 or SGS3, which makes the extraction of entities or aspects more difficult. Characteristics and Impacts of Social Media Channels. The following table gives a short overview about the impacts of each investigated social media channel: Table 5. Social media channels and their impacts Social media channel Discussion forum Microblog (Twitter) Impact The empirical analysis revealed, that discussions in forums are often organized in discussion threads, users respond to other user s questions and comments, and forum postings often contain coreferences all these factors make opinion mining more difficult and a variety of approaches have to be adopted to discussion forums. More research work is required to evaluate, which methods perform best. The characteristics of Twitter can be summarized as follows: many grammatical errors, short sentences, heavy usage of hashtags and other abbreviations. That already led researchers to taking Twitter characteristics into consideration, e.g. Davidov et al. [
Recommended
View more...
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks
SAVE OUR EARTH

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...

Sign Now!

We are very appreciated for your Prompt Action!

x