A Stoic Breakdown: Sentiment Analysis

Sheila Bugal on July 7, 2022
a stoic breakdown: sentiment analysis
AI & NLP

Unless you are a psychopath, humans are by nature emotional beings. Anger, sadness, happiness, and love are emotions that almost every human has felt. Nowadays, we have a plethora of emotional AI detection methods at hand. Sentiment analysis is one of the most widely used methods, with 59% of companies using sentiment analysis to improve their customers’ experience, according to Bain & Company. But what is sentiment analysis? Today we will look behind the fancy marketing gimmicks and get straight to the core of sentiment analysis and why Caplena’s stands out from the rest.

Sentiment Analysis 101

Sentiment analysis is a natural language processing (NLP) technique used to determine whether data has a positivenegative or neutral sentiment. In short: Sentiment analysis looks at the emotion expressed in text. For example, businesses can use this to monitor brand and product sentiment in customer feedback. Feedback of this type usually takes the form of textual data, such as reviews or responses to surveys.

‘Sentiment’ = ‘Emotions’?

A green and blue polar graph of the different emotions one experiences in life. Both positive and negative sentiment.

When we examine emotion in academia, we often see circles like the one above that show various emotional options. However, it is challenging – even for people – to make precise statements as to what kind of emotions are prevalent in a given text. As a result, there is a significant disparity between what different people attribute to the same text. To illustrate this point, we had eight people decide which emotion to assign to a given text out of eight options.

💡 The results -> contestants chose four emotions overall, not just one.

This shows a 75% perceptive discrepancy from choosing just one answer. Crazy, isn’t it?

As we can now see, even humans are incapable of determining complex emotional subtleties without images or audio. It is not recommended to classify into detailed emotions such as hate, sadness, or joy – this would be too inaccurate.

➡️ Through sentiment analysis we can see the polarity of texts and whether they are positivenegative, or neutral. There are two main types of sentiment analysis:

1 – Verbatim / Text Level Sentiment Analysis

Positive sentiment for "fascinating", negative sentiment for "awful" and neutral sentiment for "okay".
From Caplena’s in-app sentiment analysis feature.

Here we assign a positive, negative, or neutral sentiment towards an entire text, meaning a whole sentence – or paragraph. How can we use this? On the verbatim level, using an aggregated approach is recommended, so looking at 100s or 1000s of texts and saying that 30% had a negative sentiment, for example. A verbatim-level sentiment analysis may be appropriate for practices such as social listening. Typical tweets, for example, are short and have only one sentiment per sentence. For types of text that show only one emotion, verbatim-level sentiment analysis is sufficient.

➡️ In reality, however, most sentences contain more than one sentiment. Customer feedback and customer reviews are good examples of this. As a result, verbatim-level sentiment analysis can quickly become inaccurate. Assigning a single sentiment to a given sentence becomes problematic if the sentence has more than one sentiment; which leads us to the second type:

2 – Topic Level Sentiment Analysis

Review states "too costly but the service is great" reflecting a negative price topic and positive service topic.
From Caplena’s in-app sentiment analysis feature.

A more advanced version of verbatim-level sentiment analysis is topic-level sentiment analysis (also used by Caplena). Here we classify the sentiment toward a specific topic of the sentence (e.g price and service.) As shown in the example above, statements with mixed sentiments are very common in customer feedback.

Most humans would agree that it is hard to decide on just one sentiment for the above sentence. But if this was a verbatim level sentiment analysis – you would have to choose. Would negative and positive cancel each other out and make a neutral? Would it appear to be more positive than negative?

This is our humble opinion → why force a conclusion based on just one sentiment, when a sentence clearly contains multiple? This is why Caplena offers ️‍🔥sentiment analysis on the topic level️‍🔥 rather than verbatim level – because it is simply more accurate.

Topic-level sentiment analysis more accurately presents aggregated analysis findings and provides topic-level action-taking potential. If you know that 20% of your customers were unhappy with the price of your service, you could reach out to each of them specifically to offer a discount, for instance.

How Reliable is Sentiment Analysis? A Short History

Starting in the 2000’s

Determining sentiment 100% reliably and automatically is still a holy grail of text analysis. But we’re getting closer. In the past, lexicons have been used, such as SentimentWortschatz, which assigns a value between -1 and 1 to each word. The word “crash” gets a value of -0.47, while wonderful gets +0.72. By adding up the values of each word in a sentence, we get a total value that announces which sentiment prevails. This works for simple statements but fails for negations because it ignores the interplay between words. The 2000s saw several attempts to address this issue, but none could account for context. The real *boom* of sentiment analysis came only within the last decade.

2010 – Present | How Sesame Street Saved Machine Learning

Artificial intelligence, deep learning, and transformers have fundamentally changed this context issue in the last decade. It began with BERT, the character of Sesame Street – short for Bidirectional Encoder Representations from Transformers. For the first time, BERT took context into account by giving the AI many simple tasks on millions of Internet text documents. One of these tasks was to reconstruct randomly deleted words from texts. With practice, the AI gets better at guessing which word is missing from the gap, which is how machines learn context.

Popular algorithms in NLP overview: BERT; ELMo; Word2Vec; OpenAI GPT. Bert and Elmo from Sesame Street portray 2 algorithms.

Since BERT, many improvements have been made based on this, such as RoBERTa and ALBERT. One thing is clear: AI’s are becoming more complex and trained on more data, making it more difficult and expensive to build an AI from scratch. Most companies use commercial AI providers or open-source libraries. Very few perform automatic sentiment analysis on user-defined aspects (with Caplena being one of them). Some solutions additionally offer fine-tuning of the AI significantly improving the accuracy for the specific industry and question. The fine-tuning process involves manually reviewing a few sentiment assignments or injecting additional rules.

An example of fine-tuning (categorizing feedback) on the Caplena coding software platform.
Caplena provides fine-tuning in the ‘Coding View.’ Check if the topics assigned automatically are correct, incorrect, or need any additions. Within a few clicks, the AI reaches human-level accuracy.

“But What About Sarcasm?” – You Say.

Sarcasm is difficult to interpret even with the best systems. The primary difficulty is that sarcasm is very context-dependent. Let’s look at some examples of how Caplena’s AI categorizes the following statements:

"Great to see sentiment analysis being used everywhere" marked as a positive sentiment with no emoji and negative sentiment with a sarcastic emoji.
From Caplena’s in-app sentiment analysis feature.

As it turns out, our system has correctly inferred 3/4 of the cases (all but the last). What’s the takeaway?

  • AI systems can already detect irony and sarcasm in quite a few cases, although they still lack human-level performance. Without additional indicators like emojis, pictures, or the extensive use of punctuation, it is impossible even for humans to detect irony or sarcasm in text reliably.

Where does this lead us in terms of the overall analysis?

  • Irony and sarcasm should not affect the overall sentiment analysis results. Of course, mistakes happen, but in most cases, AI can detect sarcasm or irony or humans leave enough evidence of them that it is obvious.
  • Having more than a few people give sarcastic feedback is unlikely – most people want to make sure their comments are understood and thus won’t be vague in how they provide it.

What can State-of-the-Art Sentiment Analysis provide?

Customer surveys are undoubtedly valuable to your business. They produce rich, valuable insights into loyalty and satisfaction, provide actionable feedback to improve your product or service and help build a relationship with your customer base. That being said, surveys come with their own set of challenges, including accurate text analysis.

The Challenge: Understanding Your Feedback from Open-Ended Surveys

Let’s rewind for a minute and walk through what a typical customer survey might look like for your business. If you’re looking to gain insight into customer satisfaction or loyalty, you might seek to gather a measurable metric accompanied by open-ended feedback– such as NPS (net promoter score) or CSAT (customer satisfaction).

If you choose to send a net promoter score survey, for example, your survey may ask customers, How likely are you to recommend Brand X to a friend, coworker, or family member? Please provide a rating on a scale of 0-10, 0 being extremely unlikely and 10 being extremely likely.

This question will be followed up by an opportunity for respondents to explain their rating: Please explain your rating.

Then, you’ll place your respondents into three categories: Promoters (rating of 9 or 10), Passives (7 or 8), and Detractors (6 or below). After calculating your score
(% of promoters - % of detractors = NPS),
you’re now faced with the task of evaluating feedback: the open-ended text that expresses the thoughts, opinions, and advice of your customers.

Here’s where the challenge begins–accurately analyzing what your customers had to say, and producing actionable insights.

Option 1: Use a human

Of course, you could rely on a human being to perform text analysis. In theory, a human could accurately run through hundreds (or thousands) of open-ended responses to identify common themes and evaluate various sentiments.

One man is staring at the screen of one of two laptops representing the investigating of data for sentiment analysis.

The glaring issue with this approach is, of course, the fact that this takes an enormous amount of time (and money) to achieve. It’s slow, tedious, and meticulous work to pick through each individual response. And depending on the size of your customer base, this could take days or even weeks to accomplish successfully.

The second issue with human analysis is that humans are susceptible to fatigue and inconsistency. For example, a human coder might evaluate the phrase “I don’t care for product X” as extremely negative in one instance, but a few hours later, decide that this same phrase only connotes mild negativity.

In any case, human analysis isn’t ideal.

Option 2: Use a conventional algorithm

Your second option is to use conventional algorithms to evaluate open-ended feedback. In theory, an algorithm can perform the work of analysis for you and applies consistent intelligence to produce consistent results.

Conventional algorithms, however, often lack the type of intelligence required to evaluate the following:

  • Negations or words that carry different meanings depending on the context. Word-dictionary sentiment analysis cannot, for example, decipher the difference between the literal and figurative use of the word “hot” (The bread is always hot vs. The hottest place to be on a Friday night.)
  • Irony or sarcasm (such as “We only waited for 2 hours for our food to arrive…“)
  • Spelling mistakes

That being the case, a conventional algorithm may produce some basic feedback for you…but don’t count on it being accurate. 😕

The Solution: State-of-the-Art Sentiment Analysis

Unlike a conventional algorithm, state-of-the-art sentiment analysis relies on a machine-based learning system to evaluate open-ended text.

The difference lies in the system’s ability to accurately analyze the commentary of the survey-taker (i.e. The quality of the product is great, but customer service is terrible).

The right kind of sentiment analysis empowers you to accurately capture both positive and negative sentiments around various topics. In short, it allows you to really understand “what was good” and “what was bad”.

How to Choose the Right Kind of Sentiment Analysis

A sentiment analysis that takes an aspect approach gives you the richest type of insight. How so? An aspect approach understands sub-themes – not just overall sentiment.

An answer-level approach assesses the single mood of a response. But aspect-level sentiment analysis can assess multiple sentiments–or themes–within a single response.

Not only that, but a user can define different sub-themes he or she would like to identify in the feedback by creating polarized codes.

Some aspects of your product or service, such as billing, may simply be positive or negative. If a customer doesn’t appreciate your pricing, the sentiment won’t be very complex.

But other aspects of your business may carry multiple connotations. Customer Service, for example, may produce a range of emotions in your customers. Sub-themes such as “competent/incompetent,” “friendly/unfriendly,” and “knowledgeable/lack of knowledge” can be understood by sentiment analysis that takes an aspect approach, but may not be picked up by sentiment analysis that picks up on the overall mood.

Furthermore, customers might mention specific brand names in their feedback. If a customer is comparing you to a competitor, you’ll want to be able to identify the theme. Are you being compared positively or negatively? Are they drawing similarities, or wishing you offered a comparable service or product?

In any case, state-of-the-art sentiment analysis can help you quickly understand the emotions of your client base and ultimately, give you the accurate, actionable insight you need to improve your business and generate more enthusiastic customers.

Caplena: Your Answer for Detailed, Specific Analysis

Caplena is an A.I.-powered software that can offer you the kind of detailed, accurate insight you need from any type of open-ended feedback. Caplena’s state-of-the-art sentiment analysis is founded on a deep, transformer-based neural network that has been trained on big data and fine-tuned on domain-specific data. It operates on individual sub-words (“word-pieces”) to accurately evaluate a text. In other words, it takes the powerful aspect approach described above.

Caplena can help you:

  • Save significant time and money on manual coding
  • Gain high-value insights into any kind of open-ended feedback (not just keywords)
  • Streamline your feedback process
  • Boost your overall efficiency
  • Finally, tell the story of your feedback: What are customers saying about your business, and why?

How’d you like this article?  Our team will be happy to include your suggestions in future blog posts! Just email our Head of Marketing sheila [at ] caplena.com

PSST…. Would you like a FREE trial?

Caplena conducts sentiment analysis using AI to understand how your customers or employees genuinely feel so you can spend less time on the analysis and more time on the results! 🏆️
Let's Begin!

Other Useful Links:

Sentiment and Emotion Analysis for Beginners: Types and Challenges