Session 5 · Sentiment Analysis and Clustering

Objective

By the end of class, you can score texts with a sentiment dictionary, group them with hierarchical clustering, and read where the scores are useful or misleading.

Before class

• Watch Orange’s Sentiment Analysis and Text Classification tutorials (about 4 minutes each).
• In DataCamp’s Introduction to R, complete Chapters 4 Factors, 5 Data frames, and 6 Lists. Completion is checked through the classroom dashboard. No upload is required.
• Confirm that Demo 4 is pushed to GitHub.

Optional, if you want to get ahead

• Read Orange’s widget documentation for Sentiment Analysis. It lists the scoring methods and the languages each one supports.
• Open the Moon Jae-in tweets in Orange. Connect Corpus to Sentiment Analysis and choose the Multilingual method with Korean as the language. Sort the tweets by score and read a few of the highest and lowest. Do the scores make sense?

See all R-foundation deadlines and marks →

What this session is about

Session 4 grouped speeches by their words. Session 5 adds a second measure, the feeling a text expresses, and combines the two. The lecture is given by Aron van de Pol.

• Sentiment analysis. Measuring whether a text is positive, negative, or neutral, usually as a score.
• Dictionary-based scoring. A lexicon lists words as positive or negative. We use the KNU Korean sentiment lexicon. A sentence’s score is 100 × (positive words − negative words) / all words.
• Clustering, again. Hierarchical clustering groups the sentences by their nouns, as in Session 4. The mean score of each group then shows which topics are spoken about warmly and which coldly.

A score counts words. It does not read the sentence. You check it by reading the sentences that receive the highest and lowest scores.

Laptop demonstration

Standard track · Orange

Open the starter workflow. It holds the preprocessing (File, Corpus, the Kiwi Python Script, and Preprocess Text), Sentiment Analysis, and Formula. In the File widget, open assembly-nk-japan.csv. We build the rest together in class. To rebuild it later, follow the workflow image (new tab).

Preprocess Text → Sentiment Analysis → Formula → Data Table, Pivot Table, Box Plot

Select Rows → Select Columns → Corpus → Preprocess Text → Bag of Words → Distances → Hierarchical Clustering → Pivot Table

In Sentiment Analysis, choose Custom Dictionary and load the two word lists each time you open the workflow. Orange does not save them. The settings for each widget are on the slides.

BA2 review track · R

Complete this assignment in R. Orange is not permitted for assessed work. It is the same task as in Orange: score the sentences, check six of them, and make the tree on the 일본 sentences with a mean score per group. Save the script as demos/05-sentiment/workflow/s05-sentiment.R. No new packages this week.

1. Read assembly-nk-japan.csv and the two KNU word lists from data/.
2. Split the sentences with Kiwi as in Session 2, but keep verbs and adjectives (VV, VA) as well as nouns. Feelings sit in verbs and adjectives.
3. Score each sentence as 100 × (positive words − negative words) / all words. A sentence with no list word scores 0. Add a label: positive, neutral, or negative.
4. Compare the mean score of government, ruling party, and opposition speakers.
5. Take the three highest and three lowest sentences, read them, and write them to output/text-checks.csv with a manual_check column.
6. Save a chart of the scores as output/scores.png.
7. Keep the 일본 sentences and cluster them as in Session 4, from the nouns column, which is already preprocessed. Remove the words in stopwords-ko-assembly.txt and use word counts without IDF. Cut the tree into 12 groups.
8. Compute the size and mean score of each group, and read the sentences of the warmest and coldest groups.

Your numbers can differ a little from the slides, because your preprocessing differs slightly from the course script. That is expected.

Demo 5 · What does the Assembly say about Japan?

Due Oct. 30 at 09:15

In class we build a workflow on the sentences about 북한. For Demo 5 you use that same workflow to run the same analysis on the sentences about 일본 (Japan). In Select Rows, set the keyword to 일본. Keep Hierarchical Clustering at Top N = 12.

Create demos/05-sentiment/ with the following files.

• README.md naming the KNU lexicon, with your paragraph on the 일본 tree.
• workflow/ containing the Orange .ows workflow or completed R script.
• output/scores.png, a chart of the scores (the Box Plot or the cluster Pivot Table).
• output/text-checks.csv with the six checked sentences and a manual_check column.

Writing the README

Write one paragraph that answers these questions.

1. What do members talk about when they talk about Japan? (clustering)
2. With what sentiment do they speak about the different topics you found? (sentiment per cluster)
3. Why do you think some topics are warmer and others colder?

Base your answer on what your analysis shows, the clusters and their sentiment scores, and on the sentences inside them.

The six checks

Sort the scored sentences by sentiment and pick three of the highest and three of the lowest. Mark each one reasonable, questionable, or misleading in a manual_check column of output/text-checks.csv. You may take them from the 일본 sentences.

Before you submit

• The README names the scoring method or lexicon.
• The workflow or script is in workflow/, and the score chart is in output/.
• The README has a paragraph on what the 일본 tree shows.
• Six sentences were checked by hand in output/text-checks.csv.
• Every checked sentence has one of the three labels.
• The final files are pushed to GitHub.