Session 2 · Text Preprocessing

Objective

By the end of class, you can apply and document a small set of preprocessing choices and explain how those choices change the text available for analysis.

Before class

• Complete the six assigned Swirl · R Programming lessons and save the completion evidence as r-learning/swirl-r-programming.png.
• In DataCamp’s Introduction to R, complete Chapter 1 Intro to basics. Completion is checked through the classroom dashboard. No upload is required.
• Watch Orange’s current Text Preprocessing tutorial.
• Read Text as Data in Six Moves.
• Download the prepared South Korean presidential-speeches corpus from the private course corpus repository after access is granted. Keep it locally under data/private/presidential-speeches/ and follow its corpus card. Do not commit the data file.
• Confirm that Demo 1 is pushed to GitHub.

See all R-foundation deadlines and marks →

Opening survey

The ungraded questions revisit corpus boundaries, units of analysis, and provenance. Attendance is recorded separately from correctness.

Device-free lecture

We will introduce tokens, tokenization, stopwords, normalization, filtering, and language-specific choices through one small corpus. The lecture distinguishes what researchers remove, retain, or transform from what the software later counts.

Laptop demonstration

Standard track · Orange

Build one documented workflow.

Corpus → Preprocess Text → Corpus Viewer

Compare the raw and processed text, change one setting at a time, and record what disappears or changes.

BA2 review track · R

Complete this assignment in R. Orange is not permitted for assessed work. Use the supplied starter script to import the same corpus, tokenize it, remove the supplied stopwords, and export a before-and-after comparison.

Demo 2 · Reproduce and document preprocessing

Due Oct. 1 at 17:00

Create demos/02-preprocessing/ with the following files.

• README.md listing the corpus, preprocessing settings, and completed tasks.
• workflow/ containing either one Orange .ows file or the supplied R starter completed as an .R file.
• output/before-after.png showing raw and processed text.
• output/settings.md recording the required preprocessing choices.
• output/text-check.md naming three examples inspected after preprocessing.

Before you submit

• The workflow or script opens from the repository.
• The preprocessing settings are recorded.
• The before-and-after output is present and labeled.
• Three inspected examples are identified.
• Everything is committed and pushed.

The score records whether these items are complete. It does not reward a more sophisticated interpretation.