Analyzing linguistic data: schedule - Katrin Erk's homepage

Courses‎ > ‎Analyzing linguistic data, and programming for linguists‎ > ‎

Analyzing linguistic data: schedule

This schedule is subject to change.

Assignments are due at class time (2pm) on their due date. Please submit assignments online on Canvas unless the assignment tells you otherwise.

Readings can be done either before or after class (unless noted otherwise); they are chosen to support the material covered in class.

Week 1

Jan 22 Introduction

Analyzing text to gain insight about people and about language:

Jan 24 Python: Installation session

We will be installing:

Python: https://www.python.org/downloads/
numpy: https://numpy.org/
matplotlib: https://matplotlib.org/
Jupyter notebooks: https://jupyter.org/
Pandas: https://pandas.pydata.org/
Statsmodels: https://www.statsmodels.org/stable/index.html
NLTK: Installing NLTK itself: http://www.nltk.org/install.html You also need the NLTK data, see http://www.nltk.org/data.html

You can install all of these through Anaconda
Here is a jupyter notebook to check your installations: ipynb file

Week 2

Jan 27 Python: Programming first steps

Code for download: first steps in Python

Jan 29 Exploring and visualizing data

Code for download: how to use jupyter notebooks
Code for download: exploring data.
We use the inaugural dataset available on Canvas.

Jan 31 Exploring and visualizing data, continued

Week 3

Feb 3 Python basics: conditions and lists

Code for download: conditions, lists, and loops

Feb 5 Python basics: loops

We continue using the same notebook from last time: Code for download: conditions, lists, and loops

Feb 7 What can I do for my project?

Week 4

Feb 10 Python basics: dictionaries, and how to count words

Code for download: dictionaries

Feb 12 Python basics: dictionaries, continued

Code for download: from dictionaries to pandas data frames
Homework 1 due

Feb 14 Python basics: preprocessing text and counting words

Code for download: from dictionaries to pandas data frames

Week 5

Feb 17 We finish up the creation of pandas data frames, and talk about text preprocessing

Feb 19 We talk in class about your project ideas.
Feb 21 We finish up preprocessing.

Week 6

Feb 24 descriptive statistics: central tendency and spread

Readings: SE chapter 2

Do women talk more than men do? Here are some relevant Language Log posts:

Code for download: central tendency and spread
We use the Fisher telephone data available on Canvas as fisher1.tbl

Feb 26: Clustering for exploratory data analysis

Code for download: clustering
Initial project description due.

Feb 28 A preview of samples and populations.
Then: Clustering continued.

Week 7

Mar 2 Clustering: Dimensionality reduction

Code for download: dimensionality reduction

Mar 4 Topic modeling for detecting word clusters and topic clusters

Code for download: topic modeling
Homework 2 due.

Mar 6 Probability distributions. The central limit theorem

Code for download: probability distributions
Cartoon Introduction to Statistics, available on Canvas:

Samples and populations: 1-cartoon-sampling
Confounder variables: 2-cartoon-two-variables
The Central Limit theorem: 3-cartoon-central-limits
Applying the Central Limit theorem: 4-cartoon-applying-central-limits

Week 8

Mar 9 Statistical tests: the basic principle; significance thresholds

Readings: SE ch. 4,5, 6
Code for download: hypothesis testing
https://xkcd.com/539/
Cartoon Introduction to Statistics, available on Canvas: Hypothesis testing: 5_cartoon_hypothesistesting

Mar 11 The t-test

Code for download: the t-test
Readings: SE ch. 7

Mar 13 Class canceled due to Corona virus

Week 9

Spring break

Week 10

Class canceled due to Corona virus

Week 11

Mar 30 The t-test, continued

Readings: SE ch. 8
Code for download: the t-test
Pitfalls of statistical analysis: The problem of multiple testing https://xkcd.com/882/

April 1 finishing up the t-test. Hypothesis testing: Type I and Type II errors, the power of tests, and the problem of multiple testing

Readings: SE ch. 9
Pitfalls of statistical analysis: The problem of multiple testing https://xkcd.com/882/

April 3 Python list comprehsions, and how to use them with pandas

Code for download: Python list comprehensions

Week 12

April 6 Correlation and linear regression

Code for download: correlation.
Readings: SE ch. 20, 261-70
https://xkcd.com/552/
https://9gag.com/gag/2344861
Language Log on correlation and causation
And another note on correlation and causation: "Nobel prize winners eat more chocolate"??
While we are on the topic of how hard it is to draw good risky conclusions: "p-value hacking"

April 8 More linear regression

Code for download: linear regression with Python statsmodels
Readings; SE rest of ch. 20, ch. 21

Intermediate project report due.

April 10 Multiple regression, and practicing regression

Code for download: more linear regression with Python statsmodels, including multiple predictors and categorial predictors

Week 13

April 13 Practicing regression
April 15 Logistic regression

Code for download: Logistic regression
Homework 3 due.

April 17 Practicing logistic regression

Week 14

April 20 practicing regression

April 22 Model comparison

April 24 Python: defining your own functions, and structuring your programs. Python exceptions

Week 15

April 27 Python pandas: grouping, merging, appending data frames

Code for download: more pandas
Homework 4 due.

April 29 Different tests for different types of data

We look at the chi-square test. to be used for frequency data (we have a lot of frequency data)
Readings: SE ch. 19.
Code for download: the chi-squared test

May 1 ANOVA: comparing more than two variables, and how that relates to regression

Code for download: ANOVA

Week 16

May 4: Project presentations:

2:00: Nathaniel Lawson and Connor Krusleski
2:08: Marilin Ayala and Victor Rodriguez
2:24: Brandon Falcona and Noah Pang
2:32: Andrew Smith

May 6: Project presentations:

2:00: Nitesh Kartha and Sebastian Mancha
2:08: Mary Guinn and Sydney Willett
2:16: David Alvarez and Zachary Holsinger
2:24: McKenzie Burk and Dana Freeman
2:32: Samuel Smith

May 8: Project presentations:

2:00: Jackie Mares and Misty Peng
2:08: Emily Frazier
2:16: Adam Hanzel and William Culver
2:24: Gabriela O'Connor
2:32 Hyemi Jo
... and some final remarks

Final report due: Tuesday May 12, 2pm.

Comments