Week 1

• Jan 22 Introduction
• Analyzing text to gain insight about people and about language:
• Google trends
• Word watchers: tracking the language of public figures
• Language Log: more on the language of politicians
• Language Log: the language of real estate ads
  
• Language Log: the decline of passivity
  
• Language Log: decreasing definiteness
  
• Language Log: when did spelling become more consistent?
• Language Log: almost literally, and literally
  
  
• Jan 24 Python: Installation session
  
• We will be installing:
• Python: https://www.python.org/downloads/
• numpy: https://numpy.org/
• matplotlib: https://matplotlib.org/
• Jupyter notebooks: https://jupyter.org/
• Pandas: https://pandas.pydata.org/
• Statsmodels: https://www.statsmodels.org/stable/index.html
  
• NLTK: Installing NLTK itself: http://www.nltk.org/install.html You also need the NLTK data, see http://www.nltk.org/data.html
• You can install all of these through Anaconda
  
• Here is a jupyter notebook to check your installations: ipynb file
  

Week 2

• Jan 27 Python: Programming first steps
• Code for download: first steps in Python
  
  
• Jan 29 Exploring and visualizing data
• Code for download: how to use jupyter notebooks
  
• Code for download: exploring data.
• We use the inaugural dataset available on Canvas.
  
  
• Jan 31 Exploring and visualizing data, continued
  

Week 3

• Feb 3 Python basics: conditions and lists
• Code for download: conditions, lists, and loops
  
• Feb 5 Python basics: loops
• We continue using the same notebook from last time: Code for download: conditions, lists, and loops
  
  
• Feb 7 What can I do for my project?
  

Week 4

• Feb 10 Python basics: dictionaries, and how to count words
• Code for download: dictionaries
  
  
• Feb 12 Python basics: dictionaries, continued
• Code for download: from dictionaries to pandas data frames
  
• Homework 1 due
  
  
• Feb 14 Python basics: preprocessing text and counting words
• Code for download: from dictionaries to pandas data frames

Week 5

• Feb 17 We finish up the creation of pandas data frames, and talk about text preprocessing
• Code for download: from dictionaries to pandas data frames
• Code for download: preprocessing text
  
  
• Feb 19 We talk in class about your project ideas.
  
  
• Feb 21 We finish up preprocessing.

Week 6

• Feb 24 descriptive statistics: central tendency and spread
  
• Readings: SE chapter 2
• Do women talk more than men do? Here are some relevant Language Log posts:
• http://itre.cis.upenn.edu/~myl/languagelog/archives/003420.html
• http://itre.cis.upenn.edu/~myl/languagelog/archives/003607.html
• Code for download: central tendency and spread
  
• We use the Fisher telephone data available on Canvas as fisher1.tbl
  
  
• Feb 26: Clustering for exploratory data analysis
• Code for download: clustering
  
• Initial project description due.
  
  
• Feb 28 A preview of samples and populations.
  Then: Clustering continued.
  
  

Week 7

• Mar 2 Clustering: Dimensionality reduction
• Code for download: dimensionality reduction
  
  
• Mar 4 Topic modeling for detecting word clusters and topic clusters
• Code for download: topic modeling
• Homework 2 due.
  
  
• Mar 6 Probability distributions. The central limit theorem
• Code for download: probability distributions
• Cartoon Introduction to Statistics, available on Canvas:
  
• Samples and populations: 1-cartoon-sampling
• Confounder variables: 2-cartoon-two-variables
• The Central Limit theorem: 3-cartoon-central-limits
• Applying the Central Limit theorem: 4-cartoon-applying-central-limits
  

Week 8

• Mar 9 Statistical tests: the basic principle; significance thresholds
• Readings: SE ch. 4,5, 6
• Code for download: hypothesis testing
  
• https://xkcd.com/539/
• Cartoon Introduction to Statistics, available on Canvas: Hypothesis testing: 5_cartoon_hypothesistesting
  
  
• Mar 11 The t-test
• Code for download: the t-test
  
• Readings: SE ch. 7
  
  
• Mar 13 Class canceled due to Corona virus
  

Week 9

Week 10

• Mar 30 The t-test, continued
  
• Readings: SE ch. 8, 9
• Code for download: the t-test
• Pitfalls of statistical analysis: The problem of multiple testing https://xkcd.com/882/
  
  
• April 1 Different tests for different types of data
• We look at the chi-square test. to be used for frequency data (we have a lot of frequency data)
  
• Readings: SE ch. 19. 
• Code for download: the chi-squared test
• 
  
• April 3 Correlation and linear regression
• Code for download: correlation.
  
• Readings: SE ch. 20, 261-70
• https://xkcd.com/552/
  
• https://9gag.com/gag/2344861
• Language Log on correlation and causation
  
• And another note on correlation and causation: "Nobel prize winners eat more chocolate"??
• Intermediate project report due.

Week 12

• April 6 More linear regression
• Readings; SE rest of ch. 20, ch. 21 
• How not to do a scientific study: "p-value hacking"
• A  linear regression tutorial
  
  
• April 8 Practicing regression
  
  
• April 10 Logistic regression
  

Week 13

• April 13 Logistic regression, continued
• Homework 3 due.
  
  
• April 15  Model comparison
  
  
• April 17 Practicing logistic regression
  

Week 14

• April 20 Other classification techniques: Naive Bayes
  
  
• April 22 Using Naive Bayes
  
  
• April 24 Python: defining your own functions, and structuring your programs

Week 15

• April 27 Python functions, classes, and objects
• Homework 4 due.
  
  
• April 29 Python pandas: grouping, merging, appending data frames
  
  
• May 1 Python: list comprehensions, and exceptions

Week 16

• May 4: Project presentations
  
  
• May 6: Project presentations
  
  
• May 8: Project presentations