Category: Mathematics

Advanced Data Analysis from an Elementary Point of View
by Cosma Rohilla Shalizi

This is a draft textbook on data analysis methods, intended for a one-semester course for advance undergraduate students who have already taken classes in probability, mathematical statistics, and linear regression. It began as the lecture notes for 36-402 at Carnegie Mellon University.

By making this draft generally available, I am not promising to provide any assistance or even clarification whatsoever. Comments are, however, welcome.

The book is under contract to Cambridge University Press; it should be turned over to the press before the end of 2015. A copy of the next-to-final version will remain freely accessible here permanently.

Complete draft in PDF

Table of contents:

I. Regression and Its Generalizations
1. Regression Basics
2. The Truth about Linear Regression
3. Model Evaluation
4. Smoothing in Regression
5. Simulation
6. The Bootstrap
7. Weighting and Variance
8. Splines
9. Additive Models
10. Testing Regression Specifications
11. Logistic Regression
12. Generalized Linear Models and Generalized Additive Models
13. Classification and Regression Trees
    II. Distributions and Latent Structure
14. Density Estimation
15. Relative Distributions and Smooth Tests of Goodness-of-Fit
16. Principal Components Analysis
17. Factor Models
18. Nonlinear Dimensionality Reduction
19. Mixture Models
20. Graphical Models
    III. Dependent Data
21. Time Series
22. Spatial and Network Data
23. Simulation-Based Inference
    IV. Causal Inference
24. Graphical Causal Models
25. Identifying Causal Effects
26. Causal Inference from Experiments
27. Estimating Causal Effects
28. Discovering Causal StructureAppendices
    • Data-Analysis Problem Sets
    • Reminders from Linear Algebra
    • Big O and Little o Notation
    • Taylor Expansions
    • Multivariate Distributions
    • Algebra with Expectations and Variances
    • Propagation of Error, and Standard Errors for Derived Quantities
    • Optimization
    • chi-squared and the Likelihood Ratio Test
    • Proof of the Gauss-Markov Theorem
    • Rudimentary Graph Theory
    • Information Theory
    • Hypothesis Testing
    • Writing R Functions
    • Random Variable Generation

Planned changes:

• Unified treatment of information-theoretic topics (relative entropy / Kullback-Leibler divergence, entropy, mutual information and independence, hypothesis-testing interpretations) in an appendix, with references from chapters on density estimation, on EM, and on independence testing
• More detailed treatment of calibration and calibration-checking (part II)
• Missing data and imputation (part II)
• Move d-separation material from “causal models” chapter to graphical models chapter as no specifically causal content (parts II and IV)?
• Expand treatment of partial identification for causal inference, including partial identification of effects by looking at all data-compatible DAGs (part IV)
• Figure out how to cut at least 50 pages
• Make sure notation is consistent throughout: insist that vectors are always matrices, or use more geometric notation?
• Move simulation to an appendix
• Move variance/weights chapter to right before logistic regression
• Move some appendices online (i.e., after references)?

(Text last updated 30 March 2016; this page last updated 6 November 2015)