Instructor
Course Description
Pattern Recognition (a.k.a. Machine Learning I) course focuses Unsupervised Learning methods for data analysis. In particular, we cover topics such as:
- Association rules
- Density modeling
- Clustering (k-means, k-medoids, hierarchical models, spectral methods)
- Mixture models
- Distance metric learning
- Factor analysis (PCA, CCA, etc.) and other latent variable models
- Probabilistic topic models
- Matrix factorization
- Tensor models
- Clustering evaluation
Expected Work
Regular readings; mini-projects; in-class presentations; midterm and/or a final course project.
You can find examples of some of the past projects here.
Course Schedule
| Lec. # | Date | Topic | Readings |
|---|---|---|---|
| 1 | 2020-09-02 00:00:00 | Introduction & Overview | _ http://www.cs.cmu.edu/~tom/pubs/MachineLearning.pdf |
| _ M.I.Jordan and T.M.Mitchell, "Machine learning: Trends, perspectives, and prospects," Science, 17 July 2015, Vol 349 Issue 6245 | |||
| 2 | 2020-09-02 00:00:00 | Probability & Linear Algebra | _ https://content.sakai.rutgers.edu/access/content/group/c626f8f2-9099-4059-b4b9-5794524e759d/algebra.pdf |
| _ https://content.sakai.rutgers.edu/access/content/group/c626f8f2-9099-4059-b4b9-5794524e759d/probability.pdf | |||
| 3 | 2020-09-09 00:00:00 | Association Rules & | _ Chapter 14.1 of http://statweb.stanford.edu/~tibs/ElemStatLearn/ |
| Frequent Itemsets | _ Chapter 14.2-14.2.3 of http://statweb.stanford.edu/~tibs/ElemStatLearn/ | ||
| _ Chapter 6 of http://www.mmds.org/#book | |||
| 4 | 2020-09-16 00:00:00 | Density Estimation | _ Chapters 6.6 through 6.9 of http://statweb.stanford.edu/~tibs/ElemStatLearn/ |
| _ http://ned.ipac.caltech.edu/level5/March02/Silverman/Silver_contents.html | |||
| _ Chapters 4.1 - 4.4 of Duda, Hart, and Stork | |||
| — | 2020-09-16 00:00:00 | Homework #1 assigned | |
| Project proposals and in-class pitches assigned | |||
| 5 | 2020-09-23 00:00:00 | K-means | _ Chapters 9.1 and 9.2 of http://robotics.stanford.edu/~nilsson/MLBOOK.pdf |
| _ Chapters 14.3.1 through 14.3.6 of http://statweb.stanford.edu/~tibs/ElemStatLearn/ | |||
| _ Chapter 7 of http://www.mmds.org/#book | |||
| _ https://www.cs.rutgers.edu/~mlittman/courses/lightai03/jain99data.pdf | |||
| _ Chapters 13.1 and 13.2 of http://statweb.stanford.edu/~tibs/ElemStatLearn/ | |||
| _ Chapters 10.1 – 10.4 and 10.7 of Duda, Hart, and Stork | |||
| 6 | 2020-09-23 00:00:00 | Gaussian Mixtures & Expectation Maximization & Factor Analysis | _ Mixture of Gaussians: http://cs229.stanford.edu/notes/cs229-notes7b.pdf |
| _ The EM Algorithm: http://cs229.stanford.edu/notes/cs229-notes8.pdf | |||
| _ Factor Analysis: http://cs229.stanford.edu/notes/cs229-notes9.pdf | |||
| _ Chapters 14.3.7 through 14.3.9 of http://statweb.stanford.edu/~tibs/ElemStatLearn/ | |||
| 7 | 2020-09-30 00:00:00 | K-medoids & Hierarchical Clustering | _ Chapter 14.3.10 of http://statweb.stanford.edu/~tibs/ElemStatLearn/ |
| _ Chapter 14.3.12 of http://statweb.stanford.edu/~tibs/ElemStatLearn/ | |||
| _ Chapter 9.3 of http://robotics.stanford.edu/~nilsson/MLBOOK.pdf | |||
| _ Chapter 10.9 of Duda, Hart, and Stork | |||
| 8 | 2020-09-30 00:00:00 | Evaluation Metrics & Practical Issues | _ http://web.itu.edu.tr/sgunduz/courses/verimaden/paper/validity_survey.pdf |
| _ Chapter 14.3.11 of http://statweb.stanford.edu/~tibs/ElemStatLearn/ | |||
| — | 2020-09-30 00:00:00 | Homework #1 due at 11:59 PM Eastern | |
| 9 | 2020-10-07 00:00:00 | Distance/Similarity Measures & Metric Learning | _ http://web.cse.ohio-state.edu/~kulis/pubs/ftml_metric_learning.pdf |
| _ Check out the Encyclopedia of Distances on this course’s Sakai site (under Resources). | |||
| — | 2020-10-07 00:00:00 | Homework #2 assigned | |
| 10 | 2020-10-07 00:00:00 | Principal Component Analysis (PCA) & Singular Value Decomposition (SVD) | _ Chapter 14.5 of http://statweb.stanford.edu/~tibs/ElemStatLearn/ |
| _ Chapter 11 of http://www.mmds.org/#book | |||
| 11 | 2020-10-14 00:00:00 | Spectral Clustering & Graph Clustering | _ http://ai.stanford.edu/~ang/papers/nips01-spectral.pdf |
| _ http://www.cs.columbia.edu/~jebara/4772/papers/Luxburg07_tutorial.pdf | |||
| _ [Optional] http://arxiv.org/pdf/0906.0612.pdf | |||
| — | 2020-10-14 00:00:00 | Homework #1 graded | |
| 12 | 2020-10-21 00:00:00 | Kernel Principal Components & Independent Component Analysis (ICA) & Canonical Correlation Analysis (CCA) & PageRank | _ Chapter 14.5 of http://statweb.stanford.edu/~tibs/ElemStatLearn/ |
| _ ICA: Chapter 14.7 of http://statweb.stanford.edu/~tibs/ElemStatLearn/ | |||
| _ CCA: https://www.cs.cmu.edu/~tom/10701_sp11/slides/CCA_tutorial.pdf | |||
| _ PageRank: | |||
| o Chapter 5 of http://www.mmds.org/#book | |||
| o Chapter 14.10 of http://statweb.stanford.edu/~tibs/ElemStatLearn/ | |||
| o https://www.cs.purdue.edu/homes/dgleich/publications/Gleich%202015%20-%20prbeyond.pdf | |||
| — | 2020-10-21 00:00:00 | Homework #2 due at 11:59 PM Eastern | |
| — | TBD | Two-page project proposals due at 11:59 PM Eastern | |
| TBD | In-class project pitches | ||
| 13 | 2020-10-28 00:00:00 | Recommendation Systems Retrieval models | _ http://infolab.stanford.edu/~ullman/mmds/ch9.pdf |
| _ http://eliassi.org/papers/chaney-recsys15.pdf | |||
| TBD | Midterm exam | ||
| — | TBD | Project proposals & pitches graded | |
| Project presentations and reports assigned | |||
| 14 | 2020-11-04 00:00:00 | Latent Variable Models & | _ http://research.microsoft.com/pubs/67187/bishop-latent-erice-99.pdf |
| Probabilistic Topic Models | _ http://www.cs.columbia.edu/~blei/papers/Blei2012.pdf | ||
| _ http://www.cs.princeton.edu/~blei/papers/Blei2011.pdf | |||
| _ http://www.cs.columbia.edu/~blei/papers/BleiLafferty2009.pdf | |||
| 15 | 2020-11-04 00:00:00 | Latent Variable Models & | _ http://www.cs.berkeley.edu/~jordan/papers/variational-intro.pdf |
| Probabilistic Topic Models (continued) | _ http://www.cs.ubc.ca/~arnaud/andrieu_defreitas_doucet_jordan_intromontecarlomachinelearning.pdf | ||
| _ https://www.ee.washington.edu/techsite/papers/documents/UWEETR-2010-0006.pdf | |||
| — | 1900-01-13 00:00:00 | Homework #2 graded | |
| 16 | 2020-11-11 | Matrix Factorization | _ Chapter 14.6 of http://statweb.stanford.edu/~tibs/ElemStatLearn/ |
| _ http://papers.nips.cc/paper/1861-algorithms-for-non-negative-matrix-factorization.pdf | |||
| 17 | 2020-11-11 | Tensor Factorization | _ http://www.sandia.gov/~tgkolda/pubs/pubfiles/TensorReview.pdf |
| TBD | Midterm exam graded and returned at the end of lecture | ||
| 18 | 2020-11-18 00:00:00 | Sequence Models - Non IID | |
| 2020-11-25 00:00:00 | Thanksgiving Holiday – no class (Friday classes) | ||
| 19 | 2020-12-02 00:00:00 | Model Selection | _ Chapter 7 of http://statweb.stanford.edu/~tibs/ElemStatLearn/ |
| 26 | 2020-12-02 00:00:00 | Theory of Clustering | _ http://www.cs.cornell.edu/home/kleinber/nips15.pdf |
| _ http://papers.nips.cc/paper/3491-measures-of-clustering-quality-a-working-set-of-axioms-for-clustering.pdf | |||
| 27 | TBD | Project presentations | |
| 2020-12-10 00:00:00 | Last day of classes | ||
| — | TBD | Project presentations graded | |
| — | TBD | Project reports due at 11:59 PM Eastern | |
| — | TBD | Project reports graded and final grades released. |
Textbooks
| Abbreviation | Textbook Title | Author | Publisher | Year |
|---|---|---|---|---|
| PRML | Pattern Recognition and Machine Learning | Christopher C. Bishop | Springer | 2006 |
| MLPP | Machine Learning: A Probabilistic Perspective | Kevin P. Murphy | MIT Press | 2012 |
| CVMLI | Computer vision: models, learning and inference | Prince, Simon J D | Cambridge University Press | 2012 |
| DL | Deep Learning | Goodfellow, Ian and Bengio, Yoshua and Courville, Aaron | MIT Press | 2016 |
| FML | Foundations of Machine Learning | Mohri, Mehryar and Rostamizadeh, Afshin and Talwalkar, Ameet | MIT Press | 2012 |
| DHS | Pattern Classification, 2nd ed | Duda, Richard O. and Hart, Peter E. and Stork, David G. | Wiley Interscience | 2004 |
| ML | Machine Learning | Mitchell, Tom | McGraw Hill | 1997 |
| I2ML | Introduction to Machine Learning, 2nd ed | Alpaydin, Ethem | MIT Press | 2012 |
| MLAP | Machine Learning: An Algorithmic perspective | Marsland, Stephen | CRC press | 2009 |
| PTPR | A Probabilistic Theory of Pattern Recognition | Devroye, Luc and Gyorfi, Laszlo and Lugosi, Gabor | Springer | 1997 |
| ESL | The elements of statistical learning: Data mining, inference, and prediction | Friedman, J and Hastie, T and Tibshirani, R | Springer | 2009 |
| NAPR | Netlab: Algorithms for Pattern Recognition | Nabney, Ian | Springer | 2002 |
| DMPMLTT | Data Mining: Practical Machine Learning Tools and Techniques | Witten, Ian H and Frank, Eibe | Morgan Kaufmann | 2005 |
| LAA | Linear Algebra and Its Applications | Strang, Gilbert | Elsevier Science | 2014 |
| MC | Matrix computations, 4th ed | Golub, Gene H and Van Loan, Charles F | JHU Press | 2013 |
| CO | Convex Optimization | Boyd, Steven P and Vandenberghe, Lieven | Cambridge University Press | 2004 |
| ILCO | Introductory lectures on convex optimization: a basic course_ | Nestorov, Yurii | Springer | 2004 |
| GPML | Gaussian Processes for Machine Learning | Rasmussen, Carl Edward and Williams, Christopher K. I. | MIT Press | 2006 |
| ITILA | Information Theory, Inference, and Learning Algorithms | MacKay, David | Cambridge University Press | 2003 |