In an effort to learn more about machine learning, I've decided to go through the textbook Machine Learning: a Probabilistic Perspective by Kevin Murphy. I also want to learn more Python, so I've decided to write up solutions to selected exercies in Jupyter notebooks.

In general, I try to derive the mathematical results in the notebooks, too, so that the code is not completely opaque. When it comes to modeling, at times I will use an existing scikit-learn implementation, but often, I will try to implement the model myself with NumPy and SciPy when I feel that doing so is instructive. Plotting is done with matplotlib. TensorFlow is used for the neural networks.

Here's an index of some of the notebooks, which will be updated as I make more progress.

Selected Exercises

• Chapter 1: Introduction
  • KNN classifier on shuffled MNIST data
  • Approximate KNN classifiers
  • CV for KNN
• Chapter 3: Generative models for discrete data
  • Setting the beta hyper-parameters
• Chapter 4: Gaussian models
  • Whitening versus standardizing
  • LDA/QDA on height/weight data
• Chapter 7: Linear Regression
  • Sufficient statistics for online linear regression
  • Bayesian linear regression in 1 dimension with known $\sigma^2$
• Chapter 8: Logistic Regression
  • Spam classification using logistic regression
  • Spam classification with naive Bayes
• Chapter 12: Latent Linear Models
  • Latent Semantic Indexing
  • Probabilistic Principal Component Analysis versus Factor Analysis
  • Nonlinear regression for inverse dynamics

Blog Posts

• Nearest Neighbors and Discriminant Analysis
• Factor Analysis and Matrix Calculus
• SARCOS and TensorFlow