• Definition of Statistical Machine Learning
• Probability density estimation
• Definition and behavior of linear discriminant models
• Linear regression
• Logistic regression
• Support Vector Machines
• Ensemble Methods
• Basics of Neural Networks

Week 1: Introduction, Definitions, and Core Principles

In the first week, we will provide an overview of the course and introduce the fundamental concepts of machine learning. Students will learn about the different types of learning, such as supervised, unsupervised, and reinforcement learning, as well as the key steps involved in developing a machine learning model, from data preprocessing to model evaluation and optimization.

Week 2: Probability Density Estimation

In week two, students will delve into probability density estimation, an essential technique for understanding the underlying structure of data. We will cover various methods, such as parametric and non-parametric approaches, and how they can be used for building machine learning models.

Week 3: Linear Discriminants

During the third week, we will focus on linear discriminants and their use in classifying data points. Students will learn about the concept of decision boundaries and how to derive them using linear discriminant functions. We will also discuss how to solve decision problems by minimizing a least-squares objective, the limitations of the resulting linear classifiers, and introduce strategies for handling non-linearly separable data.

Week 4: Linear Regression

In week four, students will be introduced to linear regression, a fundamental technique for modeling continuous data. We will cover the basics of simple and multiple linear regression, discuss the concept of least squares estimation, and explore regularization as a measure against overfitting.

Week 5: Logistic Regression

The fifth week will be dedicated to logistic regression, a powerful technique for binary classification tasks. Students will learn how to derive the logistic regression model, perform iterative optimization using first- and second-order methods, apply regularization, and explore the relations between generative and discriminative methods.

Week 6: Support Vector Machines

In week six, we will explore support vector machines (SVMs), a versatile and very robust algorithm for classification tasks. Students will learn about the key concepts behind SVMs, such as maximum margin and kernel functions, and gain hands-on experience implementing SVMs using popular machine learning libraries.

Week 7: Ensembling Methods

During the seventh week, we will delve into ensemble methods, which combine multiple models to improve the overall performance of a machine learning system. Students will explore popular techniques such as bagging and boosting, and learn how to implement the AdaBoost algorithm.

Week 8: Neural Network Basics

In the final week, students will be introduced to the foundations of deep learning and neural networks. We will cover the basics of artificial neurons, feedforward networks, and backpropagation. This week will provide the groundwork for more advanced topics in deep learning, preparing students for further study or simple practical applications in the field.