School of Engineering and Technology, (SET)
 
AT70.9022 : Selected Topic: Machine Learning  3(3-0)
 
Course objectives:

Machine learning is the branch of artificial intelligence in which we build and use tools that let us specify a computer's behavior implicitly by giving a performance measure and/or a series of examples of how it should respond in a given situation, without specifying the algorithm needed to compute the response. Today’s applications include financial fraud detection, image recognition, speech recognition, and self-driving cars. The near future will see machine learning technologies helping drive the fourth industrial revolution, spanning health care, agriculture, transportation, manufacturing, and the services sector. The course will introduce students from a variety of science and engineering backgrounds to the fundamentals of machine learning and prepare them to perform R&D involving machine learning techniques and applications. Students will develop the data analysis and modeling skills needed for the engineering of intelligent systems incorporating models learned from data.
 
Learning Outcomes:
Students, on completion of the course, would be able to
1.     Formulate a practical data analysis and prediction problem as a machine learning problem.
2.     Identify the characteristics of the data set required for a particular machine learning problem.
3.     Train and test supervised regression and classification models.
4.     Apply the principles of deep learning to the development of supervised learning models.
5.     Train and test unsupervised learning and density estimation models.
6.     Train and test reinforcement learning models.
7.     Integrate a trained machine learning model in an online software system.
 
Pre-requisite(s):
None. Anyone with a basic understanding of calculus, linear algebra, probability theory, and computer programming (in any high-level programming language), can take the course.
 
Course Outline:
I.             Supervised Learning
1.    Generalized linear regression
2.    Generative probabilistic models
3.    Support vector machines
4.    Convex optimization

II.         Deep Learning
1.    Perceptrons
2.    Multilayer neural networks and backpropagation
3.    Convolutional neural networks
4.    Optimization techniques

III.      Learning Theory
1.    Bias-variance tradeoff
2.    Regularization, model selection, and feature selection
3.    VC dimension
IV.       Unsupervised Learning
1.    Clustering: k-means, Gaussian mixture models
2.    Principal components analysis
3.    Independent components analysis

V.          Reinforcement Learning
1.    Markov decision processes and the Bellman equations
2.    Value iteration, policy iteration
3.    Q-learning
4.    Adversarial learning and generative adversarial networks

VI.       Data Analysis Tools and Best Practices
1.    Programming and data analysis in Python
2.    Programming and data analysis in R
3.    Deep learning with Caffe
 
Laboratory Sessions:

None. Several tutorials on data analysis tools will be given during regular class.
 
Learning Resources:

Textbook:

None. Lecture notes and links to online resources will be posted online.
 
Reference Books:
1.     Bishop, C. (2006), Pattern Recognition and Machine Learning, Springer.

2.     Goodfellow, I., Bengio, Y., and Courville, A. (2016), Deep Learning, MIT Press.

3.     Hastie, T., Tibshirani, R., and Friedman, J. (2016), The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edition, Springer.
 
Journals and Magazines:
•         IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI).

•         Journal of Machine Learning Research (JMLR).

Others:

•      Proceedings of the Advances in Neural Information Processing Systems (NIPS) conference.

•       Proceedings of the International Conference on Machine Learning (ICML).

•      Ng, A. (2017), Lecture notes for CS229 (Machine Learning), Stanford University. http://cs229.stanford.edu/.
 
Time Distribution and Study Load:
In-class lecture/discussion/tutorials: 45 hours. Independent study, homework, and project work: 90 hours.
 
Teaching and Learning Methods:
1.     Use of online resources outside of class: Students will be periodically assigned online video lectures prior to the face-to-face lecture.
2.     Lectures: Lecture notes will be posted online.
3.     In-class tutorials: Tutorials on important data analysis and modeling tools will be given in class periodically.
4.     Homework: Several homework exercises requiring students to apply the knowledge acquired from lecture and discussion will be assigned and graded.
5.     Project: Students will propose and execute a plan for a significant machine learning project in groups of 1-3.
 
Evaluation Scheme:

LO
Assessment method
% marks
1, 2, 7
Term project
30
3-6
Homework
20
1-4
Midterm examination
20
1-6
Final examination
30

A grade of “A” indicates excellent and insightful understanding of the key concepts and ability to implement sophisticated systems; “B” indicates good understanding of the key concepts and ability to implement basic techniques; “C” indicates barely acceptable understanding and implementation ability; “D” indicates poor understanding and implementation ability; “F” indicates complete failure to meet the learning objectives of the class.
 
Instructor(s):
SECTION NAME