00230212: Introduction to Morden AI (现代人工智能导论) (Fall 2024)
Course Information
When: Tuesday 9:50 pm – 12:15 pm.
Where: 清华大学旧水利馆303
Instructor: Jianzhu Ma (马剑竹)
Email: majianzhu at tsinghua dot edu dot cn.
Office Hour: Wednesday 19:30 pm – 20:30 pm.or by appointment (send email) Where: TBD
Course description:
This course aims to provide a comprehensive overview of the principles, technologies, and impacts of modern artificial intelligence (AI). Beginning with historical context, the course delves into the development of AI, emphasizing the transition from theoretical concepts to practical applications. Students will learn the fundamental concepts of AI, including machine learning, deep learning, and computational learning theory, with specific applications covering computer vision (CV), natural language processing (NLP), and computational biology. Through theoretical studies and handson projects, the course aims to equip students with the skills to understand and apply AI technologies across various fields, providing a solid foundation for their application of AI in various industries in the modern world.
Prerequisites:
Students are expected to have the following background:
Basic programming skills to write a reasonably nontrivial computer program in Python.
Basic understanding of statistics, linear algebra.
Understanding of data science. Knowing how to work with data and analyze.
Optional textbooks
“Probabilistic Machine Learning: An Introduction” by Kevin Murphy
“Pattern Recognition and Machine Learning” by Christopher Bishop
“Dive into Deep Learning” by Aston Zhang, Zach Lipton, Mu Li, Alex Smola
Grading
Homework (50%)
Final projects (50%)
Assignments
There will be NINE homework assignments including both FIVE theory problems and FOUR programming problems.
Programming projects need to be written in Python. Deep learning final projects should be written in PyTorch.
Late policy
Assignments are to be submitted by the due date listed. Assignments will NOT BE accepted if they are submitted late. Additional extensions will be granted only due to serious and documented medical or family emergencies.
Final projects
Students are encouraged to work as teams of two or three. Each team can either choose the provided projects on the course website or a separate project related to Biology. If you choose to do your own project, please contact me beforehand to discuss it. Here are some possible directions:
An interesting mathematical problems around a paper.
Adopting a developed computational framework on a new dataset.
Followingup experiments of an existing work to understand its important properties.
Simple extension of an existing machine learning model, such as unsigned network to signed network, trees to DAGs, shallow neural networks to deep neural networks.
Using machine learning model to solve a problem in your own research area.
Syllabus (tentative)
Time  Topic  Contents 
09/10  Introduction & Linear Regression  (1) Introduction to AI history; (2) Introduction to Machine Learning; (3) Probability; (4) Linear Regression Optional Reading: Linear Regression Made Simple: A StepbyStep Tutorial [link] 
09/17  LASSO  (1) Biasvariance decomposition; (2) Description length; (3) Ridge regression; (4) L0, L1 and LP penalty; (5) Stepwise regression 
09/24  Gaussian Distribution  (1) Loglikelihood; (2) Revisiting linear regression; (3) Mixture Gaussian and EM algorithm 
10/08  Logistic Regression  (1) Classification & Regression; (2) Logistic model; (3) Gradient ascent/descent; (4) Regularization 
10/15  Neural Network  (1) Neural network structure (2) Backpropagation; (3) Optimization; (4) Initialization; (5) Dropout; (6) Batch normalization; 
10/22  Support Vector Machine  (1) Objective and Optimization; (2) Lagrange duality; (3) Kernel trick 
10/29  Deep Learning  (1) Convolutional Neural Network; (2) Recurrent Neural Network; (3) Graph Neural Network; (4) Transformer Optional reading: (1) Attention Is All You Need [pdf] (2) Transformer from scratch using pytorch [link] (3) TRANSFORMER EXPLAINER [link] 
11/05  Reinforcement Learning  (1) Qlearning; (2) Policy Gradient; (3) ActorCritic Algorithm 
11/12  Graph Neural Network  (1) Graph Convolutional Network; (2) Graph Attention Model; 
11/19  Boosting  (1) AdaBoost; (2) Additive regression 
11/26  Nonparametric Methods  (1) Kernel density estimation; (2) KNN; (3) Principle Component Analysis; (4) Kmeans 
12/03  Learning theory  (1) Occam’s Razor/No free lunch; (2) Basic error bounds; (3) Hoeffding’s Inequality; (4) Union bound; (5) VC theory; (6) Rademacher complexity; (7) PAC bound 
12/10  Graphical Models (I)  (1) General concepts (2) Exact inference (3) Sumproduct algorithm (4) Maxproduct algorithm (5) Conditional Random Fields Optional textbooks: (1) “Probabilistic Graphical Models” by By Daphne Koller and Nir Friedman (2) “Graphical Models, Exponential Families, and Variational Inference” by Martin J. Wainwright and Michael I. Jordan (3) Chapter 8 of “Pattern Recognition and Machine Learning” by Christopher M. Bishop 
12/17  Graphical Models (II)  (1) Structure learning (2) Gaussian Graphical Model (3) Pseudolikelihood approximation (4) Protein contact prediction (5) DeepLearningbased structure learning Optional reading: (1) Sparse Inverse Covariance Estimation with the Graphical Lasso [pdf] (2) HighDimensional Graphs and Variable Selection with the Lasso [pdf] 
12/24  Project presentations  TBD

