Standford's Machine Learning (CS 229) by Prof. Andrew Ng

0
9831

This course (CS229) — taught by Professor Andrew Ng — this course provides a broad introduction to machine learning and statistical pattern recognition.

Topics include: supervised learning (generative/discriminative learning, parametric/non-parametric learning, neural networks, support vector machines); unsupervised learning (clustering, dimensionality reduction, kernel methods); learning theory (bias/variance tradeoffs; VC theory; large margins); reinforcement learning and adaptive control.
The course will also discuss recent applications of machine learning, such as to robotic control, data mining, autonomous navigation, bioinformatics, speech recognition, and text and web data processing.
Students are expected to have the following background:

Prerequisites: – Knowledge of basic computer science principles and skills, at a level sufficient to write a reasonably non-trivial computer program.
– Familiarity with the basic probability theory. (Stat 116 is sufficient but not necessary.)
– Familiarity with the basic linear algebra (any one of Math 51, Math 103, Math 113, or CS 205 would be much more than necessary.)

Complete Playlist for the Course:
http://www.youtube.com/view_play_list…

CS 229 Course Website:
http://see.stanford.edu/see/courseInfo.aspx?coll=348ca38a-3a6d-4052-937d-cb017338d7b1
http://www.stanford.edu/class/cs229/

Stanford University:
http://www.stanford.edu/

Stanford University Channel on YouTube:
http://www.youtube.com/stanford

UPDATE (Apr 21, 2013): The online version of this course is being offered at Coursera.org.

Artificial Intelligence | Machine Learning

Instructor: Ng, Andrew

This course provides a broad introduction to machine learning and statistical pattern recognition.

Topics include: supervised learning (generative/discriminative learning, parametric/non-parametric learning, neural networks, support vector machines); unsupervised learning (clustering, dimensionality reduction, kernel methods); learning theory (bias/variance tradeoffs; VC theory; large margins); reinforcement learning and adaptive control.
The course will also discuss recent applications of machine learning, such as to robotic control, data mining, autonomous navigation, bioinformatics, speech recognition, and text and web data processing.
Students are expected to have the following background:

Prerequisites: – Knowledge of basic computer science principles and skills, at a level sufficient to write a reasonably non-trivial computer program.
– Familiarity with the basic probability theory. (Stat 116 is sufficient but not necessary.)
– Familiarity with the basic linear algebra (any one of Math 51, Math 103, Math 113, or CS 205 would be much more than necessary.)

View Lectures and Materials

Lecturer Image

Andrew Ng

Ng’s research is in the areas of machine learning and artificial intelligence. He leads the STAIR (STanford Artificial Intelligence Robot) project, whose goal is to develop a home assistant robot that can perform tasks such as tidy up a room, load/unload a dishwasher, fetch and deliver items, and prepare meals using a kitchen. Since its birth in 1956, the AI dream has been to build systems that exhibit “broad spectrum” intelligence. However, AI has since splintered into many different subfields, such as machine learning, vision, navigation, reasoning, planning, and natural language processing. To realize its vision of a home assistant robot, STAIR will unify into a single platform tools drawn from all of these AI subfields. This is in distinct contrast to the 30-year-old trend of working on fragmented AI sub-fields, so that STAIR is also a unique vehicle for driving forward research towards true, integrated AI.

Ng also works on machine learning algorithms for robotic control, in which rather than relying on months of human hand-engineering to design a controller, a robot instead learns automatically how best to control itself. Using this approach, Ng’s group has developed by far the most advanced autonomous helicopter controller, that is capable of flying spectacular aerobatic maneuvers that even experienced human pilots often find extremely difficult to execute. As part of this work, Ng’s group also developed algorithms that can take a single image,and turn the picture into a 3-D model that one can fly-through and see from different angles.

Complete Course Material Downloads:

Course Handouts: The ZIP file below contains all of the course handouts for this course. If you do not need the complete course, individual documents can be downloaded from the course content pages.

Artificial Intelligence | Machine Learning

Instructor: Ng, Andrew

Course Meetings: 20

Lecture 1    View Now >
1 hr 9 min

  • Topics: The Motivation & Applications of Machine Learning, The Logistics of the Class, The Definition of Machine Learning, The Overview of Supervised Learning, The Overview of Learning Theory, The Overview of Unsupervised Learning, The Overview of Reinforcement Learning
  • Transcript: HTML | PDF

YouTube | iTunes | WMV Torrent | MP4 Torrent
[hr]
Lecture 2    View Now >
1 hr 16 min

  • Topics: An Application of Supervised Learning – Autonomous Deriving, ALVINN, Linear Regression, Gradient Descent, Batch Gradient Descent, Stochastic Gradient Descent (Incremental Descent), Matrix Derivative Notation for Deriving Normal Equations, Derivation of Normal Equations
  • Transcript: HTML | PDF

YouTube | iTunes | WMV Torrent | MP4 Torrent
[hr]
Lecture 3    View Now >
1 hr 13 min

  • Topics: The Concept of Underfitting and Overfitting, The Concept of Parametric Algorithms and Non-parametric Algorithms, Locally Weighted Regression, The Probabilistic Interpretation of Linear Regression, The motivation of Logistic Regression, Logistic Regression, Perceptron
  • Transcript: HTML | PDF

YouTube | iTunes | WMV Torrent | MP4 Torrent
[hr]
Lecture 4    View Now >
1 hr 13 min

  • Topics: Newton’s Method, Exponential Family, Bernoulli Example, Gaussian Example, General Linear Models (GLMs), Multinomial Example, Softmax Regression
  • Transcript: HTML | PDF

YouTube | iTunes | WMV Torrent | MP4 Torrent
[hr]
Lecture 5    View Now >
1 hr 16 min

  • Topics: Discriminative Algorithms, Generative Algorithms, Gaussian Discriminant Analysis (GDA), GDA and Logistic Regression, Naive Bayes, Laplace Smoothing
  • Transcript: HTML | PDF

YouTube | iTunes | WMV Torrent | MP4 Torrent
[hr]
Lecture 6    View Now >
1 hr 14 min

  • Topics: Multinomial Event Model, Non-linear Classifiers, Neural Network, Applications of Neural Network, Intuitions about Support Vector Machine (SVM), Notation for SVM, Functional and Geometric Margins
  • Transcript: HTML | PDF

YouTube | iTunes | WMV Torrent | MP4 Torrent
[hr]
Lecture 7    View Now >
1 hr 16 min

  • Topics: Optimal Margin Classifier, Lagrange Duality, Karush-Kuhn-Tucker (KKT) Conditions, SVM Dual, The Concept of Kernels
  • Transcript: HTML | PDF

YouTube | iTunes | WMV Torrent | MP4 Torrent
[hr]
Lecture 8    View Now >
1 hr 17 min

  • Topics: Kernels, Mercer’s Theorem, Non-linear Decision Boundaries and Soft Margin SVM, Coordinate Ascent Algorithm, The Sequential Minimization Optimization (SMO) Algorithm, Applications of SVM
  • Transcript: HTML | PDF

YouTube | iTunes | WMV Torrent | MP4 Torrent
[hr]
Lecture 9    View Now >
1 hr 14 min

  • Topics: Bias/variance Tradeoff, Empirical Risk Minimization (ERM), The Union Bound, Hoeffding Inequality, Uniform Convergence – The Case of Finite H, Sample Complexity Bound, Error Bound, Uniform Convergence Theorem & Corollary
  • Transcript: HTML | PDF

YouTube | iTunes | WMV Torrent | MP4 Torrent
[hr]
Lecture 10    View Now >
1 hr 13 min

  • Topics: Uniform Convergence – The Case of Infinite H, The Concept of ‘Shatter’ and VC Dimension, SVM Example, Model Selection, Cross Validation, Feature Selection
  • Transcript: HTML | PDF

YouTube | iTunes | WMV Torrent | MP4 Torrent
[hr]
Lecture 11    View Now >
1 hr 22 min

  • Topics: Bayesian Statistics and Regularization, Online Learning, Advice for Applying Machine Learning Algorithms, Debugging/fixing Learning Algorithms, Diagnostics for Bias & Variance, Optimization Algorithm Diagnostics, Diagnostic Example – Autonomous Helicopter, Error Analysis, Getting Started on a Learning Problem
  • Transcript: HTML | PDF

YouTube | iTunes | WMV Torrent | MP4 Torrent
[hr]
Lecture 12    View Now >
1 hr 14 min

  • Topics: The Concept of Unsupervised Learning, K-means Clustering Algorithm, K-means Algorithm, Mixtures of Gaussians and the EM Algorithm, Jensen’s Inequality, The EM Algorithm, Summary
  • Transcript: HTML | PDF

YouTube | iTunes | WMV Torrent | MP4 Torrent
[hr]
Lecture 13    View Now >
1 hr 15 min

  • Topics: Mixture of Gaussian, Mixture of Naive Bayes – Text clustering (EM Application), Factor Analysis, Restrictions on a Covariance Matrix, The Factor Analysis Model, EM for Factor Analysis
  • Transcript: HTML | PDF

YouTube | iTunes | WMV Torrent | MP4 Torrent
[hr]
Lecture 14    View Now >
1 hr 21 min

  • Topics: The Factor Analysis Model,0 EM for Factor Analysis, Principal Component Analysis (PCA), PCA as a Dimensionality Reduction Algorithm, Applications of PCA, Face Recognition by Using PCA
  • Transcript: HTML | PDF

YouTube | iTunes | WMV Torrent | MP4 Torrent
[hr]
Lecture 15    View Now >
1 hr 17 min

  • Topics: Latent Semantic Indexing (LSI), Singular Value Decomposition (SVD) Implementation, Independent Component Analysis (ICA), The Application of ICA, Cumulative Distribution Function (CDF), ICA Algorithm, The Applications of ICA
  • Transcript: HTML | PDF

YouTube | iTunes | WMV Torrent | MP4 Torrent
[hr]
Lecture 16    View Now >
1 hr 13 min

  • Topics: Applications of Reinforcement Learning, Markov Decision Process (MDP), Defining Value & Policy Functions, Value Function, Optimal Value Function, Value Iteration, Policy Iteration
  • Transcript: HTML | PDF

YouTube | iTunes | WMV Torrent | MP4 Torrent
[hr]
Lecture 17    View Now >
1 hr 17 min

  • Topics: Generalization to Continuous States, Discretization & Curse of Dimensionality, Models/Simulators, Fitted Value Iteration, Finding Optimal Policy
  • Transcript: HTML | PDF

YouTube | iTunes | WMV Torrent | MP4 Torrent
[hr]
Lecture 18    View Now >
1 hr 17 min

  • Topics: State-action Rewards, Finite Horizon MDPs, The Concept of Dynamical Systems, Examples of Dynamical Models, Linear Quadratic Regulation (LQR), Linearizing a Non-Linear Model, Computing Rewards, Riccati Equation
  • Transcript: HTML | PDF

YouTube | iTunes | WMV Torrent | MP4 Torrent
[hr]
Lecture 19    View Now >
1 hr 16 min

  • Topics: Advice for Applying Machine Learning, Debugging Reinforcement Learning (RL) Algorithm, Linear Quadratic Regularization (LQR), Differential Dynamic Programming (DDP), Kalman Filter & Linear Quadratic Gaussian (LQG), Predict/update Steps of Kalman Filter, Linear Quadratic Gaussian (LQG)
  • Transcript: HTML | PDF

YouTube | iTunes | WMV Torrent | MP4 Torrent
[hr]
Lecture 20    View Now >
1 hr 17 min

  • Topics: Partially Observable MDPs (POMDPs), Policy Search, Reinforce Algorithm, Pegasus Algorithm, Pegasus Policy Search, Applications of Reinforcement Learning
  • Transcript: HTML | PDF
    [hr]

Leave a Reply