Machine Learning 10-601 in Fall 2014

Announcements

• Important announcements will be made here as well as on Piazza.
• Mid-term: there will be a block midterm, with one test shared between the two sections, scheduled for 7-9pm October 29.
• Start of classes: the first lecture for 10-601B (William's class, Tu/Th) will be Tuesday 9/2. The first lecture for 10-601A (Ziv's class, M/W) will be Wed 8/27.
• 9/10: Homework 1 has been announced and is available on the syllabus page.
• 9/19: Homework 2 has been announced and is available on the syllabus page.
• 9/26: Homework 3 has been announced and is available on the syllabus page.

Important People and Places

There are two sections for the course, which will be closely coordinated.

• For both sections:
  • Sandy Winkler, sandyw@cs.cmu.edu is the course secretary.
  • Syllabus for Machine Learning 10-601 in Fall 2014 with lectures slides and homeworks
  • We'll be using BlackBoard and Autolab for most assignments, and Piazza for general Q/A. The lectures are recorded by MediaTech.
    • Autolab - submit programs
    • Piazza - announcements and discussion
    • Blackboard page - grades and non-programming assignments
    • Video Lectures for 10-601A - MediaTech
    • Video Lectures for 10-601B - MediaTech
• For Section 10-601A
  • Instructor: Ziv Bar-Joseph, Lane Center and Machine Learning Dept
  • When/where: Mon/Wed 1:30-2:50 WEH 7500
  • TAs:
    • Abhinav Maurya ahmaurya+10601@gmail.com - PhD
    • Ying Yang yingyan1@andrew.cmu.edu - PhD
    • Qihui (Anna) Li qihuil@andrew.cmu.edu - MS
    • Jingwei Shen js1@andrew.cmu.edu - MS
    • Yipei Wang yipeiw@andrew.cmu.edu - MS
    • Henry (Harry) Gifford hgifford@andrew.cmu.edu - senior
• For Section 10-601B
  • Instructor: William Cohen, Machine Learning Dept and LTI
  • When/where: Tues/Thurs 1:30-2:50 WEH 7500
  • TAs:
    • Siddhartha Jain sjX [at] cs.cmu.edu where X=1 - PhD
    • Debjani Biswas dbiswas@cmu.edu - MS
    • Kuo Liu kuo.liu.pku@gmail.com - MS
    • Daniel Ribeiro Silva drsilva@andrew.cmu.edu - MS
    • Jin Sun jins@andrew.cmu.edu - MS
    • Xu Zhuo zhuox@cmu.edu - MS

Recitations

There are four recitations held by two TAs each week. Students can go to any section they like. Attendance is optional but highly recommended. In recitations, TAs will usually review the important contents in the lectures, cover (extra) materials that students need to know for homework, answer students' questions, and help students to prepare for the exam and project.

Time slots and locations can be found below. Content (slides and code) can be found on syllabus page (scroll down to the bottom). Please be aware that the recitation schedule changes in some of the weeks. Also, note that assignments in Sep/Oct are due on Thursdays, and assignments in Nov/Dec are due on Mondays.

• Monday 8pm: PH 125C -- ending Nov 3
• Tuesday 6pm: HH B131
• Tuesday 8pm: DH112
• Wed 7pm: GHC 4307 (in HH B103 Sept 17, Sept 24, Oct 1)
• Thursday 8pm: DH112 -- starting Nov 6

Office hours

Instructor	Day	Time	Location
William	Monday	11am	GHC 8217
Ziv	Monday	3pm (after class)	GHC 8006
Kuo Liu	Monday	8:30am	GHC 6418
Qihui Li	Tuesday	11am	GHC 7404
Jingwei Shen	Tuesday	5pm-6pm	GHC 5th floor Citadel Public Area
Sid Jain	Tuesday	5 pm	GHC 6505
Yipei Wang	Friday	3pm	GHC 6405
Ying Yang	Friday	4pm-5pm	Baker Hall 434, (follow the sign near Suite 332)
Abhinav Maurya	Saturday	10:30am - 11:30am (email by 10am if you plan to come)	Hamburg Hall - 3rd Floor Faculty Lounge
Daniel Ribeiro SIlva	Tuesdays and Thursdays	3pm (after Cohen's lectures)	GHC 5411

Description

Machine Learning (ML) asks "how can we design programs that automatically improve their performance through experience?" This includes learning to perform many types of tasks based on many types of experience, e.g. spotting high-risk medical patients, recognizing speech, classifying text documents, detecting credit card fraud, or driving autonomous robots.

Topics covered in 10-601 include concept learning, version spaces, decision trees, neural networks, computational learning theory, active learning, estimation & the bias-variance tradeoff, hypothesis testing, Bayesian learning, Naïve Bayes classifier, Bayes Nets & Graphical Models, the EM algorithm, Hidden Markov Models, K-Nearest-Neighbors and nonparametric learning, reinforcement learning, bagging and boosting, neural networks, and other topics.

10-601 focuses on the mathematical, statistical and computational foundations of the field. It emphasizes the role of assumptions in machine learning. As we introduce different ML techniques, we work out together what assumptions are implicit in them. Grading is based on written assignments, programming assignments, and a final exam.

10-601 focuses on understanding what makes machine learning work. If your interest is primarily in learning the process of applying ML effectively, and in the practical side of ML for applications, you should consider Machine Learning in Practice (11-344/05-834).

10-601 is open to all but is recommended for CS Seniors & Juniors, Quantitative Masters students, and non-SCS PhD students.

Syllabus and Text

Syllabus for Machine Learning 10-601, including lecture slides and HWs

• Syllabus for Machine Learning 10-601 in Fall 2014

Previous syllabi, for the historically-minded:

• Syllabus for Machine Learning 10-601 in Fall 2013 - William and Eric Xing's class from fall 2013
• Ziv's 701 lectures
• Ziv's class with Tom fall 2012
• Roni's 10-601 syllabus

The text is Tom Mitchell's textbook, Machine Learning. It is recommended but not required.

Prerequisites

Formal prerequisites:

• Prerequisites are 15-122, Principles of Imperative Computation AND 21-127: Concepts of Mathematics.
• Additionally, a probability course is a co-requisite: 36-217: Probability Theory and Random Processes OR 36-225: Introduction to Probability and Statistics I
• A minimum grade of 'C' is required in all these courses.

Self-assessment for students:

• Students, especially graduate students, come to CMU with a variety of different backgrounds, so formal course prereqs are hard to establish. There is a short self-assessment test to see if you have the necessary background for 10-601. We recommend that all students take this before enrolling in 10-601 to see if they have the necessary background knowledge already, or if they need to review and/or take additional courses.

Refresher material:

A few resources that can help you review the math required to do well in a machine learning course:

• Linear Algebra: Review, Cheatsheet
• Probability: Review, Cheatsheet

Some other reviews you might be interested in:

• Zico Kolter, a prof in CSD, has put up a set of video lectures that review linear algebra.
• Very recently, Aaditya Ramdas, a grad student in MLD, has put up some video reviews of multivariate calculus and multivariate probabilities and stats.

To assess whether you need to watch these, you should do the self-assessment test, which is linked to on the wiki.

Grading Policy

• Semi-final exam: 30%
  • Instead of a final exam, we have an exam in class, the evening of 10/29.
• Weekly homeworks (out Wed, due Wed): 50%
  • Late assignment policy: We will grant up to 50% credit if an assignment is less than 48 hrs late. Also, you can drop your lowest assignment grade entirely.
• Project: 20% (see below)

Projects

• Instructions (draft) - also listed on the syllabus for the week ending 11/14.

Policy on Collaboration among Students

These policies are the same as were used in Dr. Rosenfeld's previous version of 2013.

The purpose of student collaboration is to facilitate learning, not to circumvent it. Studying the material in groups is strongly encouraged. It is also allowed to seek help from other students in understanding the material needed to solve a particular homework problem, provided no written notes are shared, or are taken at that time, and provided learning is facilitated, not circumvented. The actual solution must be done by each student alone, and the student should be ready to reproduce their solution upon request.

The presence or absence of any form of help or collaboration, whether given or received, must be explicitly stated and disclosed in full by all involved, on the first page of their assignment. Specifically, each assignment solution must start by answering the following questions:

(1) Did you receive any help whatsoever from anyone in solving this assignment? Yes / No.
If you answered 'yes', give full details: _______________ (e.g. "Jane explained to me what is asked in Question 3.4")
(2) Did you give any help whatsoever to anyone in solving this assignment? Yes / No.
If you answered 'yes', give full details: _______________ (e.g. "I pointed Joe to section 2.3 to help him with Question 2".

Collaboration without full disclosure will be handled severely, in compliance with CMU's Policy on Cheating and Plagiarism.

As a related point, some of the homework assignments used in this class may have been used in prior versions of this class, or in classes at other institutions. Avoiding the use of heavily tested assignments will detract from the main purpose of these assignments, which is to reinforce the material and stimulate thinking. Because some of these assignments may have been used before, solutions to them may be (or may have been) available online, or from other people. It is explicitly forbidden to use any such sources, or to consult people who have solved these problems before. You must solve the homework assignments completely on your own. I will mostly rely on your wisdom and honor to follow this rule, but if a violation is detected it will be dealt with harshly. Collaboration with other students who are currently taking the class is allowed, but only under the conditions stated below.