24-786 bayesian machine learning 1

24-786: Bayesian Machine Learning

Instructor:
Spring 2019, 12 units Venkat Viswanathan
Department of Mechanical Engineering, Carnegie Mellon University venkvis@cmu.edu
Office hours:
Scott Hall 5109, Monday, 5 PM - 6 PM
Instructor: Professor Venkat Viswanathan
Scott Hall 5109 Teaching Assistants:
Matthew Guttenberg: mguttenb
venkvis@cmu.edu Adarsh Dave: ardave
TA Office Hours: TBD
Lecture: WEH 5415, Monday & Wednesday, 11:30 AM–1:20 PM
Graders:
Siddarth Ghiya: sghiya2
Recommended textbooks: Suyash Narain: snarain
Bayesian Data Analysis, Third Edition by A. Gelman, J. B. Carlin, H. S.
Grading:
Stern, D. B. Dunson, A. Vehtari and D. B. Rubin (CRC Press, 2014), Homeworks 30%
ISBN: 978-1-4398-4095-5. Test 20%
Project 50%
Bayesian Reasoning and Machine Learning by D. Barber (Cambridge Uni-
versity Press, 2012), ISBN: 978-0-521-51814-7. Available online at:
http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=Brml.Online

Description and Objectives

The goal of this course is to introduce Bayesian inference starting from

first principles. The course will cover efficient current approaches to
Bayesian modeling and computation and how can they applied to var-
ious areas of engineering. The topics that will be covered include
Bayesian vs frequentist philosophy, Bayes factors, credible intervals,
Bayesian Analysis of variance (ANOVA), comparison of means, mea-
surement systems analysis (MSA), control charts, survival/reliability
analysis and experiment planning. The course will cover efficient cur-
rent approaches to Bayesian modeling and computation and how can
they applied to various areas of engineering. Students successfully
completing this course will be able to:

• Bayesian vs frequentist philosophy

• Bayes factors (BF)
• Credible intervals
• Measurement systems analysis (MSA)
• Control charts
• Reliability analysis and experiment planning

The prerequisite for this course is an undergraduate-level understand-

ing of statistics, numerical methods and programming.
 24-786 bayesian machine learning 2

Logistics

Class Time
There are two lectures per week. You are responsible for all material
discussed in class, whether you attended or not.

Website
Materials related to the course will be found on Canvas. You are re-
sponsible for all material posted to Canvas.

Communication
The subject of any email sent to the instructor or the course staff should
start with “24-786:"

Grades
Any grading disputes will be handled by the instructor. Any request
for a grade change should be made to the instructor, in writing, within
one week after the graded work is returned. Your entire submission
will be subject to regrading.

Cheating and plagiarism is unethical behavior and is not tolerated in

this course or at Carnegie Mellon University. The Carnegie Mellon
University Policy on Academic Integrity will be strictly followed. Stu-
dents are advised to read and adhere to the policy, which can be found
at http://www.cmu.edu/policies/student-and-student-life/academic-
integrity.html.

(a) Homework assignments: 5, Coding Assignments: 4

There will be 5 homework assignments, all to be submitted in class.

There will also be 4 coding assignments. Your lowest homework as-
signment score will be dropped. You can not drop any coding assign-
ments. Students may work together but must submit their own work
for grading. Discussion about the homework will take place on Piazza
discussion forum. Do not email the instructor or TA with questions.
The instructor and TAs will check Piazza at the end of most days.

(b) Test

In-class test lasting 1 hour and 50 minutes. You must make the instruc-
tor aware of conflicts as soon as possible.
 24-786 bayesian machine learning 3

(c) Project

Course projects are intended to be full-fledged research projects under-

taken by groups of students over the semester. The general theme and
direction of each project is specified by a mentor based in the industry.
The progress and outcomes of the project will be assessed periodically.
Students will be notified about other details on the course project over
the upcoming weeks.

Teaching Philosophy

Students are welcome to ask questions at all times. Don’t be afraid to

interrupt if a point is not clear.

Course Contents

The following topics will be covered [Subject to Change]:

Topic
Introduction
• Hypothesis Testing
• T-test, Z-test, Chi-square test
• Confidence Interval
• P-value
Comparison of Means
Comparison of Distributions
Measurement Systems Analysis (MSA)
Control Charts
Common Pitfalls
Survival/reliability Analysis
Bayesian vs Frequentist Philosophy
Experiment Planning (Data Curation)
ANOVA
Polynomial Chaos Expansion
Normality Testing
Bayesian Inference
Bayesian Machine Learning
• Information Theory
• Regression - Linear, Logistic
• Neural Networks - CNN, RNN
• Bayesian Error Estimation