
^{4 credit points}
_{Start: September 2009}
_{End: }_{December 2009}
_{Frequency: Annually, fall semester}
Angela Schoellig, Sebastian Trimpe
Wednesdays
13:0016:00, ML H 44
Dynamic Programming and Optimal Control by Dimitri P. Bertsekas, Vol. I, 3rd edition, 2005, 558 pages, hardcover.
Description:
Dynamic Programming Algorithm; Deterministic Systems and Shortest Path Problems; Infinite Horizon Problems; Value/Policy Iteration; Deterministic ContinuousTime Optimal Control; Problems with Perfect State Information.
Book: Dynamic Programming and Optimal Control by Dimitri P. Bertsekas, Vol. I, 3rd edition, 2005, 558 pages, hardcover.
Requirements: Knowledge of advanced calculus, introductory probability theory, and matrixvector algebra.
Feb 18 
Final Grades:

Jan 26 
Summary of information regarding the final examination:

Jan 15 
For the final examination, all material that was covered in the lecture or the exercise class is relevant. 
Jan 13 
During the semester break, we offer office hours on Jan 19 (Tue), Jan 25 (Mon) and Jan 26 (Tue) at the usual time and location (see section 'Office Hours' below). 
Dec 17 
Final examination The final exam takes place on Jan 28, 2009 from 09:0011:30. You can bring one A4 sheet of paper (double sided). No calculators or other aids are allowed. Last exercise class (Dec 16) Programming Exercise 2 
Dec 14 
This year's last office hour is on Tuesday, Dec 15. We will offer office hours again in the week before the exam. We will publish more details on time and location soon. 
Dec 14  The results and solutions of Quiz 2 are available for download. See section Quizzes and Programming Exercises. You may take a look at the correction of your quiz during office hours. 
Dec 11 
The Problem Set Problems with Perfect State Information is online. 
Dec 09 
The results of Programming exercise 2 are online (see Section Quizzes and Programming Exercises).
The students whose initially submitted code was not correct have the option of resubmitting their solution until Dec 16. Please submit by email to aschoellig(at)ethz.ch. 
Dec 03 
Information regarding the second quiz taking place on Dec 09:

Nov 26 
The results of programming exercise 1 have been updated. Resubmissions of the code were taken into account. 
Nov 18 
The Problem Set Deterministic Continuoustime Optimal Control is online. 
Nov 18 
The results of Programming exercise 1 are online (see Section Quizzes and Programming Exercises).
The students whose initially submitted code was not correct have the option of resubmitting their solution until Nov 25. Please submit by email to strimpe(at)ethz.ch. 
Nov 18 
Guideline for Programming Exercise 2: The value iteration algorithm can be terminated if J_{k}(i)J^{*}(i)<0.1. 
Nov 16 
Note: In Programming Exercise 2, script.m, it should read: [valueIterationJ, valueIterationF] = ssp_ValueIt(P, policyIterationJ). A corrected version is online. 
Nov 16  Slides on the Stochastic Shortest Path Problem are found here. 
Nov 13 
The Problem Set Infinite Horizon Problems is online. 
Nov 03 
The Problem Set Infinite Horizon Problems will include the problems 7.1, 7.3, 7.12, and 1.23 from the class textbook. The solutions will be available next week. 
Oct 30 
The results and solutions of Quiz 1 are available for download. See section Quizzes and Programming Exercises. You may take a look at the correction of your quiz during office hours. 
Oct 21 
The programming exercise 1 is online. It is due on Nov 04. 
Oct 15 
An updated version of the Problem Set 1 has been uploaded. Please keep reporting errors that you may find in the problem sets to the TAs. We will keep a list of all corrections of previous versions of the problem sets, which you can download at the end of the problem sets section or here. 
Oct 13 
There is a typo in the fact sheet that we distributed in the first lecture. Instead of "Imperfect State Information" in lecture 12 and 13 and the corresponding problem set, it should say "Perfect State Information." The updated sheet can be downloaded here. 
Oct 08 
Information regarding the quiz taking place on Oct 14:

Oct 07  The slide shown in the exercise class which summarizes the Dynamic Programming Algorithm is found here. 
Sep 24 
We got a new class room: from now on, the lectures and exercises will take place in ML H44. 
Sep 23 
The class textbook is being sold by the Polybuchhandlung during the break of the next lecture (Sep 30, 14:00). It will cost 159 CHF (139 CHF for Master students) and has to be paid in cash. You can also order the book through the publisher (Athena Scientific), or any other store you like. 
Sep 22 
The first lecture takes place on Sep 23. Exercises and office hours start the week after.
The fact sheet that will be handed out in the first lecture can be downloaded here. 
Sep 22 
The website has been updated (office hours, quizzes/programming exercises, problem set 1 uploaded). 
Aug 19  The class website has been updated with information on grading, quizzes/programming exercises, and problem sets. 
Jul 22 
There will be more information about the class Dynamic Programming and Optimal Control (Fall 09) on this page soon.
Last year's class website (Fall 08) can be found here. 
Instructor  Prof. Raffaello D'Andrea 
Teaching Assistants  Angela Schoellig, Sebastian Trimpe 
Lecture 
Wednesday, 13:15 to 15:00, ML H44 
Exercise 
Wednesday, 15:15 to 16:00, ML H44 
Office hours 
Monday, 18:30 to 19:30, ML K37.1 (IDSC library), Angela
Tuesday, 16:00 to 17:00, ML K37.1 (IDSC library), Sebastian During the semester: every Monday and Tuesday; After the semester: Jan 19, Jan 25, Jan 26. 
Exam 
Final written exam during the examination session, covers all material 
Grading 
40% quizzes/programming exercises, 60% final exam if the grade for quizzes and programming exercises is better than the grade in the final exam; 100% final exam otherwise. 
PhD students will get credits for the class if they pass the class (final grade of 4.0 or higher). 

Repetition 
The final exam is only offered in the session after the course unit. Repetition is only possible after reenrolling. Students who took the class in Fall 08 and have to retake the course have the option of keeping their old homework grades. They have to inform the teaching assistants before the beginning of the new class. 
#  date  topic  reading 
1 
Sep 23 
The Dynamic Programming Algorithm 
1.1  1.4 
2 
Sep 30 
The Dynamic Programming Algorithm 
1.1  1.4 
3 
Oct 07 
The Dynamic Programming Algorithm 
1.1  1.4 
4 
Oct 14 
Deterministic Systems and the Shortest Path Problem 
2.1  2.3 
5 
Oct 21 
Deterministic Systems and the Shortest Path Problem 
2.1  2.3 
6 
Oct 28 
Infinite Horizon Problems, Value Iteration, Policy Iteration 
7.1  7.3 
7 
Nov 04 
Infinite Horizon Problems, Value Iteration, Policy Iteration 
7.1  7.3 
8 
Nov 11 
Infinite Horizon Problems, Value Iteration, Policy Iteration 
7.1  7.3 
9 
Nov 18 
Deterministic ContinuousTime Optimal Control 
3.1  3.4 
10 
Nov 25 
Deterministic ContinuousTime Optimal Control 
3.1  3.4 
11 
Dec 02 
Deterministic ContinuousTime Optimal Control 
3.1  3.4 
12 
Dec 09 
Problems with Perfect State Information: Linear Systems and Quadratic Cost 
4.1 
13 
Dec 16 
Problems with Perfect State Information: Linear Systems and Quadratic Cost 
4.1 
During the semester, there will be graded quizzes and programming exercises, which can be used to improve the final grade for the course (see "grading"). Quizzes will take place at the beginning of the lecture and will test the student's understanding of the corresponding topic. The programming exercises will require the student to apply the lecture material.
Up to three students can work together on the programming exercises. If they do, they have to hand in one solution per group and will all receive the same grade.
#  type  topic  dates  download 
Q1  Quiz  The Dynamic Programming Algorithm 
Oct 14 
Results Solutions 
P1  Programming  Deterministic Systems and the Shortest Path Problem 
Oct 21 (issued) Nov 04 (due) 
Exercise MatlabTemplate 
P2  Programming  Value/Policy Iteration 
Nov 11 (issued) Nov 25 (due) 
Exercise MatlabTemplate 
Q2  Quiz  Deterministic ContinuousTime Optimal Control 
Dec 09 
Results Solutions 
We will make sets of problems and solutions available online for the chapters covered in the lecture. It is the student's responsibility to solve the problems and understand their solutions. The TAs will answer questions in office hours and some of the problems might be covered during the exercises.
# 
topic 
download 
1 
The Dynamic Programming Algorithm 
ProblemSet1.pdf 
2  Deterministic Systems and the Shortest Path Problem 
ProblemSet2.pdf 
3  Infinite Horizon Problems, Value Iteration, Policy Iteration 
ProblemSet3.pdf 
4  Deterministic ContinuousTime Optimal Control 
ProblemSet4.pdf 
5  Problems with Perfect State Information: Linear Systems and Quadratic Cost  ProblemSet5.pdf 
List of corrected errors in previous versions of the problem sets.
Wichtiger Hinweis:
Diese Website wird in älteren Versionen von Netscape ohne
graphische Elemente dargestellt. Die Funktionalität der
Website ist aber trotzdem gewährleistet. Wenn Sie diese
Website regelmässig benutzen, empfehlen wir Ihnen, auf
Ihrem Computer einen aktuellen Browser zu installieren. Weitere
Informationen finden Sie auf
folgender
Seite.
Important Note:
The content in this site is accessible to any browser or
Internet device, however, some graphics will display correctly
only in the newer versions of Netscape. To get the most out of
our site we suggest you upgrade to a newer browser.
More
information