Final Project Information

The final project will be your opportunity to take what you have learned in the class and put it towards an application of your choice.

In previous years, it has been very popular to choose a problem that you are passionate about (e.g. a dolphin surfing on waves, controlling a yo-yo, brachiation as upside-down walking, ...), then find the absolute simplest way to formulate a model/problem for this system, then apply some of the tools from class to that problem (trajectory optimization, region of attraction, control design, ...).

Here are some guidelines:

  • Apart from rare exceptions, your problem should have apply to a system with dynamics, and should take an algorithmic approach to analyzing/controlling those dynamics.
  • You are welcome to work on the project in small teams (2-3 people). If you plan on doing a team project, please make this clear in your proposal. Our expectations for your project will be proportional to the number of people on the team.
  • The project is only over 4-5 weeks, so scope your project carefully.
  • Developing novel algorithms/systems is great, but reimplementing published work and/or applying a published algorithm to a model system from class is also great. You will be evaluated based on whether the project forced you to understand new concepts, not based on the novelty of the approach.
  • Do not underestimate the time it can take to set up a simulation in place. Using/modifying an existing example from the textbook is great. Using Drake is not a requirement, but the staff can help you more if you choose this route.
  • Some of you will be able to do a project that is closely related to your current research. This is completely acceptable, as long as the subproblem you are solving for this class is clearly defined. If you are using part or all of your work for research or as part of another class, you must state this explicitly in your final report. We expect you to do something new that you would not have done if you hadn’t taken this class (e.g., I was already working on robot X, but I was using RRT for trajectory planning, and here I tried trajectory optimization).
  • The goal of the project is to deepen each student's mastery of some topic from class. If you do work in a similar field, and submit an incremental paper in that line of work that makes no connections to the class material, then that does not constitute a successful project.
  • Please don't underestimate the time it could take to get a basic simulation of your system in place. Using/modifying an existing example from the textbook is a great idea. Using Drake is not a requirement, but the staff can help you more if you choose this route.
  • We encourage to build on code and tools of the community, but remember that you must clearly distinguish your contribution and acknowledge the work of others to avoid plagiarism. If you do improve an open-source tool or example, continue contributing back!
  • You can find some great final projects from previous semesters here. I would love to add your work to the list!

The course staff will provide you with feedback on the scope on direction of your proposal shortly after we receive it.

Project Ideas

Here are a set ideas we believe might be worth investigating, but do not feel the need to grab one of these!

  • Sums-of-squares approximate dynamic programming. I think this idea can go farther than what I've shown in the course notes; but we don't actually know how far. You can help us find out! We would start with some simple problems (like the pendulum), but try to scale up from there.
  • Explore different control parameterizations for gradient-based policy search. How well do deep nets solve our favorite benchmarks vs e.g. polynomial feedback?
  • PyTorch trajectory optimization. There are many interesting variants to explore that would exploit parallelization. Talk to Russ if you're interested.
  • System identification / model learning. Try comparing multibody methods vs linear or deep models for a task of your choice.
  • Value iteration on systems with discontinuous value functions. How well can we make neural value iteration (example from the text) work? Are ReLU networks really a natural representation for this type of problem? Are there better representations? Or study some of the more classical (non-neural) algorithms like upwind discretization or variable-resolution value iteration.
  • Racecar. A number of groups have been converging on a model car driving quickly around a small track as a benchmark problem for e.g. model-predictive control. This paper could be a starting point.
  • Bicycle/Airplane/Quadrotor driving/flying through a random forest/building. Via online model-predictive control? Or could you find a simple feedback controller that does not depend on knowing/estimating the map?
  • Simple legged-robot walking on rough terrain. The RABBIT model from Grizzle might be a nice place to start.
  • System-level Synthesis. Implement SLS feedback control in place of LQR or time-varying LQR for one or more of the model systems.
  • Can you find an optimization-based interpretation of a classical controller? E.g. Mark Spong's swing-up controllers? Marc Raibert's hopping control? For instance, the vertical dynamics of the hopper (from Assignment 4) are hybrid linear; is there a natural problem formulation for which mixed-integer MPC produces the same control inputs? If not, how are they different?
  • Robust trajectory optimization. We've talked about trajectory optimization and robust control, mostly separately. There are a few approaches that attempt to incorporate robustness into the trajectory design (search google for e.g. Robust MPC). Try one of these algorithms on a model system.

Example Projects

Each semester students put together a final project using the tools from this class. Many of them are fantastic! Here is a small sample.

Spring 2020:

Even before we started posting project presentations on YouTube, some great projects from class have turned into full publications. Here are a few examples:

Requirements

  • A short project proposal is due on Friday, March 18.
  • A revision of the proposal is due on Wednesday, April 6.
  • Brief (1-paragraph) progress updates is due Wednesday, April 13.
  • There will be a face-to-face checkin with the staff the week of April 25.
  • A video presentation of your results is due of Friday, May 6th
  • A final report along with any code you'd like to provide (we'll love it!) is due on Tuesday, May 10.

Project Proposal

The project proposal is intended as a forcing function for you to crystalize a project idea. Moreover, it gives us a chance to offer you a feedback and make sure that your plan is plan is feasible in the time alloted to the project. It should be approximately one page (at most, say, 500 words) detailing your idea for the project. It should:

  • Define exactly what the project deliverable is.
  • Briefly describe why the project is interesting.
  • List the topics that we have studied (or that we will study) in this class that are covered by your project. For example: dynamic programming, Lyapunov theory, legged robots, reinforcement learning etc.
  • Discuss any related prior work you have found that is relevant. If this project is related to your research or a project you are doing concurrently for another class, tell us about that now (and read the guidelines for the final report below)..
  • Define specific goals that you expect to have accomplished before each of the progress updates.
It does not have to be extremely polished, but it does need to provide us with enough information to understand what you're hoping to do. Otherwise, we won't be able to help you!

Progress Updates

We will expect 1-paragraph project updates in order to keep us up to date with how you have progressed.

Final Presentation

A video presentation of your project is due by: Friday, May 6. The presentation must have duration of 3 minutes for individual projects and 5 minutes for group projects. Please try to share the 5 minutes evenly between the various members of a group. This year we will require you to upload presentations on YouTube, more details on this will be given on Piazza.

Final Report

We expect you to provide a final report of your project by Tuesday, May 10. Using the IEEE template for conference proceedings (probably the Latex one, unless you really enjoy using MS Word), write a summary of what you accomplished during your project. Write it, as much as possible, like a conference paper. You should include:

  • An abstract.
  • An introduction of your project and why you think it is interesting.
  • A related work section (at your discretion -- this part will depend a lot on the nature of the work you do).
  • Your results (partial or work-in-progress results is expected, and fine!).
  • A discussion of your results and potential next steps.
  • For team projects: please describe the contributions made by each team member.
  • For projects related to your research or a project in another class, please clearly denote what parts of the project overlap (or not) with the other efforts.
For projects that do overlap with your research, please make connections to the topics in the class. A report that might have been written if you had not taken the class does not constitute a successful project. The goal of the project is to demonstrate some level of mastery of the material from class.

We don't have any requirements on length, but realize that this is worth a very large portion of your overall grade, so be as thorough as possible in demonstrating mastery of the course material. For reference, reasonable lengths could be anywhere between 3 and 8 pages, with figures and a few citations where relevant. Videos are encouraged as well, as a way to showcase simulation/hardware results and provide visual explanation.