This past fall, I took Stanford’s class on machine learning. Overall, it was a terrific experience, and I’d like to share a few thoughts on it:
- A lot of participants were concerned that it was a watered down version of Stanford’s CS229. And, in fact, the course was more limited in scope and more applied than the official Stanford class. However, I found this to be a strength. Because I was already familiar with most of the methods in the beginning (linear and multiple regression, logistic regression), I could focus more on the machine learning perspective that the class brought to these methods. This helped in later sections where I wasn’t so familiar with the methods.
- The embedded review questions and the end of section review questions were very well done, with some randomization algorithm making it impossible to guess until everything was right.
- Programming exercises were done in Octave, an open source Matlab-like programming environment. I really enjoyed doing this programming, because it meant I essentially programmed regression and logistic regression algorithms by hand with the exception of a numerical optimization algorithm. I got a huge confidence boost when I managed to get the backpropagation algorithm for neural networks correct. Emphasis on these exercises was on the loops, which you could code using “slow” loops (for loops, for instance), but then really needed to vectorize using the principles of linear algebra. For instance, there was an algorithm for a recommender system that would take hours if coded using for loops, but ran in minutes using a vectorized implementation. (This is because the implicit loops of vectorization were run using optimized linear algebra routines.) In statistics, we don’t always worry about implementation details so much, but in machine learning situations, implementation is important because these algorithms often need to run in real time.
- The class encouraged me to look at the Kaggle competitions. I’m not doing terribly well in them, but now at least I’m hacking on some data myself and learning a lot in the process.
- The structure of the public class helps a lot over, for example, the iTunes U version of the class. But now I’m looking at the CS 229 lectures on iTunes U and am understanding them a lot more now.
- Kudos to Stanford for taking the lead on this effort. This is the next logical progression of distance education, and takes a lot of effort and time.
I also took the databases class, which was even more structured with a mid-term and final exam. This was a bit of a stretch for me, but learning about data storage and retrieval is a good complement to statistics and machine learning. I’ve coded a few complex SQL queries in my life, but this class really took my understanding of both XML-based and relational database systems to the next level.
Stanford is offering machine learning again, along with a gaggle of other classes. I recommend you check them out. (Find a list, for example, at the bottom of the page of Probabilistic Graph Models site.) (Note: Stanford does not offer official credit for these classes.)
CS229 Final Project Information
One of CS229's main goals is to prepare you to apply machine learning algorithms to real-world tasks, or to leave you well-qualified to start machine learning or AI research. The final project is intended to start you in these directions.
For group-specific questions regarding projects, please create a private post on Piazza, or email email@example.com. Please first have a look through the frequently asked questions.
Your first task is to pick a project topic. If you're looking for project ideas, please come to project office hours, and we'd be happy to brainstorm and suggest some project ideas. In the meantime, here are some suggestions that might also help.
Most students do one of three kinds of projects:
- Application project. This is by far the most common: Pick an application that interests you, and explore how best to apply learning algorithms to solve it.
- Algorithmic project. Pick a problem or family of problems, and develop a new learning algorithm, or a novel variant of an existing algorithm, to solve it.
- Theoretical project. Prove some interesting/non-trivial properties of a new or an existing learning algorithm. (This is often quite difficult, and so very few, if any, projects will be purely theoretical.)
Many fantastic class projects come from students picking either an application area that they're interested in, or picking some subfield of machine learning that they want to explore more. So, pick something that you can get excited and passionate about! Be brave rather than timid, and do feel free to propose ambitious things that you're excited about. (Just be sure to ask us for help if you're uncertain how to best get started.) Alternatively, if you're already working on a research or industry project that machine learning might apply to, then you may already have a great project idea.
A very good CS229 project will be a publishable or nearly-publishable piece of work. Each year, some number of students continue working on their projects after completing CS229, submitting their work to a conferences or journals. Thus, for inspiration, you might also look at some recent machine learning research papers. Two of the main machine learning conferences are ICML and NIPS. You can find papers from recent ICML conferences online: https://2017.icml.cc/Conferences/2017/Schedule. All NIPS papers are online, at http://books.nips.cc/. Finally, looking at class projects from previous years is a good way to get ideas.
Once you have identified a topic of interest, it can be useful to look up existing research on relevant topics by searching related keywords on an academic search engine such as: http://scholar.google.com. Another important aspect of designing your project is to identify one or several datasets suitable for your topic of interest. If that data needs considerable pre-processing to suit your task, or that you intend to collect the needed data yourself, keep in mind that this is only one part of the expected project work, but can often take considerable time. We still expect a solid methodology and discussion of results, so pace your project accordingly.
Notes on a few specific types of projects:
- Deep learning projects: Since CS229 discusses many other concepts besides deep learning, we ask that if you decide to work on a deep learning project, please make sure that you use other material you learned in the class as well. For example, you might set up logistic regression and SVM baselines, or do some data analysis using the unsupervised methods covered in class. We may grade these projects using different criteria to make sure that grading is fair for students who have not had exposure to DL before. Finally, training deep learning models can be very time consuming, so make sure you have the necessary compute. Unfortunately, we will not be providing compute in this course, although both Google Cloud and Microsoft Azure offer free academic units.
- Preprocessed datasets: While we don't want you to have to spend much time collecting raw data, the process of inspecting and visualizing the data, trying out different types of preprocessing, and doing error analysis is often an important part of machine learning. Hence if you choose to use preprepared datasets (e.g. from Kaggle, the UCI machine learning repository, etc.) we encourage you to do some data exploration and analysis to get familiar with the problem.
- Replicating results: Replicating the results in a paper can be a good way to learn. However, we ask that instead of just replicating a paper, also try using the technique on another application, or do some analysis of how each component of the model contributes to final performance.
Project Parts: Proposal, Milestone, Poster, & Final Report
This section contains the detailed instructions for the different parts of your project.
Submission: We’ll be using Gradescope for submission of all four parts of the final project. We’ll announce when submissions are open for each part. You should submit on Gradescope as a group: that is, for each part, please make one submission for your entire project group and tag your team members.
We will not be disclosing the breakdown of the 40% that the final project is worth amongst the different parts, but the poster and final report will combine to be the majority of the grade. Projects will be evaluated based on:
- The technical quality of the work. (I.e., Does the technical material make sense? Are the things tried reasonable? Are the proposed algorithms or applications clever and interesting? Do the authors convey novel insight about the problem and/or algorithms?)
- Significance. (Did the authors choose an interesting or a “real" problem to work on, or only a small “toy" problem? Is this work likely to be useful and/or have impact?)
- The novelty of the work. (Is this project applying a common technique to a well-studied problem, or is the problem or method relatively unexplored?)
In the project proposal, you'll pick a project idea to work on early and receive feedback from the TAs. If your proposed project will be done jointly with a different class' project, you should obtain approval from the other instructor and approval from us. Please come to the project office hours to discuss with us if you would like to do a joint project.
In the proposal, below your project title, include the project category. The category can be one of:
|Project mentors||Based off of the topic you choose in your proposal, we’ll suggest a project mentor given the areas of expertise of the TAs. This is just a recommendation; feel free to speak with other TAs as well.|
Your proposal should be a PDF document, giving the title of the project, the project category, the full names of all of your team members, the SUNet ID of your team members, and a 300-500 word description of what you plan to do.Your project proposal should include the following information:
|Grading||The project proposal is mainly intended to make sure you decide on a project topic and get feedback from TAs early. As long as your proposal follows the instructions above and the project seems to have been thought out with a reasonable plan, you should do well on the proposal.|
The milestone will help you make sure you're on track, and should describe what you've accomplished so far, and very briefly say what else you plan to do. You should write it as if it's an “early draft" of what will turn into your final project. You can write it as if you're writing the first few pages of your final project report, so that you can re-use most of the milestone text in your final report. Please write the milestone (and final report) keeping in mind that the intended audience is Profs. Boneh and Ng and the TAs. Thus, for example, you should not spend two pages explaining what logistic regression is. Your milestone should include the full names of all your team members and state the full title of your project. Note: We will expect your final writeup to be on the same topic as your milestone.
|Contributions||Please include a section that describes what each team member worked on and contributed to the project. This is to make sure team members are carrying a fair share of the work for projects. If you have any concerns working with one of your project teammates, please fill out this optional teammate evaluation form (only seen by the teaching staff).|
|Grading||The milestone is mostly intended to get feedback from TAs to make sure you’re making reasonable progress. As long as your milestone follows the instructions above and you seem to have tested any assumptions which might prevent your team from completing the project, you should do well on the milestone.|
|Format|| Your milestone should be at most 3 pages, excluding references. Similar to to the proposal, it should include |
The class projects will be presented at a poster presentation on 12/12. Each team should prepare a poster, and be prepared to give a very short explanation, in front of the poster, about their work. At the poster session, you'll also have an opportunity to see what everyone else did for their projects. We will supply poster-boards and easels for displaying the posters.
|Format||Here are some poster guidelines (please note that 36x24in means 36in wide by 24in tall, i.e. it's better if your poster is formatted landscape). You can also look at posters from previous years. Note: Despite example given in guidelines, posters with nice, illustrative figures are preferred over posters with lots of text.|
|Grading||We will be grading posters on the poster quality and clarity, the technical content of the poster, as well as the knowledge demonstrated by the team when discussing their work with teaching staff at the poster session.|
Because the teaching staff will have only a few hours to see a large number of posters at the poster session, we'll only be able to get an overview of the work you did at the session. We know that most students work very hard on the final projects, and so we are extremely careful to give each writeup ample attention, and read and try very hard to understand everything you describe in it.
After the class, we will also post all the final writeups online so that you can read about each other's' work. If you do not want your write-up to be posted online, then please create a private Piazza post or contact us at firstname.lastname@example.org at least a week in advance of the final submission deadline.
Final project writeups can be at most 5 pages long (including appendices and figures). We will allow for extra pages containing only references. If you did this work in collaboration with someone else, or if someone else (such as another professor) had advised you on this work, your write-up must fully acknowledge their contributions. For shared projects, we also require that you submit the final report from the class you're sharing the project with.Here's more detailed guidelines with a rough outline of what we expect to see in the final report: cs229-final-report-guidelines.pdf.
|Contributions||Please include a section that describes what each team member worked on and contributed to the project. If you have any concerns working with one of your project teammates, you can also fill out this optional teammate evaluation form (only seen by the teaching staff). We may reach out and factor in contributions and evaluations when assigning project grades.|
|Code||Please include a zip file or peferably a link to a Github repository with the code for your final project. You do not have to include the data or additional libraries (so if you submit a zip file, it should not exceed 5MB). If you have a private repository, please add the CS229 staff account as a collaborator.|
|Grading||The final report will be judged based off of the clarity of the report, the relevance of the project to topics taught in CS229, the novelty of the problem, and the technical quality and significance of the work.|
After CS229, if you want to submit your work to a machine learning conference, the ICML deadline will probably be in early February next year (http://icml.cc), and the NIPS deadline is usually in early June (http://nips.cc/). Of course, depending on the topic of your project, other non-machine learning conferences may also be more appropriate.
- What are the deliverables as part of the term project?
- The project has four deliverables:
- Final report
Please refer to the course schedule page for information about deadlines. We will post more details about each each on the website and on Piazza.
- Should final project use only methods taught in classroom?
No, we don't restrict you to only use methods/topics/problems taught in class. That said, you can always consult Project Tif you are unsure about any method or problem statement.
- Is it okay to use a dataset that is not public ?
We don't mind you using a dataset that is not public, as long as you have the required permissions to use it. We don't require you to share the dataset either as long as you can accurately describe it in the Final Report.
- Is it okay to combine the CS229 term project with that of another class ?
- In general it is possible to combine your project for CS229 and another class, but with the following caveats:
- You should make sure that you follow all the guidelines and requirements for the CS229 project (in addition to the requirements of the other class). So, if you'd like to combine your CS229 project with a class X but class X's policies don't allow for it, you cannot do it.
- You cannot turn in an identical project for both classes, but you can share common infrastructure/code base/datasets across the two classes.
- Clearly indicate in your milestone and final report, which part of the project is done for CS229 and which part is done for a class other than CS229. For shared projects, we also require that you submit the final report from the class you're sharing the project with.
- Do all team members need to be enrolled in CS229?
No, but please explicitly state the work which was done by team members enrolled in CS229 in your milestone and final report. This extends to projects that were done in collaboration with research groups as well.
- What are acceptable team sizes and how does grading differ as a function of the team size ?
We recommend teams of 3 students, while teams sizes of 1 or 2 are also acceptable. The team size will be taken under consideration when evaluating the scope of the project in breadth and depth, meaning that a three-person team is expected to accomplish more than a one-person team would.
The reason we encourage students to form teams of 3 is that, in our experience, this size usually fits best the expectations for the CS229 projects. In particular, we expect the team to submit a completed project (even for team of 1 or 2), so keep in mind that all projects require to spend a decent minimum effort towards gathering data, and setting up the infrastructure to reach some form of result. In a three-person team this can be shared much better, allowing the team to focus a lot more on the interesting stuff, e.g. results and discussion.
In exceptional cases, we can allow a team of 4 people. If you plan to work on a project in a team of 4, please come talk to one of the Project TAs beforehand so we can ensure that the project has a large enough scope.
- Do I have to be on campus to submit the final report?
No, the final report will be submitted via Gradescope.
- How do SCPD students submit the poster?
As an SCPD student you have the choice to either (1) attend the poster session or (2) submit the poster PDF on Gradescope.
- Is it okay for non-SCPD students to miss the poster session?
- Part of your project grade part depends on your presentation at the poster session, so we really urge you not to miss it. That said, if (and only if) you have a final exam conflict there are a few possibilities:
- If your other class offers an alternative time for the exam, you should choose that.
- If you are working on the project as a team, the rest of the team could present the poster without you there.
- If none of above options work for you, come talk to one of the Project TAs or send us an email at email@example.com.
- What fraction of the final grade is the project?
The term project is 40% of the final grade.