Foundations of Machine Learning

Taught by Emtiyaz Khan, AIP-RIKEN and OIST
E-mail at emtiyaz.khan@riken.jp
TA: Thomas Burns, and David Pere Tomas Cuesta
Goals Lecture-Notes Project-Info

News

Lecture Notes

Introduction to Machine Learning Slides Slides with Answers
Course Information Slides Annotated Slides
Linear Algebra (Handout) Slides
Week 1-6 (Linear Regression, SGD, Least-Squares, Ridge Regression, Cross-Validation, Bias-Variance decomposition) Notes Annotated
Week 7-8 (Classification, Logistic Regression, k-NN, DNNs) Notes Annotated

Course Goals

Evaluation is based on the following methods:
  • [40%] Class summary: Students will summarize every two weeks of lecture in their own words (a total of 5 such reports). This needs to be a summary based on understanding and can be as short as 2 pages.
  • [40%] Project report and presentation: students will submit a final project report in Week 13, and present their work in Week 14. The grading will be based on constructive feedback from the class on the project and presentation.
  • [20%] Class discussions

Students successfully completing this course will be able to:
  • Explain a few methods for Regression and Classification.
  • Implement and apply these methods to real data.
  • Discuss fundamental principles of machine learning.
  • Create an assessment of current skill level, and devise a plan for ongoing learning.

Project Information

Every (registered) student is supposed to complete a project, either alone or with another student in a team of two. Grading will be based on the constructive feedback from the class on the project report and presentation.

The main goal of the project is for students to get hands-on experiences on some of the concepts taught in the course. Students are advised to pick a project in consultation with the teaching team. Our hope is that the project will not only teach you but also give you a taste of how it is like to work on ML research projects, and how the submission and review process on conferences like NeurIPS/ICML/ICLR works.

One strategy could be to first think about a concept and topic that you might want to learn about, and choose a project that can be realistically finished in 2 months time. Our recommendation is to not choose too ambitious project, rather some simple and realistic. Remember that it takes (almost always) much more time than what we often anticipate. So one should leave enough room for uncertainties that may arise during the execution of the project.

The 4-page final project report (with supplementary and code) is due on Aug 12, 5pm JST.

To avoid last-minute rush to the deadline, we recommend students to follow the following rough schedule,

  • [June 17] Submit the team information, a potential title and abstract of the project (format to be provided later). After this date, you can still change the title/abstract, but not the team.
  • [July 1] Submit 1 page report with a title, abstract, and a plan. After this date, you can not change the project.
  • [July 15] Refine the plan (resubmit the 1-page report with a refined plan).
  • [Aug 12] Submit 4-page final project report, a supplementary (no page limit), and all the code. You will be graded only on the report, but we may look at the other material whenever necessary.
  • [Aug 16/17] Project presentation (will decide the exact duration based on the number of projects).

Tips for writing the report

You must detail your analysis in a report. You should include complete details of what you did. You should clearly state your conclusions. You should argue that the results you get make sense (or do not make sense), and what could be the reason behind it.

Your report should not be longer than 4 pages!

You must use the latex style given below (in the latex source). We have also given you a sample report that shows the format of this style file. The report details the demo done during an exercise session. Do not copy the content and figures or even the analysis of this report. This report is for illustration purpose only!

Sample Report (from an older course)

Latex Source

Bonus: here is a sample marked report

You can learn Latex using tutorials given below. We will help if you ask us.

You can read the following paper on how to write a machine learning paper. Section 2 and 4 are highly relevant.

Latex Resources

http://www.maths.tcd.ie/~dwilkins/LaTeXPrimer/ - tutorial on Latex

http://www.stdout.org/~winston/latex/latexsheet-a4.pdf - cheat sheet with useful commands for Latex

http://mirror.switch.ch/ftp/mirror/tex/info/first-latex-doc/first-latex-doc.pdf - example how to create a document with Latex

http://en.wikibooks.org/wiki/LaTeX - detailed tutorial on Latex

Checklist for a good report

Here are a few things to check to make sure your report is good enough. Read the following carefully.

  • Your report should not be longer than 4 pages!
  • Your report should include the details of your work, e.g. you can include the following points:
    • What feature transformation or data cleaning did you try? And why?
    • What methods you applied? Why?
    • What worked and what did not? Why do you think are the reasons behind that?
    • Why did you choose the method that you choose?
  • You should include complete details about each algorithm you tried, e.g. what lambda values you tried for Ridge regression? What feature transformation you tried? How many folds did you use for cross-validation? etc.
  • You should include figures or tables supporting your text and your conclusions.
  • Make sure that the captions are included in the figure/tables. A caption should clearly describe the content of its corresponding figure/table.
  • Please label your figures and make sure that the labels and legends are large enough to be read clearly.
  • Make sure that the tick marks and labels are large enough to be clearly read.
  • Your sentences in the report should be clear, concise, and direct.
  • You should clearly state your conclusions.
  • You will loose marks if you did not do things mentioned above.
  • You will loose marks if your written text is vague and not understandable!