DD2434 Machine Learning, Advanced Course

Resultat av kursutvärdering


    Thanks for filling out the student survey, it is quite long, but your input is extremely important and valuable!


  1. Was the scope of the course clear from the start?

    1. 29% (12 st) Yes.
    2. 45% (19 st) Relatively clear.
    3. 24% (10 st) Not so much.
    4. 2% (1 st) Not at all.

  2. Did you have the right prerequisites for the course?

    1. 76% (32 st) Yes, completely agree
    2. 19% (8 st) Somewhat.
    3. 5% (2 st) No, not at all.

    Comments on the difficulty level of the course with respect to prerequisites:

    Having strong mathamatical background from first years of university
    ---
    The course is quite difficult especially for students being in their first year of studies.
    ---
    Some parts required quite a lot of math/derivations without enough guidance. Jens's part (graphical models) remains a mistery to me... we didn't do a lot of practice for it so I don't see how I could use that on an actual project.

    Assignements' wise, Carl Henrik's required *WAY* too much work to get an E. I spent a couple sleepless nights to just get to that level.

    ---
    I missed a lot of background. Especially the first part part by Jen's made me watch hours of lectures more material selected (except the book) would be very helpfull, to study what is needed and not look for material to study with.
    Carl's part was easier because a lot of literature was available, but I was lacking a lot of time.
    Hedvig's part seemed easier, the concept was mathematically simpler and by then we know all basics anyway so it had to feel more simple.

    ---
    Requires better understanding of statistics and probability theory than required. Which I had so to me the course was great.
    ---
    Prerequisites were not properly explained in the course page
    ---
    For me it was fine but it seems there are students without multivariate calculus that might find it hard.
    ---
    My bachelor is Computer Science so some of the mathematics in Assignment 2 was quite hard for me but the rest of the course was OK.
    ---
    I did not find it too difficult. As far as general machine learning knowledge goes, the basic course (DD2431) was enough to be able to understand the new ML concepts introduced. I did not have a problem with the maths either (but as an engineering physics student I imagine I have a more extensive maths background than is required for this course).
    ---
    The prerquisites were not clear. I think a general background in probability theory, statistics and programming was necessary. It was not necessarry to take the first machine learning course.
    ---
    Medium-high, and I'd like to say that being a last year student in the Machine Learning master's degree and NOT HAVING all the prerequisites sounds crazy for the purpose of this course.
    ---
    The course difficulty level was OK, the only problem was that it varied a bit too much between all three parts.
    ---
    I like when it was difficult in the way that you knew what you were supposed to do, just how to do it was difficult. The difficulty "What am I supposed to do?" or "What does htis question really ask?" was sometimes annoying or took too much time.
    ---
    Probabilities background was what was missing for me understanding the whole context of the class much easier.
    ---
    The basic Machine Learning course did in no aspect prepare students for the level of this course. It is very basic and goes not into detail.
    Probability theory and statistics on Bachelor level do not provide sufficient knowledge to understand the contants of this course - mainly get the gist of it. Another student asked me what a vector norm is - that is the mathematical level of some students.

    ---
    Not difficult.
    ---
    I had taken the basic machine learning course, along with several other courses with ML elements, and I had no idea what I was doing in this course.
    ---
    I think I had, but it was still a hard course. Learned a lot.
    ---
    The assignments took some fair time to complete. I would say that the workload over the course was more than 7.5 credits
    ---
    I had some problems with the linear algebra part which involved Schur complement and the matrix differentiation, but it was just a small part. I also didn't know Dynamic programming prior to this course.
    ---
    I am studying the maths master, and my answer is therefore biased. I would assume that one has a rather good grasp of the mathematics required after taking the Probability Theory course. Especially as the probability theory course has Fourier analysis as a prerequisite, which should imply knowledge of most vital parts of the course (even those noted as difficulties on the course web, such as complex numbers).
    ---
    I found this course very difficult, and was not prepared for it solely based on the first machine learning course I took at KTH. I thought that this course would build more on the concepts we learned in the first ML course, but instead it seemed to introduce a few disjoint techniques in 3 separate ML areas, with no common foundation or explanation of where these techniques fit overall in the field of Machine Learning. There was no continuity. I think the level of this course would be right for PhD students, but not for students who have only taken the first ML course at KTH. Especially the level of probability theory knowledge expected was very high. Most of us attending this course had only seen Bayes rule once or twice before, we had no practical experience with it, and especially the first and second part of the course were very overwhelming with the amount of theory covered, and the amount of theory we were "expected" to have known already.
    ---
    The course made extensive use of multivariate calculus and linear algebra. None of which are listed as prerequisites (recommended or otherwise) and the course provided no help or guidance in reviewing the necessary parts. Furthermore both the book and the lectures assumed perfect familiarity with ML concepts and abreviations suggesting that extensive prior experience with ML would be a requisite.
    ---
    After having taken the basic machine learning course it was one step up in the difficulty level. but that was expected since this is an advanced course.
    ---
    Much more theoretically difficult than basic course. Good for us with a more mathematical background, probably worse for others.
    ---
    Way too difficult with way too much material for the assignments. This was at least 9cr worth of work.


  3. Did you find the course content relevant as a part of your degree?

    1. 81% (34 st) Yes, completely agree
    2. 19% (8 st) Somewhat.
    3. 0% (0 st) No, not at all.

  4. Did you enjoy the course?

    1. 40% (17 st) Yes, very much so.
    2. 33% (14 st) Yes.
    3. 12% (5 st) Neutral.
    4. 2% (1 st) Not particularly.
    5. 12% (5 st) No not at all.

    Comments on the focus of the course, e.g., the focus on theory vs practice.

    The assignment and project are too theoritical , some practical example could be brought to student (see my remarks on project below)
    ---
    I think the focus on theory is still too heavy on some parts (Jens's part and some of the parts we saw with Carl Henrik) but I guess that's a good thing.
    ---
    It gave me an exciting insight and I don't want to miss that knowledge. To see the "crazy" math and then discover how it really works was great. I guess it was good that way.
    ---
    I liked that it took time to explain the principles behind the methods (especially second part with Carl Henrik)
    ---
    Good combination
    ---
    Assignment work load is much higher than the given time to solve.
    ---
    Good balance.
    ---
    My master is in Machine Learning so the course was very relevant and also very interesting.
    ---
    I liked the computer assignments and the project very much. The lectures were ok, but too few for to many different topics.
    ---
    I expected the course to focus more on theory. I wish we had at least a part that goes into Statistical Learning theory.
    ---
    I wish the course went deeper into theory.
    ---
    I spent way too much time implemented stuff that was not even evaluated, I would suggest allowing the use of a library instead, like sci-kit and have a focus on practical usage of these algorithms on REAL problems.
    ---
    I think that theory and practice were well balanced between each other. Nevertheless, the amount of effort needed for a student to accomplish a good result was too much.
    ---
    Alright!
    ---
    Too much focus on theory with not enough practice to back it up. The learning curve was far too difficult for all parts of the course.
    ---
    I enjoy it know, because I would not have liked to leave KTH without the knowledge I got during the course, but the assignments was hard for me.
    ---
    The theory vs practice weight where good. I believe that the course could be improved by using more concepts with smaller labs assigned to each concept.
    ---
    I felt it was a good combination.
    ---
    the course had a nice balance between theory and implementation and the assignments were tough and great.
    ---
    I really enjoyed the course and found it to be one of the most interesting courses I've taken. Especially the focus on hand-ins rather than exams, as I believe that knowledge of machine learning might be hard to test in an exam setting. Hand-ins allowed me to focus on deeper knowledge of the methods.

    Furthermore, I really liked that the assignments did not guide us excessively towards a solution, but was left a bit open-ended.

    ---
    I liked the idea of the homework assignments as I feel that practical work is the best way to learn. So from that perspective the course had the right focus (having homework assignments and a project where we got to implement in practice what we were learning in theory). However, none of the assignments (except maybe 3) were appropriate for the amount of time we had to complete them. Especially assignment 2 was completely squashed out by assignment 1, and I am sorry but telling students that they need to be working on assignments 1 and 2 at the same time is just not a reasonable resolution to this problem.
    The theory we learned for this course: I wish it was more grounded, and I wish each section had some very well explained fit into the overall picture. As it is, the course felt disjoint, and no one had an explanation as to why we were studying these 3 main topics instead of other ML topics. I guess my main point is: there was no structure to the knowledge presented, and no vision given on how everything fits together in the big picture of the ML field.

    ---
    The fact that the course was the most difficult and with the heaviest workload that I've ever experienced completely overshadowed any focus it may have had. In combination with the super-strict, tight and grade affecting deadlines this course not only made me feel discriminated but the constant stress followed by repeated failure to meet the deadlines regardless of my best and constant efforts also made me genuinely hate Machine Learning - the subject that I previously found fascinating enough to pick the machine learning master. Adding insult to injury I had to see how students who resorted to solve the situation in less honorable ways were awarded with higher grades.
    ---
    It would have been better if the assignments involved more self thinking, for example, if you taught us several machine learning techniques and then gave us a data set and said, use whatever technique you want to get the best results.
    ---
    Constant struggle with poor assignment instructions, nearly nonexistent help from theachers and short deadlines


  5. What did you think about the book (Murphy)?

    1. 12% (5 st) Fantastic!
    2. 38% (16 st) OK, I learned a lot from it.
    3. 24% (10 st) OK, but would have preferred another book.
    4. 19% (8 st) I did not learn much from it.
    5. 5% (2 st) It did not give me anything at all.

    Comments on the book, e.g., suggestions of alternative books:

    It was OK in the sense that it corresponded to the goal of the course which was exactly machine learning from a probabilistic perspective. Though, I found it hard to understand and not thorough when it comes to basic concepts.
    ---
    I would've prefered something more practical maybe, that would still show how different methods work but focus on when they work better (and why, without getting into derivations) and such.

    The only examples I have in mind are made for developers and not necessarily master students (like Bayesian methods for hackers) so I don't think they'd necessarily be a good fit

    ---
    I often used other sources to understand the idea and that made me able to understand the pure mathematical explanation of the book.
    ---
    The book could use an editors touch, it seem like a compilation of different short introductions with different notations and names for the method in every section. However the content of the book is really good and I think you should keep it. I have already used it to read up on other methods.
    ---
    The diposition of the book is the only good thing, otherwise it is awful.
    ---
    I used different books for each topics where I felt more comfortable.
    ---
    It is not good on teaching new concepts, just a bunch of equations, we are humans not machines. But it was good to use after lectures which were good at introducing concepts.
    ---
    I'm not a huge fan of reading entire course literture and learned more from implementing and the labs. However, the parts I did read I found useful and overall I liked the book.
    ---
    Maybe combine Murphy's book and Bishops book.
    ---
    It had quite some errors and confusing notation. Many chapters were just copied from other sources and not properly integrated in Murphys "book". It foucses too much on formulas instead of intuition.
    ---
    I totally hate Murphys book, it’s utterly worthless. It feels like a copy-paste, from all kind of different sources, very inconsistent. I would much rather see Bishops pattern recognition book.
    ---
    I never got the book so I never learned anything from it.
    ---
    I have no suggestions for other books. But my "problem" with Murphy was that it wasn't always comprehensible!
    ---
    The book was good and I learned a lot from it, but I think that maybe a book that will explain in a more simple language the terms and facts of this course would have been a better choice. At least for people that are not Machine Learning students, but maybe come from a different department.
    ---
    I like it. But I think it is not very educational, when you do not know a lot of mathematics. Variables just turn up out of nowhere and you have to guess and accept facts in order to get through the derivations.
    ---
    Murphy's book is excellent but some parts of it is hard to grasp for lack of related background.
    ---
    Maybe point to Bishop?
    ---
    I only took a few informations out of the book and had to search the rest in the internet. There I found better sources for the most topics.
    ---
    It was harder reading than I've encountered in other courses, which made it difficult. But I learned from it a lot! It was a good complement to Marsland's course book from the first ML course.
    ---
    I would say that is more of a encyclopedia manner. Which is a good since since you can read about a lot of interesting parts and still constrained to a manageable amount of pages
    ---
    Bishop Pattern Recosgnition is also a great book with simpler language in a lot of concepts.
    ---
    The book is a bit dense, and not necessarily always clear. However, as it seems to be the definitive book in this topic, I would not have preferred another.
    ---
    The Murphy book seems to collect concepts introduced in other books, and not explain them very well (or at least not as well as the original book authors explained them). For example, I found Bishop's book much more accessible when doing assignment 2 (which was where the figure was taken from in Murphy's book). At least from my perspective I liked Bishop's book a lot better.
    ---
    Since I was always confounded during lectures the natural reaction would be to fall back to the book and try to get a grip on things. There are, however, quite a few reviews on amazon clearly stating that this is not a book for anyone who's not already very engaged in machine learning on a daily basis. Some argue it reads more like a reference but it's constant failure to introduce and define concepts, abbreviations and even mathematical variables before using them makes it worthless even as such. When you already know what you want to do and how to do it this book gives you a mathematical formulation of how to do it, nothing more. Trying to learn anything for the first time from this book will waste a student's time and enthusiasm. Most papers I've read are more pedagogical and well-formed although I would in no way recommend them as the main course literature.

    I have found but one alternative and it was the sole thing that managed to ever so slightly rekindle the enthusiasm I once had for ML. The YouTube channel mathematicalmonk goes through the concepts in an equally formal way but with excellent examples, intuitions and structure. And most importantly, he _never_ assumes that the reader already knows what he means by an abbreviation or the variable name x.

    ---
    It might have been a good book to read if it was a small book and we were needed to read most of the content. But it was a bit difficult to read read bits and pieces from the book. I felt like it broke the flow of the book. It was hard to follow his style of writing by just reading bits and pieces. Which is why I opted to study most of the material from online courses
    ---
    It doesn't add much to the lectures, I was hoping for more detailed explanations and examples


  6. What did you think of the course design (3 computer assignments 4 ECTS performed individually, project 3.5 ECTS performed in group, no written exam)?

    Perfect, but deadline were too tight.
    ---
    I liked the course design, I also believe that a project is always a better learning experience than an exam. However I disagree with:
    1) the scheduling of the 3 assignments, we did not have enough time to actually comprehend what we were learning
    2) The project groups should have been formed not by the grade of each student but by their actual interest. E.g. I worked on a paper that I was not at all interested in because I had to respect my team's interest. I believe that a team interested on the same thing would have a higher motivation than a team just aiming for the same grade

    ---
    I think it was good! Please don't add an exam, that just adds a lot of stress on us and forces us to remember things by heart (unlike projects that let us do research and implementation, which is more like what we'd do in our future careers)
    ---
    No written exam is awesome! The 3 assignments kept me working all the time, which is good (no exam focused learning).
    ---
    Really good.
    ---
    Great!
    ---
    First assignment : I did more work , but I didnt get a time to explain it. It made me upset after the personal interview. The ultimate aim of the interview to explain what we have learned, but that was not actually happened. But for Assignment 2 and 3, It was much better and satisfied.
    ---
    It was good, but the project destroyed christmas as I did nothing but debugging code for 10 whole days.
    ---
    The course design was good but I think that the 3 assignments were a bit to large all together and each of them took a lot of time to complete. It was good that the criteria for the different grades was changed to only account for the two best assignments.
    ---
    The idea and concept is good, I like the focus on labs and 'learning by doing'. Doing the project over christmas was not ideal though, as we (my group) were not all in Sweden during this time and therefore not able to meet up as much as maybe we would have wanted to.
    ---
    Good but not enough support for project. When given a project I believe simply giving a journal article is not enough. A project description is required and a supervisor is needed. Also I would have prefered to recieve the project from the start of the course not a few days before christmas as if we are not expected to rest for holidays. We compensated by working 7days a week the last 2-3 weeks to coop with other courses and labs. This course was respectless. Also in 3.5 credits you are supposed to work about 93 hours maximum. I worked 103 hours for these 3.5 credits and my other group members probably worked even more. Still we got a low grade.
    ---
    Homeworks were well designed. They were helpful to see how the theories that we learned in the class would work in practice. The project thought us how to analyse paper from start to end and apply it.
    ---
    The design is good.
    However, the difficulty of the assignments may be better to be incremental.

    ---
    It was fine, only the deadlines for the assignments were too quickly.
    ---
    I think that the time given for each assignment is not enough, given that we have other courses and assignments in the same period. It would be nice to have some liberty on when to start in order to respect the deadlines by beginnining to work earlier when we don't have other stuff to do, even if the course has not yet covered the topic of the assignment.
    ---
    I prefer this design to most other course designs.
    ---
    I think the design is great. Not having to worry about a written exam allows to dig deeper in the topics and really get an understanding of them.
    ---
    Good
    ---
    The course design was good. However, it could have been organized in a better way. The assignments were challenging and had less time to complete it.
    ---
    It was a good system, which challenged the students individually and forced them to work in a large group.
    ---
    I absolutely love to have assignment and a project instead of an exam. But this course was a total mess here. Group projects should be graded P-F and don’t have these crappy tables for lower grades when presenting later. THEY ARE TERRIBLE and makes it a harder to plan for. Instead have a set of passing assignments, and a set of bonus assignments. You could then have a score table, something like this.
    A 60
    B 50-60
    C 40-30
    D 20-40
    That allow me to focus on assignments I find interesting, and allows for a wide verity of assignment to be created. If I like graphical models. I can specialize in that. Of course have a set of minimum mandatory assignment for E. Have 1 hard deadline at the end of the course and a set of lab sessions when you can present your result. There is also no point in having individual assignments when people are working together anyway.

    ---
    I love it =)
    ---
    It is a very good design!
    ---
    The course design was good. In my opinion, substituting the exam with project and computer assignments is much better as engages the student to study through the whole period/semester and not just for the exam. Nevertheless, the effort needed was too much and maybe you should consider "spreading" the course in two periods instead of one, cause the pressure is too high in conjunction with other courses that still have projects and assignments too.
    ---
    That is a lot of work for 7.5 credits as the labs are very heavy. I think I spend at least 2-3 times as much time on the labs compared to the project.
    ---
    ok
    ---
    I very much enjoyed the project, but maybe 2 assigments might be enough, the course was really quite stressful.
    ---
    Assignments are fine (although they need a lot of work to be fair). I personally spent upwards of 15 hours on each assignment, and was unable to complete the first two on time, and will probably end up with an E in the end anyway. For implementation work there needs to be a much better explanation in the lectures or in the assignment itself, so that it's possible to do it without spending hours and hours trying to work out what to do. The whole point of assignments is supposed to be to give more understanding, not lead to confusion and stress. Break things down more and do the implementation parts in small steps so that we can understand what we are doing. The project is a bad choice, especially because of the random group choice and the difficulty of re-implementing a published paper. That depends on the paper, of course, but from experience barely anyone works over the christmas break so work gets compressed and rushed. If the project is more sensible then it is preferable to an exam.
    ---
    It is a nice design. I liked it, because in the homeworks you learn the most.
    ---
    it was good. A lot of hard work, but you learned a lot. I didn't like that the parts were overlapping, because I really had to concentrate on the current assignment all the time, so it was for example hard to take in the information from Part 2s lectures when you were working full time with Part 1 assignment and the same about starting with the project when you were full time working with assignment 3.
    ---
    I do not think that I got much out of the Project work. I would vote for more labs
    ---
    The work load of the three computer assignments were far from 4 ECTS. Even if that is the credits we earn from them. The course felt more like a 9 - 10 ECTS
    ---
    I believe that the focus on hand-ins rather than exams was really beneficial to the subject seeing as knowledge of machine learning might be hard to test in an exam setting. Hand-ins allowed me to focus on deeper knowledge of the methods.
    ---
    I liked the course design. But the implementation left a lot to be desired. The 3 assignments were of very high difficulty and time consuming. The way groups were formed for the projects is questionable, as one can argue that at the end, people ended up being in groups of other people they may or may not have wanted to work with, and working on a project they had little input in choosing. So some students end up in a group they didn't want, working on a project they have little interest in working on. I think students should be able to pick their own groups, and their own projects - so what if multiple groups work on the same project?
    ---
    While I like concept and believe that, if done right it, it has the potential of leaving students of this course with more long-lived and in-depth knowledge than any pre-exam cramming can give I strongly believe that the deadlines should not give anything beyond a few bonus points. The grade affecting deadlines encourages cheating, causes unnecessary stress and severely discriminates against those who happen to have a tight schedule before the deadlines or happen to have more prior knowledge. It also discourages good coding practices and heavily penalises effort and the ambition of attaining solid understanding.

    The group project was good although, given the impact it potentially had on the grade there should be an opportunity for individual supplemental assignments to raise the shared grade.

    ---
    I thought the no written exam part was excellent. We learn so much more when we actually sit in front of a data set and understand it and apply techniques to it. An exam would not be able to assess us very well. Like I mentioned before the assignments could have been made more about understanding machine learning techniques instead of derivations and stuff.
    ---
    The assignments need to be shorter, have more instructions and have specific help sessions.


  7. How many lectures did you attend in Part 1 of the course (Lectures 2-5 followed by Assignment 1, held by Jens Lagergren)?

    1. 76% (32 st) 3-4.
    2. 17% (7 st) 1-2.
    3. 2% (1 st) 0.

  8. How did you find the lectures in Part 1 of the course (Lectures 2-5 followed by Assignment 1, held by Jens Lagergren)?

    1. 2% (1 st) Excellent.
    2. 10% (4 st) Good.
    3. 17% (7 st) Ok.
    4. 40% (17 st) Not so good.
    5. 29% (12 st) A waste of time.

    Comments on Lectures 2-5:

    Lack of pedagogy from the teacher.
    ---
    Very scientific without letting us understand the underlying motivation.
    ---
    Sorry Jens, but I really couldn't understand the subtlties of graphical models by coming to these lectures. I still don't understand those algorithms to get independance sets and how all this is supposed to work.
    ---
    He really explained the math, but I was missing the general idea of the EM-Algorithm framework. It took me a very long time to study and fully understand the concepts from the lecture.
    ---
    Did not explain the concepts thoroughly and sometimes seemed confused himself. Should put more explanations on the slides instead of just putting on the word he is talking about without any content.
    Very good that he had found specific sections in the book for each lecture. It made it feasible to read before the lecture.

    ---
    Impossible to understand Jens
    ---
    If professor don't have much time to evaluate the assignment through personal interview , Please make a team of TAs to do so.
    ---
    He did not seem prepared on the examples in his own material and had to figure it out as he went on. I think the content was fine otherwise.
    ---
    Mainly a lack of structure and clarity.
    ---
    Lecture 1: Crowded and uncomfortable(no bench). I did not see the screen etc. Perhaps it got better after the first lecture, I don't know.
    ---
    Jens knows well about the content himself. However, the pace of the lecture and the way to present certain course content is not perfect. The important content should be explained more clear. And slides with only equations can be replaced by derivation on the board with more explanation.
    ---
    It is a very difficult topic which was presented in a slightly confusing manner.
    ---
    Jens doesn't know what means (and eventually how) to teach something. The purpose of a teacher is to make a topic easier to understand for the students, not to make it more complicated than the book(s).
    ---
    The lecturer was confused, unprepared, unprepared examples, little or no intuition was given, we wasted time going through derivations.
    ---
    Unprepared lecture notes. Lots of mistakes. Bad examples.
    ---
    Jens could improve his ways of teaching and could make it more interactive.
    ---
    Jens is a smart guy, I can see that, but he really is hard to understand. Too bad-- I wish I could learn more from him.
    ---
    Jens needed to practice his lectures more. When I was presenting to him he was very knowledgeable, focused and I learned a lot. However when teaching he felt insecure and vague. It was a shame really because the impression he gave me when I presented was that he could be a very good teacher.
    ---
    I could not follow or understand Jens!
    ---
    Understanding the mathematical part of the derivations explained in the slides wasn't so difficult. It would be much better if except o focusing so much in how we derive some formulas we were focusing more on why we need to do that and what those mathematical derivations actually symbol.
    ---
    The learning curve was negative. I went into the lecture, thinking I knew the concepts and when leaving, I was confused.
    ---
    The lectures may be better structured.
    ---
    The teacher did not seem very prepared sometimes and with no clear direction.
    ---
    Rushed, often unintelligible and difficult to connect to anything.
    ---
    The lectures where really hard to follow.
    ---
    Jens came across as unprepaired and lost. Quite the opposite of his performance during the homework review.
    ---
    More Exercises could have helped.
    ---
    These lectures were of very poor quality. The teacher spent most of the time just reading formulas aloud from the board, without even explaining what the formula terms represent. Not at all productive. Time should be spent explaining the basic concepts first, and the formulas, not just reading them off the board. And no, I am sorry, we don't already know all this stuff, and yes, please start form the basics, and progress very slowly. It's better that we grasp a few concepts very well, than be overwhelmed with high complexity and understand nothing at all in the end.
    ---
    I'm sorry Jens, while your slides were super sleek and one could feel your enthusiasm the lectures felt ill rehearsed and lacked the pedagogics. Variable introductions and definitions were also glossed over all to often and abbreviations were used exclusively too early. (Even now I struggle to remember if a CPD is a categorical probability distribution or a conditional probability distribution. I should struggle to understand your fancy math and awesome concepts, not to decipher the lossy encoding of elementary concepts into abbreviations.)
    ---
    I think Jens needs to find a way to make the lectures a little more enthusuastic.
    ---
    Below average lecture notes with errors and messy logic, and lack of useful exercise.


  9. How well did Assignment 1 align with Lectures 2-5?

    1. 40% (17 st) Well.
    2. 52% (22 st) Somewhat
    3. 5% (2 st) Not very well.

  10. How well did the examination of Assignment 1 reflect what you learned?

    1. 33% (14 st) Well.
    2. 38% (16 st) Somewhat
    3. 26% (11 st) Not very well.

    Comments on Assignment 1:

    Long and difficult. Need more instructions
    ---
    I liked assignment 1 and the examination was fair. However there were parts on assignment 1 which was very unclear resulting in a lot of students making the same mistake. Plus, I would really appreciate to get the answers of the assignment after the examination so that I could really learn what my mistakes were.
    ---
    I got an average mark (after a lot of work on the assignements) which was pretty fair.
    ---
    I learned a lot, but it was HARD!
    ---
    Assignment 1 was very hard and took way too much time. The tools or way of thinking needed to complete the assignment had not been presented well enough in the lectures.
    ---
    I think the assignments were good, but way hard for the lower grades. But it was reasonable if you're aiming for A.
    ---
    I felt that assignment 1 was fairly easy and what we learnt in the lectures reflected pretty much everything in the assignment
    ---
    Not all the excercises felt really thought through. The examination took far more than the appointed 10 minutes (which obviously stacked up and lead to huge delays).
    ---
    Did not attend.
    ---
    The content aligns. But the examination point in the assignment was not addressed very well in the lecture.
    ---
    When I started with the assignment 1, I had to start from zero more or less.
    ---
    It was unclear what was to be done. The first exercise (1.1) was actually presented in Jens' last lecture.............
    ---
    The evaluation was very subjective. Me and two other friends worked together, presented the exact same answers, one of us got A, another got B and the third got C!!
    ---
    Great to had some problem solving tasks.
    ---
    The assignment was very difficult and little guidance was given in the assignment description.
    ---
    Content was very challenging to understand n had lots of ambiguities.
    ---
    Assignment 1 was a bit strange with the Viterbi-like algorithm, it was not easy to understand what is similar and which is not. I am still not sure what answer Jens was looking for and I would like to see a solution on that particular question. Otherwise the problems were direct applications of the lectures and exercises.
    ---
    Again a model were you present different assignment would make this tons easier instead of bulking everything in to one.
    ---
    Don't know only attended one lecture
    ---
    I feel that I didn't have the chance to discuss about what I did, or did not understand and as such, I feel that my grade does not reflect neither my effort nor what I understood!
    ---
    It was good that there was at least one challenging exercise that apparently almost no-one could solve. I think that this gives you, as professors, the chance to distinguish the really good students. On the other hand, in my opinion it wasn't such a good practice to accept as correct a solution that wasn't correct(I was on of those students that had a wrong(but still accepted) solution), just because almost noone solved the exercise.
    Moreover, in the 1st assignment it wasn't so clear which were the grading criteria, which may gave a bad impression.

    ---
    As everyone had problems with Viterbi, it would be very nice to see a working solution and an explanation of where we went wrong.
    ---
    The examination was really demotivating. The assigment was quite ill posed (as the viterbi-like-problem was far too hard to solve in reasonable time) and despite me spending a rather large amount on the assignment and getting almost everything right, I have the feeling I did not get an A because I was honest and clear about the fact that I thought one solution was not quite correct.
    ---
    Way too difficult to complete in time, especially since the lectures were useless in helping to understand what was going on. Needed far more explanation.
    ---
    It took a long time to understand what the questions actually asked for.
    ---
    I was not able to get any nice probability expressions for the Viterbi like algorithm, and it seems like the problem was not worked through before presented as an assignment.
    ---
    It was good. The last two problems might have been discussed more thoroughly during lectures. More time was needed for the assignment.
    ---
    There was no formal request for report on assignment 1, yet the teacher seemed annoyed when I presented hand-written notes.
    ---
    I spent a couple of days trying to understand what was even asked for in the first question.
    ---
    The derivation was unnecessary to ask. I would have preferred implementation and analytic reasoning tasks
    ---
    Assignment 1 was poorly defined and no real help was received. it needs to be shortened by 25-30%


  11. How many lectures did you attend in Part 2 of the course (Lectures 6-9 followed by Assignment 2, held by Carl Henrik Ek)?

    1. 86% (36 st) 3-4.
    2. 7% (3 st) 1-2.
    3. 5% (2 st) 0.

  12. How did you find the lectures in Part 2 of the course (Lectures 6-9 followed by Assignment 2, held by Carl Henrik Ek)?

    1. 67% (28 st) Excellent.
    2. 17% (7 st) Good.
    3. 12% (5 st) Ok.
    4. 2% (1 st) Not so good.
    5. 0% (0 st) A waste of time.

    Comments on Lectures 6-9:

    The lectures were perfect. Intuitive reasonning give more interest to students. Should be the first part of the course. I think it is a good introduction to the course to start with it. I learned more on being baysian at this part than at the first one.
    ---
    Carl Henrik Ek is an excellent speaker. He was really able to motivate us.
    ---
    Great lectures, really lively. One of the lectures was 100% derivations and was (IMHO) a waste of time because it was just impossible to follow all what was happening on the board.
    ---
    Too fast sometimes. I know we can interrupt, but most of the times I just don't do it.
    ---
    Really good lectures, you did a very good job. I very much enjoyed them and learned a lot from them.
    However the assigned readings for each lecture could be more specific. When 2-3 chapters are assigned each time it is simply not feasible to read it all.

    ---
    Assignment interview was good and satisfied
    ---
    Carl Henrik speaks very clearly during lectures and seems very keen that all the students understand the concepts. Recap on the previous lecture is a great idea. Even so, gaussian processes and representation learning was a little bit hard to grasp during the lectures, but the students' questions really helped. There were new equations on each slide and it was hard to keep track of why the kernel had capital K/small K and what different letters were and which priors we assumed for them previously etc.
    ---
    Good lectures but I felt that this part was harder to follow than part 1, but this may be because I'm from a Computer Science background and there was a lot of math.
    ---
    Great lectures, both instructive and inspiring! If anything, I think the content of the last lecture (of part 2) was a bit rushed (which was a shame, I would have liked to get more insight).
    ---
    Did not attend.
    ---
    Great!
    ---
    I like Carl Henrik's way of giving intuition instead of plain formulas. I enjoyed the lectures very much.
    ---
    Carl is really a good teacher but given that I had not been able to get a good grasp of the first part of the course in time, going to Carl's lectures was partly a waste of time.
    ---
    Lots of intuition, very well presented and well prepared slides.
    ---
    It was very good. I really liked the way he presented the topics.
    ---
    The lectures were great for understanding and reasoning about the topics at hand.
    ---
    I felt Carl Henrik is good teacher, maybe he crammed a bit too much content into his part. It was also really terrible how part 1 and part 2 were overlapping. Very confusing.
    ---
    I stopped going because I thought it would be the same teacher, but when I went to his practice session I immediately regretted my decision.
    ---
    Carl Henrik is an amazing lecturer!
    ---
    Introduction of the variables would be very nice - in both the lecture and the exercises. It is sometimes hard to follow with which dimensions we are currently playing.
    ---
    Great lectures!
    ---
    Much better than the first part, but far too much content and not enough explanation for me to understand. I understand the concepts to some extent but nowhere near enough to do the questions on the assignment.
    ---
    I am sad, that I could not join the lectures, because of other courses.
    ---
    The lectures were really really good! One thing that made it hard to remember all the good things that was said during the lectures was that the lecture slides had too little information. Since we were working on assignment 1 during some of part 2s lectures it was hard to learn everything right away, and then later on when working on assignment 2 it was hard to remember what was said during the lectures. The recap at the start of each lecture was really great!
    ---
    The lectures where quite fast which is nice but I would like if it was possible to read the slides before attending the lecture
    ---
    Wonderful. Clear and engaging.
    ---
    Well a short introduction on the Linear ALgebra methods used for that part of the course could have aided understanding of the concepts.
    ---
    Carl Henrik's lectures suffered from a different problem. While it was clear that he sincerely cared about our learning from these lectures, I felt that the slides don't present the information well, especially whole sequences of slides that just had a single figure, with no explanation whatsoever of what is going on. His lectures were at a very high level, I felt like perhaps there was a whole probability class I was missing that I should have taken before I grasp these concepts. But overall Carl Henrik should be commended for his effort, and for his willingness to help students when we got stuck on assignment 2. My advice for next year is to take it slowly, reduce the number of slides by 3/4 of the current number, and only present the most important concepts, very clearly.
    ---
    Excellent work with these lectures! Rememberable examples and intuitions were spot on and the lectures inspired greatly. Still could be more formal with the math though. I would recommend keeping the variable definitions on all slides where they are used so noone has to miss an entire explanation because they spaced off when the variables where introduced. It could also be clearer what is a matrix or a vector or a scalar and if a matrix ha row vectors or column-vectors and such.
    ---
    The lectures were very good. The reason I had to miss some lectures were because I was so overloaded with the assignment work that I just couldn't manage it.
    ---
    Beautiful structure and good story, difficult to implement as there were not enough examples


  13. How well did Assignment 2 align with Lectures 6-9?

    1. 67% (28 st) Well.
    2. 24% (10 st) Somewhat
    3. 5% (2 st) Not very well.

  14. How well did the examination of Assignment 2 reflect what you learned?

    1. 60% (25 st) Well.
    2. 26% (11 st) Somewhat
    3. 10% (4 st) Not very well.

    Comments on Assignment 2:

    Very long, strong mathematical background required. Should be a little bit more help for these parts in the assignment instructions.
    ---
    I was really sad that there was so limited time to work on Assignment 2. Although I did well on the examination I feel that, because of the time limitations, I could not absorb the 100% of what I was supposed to learn.
    ---
    SO much work was required to get to an "E"... so I feel this doesn't reflect what I understood (as I think I would've been able to answer the other questions too if the first part wasn't so time consuming and requested so many derivations that were hard to do)

    Less focus on theory might help, but this is just a student speaking!

    ---
    Sometimes, I had to look up things like "spherical covariance matrix". It happened too often and took too much time. After reading and thinking a lot, I got the picture of it, but I think it was not intended.
    ---
    Really good assignment. Gave a great new look into the theory. I wish I had had time to complete it all, but assignment 1 simply took too much time.
    ---
    Well designed.
    ---
    I found Assignment 2 harder than Assignment 1 and I also found that the implementation part of Assignment 2 was hard.
    ---
    Very good lab which really helped to understand the content of part 2. Unfortunately I did not have the time to complete the last task of the assignment - maybe a little more time (not competing with assignment 1) would be good.
    ---
    I love assignment 2. I learned a lot from it.
    ---
    The lecture helped me a lot to do assignment 2.
    ---
    I couldn't complete it on the first deadline because it was too difficult to answer for me.
    ---
    Assignment 2 was an awesome tutorial, I believe its one of the most well written assignments I've had in my entire academic studies.
    ---
    Very well written assignment.
    ---
    Assignment 2 was well formulated. However, it was too exhaustive and consumed more time than expected.
    ---
    The mathematics in assignment 2 can have been quite challenging for many, at least that is what I expect.
    ---
    Way too much content, crazy amount. A lot of people were really frustrated. Me included. Felt I missed a lot of important stuff just to stress trough this assignment.
    ---
    Assignment 2 was a bit too long.
    ---
    It was fun!
    ---
    Better than the first assignment - at least there was some explanation, although the questions were often vague. Still much too difficult even to get a passing grade.
    ---
    I learned a lot more than I was able to actually do in actual tasks in the assignment, mainly due to too short time for that assignment than I apparently needed.
    ---
    The assignment was quite lengthy. I was able to finish all parts except the optional until my presentation, a few days after the first dead line with many late nights.
    ---
    Fun and interesting. More time was needed for the assignment.
    ---
    The assignment was quite difficult, especially to someone who is new to terms like multi-variate gaussians, and gaussian processes, could be a bit over-whelming if it is taught in just 2-3 lectures, and we hardly had any time between the first assignment submission and the second one.
    ---
    The GP part was fascinating.
    ---
    Assignment 1 cannibalized the time for assignment 2. I guess dropping the lowest grade was added as a solution to this, but I would have preferred to actually have enough time to work on assignment 2 since it was very interesting. However, assignment 2 was the most challenging to me of all assignments. The teacher had very high expectations of us with very little investment (due to the obvious time constraints). We can't possibly learn all this material from 4 lectures in less than 2 weeks. Also the assignment was written in a way that left us digging for hours in order to grasp the simplest concepts, and search for the right formulas to use. It would have been so helpful to include these bits in the assignment description, so we can focus on the most important parts. Also the theoretical questions were a bit pointless to me, I would prefer more implementation. I also thought that the parts for E grade should have been much simpler, and focused on a problem that lets us play with the concepts of PRIOR and POSTERIOR in a much more basic way than what it did. We had no sense or experience of what a prior was, or a posterior, other than a few symbols from the Bayes rule. We should have played with an obvious and simple example, something involving blue and red balls or oranges or whatever, to get a solid understanding and feel for the prior, likelihood and posterior, before we moved onto the current assignment for E. The current assignment for E should be moved to be up to a C grade. And for A and B, something with more practical implementation would have been better.
    Overall, the amount of work required to complete this assignment should be cut by half.

    ---
    While the intuition was attained the required formality was not and since the book was useless I had to resort to copying from whatever parts of the book I could decipher or look elsewhere for understanding.
    ---
    I think considering the general student's mentality, everyone will only start working on assignment 2 after completing assignment 1. So 1 week for assignment 2 became very diffucult to handle. You could have given 2 weeks each for the 1st and 2nd assignment instead of 3 weeks and 1 week.
    ---
    Assignment 2 was by far the best explained, but it was RIDICULOUSLY long and difficult.needs to be shortened by 50%, needs more help sessions, needs more numerical and implementation examples.


  15. How many lectures did you attend in Part 3 of the course (Lectures 1,10-12 followed by Assignment 3, held by Hedvig Kjellström)?

    1. 60% (25 st) 3-4.
    2. 31% (13 st) 1-2.
    3. 7% (3 st) 0.

  16. How did you find the lectures in Part 3 of the course (Lectures 1,10-12 followed by Assignment 3, held by Hedvig Kjellström)?

    1. 29% (12 st) Excellent.
    2. 38% (16 st) Good.
    3. 21% (9 st) Ok.
    4. 7% (3 st) Not so good.
    5. 2% (1 st) A waste of time.

    Comments on Lectures 1,10-12:

    The lectures were quite easy and revisited.
    ---
    I really liked Hedvig's Lectures. They were easy to understand and Hedvig could give a simple yet understanding explanation of the whole subject.
    ---
    Lectures were really nice, with both theory and practice.
    ---
    I wish the sampling would have been discussed earlier.
    ---
    Overall good lectures but it was a shame that it all became a little rushed towards the end of the semester. Would be better to have more lectures earlier since the lectures in the last week was not really used for anything because everyone was stressed about the assignment.
    ---
    I think the contents were fine and the student discussion is a good idea. Hedvig just needs to think about how she is talking when presenting the subject and prepare what she will actually say beforehand. She stops sentences midway, thinks for a while, and starts another sentence instead quite often.
    ---
    Some parts of the content felt a bit rushed, but good and well structured lectures in general.
    ---
    Did not attend.
    ---
    Due to a Workshop, I couldnt attend most of Hedvigs lecture. The ones in the end were fine. She gave some good advice for the assignments.
    ---
    A lot of slides were useless. The particle filter was explained very poorly.
    ---
    I guess the topics could've used more time, a little more time for preparation as well.
    ---
    Most of the topics were covered in Machine Learning course which we had previously. So I felt it was repetitive.
    ---
    With the grading system we ended up with I felt like Hedvig's part got less attention. Now when I think back it feels like I remember more from the first 2/3 of the course, so I do not think you should have a grading system which only contains the grade from two of home assignments.
    ---
    A lot of the content in the lectures didn’t seem relevant for the assignment. A bit confusing.
    ---
    Never attended any but I heard it was good.
    ---
    Hedvig is an amazing lecturer!
    ---
    Way too brief. Particle filters can fill half a course, and to go over them in 10 minutes is definitely not enough.
    ---
    Had a lot to do, for another course, so that I could not attend the lecture.
    ---
    At the lectures I manage to attend I though too much time was spent on describing things that I had seen before and too little time was spent on the things that were new to me. Of course, that is personal. It depends on what you have done before. but I got that feeling from more people in the course.
    ---
    The lectures felt not well prepared and there was not much material presented. This is solely based on lecture one and I have previously encountered Ensemble methods
    ---
    Lectures were ok but questions were barely answered.
    ---
    We had learnt about Adaboost already in the basic ML course, so it wasnt really needed. ALso particle filter could have been devoted more time by the teacher.
    ---
    Lectures were OK.
    ---
    While having pedagogical potential the lectures were confused and repeatedly made deep dives into the trivial parts while glossing ocher the difficult ones.
    ---
    I did not like that the lecture on particle filters was rushed so much. This was a major part of the assignment and we were even told that we will not be given any help regarding this. Atleast the lecture should have been informative since we were not allowed to ask for help.
    ---
    Too much material, otherwise all good.


  17. How well did Assignment 3 align with Lecture 10 (Lectures 1,11-12 were not directly related)?

    1. 64% (27 st) Well.
    2. 31% (13 st) Somewhat
    3. 2% (1 st) Not very well.

  18. How well did the examination of Assignment 3 reflect what you learned?

    1. 71% (30 st) Well.
    2. 24% (10 st) Somewhat
    3. 2% (1 st) Not very well.

    Comments on Assignment 3:

    I didn't do very well on Assignment 3 due to the fact that I felt exhausted from the other 2 assignments. However, assignment 3 was the most interesting one, mostly because of the topics it involved.
    ---
    Nice assignement with just the right amount of work, no "blocker" questions (you could just skip a question and move to the rest and get back to it later), no "read 100 pages in the book to understand one exercise"
    ---
    I liked assignment3 because it gave me a feeling for the concepts discussed in the lecture. The assignment had the least frustration factor of all of them (I did net get stuck that often).
    ---
    Ok assignment. The was not given out in a very good format so too much of the time spend on the assignment was spend trying to read that in. And therefore less time actually on the assignment.
    ---
    I as many other students only went for an E on this assignment since it didn't matter. And the little work you have to do for an E is ridiculous, it should be harder to pass courses.
    ---
    I really enjoyed Assignment 3, the questions were very interesting.
    ---
    I understand the idea of studying parametric vs non-parametric methods, however it felt like a bit of a repetition from part 2 (and implementing kNN/adaboost was for me not really anything new and therefore I did not learn that much from it). I would have liked to see assignment 3 deal some more with the new concepts of part 3 instead.
    ---
    Since it is the last assignment, the difficult level can be increased next time.
    ---
    I liked that there was a lot of programming and the opportunity to get creative in the last question.
    ---
    It was a good assignment, with the exception of the particle filter's requirements that were a bit unclear, given the lectures.
    ---
    Assignment 3 was good and I enjoyed it.
    ---
    Like I said, wish we could have used sci-kit instead. And focus more on a practical understand of using ML algorithms on real problems
    ---
    Fun as well! But I was done before the new data set was uploaded and not too happy to do all the plots again.
    ---
    The assignment 3 was somewhat overshadowed by the project work and I could not find the time to do it properly, sadly, because it seemed really interesting.
    ---
    Annoying to do because of the large numbers of experiments required, spent a lot of time writing code to automatically produce figures after computing stuff.
    ---
    It was good
    ---
    Most of the work was done realizing that I needed to write an optimizer since there was not an easy expression for the objective function (at least not in the heat of the assignment). The Particle filter looked really interesting I was however not able to finish that part.
    ---
    Great! More time was needed for the assignment.
    ---
    The data was quite a mess to begin with, and we could have been given some more time to report it.
    ---
    I particularly enjoyed the car in city sequential importance resampling project.
    ---
    I think this was the best of the 3 assignments, however it had its own issues. The time I spent implementing the weak classifier, and the contour plot if the adaboost decision boundary was longer than the time I spent on part A and B of the assignment. We should be given code for this as to not waste time, since this is really not an important part of the assignment. I liked that the assignment was very heavy on the practical implementation side.
    ---
    An incredible amount of time went into implementation. The desired answer was not clear from the question.
    ---
    I did not understand why we were told that no help would be given for the particle filters part of assignment 3. we could have just been given a hard problem for higher grades. I think teachers should be available to help students when they need help. Helping us with our questions is not like helping us with our grades.
    ---
    Assignment 3 was also unreasonable if you've never studied all the content previously. There are whole courses on particle filters alone...


  19. What did you think about the project (performed in groups of around 5 people, examined with a report and an oral presentation)?

    1. 17% (7 st) Fantastic!
    2. 71% (30 st) OK, I learned a lot from it.
    3. 7% (3 st) OK, but would have been better with a written exam.
    4. 2% (1 st) Not so good.
    5. 2% (1 st) It did not give me anything at all.

    Comments on the project:

    It was interesting to have the kind of project that we had this year. Although, I think students would be more interested if some more practical project are built. For example, building some recommendation engine, or any project where students can see how the theory can be used in practice (as the assignments are already very theoritical , I think you can afford this). A lot of students seem to get lost with the lack of mathematical background and it is a factor of disapointment and frustration. These kind of project can bring interest for all students.
    ---
    I liked the idea of a project a lot more than an exam. 5 people is a perfect number for a team, and the concept of the report and oral presentation is something typical which cannot be avoided. I didn't like the fact that the groups were fixed according to the assignment grade, as I also mentioned before.
    ---
    The project was OK, but we've got a paper only one member of the team asked for which was kind of disapointing for the rest of us.
    ---
    We had someone studying applied mathematics, very helpful. Our topic was very interesting.
    ---
    Great idea to recreate articles. However to me it highlighted that the prerequisites should include more statistics or probability theory since I spend a lot of time explaining basic concepts to my group members.
    ---
    Way to low grades for the project for everyone, otherwise great!
    ---
    The format was great, but all my debugging time could have been put to better use. I don't know if I can say that the method was too hard to implement or what to do about it.
    ---
    I think that the project was good, but it was somewhat difficult to divide the work between all 5 people. I think that a group size of 3-4 people would be more effective.
    ---
    The main reason I really enjoyed the project was probably the group I worked with. This is obviously a bit a luck, but I also like the way you made up the groups (based on assignment grades, really nice to be with people on the same level of ambition). We also ended up with an interesting topic - I like the fact that you (the teachers) suggested topics, rather than having us trying to come up with something ourselves (usualy hard to find something on the right difficulty level). About the presentation my only remark is the strict time constraint, but I understand the reason why and I appreciate the fact the you were very much aware of it and took it into consideration when specifying what should be included, it worked out well under the circumstaces. My only other comment would be that I don't think it's ideal to run the project over christmas. Half my group was not in Sweden during most of the time and therefore we were not able to meet up as much as we would have wanted to. It was a good excercise in managing a project without being able to meet up, and it worked fine for us, but I imagine this was not the primary aim of doing this project as a group.
    ---
    I prefer project before exam. But please prepare each project in a better way and make supervisor available. For e.g. give advice on tools beforehand and mention other stuff as one does usually in a project description. For example see course DD2399 Omic data and systems biology.
    ---
    Student can be encouraged by extra credits or in other ways if they can extend/improve the project.
    ---
    So far, the best part of the course.
    ---
    Project was great !!
    ---
    Working with 5 people is hard and perhaps that is what you are aiming at, but I almost feel like that at such a small level of work the final result will be better with fewer people in the group.
    ---
    I had a really good group. I always learn from projects. The only issue I have is it should have been P or F. You really is put on the spot, if the group chemistry is not working. Grading should not be a gamble.
    ---
    Groups based on performance might not be the best idea. Just because someone managed to get good grades or was very busy and had little time for the labs does not mean that they will work well together. Even A candidates like to postpone the work until last minute.
    ---
    It was a realy joy to work with my team and everything worked out quite well. Carl Henrik was quite helpful and was willing to do a skype meeting - thank you!
    ---
    Working with random people is annoying, working with random people over the holiday is even worse, especially if nobody bothers to do any work. Highly dependent on the paper you choose what the quality of work ends up being. Didn't end up learning anything other than C because we just had to do loads of implementation and barely any ML.
    ---
    Great experience, but the papers has variant difficulties this results in some groups working on easy/hard problem compared to the other groups
    ---
    It took way more time to programm the simulator, than the actual learning algorithm.
    ---
    I found my project too hard,since the working load is hard to divide equally when you do a short project and during a period where you have vacation and also since it was about thing we hadn't gone through much during the course. But I learned a lot by doing the project. And I liked the task formulation, to read about someone else’s work and re-implement it. I haven't done that before.
    ---
    It was nice to read an interesting article and to collaborate new faces. However I am not sure that it was an effective method for learning machine learning. I think it is hard to have an exam on a machine learning course since to illustrate different methods takes some time data etc. I vote for more labs, and maybe some small hand in assignments about the theory of different parts of the material.
    ---
    Instructions on what we needed to accomplish could have been clearer.
    ---
    It was a nice thing to read the research papers and analyse them.
    ---
    I really rather liked having to recreate a current topic that is not artificially created for this course, allowing at least some insight into the 'aktuella forskningsläget' and some deeper learning of good methods that would otherwise have been outside the course material. Furthermore, every group having a separate topic made for an interesting presentation seminar.
    ---
    I already commented on the project, but here are the main points:
    1. Remove the meritocracy when creating project groups. Let people choose their groups.
    2. Let people choose their project topics.

    Also ensure that the articles given require a similar effort to replicate. It seemed like this time around, some groups drew a "short straw" and a lot more work than other groups.

    ---
    Was fun and rewarding. I can see how the shared grade could be a problem for people though. The ability of amending that grade individually should probably exist.
    ---
    I loved doing this. We realise so many things when we try to repeat someone else's research. It was very challenging and a lot of fun.
    ---
    Interesting way of dividing the groups. I sort of like it, but I can see where it could be problematic (it puts a lot of weight in the first two assignments - do well and end up in a competent group).
    ---
    I learned a lot, but I don't think it was graded fairly.


  20. Now there is room for comments that do not fit with any of the above queries. Please mention aspects of the course that you thought were good and that should be retained for next year:

    Well distributed parts
    Assignments
    Very good course!!!

    ---
    The topics of the course should definitely stay the same.
    The idea of the assignments and the final project should not change.

    ---
    I think this course was really good! Sorry if some of the comments might seem harsh.
    ---
    Keep the course as a challenge. It was very hard for me, but that steep learning curve was exactly what I was looking for.
    You responded very well to emails and in the forum, thank you! You put a lot of effort into it and supported the students and where fair, keep that.

    ---
    I think you should raise the prerequisites instead of lowering the level of the course. It is not an easy waltz through course, but the best ones never are.
    ---
    Format of lectures assignments and projects were all good.
    ---
    I think that Assignment 3 was very good and should be kept for next year.
    ---
    Over all a nice course. I think the balance between depth/width on topics was quite good. Assignment 2 was great, assignment 3 was good.
    ---
    *Labs project is a good model.
    *Interesting and advanced articles is good. Keep it advanced, but more guidance and time. Solving problems and understanding concepts takes time and we should have started the project as soon as the course started.
    *I like the ability to postpone the labs. Although I dont see why the grade should be lowered because of that. I did not have the possibility to read this course at full speed so I had to do the labs later whenever I had time.

    ---
    Project in groups. Tutorial sessions given by Carl Henrik about how to solve the equations in these machine learning theories.
    ---
    It was a challenging, but still very interesting course. I especially liked all the project presentations in the end which gave me a nice impression on how research in this field could look like.
    ---
    3 professors can fail a course even if separately they're good but together they don't make a good team. Do you? I don't thnk so. Sorry. My grade for the course is Fx. Try again next year.
    ---
    Gaussian Processes. Graphical models IF they were presented in a better manner.
    ---
    Overall I think it was one of the best course in my master. I really learned a lot from it.
    ---
    First and Second parts.
    ---
    You could revisit the course contents and see to it that it does not clash with the Machine Learning course contents.
    ---
    I think the content was good. It was also very interesting when applications were presented during the lectures, perhaps with images or videos, and I suggest that you can even add some more fascinating applications and results.
    ---
    There was a huge problem with time organization! Even though the content of the course is very interesting and the assignments are worth spending time on, there was literally no time for many students to prepare themselves as much as they could/would. I feel that this is a 2-term course, so that people have the time to digest the new things they learn and spend time on the assignments and project.
    ---
    The whole structure of the course was very good, and should remain the same.
    ---
    - The exercises on the blackboard.
    - Labs
    - Openess

    ---
    The scope of the course is really great. I got the feeling that I now understand bayesian reasoning far better, which is the main thing I took from this course.

    Thank you for this course, I enjoyed it immensely!

    ---
    The scope of the course was broad, and covered a lot of topics which I think are potentially useful.
    ---
    I liked the course alot. But an Advice in the first Machine Learning course, that the secound course take way more time would be nice. I had another project course parallel so that I could not invest enough time
    ---
    The things taken up in this course were really good. I would not have wanted to leave KTH without this knowledge.
    ---
    I think labs is a great way to learn machine learning. There are some interesting competitions found on Kaggle.

    About the prerequisites I think it would be nice to have read at least some probability/time series analysis or similar. As well as all basic math courses. (Multivariable Calculus included). I do not think this course should be considered as a first year course for the Machine Learning master students since it is quite advanced and it should neither be made easier. (booth basic ml course and ANN are in my opinion to easy in the sense that they don not cover any real applications or at least feel very far from real applications)

    ---
    The material covered was excellent and interesting. More filtering would have been fun.
    ---
    The assignments were good and quite relevant to what we studied, and this course should be a mandatory course atleast for the Machine Learning Masters program. The project part was also a nice part in this course, as it helped me to learn LATEX.
    ---
    I do feel a bit like I'm not necessarily the student that should be the primary focus of the course, seeing as I am a math major rather than machine learning major. However, from my biased perspective: please do not 'dumb down' the course. This course was really enlightening to me, and a big part of that has been the high course load with questions that cannot be answered mechanically, but rely on intuition and understanding.

    The project part was also rather brilliant being based upon real articles. Keep that one.

    ---
    The format of the course is good, with 3 assignments and a project. Some of the teachers set up help sessions for the assignment, this was very good too.
    ---
    Assignment 2 was excellent in its entirety. It needs more time though and a more thorough introduction, you could use material from mathematicalmonk to great avail for refreshing peoples memory of probability and introducing the multivariate normal.
    ---
    The 3 assignments and 1 project format.
    ---
    interesting scope

  21. Please mention aspects of the course that you thought were less good and that should be changed to next year. Provide constructive suggestions for changes that could be made:

    -If this course is in relation with the first Machine Learning course, some linear algebra recap should be presented in the first course or add more exercise session on linear algebra in this course(4 hours of recap , 4 hours of exercise session seem to be necessary)
    -Verify doability of assignment before giving them :p and scale them properly
    -Strongly advise students on prerequisites

    ---
    The time limitations.
    This course is impossible to be taught in one period also being qualitative. It's just too much information and the time cannot allow us to absorb everything in two months.

    ---
    Carl Henrik's assignement should really be re-done to require less derivations and a more conceptual approach ("What do Gaussian processes represent, when do they work well compared to other models, etc." vs "Derivate the likelihood formula for such and such").

    Jens's lectures should also be more accessible, again less derivations on the board and a more high-level approach, maybe?

    ---
    I had problems with the slides. They are very useful with spoken words, but did not have enough information afterwards. Maybe record the lecture.
    The frustration level was high, I often got stuck. Maybe provide more and different kinds of material (even the most simple things but as short in precise as possible) to prevent students getting stuck and therefore become frustrated.

    ---
    Start the assignments earlier such that they are more spread out. We did not do much in the first couple of weeks of the course.
    Explain concepts thoroughly in the lectures (especially applies to first part)

    ---
    Its better to make the course as domain independent, I guess most of the examples and problems only deals with Image processing. But lot of students come from different domain.
    ---
    It needs a good overview of all methods in the course, how should one think when choosing machine learning methods for a specific problem, because the book is just a big sac of methods and you don't know what should be used where and when.
    ---
    There was not enough time between some of the assignments, especially Assignment 1 and 2.
    ---
    Part 1 needs some rearrangeing/restructuring in order to feel really relevant. The same goes for assignment 1.
    ---
    *Starting a project over christmas holidays is respectless. Star the project sooner please.
    *It would be good to have a project description. And the supervisor should give hints on how to solve a problem. This is why its important to start the project early in the course so that one allows communication and problem solving to evolve.

    ---
    There is an aspect that I found less good. One suggestion may be separate homework tutorial sessions which is given by teaching assistants. Asking questions in forums is helpful but if there are sessions dedicated to only that for one or two hours, it would be more helpful.
    ---
    The structure of the course was confusing. There were too many different topics in a very little time frame. It was hard to grasp the essence of the course/lectures sometimes. This should be improved. Other than that, I think the course has the potential to be a very interesting course!
    ---
    Try to be only one professor or a better team of professors. Publish those slides before time, Hedvig's policy on the material releasing time is ridiculous: hand it all out and let the students work earlier if they have time for it. Often they have other courses and deadlines at the same time, and you cannot presume to be the only main existing course with deadlines.
    ---
    KNN and Boosting are not advanced topics. I would suggest online learning, knowledge transfer, and go a bit into Statistical Learning Theory.
    ---
    Third part belongs in a normal course, not advanced. Do more theory
    ---
    Think trough the assignments so that the workload is not too large on a single assignment.
    ---
    Everything was good about the course except the first lectures and that some lab-questions where unspecific in a way that was just annoying and did not allow me to learn anything.
    ---
    I believe that the way the groups for the project are arranged is wrong.
    First of all, the groups were formed when only two of the three assignments were examined. For example, I had a C and an E in the first two, which meant that I was placed in a "group for D". Whereas I got a B for the third assignment which resulted in me having a B for the assignments. Particularly some of the people of my group didn't even have all the grades for the assignments.
    Actually my problem was not that bit (I actually don't think that the grade is always so important).

    What I believe is not very good is the fact that we don't get to choose our teammates. There may have been student that have worked together before and already know that they can co-operate nicely. But putting "strangers" in a team is not always a good thing. For example, there were 5 people. Two of these people did absolutely nothing. Were I to choose a group, I would choose people that I knew that we can work together and help and try to improve each other!!

    I just want to say, that in total I really enjoyed the course and I can understand that as a new course, there are several issues that have to be dealt with!

    What I said in this evaluation form, is all well-intended.

    ---
    I think that the course should be extended in the 2 periods, if the amount of effort required remains the same as this year. In that way students will have the time to really absorb all this knowledge that they take from this course.
    ---
    Check the content of artificial intelligence and machine learning. Some graphical models, Hidden Markov Models and Adaboost are already taught there. Instead of these topics, more theory and higher statistics would be nice.
    Maybe just two labs instead of three.
    All mathematical parts on the blackboard. MDS and EM should be derived it hand. In general, I think that Machine Learning as such a mathematical topic should mostly be taught on the blackboard. If students do not write down the derivation themselves, they will not follow it. A nice example is the ML course by Ng (Stanford University) on Youtube.

    ---
    I'm not really sure, but I think part 3 of the lecture should be taught for the whole period. That's where I have learned something. The task is precise, the algorithm is presented clearly. Part 1 is the worst.

    The theme tackle in project is pretty good but I prefer written exam that covers the whole topics.

    ---
    I understood only a couple of concepts in the course, and I was stressed and hated the course the whole way through. Make it easier to get a passing grade in the assignments - spending 15 hours or more just to get an E is not cool. The lectures either have too much content which is far too hard to understand in a short time (part 2) or too little, which leaves you not knowing what to do (part 3). The project needs to be improved somehow as well, especially if groups are going to be divided depending on the grades of a person
    ---
    Maybe this is more because I am not used to this format but I really do prefer the approach of working in small groups for the labs and then a written exam at the end, that way the labs can have P/F.
    This way you can discuss much more about the problems with your peers. And the examiner of the labs can help you understand the topics better.

    ---
    Having different groups solving different problems with different difficulty levels will result in a a situation where a group may have worked hard in a hard problem to be perceived to be performed poorly compared to a group worked in an easy problem.
    ---
    The course was really tough. By far the hardest course I have taken, regarding to perform difficult tasks in short time.
    ---
    It would perhaps be nice to have some hand ins combine with labs where the hand ins reflects certain theory parts of the course and the labs to get some proof of concept/working experience
    ---
    We needed a lot more time for the assignments. Less adaboost, this was already covered in the course Machine Learning.
    ---
    I felt the course was overwhelming at times, especially because we didn't have enough time for the assignments.
    ---
    I do understand that the course load was high and maybe excessively so for some students. For these, maybe add more informal tutoring sessions (seminars with quiet work, where the teacher walks the room answering questions, rather than discusses solutions in front of the board).

    Furthermore, while this information was a part of each assignment, really make sure that the students get that the assignments are supposed to take one week of work, and unlike many other assignments at other courses, cannot be finished quickly.

    ---
    1. Have Carl Henrik Ek's part first. Then, people at least have a chance for one decent grade before they have to face the current assignment 1. What happened this time was, a lot of people spent a lot of time on assignment 1, got a bad grade, then didn't have time to finish assignment 2, and got another bad grade. Don't overestimate people's knowledge in probability theory, start from the basics. Make assignment 2 E and D parts focus on a few simple practical problems that give people a feel for prior, posterior, likelihood, what it means to "sample the prior", "plot the posterior", etc. The parts for E and D of the assignments should be relatively easy.
    2. While the topic of the current part 1 of the course (Jens Lagergren) is very interesting, the lectures and assignment leave a lot to be desired. The teacher should be more helpful and set up help sessions like Carl Henrik, and review how the lectures are presented, focusing less on formulas and more on clarifying the main concepts.
    3. There is no continuity between the ML 1 and advanced ML course, perhaps the teachers should get together and come up with a common curriculum plan for both courses that is more cohesive.
    4. The current project group selection based on "meritocracy" can act as a very strong bias when the teachers assign the final project grades. It is hard to ignore such bias when grading the projects, and instead of arguing the teacher's abilities to ignore the bias, it is best not to introduce it at all - let people be able to choose their own groups.
    5. I did not understand what study the idea about lecture notes not being published before the lectures was based on. I believe students should be made aware of this study, but still given the option of whether or not to bring the lecture notes in class, and take notes on them as they see fit. We are all mature individuals with very diverse learning styles, and are capable of making the best choice of whether or not to bring the lecture notes for ourselves, given the information from the study. Personally, I felt that not having the lecture notes in class to take notes on affected my studying process negatively, especially since there were a lot of formulas and charts in the notes, and whenever an explanation of a formula term or a specific topic was given, I didn't have the chance to write it down before I forgot (70% of what I wanted to write down) after the lecture. Or, I was forced to write notes on paper pointing to slide numbers (when available), which was really unnecessary.

    ---
    The grade affecting deadlines really should go. If you would like to keep the deadlines, which is probably a good idea to get ppl to start early, make it instead so that it is easier to attain a maximum grade if the deadline is met but still possible after the deadline.

    Switch the book for online resources. I believe you'll be better off with papers, wikipedia, mathematicalmonk and online guides.

    ---
    Some contents of the assignment, have more implementation or analytic reasoning questions instead of derivations. The deadlines for the assignment can be made 2 weeks and 2 weeks for 1st and 2nd assignment.
    ---
    no integration of part 1,2 and 3. there is no transitioning material
    Also, this course needs TAs to lead help sessions and proper exercise sessions


hedvig@csc.kth.se

Denna sammanställning har genererats med ACE.