•  Positive effects of criterion-referenced grading criteria.
•  Pitfalls in connection with introduction of grading criteria.
•  How expected learning outcomes, grading criteria and examination are related to one another and to the students' learning.
•  Examples of how criterion-referenced grading criteria have been developed and are used in courses at KTH Royal Institute of Technology and Uppsala University.
•  How one writes criterion-referenced grading criteria.
•  Student perspective on grading criteria.
•  What is needed to make the grading criteria support the learning?
•  Combining assessments to a final mark.
•  Guidelines and support for criterion-referenced grading criteria at KTH Royal Institute of Technology
•  Work on grading criteria and modified assessment of the course participant's own course.

After passing the course, the course participant should be able to:

·         demonstrate how intended learning outcomes, grading criteria and examination are related

·         formulate criterion-referenced grading criteria for a course

·         suggest how examination items should be designed so that they are assessed according to specified grading criteria

·         critically review and reflect on one's own specified grading criteria and those of others

·         give an account of how assessments can be weighed together to a final mark and reflect on the possible effects of this

·         reflect on how the grading criteria can be used in one's own teaching

in order to be able to design criterion-referenced grading criteria and examination that can be used directly in an own course.

There will be three meetings in the course. There are preparations to be done before each meeting. A chronological view of all deadlines and meetings can be found under Syllabus/Kursöversikt.

Language

The main language of the course is English. However, in group discussions it is allowed to speak other languages as long as all in the group understand. The terminology will be given in both Swedish and English, and you are allowed to write your assignments in either Swedish or English. We recommend that you use the same language in your assignments as the course you are writing your grading criteria for is using.

Schedule

A course in teaching and learning in higher education, for example LH231V Teaching and Learning, is recommended but is not a requirement.

Before the first meeting, please do the following preparations:

• Read the article Chris Rust: Purposes and Principles of Assessment, available at academia.edu (also attached to the welcome email).
• Review the concept of constructive alignment and read about how intended learning outcomes should be written. Please look at these slides to refresh your memory about this. If you need more explanation and examples, please look at the reading list on the last slide.
• In this course you will be working with the design of grading criteria in a course. Decide which course you would like to work with, and go through the current intended learning outcomes (from the course syllabus). If your course is given in Swedish, you should look at the Swedish version of the intended learning outcomes (ILOs).
• Check whether the intended learning outcomes are result oriented, assessable and at a reasonable level. According to the KTH regulations, all intended learning outcomes have to be assessed in the course, and only students fulfilling all intended learning outcomes should pass the course.
  Modify if needed the ILOs so that they are possible to assess. 
• Bring your (modified) intended learning outcomes to the first meeting (in the language of instruction of your course), or if you participate on-line: prepare to share them in a window in Zoom.

Main reading

• Chris Rust: Purposes and principles of assessment, Oxford Brookes University, 2002.
• John Biggs and Catherine Tang: Teaching for Quality Learning at University, 4:th edition, Open University Press, McGraw Hill Education, 2011. Chapter 11 and 12. Available online through KTHB.
• Johanna Bergqvist: Att sätta praxis på pränt: En handbok i att skriva betygskriterier. (in Swedish)

Useful tips and examples of grading criteria

Note that grade descriptors or grading rubrics sometimes are used instead of grading criteria in the texts below.

• Examples of grading criteria at KTH, mostly from former participants of LH216V.
• Stefan Ekecrantz: Målrelaterade betyg - Att arbeta med betygskriterier och bedömning i sju grader, UPC-rapport 2007:1, Stockholms universitet, 2007. (in Swedish)
• Taxonomies for learning, useful when formulating grading criteria at different levels.
• Swedish active verbs from KTH course syllabuses,  and English active verbs categorized using the revised Bloom's taxonomy.
• Grading student work - tips and examples of Rubric creation from Berkeley.
• Graham Gibbs: Learning in Teams Grading of team projects, among other things.
• Phillip Dawson: Assessment rubrics: towards clearer and more replicable design, research and practice (available online through KTHB).
  Describes and discusses grading rubrics and gives 14 design elements related to rubrics. A useful paper, but a bit hard to read if you are not used to read educational research papers.  
• Lab Report Rubric. An example of a well designed grading rubric from MIT. The lab course is described, together with the ILOs. The scale 0-3 is applied on the eight types of criteria used. How the criteria relate to the ILOs is shown, together with the formula for the final scoring. All in all, this rubric looks like what you are expected to develop for your course within this course (LH216V).
• Scoring rubrics for professional writing. Nine criteria and four scoring levels that can be useful for inspiration on how to work with marking (or peer marking) where writing skills are to be trained and assessed. 
• Scoring rubrics for professional presentations. Eight criteria and four scoring levels that can be useful for inspiration on how to work with marking (or peer marking) where oral presentation skills are to be trained and assessed. 
• Natasha Haugnes and Jennifer L. Russell: Rubrics for artists and designers (available online through KTHB).
• Anders Jonsson and Gunilla Svingby: The use of scoring rubrics: Reliability, validity and educational consequences. A research review showing which benefits of rubrics that have been confirmed in research.

Students at KTH with a permanent disability can get support during studies from Funka:

Funka - compensatory support for students with disabilities

P, F

• INL1 - Assignment, 1.5 credits, Grading scale: P, F

Based on recommendation from KTH’s coordinator for disabilities, the examiner will decide how to adapt an examination for students with documented disability.

The examiner may apply another examination format when re-examining individual students.

An individual assignment in the own subject area is included in the written assignment.

There are three mandatory assessments in the course. It is important that you submit the assignments before the stated deadline in order for the course to work. Please contact the course coordinator if you need to submit a hand-in late.

1. Assignment 1a (ILOs and draft of new grading critieria), submitted before 22 April 2022 at 18:00 in Canvas.
2. Assignment 1b (feedback to peers), submitted before 27 April 2022 at 18:00 in Canvas.
3. Assignment 2 (final version of grading criteria etc.), submitted before 30 May 2022 at 18:00 in Canvas.
   Assessment (with respect to the six grading criteria below) and constructive feedback on your assignment 2 will be given by the teachers before meeting 3.
   A submission fulfilling all criteria will pass. A submission not fulfilling any criterion will fail. A submission fulfilling at least one but not all criteria will receive Fx, and could be completed (see below).

Active participation at course meetings 1 and 2 is compulsory.

Since this is a pass/fail course, the intended learning outcomes will be used as grading criteria. Below, you can see how the critera are linked to the assessments.

• account for how intended learning outcomes, grading criteria, and assessment fit together (assessed in assignment 1a (formatively) and assignment 2)
• formulate outcomes based criterion-referenced grading criteria for a course (assessed in assignment 1a (formatively) and assignment 2)
• suggest how elements of the examination should be designed so that they assess the student according to the grading criteria (assessed in assignment 2)
• review critically and reflect on ones own and others' grading criteria (assessed in assignment 1b (others') and 2 (own))
• describe how to combine assessments to determine a final grade and reflect on which effects this can result in (assessed in assignment 2)
• reflect on how the grading criteria can be used in your own teaching (assessed in assignment 2).

A submission of assignment 2 fulfilling at least one but not all criteria will receive Fx, and could be completed (submitting a new version in Canvas) within three weeks (29 June 2022). A new version that is fulfilling all criteria will pass. A new version fulfilling all criteria except one or two may be completed one more time (latest 15 August 2022). Otherwise the submission will be failed.

• All members of a group are responsible for the group's work.
• In any assessment, every student shall honestly disclose any help received and sources used.
• In an oral assessment, every student shall be able to present and answer questions about the entire assignment and solution.

• Use a comment field in the criteria matrix in the final assignment to explain why the criterion was not met.
• Adapt the course to a hybrid format, so that it will be possible to participate both physically (which we prefer) and on-line.

Advice from participants from the previous course offering

• Put aside more time than you think you need for the course. The more seriously you take the course, the better it gets.
• Really think carefully before you choose what course to work with in this course. If you are engaged as a teacher in a course using grading
  A-F (rather than P/F), my advice would be to choose the A-F course in order to get most out of this grading course.
• Try to challenge yourself. Similar to amount of time, you will learn more if you go outside of your comfort zone. Even though it may be more
  pragmatic to use a course that is mostly in need of some adjustments, using a course where you wish to try something new is very valuable.
• Look at the many interesting examples of previous participants that are provided on the course website. This can definitely help when
  preparing the own final assignment. 
• Keep track of deadlines.

Course evaluation and course analysis

By the end of meeting 1, a mini-evaluation will be carried out. At the end of the third meeting, we will evaluate the course together, which will constitute the course meeting of the course (as required by the KTH regulations for course evaluation and course analysis). After the third meeting, there will be an ordinary Learning Experience Questionnaire (LEQ).

After the course, the course coordinator will write a course analysis, that will be published on the official course web page in the course catalog.