Designing an assessment program
What is an assessment program?
An assessment program is a plan of how you are going to find out how
well students have achieved the learning
goals of your course and how you are going to provide feedback
to students to help them to achieve these goals. The program may be as
minimal as a timetable listing when and how students are to be assessed,
or it may consist of a detailed portfolio, including the actual texts of
assessment items; analyses of the ways in which goals are tested by these
items; categorisation of the items as strictly for feedback, strictly for
grading (not recommended), or an amalgam of feedback and grading; feedback
pro formas; and other items. For further discussion of the nature of
assessment programs and of the constraints on them generally and at UQ
read the PDF document titled Assessment programs: Characteristics and constraints.
There is no simple recipe for designing an assessment program. Even if
you restrict yourself only to a program aimed at giving you enough
information to assign valid grades to students' achievement, you will
still find the task a complex balancing act. You will be trying to balance
what might be desirable were you and the students not human and fallible
against what is realistic given various constraints.
In these pages we will try to provide tools and ideas to help in this
planning and balancing process. The assessment portfolio is a key tool in
documenting your program.
The assessment portfolio template
Download an assessment
portfolio template. This template contains a set of tables and other
documents which you may complete in order to compile a useful record of
your assessment program. A fairly complete portfolio would include the
tables as well as the actual texts of the assessment tasks, course level
and task level assessment criteria, marking schemes, and a means of
assigning grades to students' work in the course. This portfolio is a
private document which you will use to plan and document
your assessment program for a course. Unlike the Course profile it is not
a published public document.
The assessment portfolio
A completed assessment portfolio will contain:
- a draft assessment program
- the learning goals for the course
- assessment criteria for these learning goals
- a set of assessment tasks which make up the substance of the
assessment program
- for each assessment task, details of the criteria which will be used
in assessing students' attempts at completing the task
for each
assessment task, details of the marking scheme which will be used in
assigning grades to students' attempts at completing the task and/or
giving students feedback on their attempts
- a timetable showing week by week what assessment tasks are to be
attempted by students
- a table showing the relationship between the assessment tasks and
the course learning goals
- a table showing the relationship between the course level assessment
criteria and the criteria used for the individual assessment tasks
- a scheme for translating a student's results on the various
assessment tasks into a grade for the course
- a final assessment program.
Completing the assessment portfolio for a course you've already
taught
You will need to use the template to create a new
assessment portfolio document for each course you plan to work on. If you
have already taught the course then you may already have an assessment
plan for it. Your goal in working with this portfolio will then be to
complete and to fine tune that plan.
- Gather together all the assessment tasks you used in the most recent
offering of the course (examinations; other assessment tasks, both
during and outside semester; tasks which you or others marked solely for
feedback purposes).
- Classify each according to its main purpose: to help assign a grade
to a student; to provide feedback to students on their learning; both of
these.
- Describe what each task is designed to assess.
- Where the task is at least in part to help assign grades, write a
brief description of how it feeds into a student's grade.
- For all tasks describe briefly how, if at all, students are to
receive feedback on their attempts.
- Enter all this information into the table in the assessment
portfolio for this course.
Completing the assessment portfolio for a new course
If the course is completely new then your goal in
completing this portfolio will be to define and to set out in detail an
assessment plan. You should use the same table as that for an existing
course.
- Start by trying to formulate at least some possible assessment tasks
for the course. Possible sources of inspiration are:
- tasks others have used in teaching previous versions of this or of
somewhat similar courses
- tasks you encountered in this area while yourself a student
- tasks in text or reference books
- common practice in your School
- consultation with others (in other universities, for example)
teaching similar courses.
- You will be seeking to include in your assessment program tasks
serving the dual purposes of grading students' achievements in the
course and helping them learn, as well as tasks aimed solely at helping
students to learn. Tasks aimed solely at assigning grades are not
recommended; where the task is a final examination, feedback is still
desirable, even though the resultant learning may not take place until
after grades have been decided.
- Your assessment program should include a variety of tasks to be
completed under a variety of conditions. For example, it might include
tasks to be attempted:
- during semester, after semester, and during mid-semester breaks
- in class and out of class
- under examination conditions (or not)
- by students individually or in groups.
- Try to classify each according to the main purpose you see for it:
to help assign a grade to a student; to provide feedback to students on
their learning; both of these.
- Describe what each task is designed to assess.
- For all tasks describe briefly how, if at all, students might
receive feedback on their attempts.
- Where a task is at least in part to help assign grades, write a
brief description of how it feeds into a student's grade. This may not
be possible until you have reduced your array of possible tasks to a
program.
- Enter all of this information into the table in the assessment
portfolio for this course.
You are now in the same situation as a teacher refining the assessment
program for an existing course. Your goals now are to refine and to fine
tune the draft program you have just developed in your assessment
portfolio.
Completing the assessment program
Goals and criteria
You need a set of learning
goals for the course (statements of what the students
ought to achieve as a result of studying the course). If you are already
working on a course
profile then probably you will have written the learning goals
already. They then simply need to be copied from the course profile to the
assessment portfolio. You also need a set of criteria
against which the extent of achievement of the learning goals can be
judged for each student. Assessment criteria might be thought of as
dimensions along which students' achievement of the course learning goals
can be judged, or as yardsticks against which such achievements can be
measured. The possible extents of a student's achievements on each
criterion are called the standards which might be achieved. These might be
thought of as the lengths marked on the yardsticks.
Ideally the criteria are associated with the learning goals, and the
relevant criteria are stated more specifically for each assessment item.
In reality assessment criteria are often specified only in relation to
specific assessment tasks.
Assessment tasks, marking criteria, feedback and marking schemes
Before adding these to your assessment portfolio please browse through
the PDF document on Assessment programs: Characteristics and constraints.
This document includes information on commonly used assessment methods,
together with their generally claimed strengths and weaknesses.
In your portfolio you should try to include for each
task:
- the text of the task. Where the task is to be given to students as a
handout, ideally that handout will be written now and included in the
portfolio. Where the task is to complete exercises from another source
(textbook questions, for example) only the relevant references need to
be included
- the criteria against which students' attempts at the task are to be
judged
- a feedback scheme and, if some kind of mark or set of marks is to be
provided, a marking scheme, where marks and feedback are linked clearly
to the specified criteria.
Assessment timetable
At this stage you almost have a complete draft of the
assessment program for the course. All that remains is to timetable that
program over the period in which the course is to be attempted and to
design a scheme for allocating grades to students. In defining the
timetable you probably will consider:
- the teaching and learning program in the course - assessment is part
of this program
- the demands on the students, especially their likely assessment
loads from other courses. In unit programs (Arts and Science, for
example) there may be no common pattern of course choice among students
studying your course, so this may not be possible
- the demands on your own time over the semester (and the demands on
the time of others likely to be involved in marking and in giving
feedback)
- common practice in your School (always remembering that 'common'
practice is unlikely to be mandatory).
Grading scheme
For many courses this is the most difficult step in the design of an
assessment program. It is also an important one. The grade a student gains
in your course may help to determine or even to constrain the future
possibilities for that student. It may influence the courses the student
studies in the future and even the nature of the student's future career.
However, it is possible to overstate that importance; one of your main
tasks is to help students to learn your course - given their inclinations
and interests, to help them to achieve to the best of their abilities the
learning goals for the course. A final grade is not the only, nor
necessarily the main outcome, of a student studying your course.
Grades are an attempt to reduce the complex process of evaluating what
a student has achieved in studying your course to the awarding of a single
label. When seen this way, it is clear that the grading process is bound
to be somewhat unsatisfactory and simplistic. A good grading method will
reflect as accurately as is feasible the student's achievements.
The grade will be linked to the extent to which the student has
achieved the learning goals in the course. The student's results on the
various assessment tasks will be the input to any grading method.
Assessment criteria and standards at both course and assessment task level
will be used to ensure that these results and the ultimate grade will be
as valid as is feasible.
Refining the assessment program
In refining the draft assessment program several questions need to be
asked. They may even need to be re-asked several times in successive
cycles of refinement. The questions are:
Is the timetable feasible and reasonable for both staff and
students?
This question needs to be revisited repeatedly as
other aspects of the program are brought closer to finality. If the time
and work demands either on yourself or the students are markedly excessive
then tasks simply will not be done or, if done, will not be done on time.
It is especially important that feedback on students' assessment attempts
be provided at appropriate times. See the PDF document Grades and feedback for further information on
feedback.
Do the assessment tasks assess all of the course learning goals
that you have decided to assess?
There is a table in the
assessment portfolio designed to help you to check this. Remember that it
is not necessary or even necessarily desirable to assess all course
learning goals for grading purposes. Some may be assessed only for
feedback purposes; some may be completely unassessed, but included to show
students what they might aspire to gain from the course.
Are the course level assessment criteria consistent with the
task criteria - for example, do the task level criteria reveal that there
should be other course level criteria, or even another course learning
goal?
There is a table in the assessment portfolio designed
to help you check this. One criterion often included at the task level,
but omitted at the course level, is the quality of students' expression.
If quality of expression is used to determine a student's marks on various
tasks and, therefore, to determine grades, perhaps it is worth including
as an assessment criterion at the course level. Doing this may then lead
you to include being able to write clearly in the course area as a course
learning goal. In turn, this will require that you provide students with
the opportunity to learn to do so.
Does the grading scheme provide at the very least a defensible
evaluation of each student's achievements in the course?
This
is a crucial question. If you have written grade descriptors (descriptions
of what a student must achieve to be eligible for each grade) then you can
check to see whether your grading method awards grades appropriately
according to the descriptors.
The final assessment program
When you have answered all of these questions to your satisfaction, you
are then in a position to complete in the assessment portfolio the table
which sets out your final assessment program. Now you should also modify
the timetable to reflect the final version and modify the grading scheme
to reflect your final decision. All tasks included in the program should
be documented in as much detail as possible, including the marking
criteria for each task, and, where possible, marking schemes. When they
become available, insert copies of the actual handouts given to students
at the various stages of each assessment task.
If, in the course of finalising the assessment program, you have
discarded some assessment tasks, do not throw these away. They may be a
useful source of ideas and inspiration when you redesign the program in
future years.
|
