|
Do as I Say,
Not as I Do? Student Assesment in Problem Based
Learning |
|
Student Assessment in Problem-based
Learning Jeffrey A. Nowak and Jonathan A. Plucker Indiana
University
Originally Submitted: October 2, 1998 Revision
Submitted: June 7, 1999
Address Correspondence to the First
Author at: Indiana University School of Education 201 N. Rose
Avenue Bloomington, IN 47405-1006 (812) 857-6730 jplucker@indiana.edu jefnowak@indiana.edu
|
Abstract |
|
Problem-based learning (PBL) is an
increasingly popular curricular technique for developing scientific
and mathematical talent. Aligning PBL activities and subsequent
student assessment often proves to be difficult for teachers, with
many PBL activities followed by traditional, pencil-and-paper
assessments. This misalignment confuses students by disrupting their
understanding of teacher expectations. In this paper, we discuss the
importance of instruction-assessment alignment during PBL and
provide detailed examples of exemplary units.
|
Do as I Say,
Not as I Do? |
|
A promising recent development involves
the application of problem-based learning as a curricular vehicle to
develop student talent. Problem-based learning (PBL) is common in
professional education, such as medical, law, and business schools,
and is becoming increasingly common in pre-college education.
Several educators have suggested that PBL techniques may be
especially effective in enhancing scientific and technological
creativity (Gorman & Plucker, in press; Plucker & Gorman,
1995, in press; Plucker & Nowak, in press).
As the label implies, problem-based learning is an educational
approach where an ill-structured problem initiates learning. PBL is
necessarily interdisciplinary: By addressing real-world problems,
students are required to cross the traditional disciplinary
boundaries in their quest to solve the problem.
The primary characteristic of PBL is that the instructional unit
or activity is anchored to an authentic, ill-structured problem
(i.e., information readily available to the students is not
sufficient to solve the problem; a single, correct process for
solving the problem is not readily apparent or does not exist; the
problem may change as the students attempt to solve it). During PBL,
the teacher generally serves as a facilitator or "metacognitive
coach" (Gallagher & Stepien, 1996, p. 261), student choice
promotes ownership of learning, thinking and procedural skills
training is available as needed, and social interaction and
collaboration are required to solve the problem.
In light of the recent flurry of attention to problem-based
learning, educators should note that PBL has been used in gifted
education and science education - although not with the same label -
for many years (Sitkoff, 1988). Indeed, one of the most popular
models for educating gifted students, the Schoolwide Enrichment
Model (Renzulli & Reis, 1985, 1986), stresses the need for
experiences through which students solve real-world problems. Many
of these programs have been extensively evaluated (e.g.., Renzulli
& Reis, 1994). When considered in conjunction with other recent
evaluations (Barron et al., 1998; Gallagher & Stepien, 1996;
Krajcik et al., 1998), the research literature strongly suggests
that PBL is an effective curricular approach to developing
scientific, mathematical, and technological talent and creativity.
PBL descriptions and suggestions for implementation are readily
available (Gallagher & Stepien, 1996; Gorman, Plucker, &
Callahan, 1998; Plucker & Nowak, in press; Savery & Duffy,
1995; Savoie & Hughes, 1994; Stepien & Pike, 1997). Rather
than provide another description of a PBL unit or describe its
effectiveness, our purpose in this paper is to address an
often-neglected aspect of PBL: Aligning instruction and assessment
during problem-based approaches to learning.
|
Assessment
and PBL |
|
A major weakness of historical and
contemporary PBL efforts is the lack of formal student evaluation
(Reis & Renzulli, 1991). Indeed, assessment of student progress
is often haphazard or non-existent. When assessment is formally
planned, it often does not align well with the objectives of the
problem-based learning that preceded it. Although recent work on the
evaluation of products that emerge from students? PBL efforts is
impressive (Reis & Renzulli, 1991; Treffinger, 1989; Westberg,
1991), it is infrequently applied to PBL units.
In the remainder of this paper, we describe the various types of
misalignment, provide suggestions for aligning instructional
activities and assessment, and describe three examples of PBL
instruction-assessment alignment.
To illustrate the problems caused by ineffective use of
assessment during PBL activities, consider the following three
scenarios: In the first, students are told that the year is 1876,
and Alexander Graham Bell is about to patent his telephone. Provided
with Bell?s notebooks, several phone patents, and various electrical
supplies and other materials, students design a variation of the
telephone, build a working prototype, write a patent application,
and present and defend their design and prototype to their peers and
a person acting the role of a patent examiner. In this situation,
both instruction and assessment are problem-based.
In the second scenario, the teacher guides students through the
same PBL invention unit, but she then administers traditional
multiple-choice and short-answer questions to evaluate student
achievement. This lack of consistency will eventually be
self-defeating for the teacher. If a purpose of PBL is to allow
students to learn authentically (i.e., as people learn when they are
outside of school), reverting to less authentic assessments defeats
the purpose of using PBL in the first place. This misalignment is
quite common when teachers first attempt to implement PBL units, and
it results from teacher concern that students are not learning as
much content as they would during a traditional lecture. However,
research suggests that this concern is overstated (see Dods, 1997;
Gallagher, 1997; Gallagher & Stepien, 1996).
In the third scenario, the teacher covers some of the same
material that students in the first two scenarios explored (i.e.,
circuit design), but in a lecture-discussion format. He assesses
student achievement at the end of the unit with an elaborate
problem-based activity that requires students to apply the lecture
material to a real-world situation. As in the case of the second
scenario, students are confused and perform poorly, this time
because they are not prepared to apply the material in such a
complex, authentic way. Many teachers who enthusiastically embrace
PBL decide to ?start small? and only implement PBL principles in
part of their teaching, rather than promote consistent student
expectations by thoroughly implementing PBL both to instruction and
assessment.
|
Suggestions
for Aligning PBL Instruction and Assessment
|
|
Suggestion One: Stress
that students are professionals in the field in which the
ill-structured problem exists and assess them as if you were their
supervisor
One of the greatest challenges to teachers is to keep their
students motivated and engaged in classroom activities. Having
students operate as professionals in the field in which the
ill-structured problem arises increases student enthusiasm and
ownership for learning. For instance, if studying a unit on solar
home design, the students could play the role of architects. As part
of the role-playing, teachers acting as facilitators and real-world
supervisors hold positions that make assessment both appropriate and
realistic. Also in this context, the activity is no longer unrelated
to anything outside of the classroom. Students in this scenario can
now see that their efforts relate to issues that society has faced
or is facing.
Suggestion Two: If instruction is
problem-based, assessment should be similarly structured
If students are asked in the course of a unit to solve
ill-structured problems through hands-on activities, the assessment
should include how well they complete that task. That is not to say
an evaluation of their ability to learn factual and foundational
information important to solving the problem should not be
completed. Rather, an interdisciplinary, real world and hands-on
approach to learning should be evaluated largely in the same manner
it is taught. To have students design and build a model of a better
solar home and then have the assessment be based solely on a true
and false, multiple choice test undermines the creative process and
sends mixed messages to students about the importance of the PBL
activity. The instructor should instead act as a building inspector
and qualitatively and quantitatively evaluate the students' work.
Suggestion Three: Provide reasonable guidelines
regarding your expectations for the students
A single path to the solution of a real world ill-structured
problem rarely exists, whether it relates to what scientists face in
the laboratory or professionals encounter in the field. Teachers
engaged in PBL should present student expectations before the unit
begins so the students will understand their goals and how their
progress will be assessed. Of course, ridiculously detailed goals
and solution criteria are antithetical to PBL, so expectations
should be flexible enough to allow for student exploration.
Providing students with this allowance for creativity while
maintaining a realistic timeline fosters growth and inventiveness
that is not easily achieved within a cookbook lab or worksheet based
curriculum. For example, in the case of designing a better solar
home, create a list of open-ended goals or "building code
specifications" with the students at the beginning of the PBL
activity.
Suggestion Four: Don?t hold off on assessment
until the end of the activity or unit; model real-world behavior, in
which ongoing assessment occurs
In traditional classroom teaching, assessment of student learning
is often relegated to the end of a given unit. This assessment tends
to stress student recollection of factual knowledge, in direct
opposition to current beliefs that significant amounts of learning
take place during the process of solving a problem. The emphasis on
the use of factual knowledge in conjunction with real world problem
solving skills makes PBL an advantageous approach to teaching and
assessment. As shown in the following three examples, instructors
need to assess students continuously during the course of their
problem solving, much as real-world supervisors would oversee a
project. For instance, the instructor as building inspector is free
to examine how well the students address the established goals while
also having the freedom to suggest modifications or ?sign off? on
student developments. This role allows the teacher to act as a
facilitator, asking guiding questions that allow students to
approach a solution or solutions to the problem at
hand. |
Three
Examples |
|
The following examples are taken from our
work with problem-based learning. Although all have
interdisciplinary elements, the examples focus primarily on earth
science, life science, and physical science, respectively.
|
Earth
Science: Identifying Minerals |
|
Traditional methods for teaching K-12
students how to identify minerals involve providing students with
several tools (e.g., mineral key, pictures of crystal shapes, a
streak plate, glass plate, nail, penny, weak acid, hand lens), a
step-by-step demonstration about how to use the tools, and an
activity in which students identify the minerals based on the
demonstrated procedure.
In a PBL unit designed to cover the same curriculum, students
were told that they are playing the roles of geologists. Their task
was to identify the minerals at a couple of local sites in order to
facilitate the modification of local zoning ordinances. All of the
above mentioned tools and various minerals were made available, and
the students worked in small groups to identify the minerals. As
student curiosity became engaged, the freedom to explore eliminated
student fears of not using the ?correct? method. After preliminary
attempts to identify the materials, the instructor encouraged
students to discuss the various methods they employed (i.e.,
comparing similarities and differences, classifying the various
characteristics). At this point, the teacher introduced the standard
method of mineral identification and had students probe the method?s
strengths and weaknesses. Students were then given the opportunity
to apply and expand their knowledge by identifying a new group of
minerals.
At the end of the unit, after several similar PBL activities,
student achievement was assessed by bringing students to a local
quarry and a wilderness area and requiring them to identify as many
different minerals as possible. In this setting, the students - who
were addressed as geologists throughout the unit - now had the
opportunity to work in the field as real geologists. In addition to
the teacher?s field-based observations, the students were also asked
to log in their field notebooks what they found, where it was found,
and describe how and why they identified it as they did. Finally, a
debriefing discussion during the next class meeting in which
individual students and small groups reflect on and share their
experiences with the rest of the class helps students to compare and
contrast the creative skills employed. The authenticity of applying
skills to a real world problem is what makes this PBL assessment
valid and appealing to students.
|
Life
Sciences: Virtual Pet Dissection |
|
One trend in science education is to find
alternatives to animal dissection. However, using computer software,
computer generated graphics, or Web-based simulations leaves most
teachers and students with the sense that they missed doing a
hands-on activity proven to keep students engaged and learning.
In this activity, fourth to sixth grade students were told they
were scientists with the important job of determining whether or not
the virtual pet constituted a robot. Was it a benign software
program or a real robot worthy to be called a pet? In this quest, it
was important for students to draw analogies between the form and
function of the components existent in this virtual pet with the
anatomy of a real vertebrate pet. Ethical issues were also raised
within the context of popular movie plots: Was this a harmless
software game? Or could a robot take its artificial intelligence and
turn against humans in the process of seeking self-preservation? The
beauty of the context of this PBL activity is that there is no
obvious single correct process for addressing these complex
dilemmas.
Groups of four students were given a virtual pet, a small
screwdriver, a petri dish to hold small parts so that they did not
get lost, and a journal to record perceived analogies between the
anatomy of real vertebrate pets and the virtual pet. Whenever
possible, the instructors answered student questions with leading
questions without directly answering the student question. The
instructors also had texts on robots and human anatomy available for
students to use as references. The students quickly took ownership
of the task at hand and soon realized that their solutions to the
ill-structured question were dependent on their ability to perform
as good scientists.
As an assessment of this PBL activity, the instructors were able
to supervise student engagement and interaction during the activity,
examine what was written in each student's scientific journal, and
monitor large group discussion at the close of the activity. Through
these various assessment techniques, the instructors observed
considerable student growth in critical thinking and research
skills. Collaboration encouraged students to reflect on peers' and
their own plausible solutions to the question of whether or not the
virtual pet constituted a robot , and interpersonal skills were also
honed as each member enjoined, or sought to discuss why not to
enjoin, other members' insights into their eventual solutions to the
question. Ultimately, different groups came up with different
solutions to the problem. The students also said they really loved
the activity and voiced a greater appreciation for the scientific
endeavor!
|
Physical
Science: Energy in Chemical and Physical
Changes |
|
In this PBL activity, undergraduate
pre-service teachers were asked to address the problem of heating
and cooling a house at a point in the future when fossil fuels have
been exhausted. The house was situated in a region where cloud cover
and a lack of consistent winds and fast moving water prevented the
use of solar, wind, and water power. As a result, students had to
design a house that was heated and cooled solely by chemical means.
The chemicals made available to the students were road or melting
salt (CaCl2), baking soda, and water. The students were not given
any information about the exothermic or endothermic properties of
these chemicals. The students had access to the popular Texas
Instruments TI-83 calculators, calculator-based laboratories with
temperature probes, desktop computers, chemical indexes, and physics
texts.
In the course of the lab, students determined the solubility of
the chemicals and amount of heat produced and absorbed by varying
amounts of the chemicals in varying amounts of water, as well as the
solubility of these chemicals. If the students were not using
conventions such as specific heat or calories they were encouraged
to do so through the use of guiding questions from the instructor.
Students also contemplated different approaches for dissipating
energy to different areas of the house, as well as how to dispose of
the chemical waste, which are both inherent dilemmas to this
problem.
As an assessment of this PBL activity, in addition to observing
students' progress, the instructor was able to examine the equations
recorded in student journals. Since this activity was performed in a
pre-service teacher course, the instructor asked the students to
evaluate the activity and describe what they thought were positive
and negative outcomes of this approach. Students mentioned that this
approach seemed frustrating when they were uncertain of whether
their attempts to arrive at solutions were adequate, but they
thought the process was much more realistic and true to scientific
inquiry. They appreciated the freedom to design and test different
possible solutions to problems. During subsequent class discussion,
the students also mentioned how they were better able to understand
and recall the material from the lab because they had created the
testing procedure as opposed to having it provided to them at the
outset.
|
Discussion |
|
In these PBL examples, the alignment of
instruction and assessment is paramount. Most people would be
puzzled by an assessment of mountain climbing ability that involved
a test of the prospective mountaineer's ability to paint a
watercolor of the mountain. Teachers should find it just as
questionable to evaluate a student's ability to solve a real-world
problems with a pencil-and-paper, true and false or multiple choice
examination, especially after introducing the targeted skills and
content with PBL activities. In order to help students gain the
social and intellectual skills needed to solve real-life problems,
they need to be trained and assessed in real-world scenarios.
Consider if, for example, students in the Identifying Minerals
example were never given the opportunity to classify minerals first
in their own way. Student understanding of the difficulty
experienced by the first geologists to classify minerals would be
nonexistent. They could reasonably conclude that all existing
minerals have been found and identified. They may even leave with
the misconception that traditional conventions in science have
achieved perfection and that no further evaluation of these
processes is necessary. By allowing the students to function as
geologists with the role of classifying minerals for the first time,
however, an appreciation for these realities can be achieved.
Providing students with opportunities to demonstrate their
ability to identify minerals both during the unit and at the end of
the unit is also important. If the hands-on process of identifying
minerals was abandoned at the end of the mini-unit and again at the
end of the entire unit and was replaced solely with a summative
paper and pencil test, most educators would agree that little
assessment of what was really learned would take place. Yet,
whenever a PBL activity is performed without both formative
(ongoing) and summative (end-of-activity) assessment this is
precisely what occurs!
Having both summative and formative problem-based assessments is
crucial to the success of a PBL activity. For example, consider how
the physical science activity would have been different if students
were never asked to solve the heating/cooling problem and were
instead taught the physical and chemical concepts via a
definitionally-based, step-by-step lab. Student ownership of the
activity and relative engagement in the creative process would be
poor. And what if the unit were changed to include PBL but with only
summative assessment? Student ownership and creative engagement
would be reintroduced, but the narrow assessment focus would provide
little guidance for students and may even result in confusion over
teachers' expectations. By not conducting an ongoing evaluation,
students could even be discouraged from investing the creative
energy it takes to be real-world problem solvers.
|
Conclusion |
|
Problem-based learning, when used as a
curricular approach to developing scientific talent, provides
students with an authentic learning experience that also enhances
collaborative and problem-solving skills. Evaluation of student
achievement is an important aspect of education, and the skills
inherent in the process of solving real-world problems must also be
included in that assessment. The alignment of instruction and
assessment during PBL is essential if this approach is to be used.
If not, the situation is one in which students are taught to value
authentic approaches to learning, but only through our admonitions
and not our actions -- Do as I say, not as I do. As learning theory
suggests (e.g., Bandura, 1986), problem-based learning will be more
effective if students are surrounded by models that are practicing
what they preach.
|
References |
|
Abeles, S. (1988). Avenues to opportunity
for self-identified, high-ability secondary school students. In P.
F. Brandwein & A. H. Passow (Eds.), Gifted young in science:
Potential through performance (pp. 189-200). Washington, DC:
National Science Teachers Association.
Bandura, A. (1986). Social foundations of thought and action: A
social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.
Barron, B. J. S., Schwartz, D. L., Vye, N. J., Moore, A.,
Petrosino, A., Zech, L., Bransford, J. D., & The Cognition and
Technology Group at Vanderbilt. (1998). Doing with understanding:
Lessons from research on problem- and project-based learning. The
Journal of the Learning Sciences, 7, 271-312.
Brandwein, P. F., & Passow, A. H. (Eds.). (1988). Gifted
young in science: Potential through performance. Washington, DC:
National Science Teachers Association.
Dods, R. F. (1997). An action research study of the effectiveness
of problem-based learning in promoting the acquisition and retention
of knowledge. Journal for the Education of the Gifted, 20, 423-437.
Eilber, C. R. (1987). Phi Delta Kappan, 68(10), 773-777.
Gallagher, S. A. (1997). Problem-based learning: Where did it
come from, what does it do, and where is it going? Journal for the
Education of the Gifted, 20, 332-362.
Gallagher, S. A., & Stepien, W. J. (1996). Content
acquisition in problem-based learning: Depth versus breadth in
American studies. Journal for the Education of the Gifted, 19,
257-275.
Gorman, M. E., & Plucker, J. (in press). Teaching invention
as critical creative processes: A course on technoscientific
creativity. In M. A. Runco (Ed.), Critical creative processes.
Norwood, NJ: Ablex.
Gorman, M. E., Plucker, J., & Callahan, C. M. (1998). Turning
students into inventors: Active learning modules for secondary
students. Phi Delta Kappan, 79(7), 530-535.
Kopelman, M., Galasso, V. G., & Schmuckler, M. (1988). A
program for stimulating creativity in a citywide high school: The
Bronx High School of Science. In P. F. Brandwein & A. H. Passow
(Eds.), Gifted young in science: Potential through performance (pp.
209-216). Washington, DC: National Science Teachers Association.
Krajcik, J., Blumenfeld, P. C., Marx, R. W., Bass, K. M.,
Fredricks, J., & Soloway, E. (1998). Inquiry in project-based
science classrooms: Initial attempts by middle school students. The
Journal of the Learning Sciences, 7, 313-350.
Maker, C. J. (Ed.). (1986). Critical issues in gifted education:
Defensible programs for the gifted. Austin, TX: Pro-Ed.
Passow, A. H. (1988). School, university, laboratory, and museum
cooperation in identifying and nurturing potential scientists. In P.
F. Brandwein & A. H. Passow (Eds.), Gifted young in science:
Potential through performance (pp. 245-253). Washington, DC:
National Science Teachers Association.
Plucker, J. A., & Gorman, M. E. (1995). Group interaction
during a summer course on invention and design for high ability
secondary students. The Journal of Secondary Gifted Education, 6,
258-272.
Plucker, J. A., & Gorman, M. E. (in press). Invention is in
the mind of the adolescent: Evaluation of a summer course one year
later. Creativity Research Journal.
Plucker, J. A., & Nowak, J. A. (in press). Creativity in
science for K-8 practitioners: problem-based approaches to discovery
and invention. In M. Lynch & C. R. Harris (Eds.), Teaching the
creative child, K-8. Boston, MA: Allyn & Bacon.
Reis, S. M., & Renzulli, J. S. (1991). The assessment of
creative products in programs for gifted and talented students.
Gifted Child Quarterly, 35, 128-134.
Renzulli, J. S. (Ed.). (1986). Systems and models for developing
programs for the gifted and talented. Mansfield, Center, CT:
Creative Learning Press.
Renzulli, J. S. (1994). Schools for talent development: A
practical plan for total school improvement. Mansfield Center, CT:
Creative Learning Press.
Renzulli, J. S., & Reis, S. M. (1985). The schoolwide
enrichment model: A comprehensive plan for educational excellence.
Mansfield Center, CT: Creative Learning Press.
Renzulli, J. S., & Reis, S. M. (1986). The enrichment
triad/revolving door model: A schoolwide plan for the development of
creative productivity. In J. S. Renzulli (Ed.), Systems and models
for developing programs for the gifted and talented (pp. 216-266).
Mansfield Center, CT: Creative Learning Press.
Renzulli, J. S., & Reis, S. M. (1994). Research related to
the schoolwide enrichment triad model. Gifted Child Quarterly, 38,
7-20.
Savery, J. R., & Duffy, T. M. (1995). Problem based learning:
An instructional model and its constructivist framework. Educational
Technology, 35(5), 31-38.
Savoie, J. M., & Hughes, A. S. (1994). Problem-based learning
as classroom solutions. Educational Leadership, 52(3), 54-57.
Sitkoff, S. (1988). Science centers - An essential support system
for teaching science. In P. F. Brandwein & A. H. Passow (Eds.),
Gifted young in science: Potential through performance (pp.
179-187). Washington, DC: National Science Teachers Association.
Stepien, W. J., & Pyke, S. L. (1997). Designing problem-based
learning units. Journal for the Education of the Gifted, 20,
380-400.
Treffinger, D. J. (1989). Student invention evaluation kit: Field
test edition. Sarasota, FL: Center for Creative Learning.
Westberg, K. L. (1991). The effects of instruction in the
inventing process on students' development of inventions.
Dissertation Abstracts International, 51. (University Microfilms No.
9107625)
|
Authors'
Note |
|
The authors appreciate the constructive
feedback of Tom Keating and several anonymous reviewers. The
scenarios described early in this paper are based on a problem-based
invention module developed by Michael Gorman and his colleagues at
the University of Virginia (see Gorman, Plucker, & Callahan,
1998). Interested educators are encouraged to visit the related Web
site http://jefferson.village.virginia.edu/~meg3c/id/id_sep/id_sep.html.
However, all but Scenario 3 are fictitious but based on similar real
occurrences (all identifying information has been deleted or
altered).
PBL Assessment
jplucker@indiana.edu
|
|