Assessment in Problem Based Learning


Q515
LWLR Home
Jodi Huebert
HS Curriculum Overview
HS Intro. Programming Curriculum
Dana Calfee
Webquest: Feeding Frenzy
Jeff Nowak
Assessment in PBL

LEGOBots
Home
Photohistory
Papers & Pres.
LEGOBots Links

LEGOs at IUB
A110 TecTrac
A290: Intro to LEGO Robotics
Q514/515: Learning with LEGO Robots
WebWorms!

The Technology
LEGOScheme

Do as I Say, Not as I Do?
Student Assesment in Problem Based Learning

 
Student Assessment in Problem-based Learning
Jeffrey A. Nowak and Jonathan A. Plucker
Indiana University

Originally Submitted: October 2, 1998
Revision Submitted: June 7, 1999

Address Correspondence to the First Author at:
Indiana University School of Education
201 N. Rose Avenue
Bloomington, IN 47405-1006
(812) 857-6730
 jplucker@indiana.edu
jefnowak@indiana.edu

Abstract

  Problem-based learning (PBL) is an increasingly popular curricular technique for developing scientific and mathematical talent. Aligning PBL activities and subsequent student assessment often proves to be difficult for teachers, with many PBL activities followed by traditional, pencil-and-paper assessments. This misalignment confuses students by disrupting their understanding of teacher expectations. In this paper, we discuss the importance of instruction-assessment alignment during PBL and provide detailed examples of exemplary units.

Do as I Say, Not as I Do?

  A promising recent development involves the application of problem-based learning as a curricular vehicle to develop student talent. Problem-based learning (PBL) is common in professional education, such as medical, law, and business schools, and is becoming increasingly common in pre-college education. Several educators have suggested that PBL techniques may be especially effective in enhancing scientific and technological creativity (Gorman & Plucker, in press; Plucker & Gorman, 1995, in press; Plucker & Nowak, in press).

As the label implies, problem-based learning is an educational approach where an ill-structured problem initiates learning. PBL is necessarily interdisciplinary: By addressing real-world problems, students are required to cross the traditional disciplinary boundaries in their quest to solve the problem.

The primary characteristic of PBL is that the instructional unit or activity is anchored to an authentic, ill-structured problem (i.e., information readily available to the students is not sufficient to solve the problem; a single, correct process for solving the problem is not readily apparent or does not exist; the problem may change as the students attempt to solve it). During PBL, the teacher generally serves as a facilitator or "metacognitive coach" (Gallagher & Stepien, 1996, p. 261), student choice promotes ownership of learning, thinking and procedural skills training is available as needed, and social interaction and collaboration are required to solve the problem.

In light of the recent flurry of attention to problem-based learning, educators should note that PBL has been used in gifted education and science education - although not with the same label - for many years (Sitkoff, 1988). Indeed, one of the most popular models for educating gifted students, the Schoolwide Enrichment Model (Renzulli & Reis, 1985, 1986), stresses the need for experiences through which students solve real-world problems. Many of these programs have been extensively evaluated (e.g.., Renzulli & Reis, 1994). When considered in conjunction with other recent evaluations (Barron et al., 1998; Gallagher & Stepien, 1996; Krajcik et al., 1998), the research literature strongly suggests that PBL is an effective curricular approach to developing scientific, mathematical, and technological talent and creativity.

PBL descriptions and suggestions for implementation are readily available (Gallagher & Stepien, 1996; Gorman, Plucker, & Callahan, 1998; Plucker & Nowak, in press; Savery & Duffy, 1995; Savoie & Hughes, 1994; Stepien & Pike, 1997). Rather than provide another description of a PBL unit or describe its effectiveness, our purpose in this paper is to address an often-neglected aspect of PBL: Aligning instruction and assessment during problem-based approaches to learning.

Assessment and PBL

  A major weakness of historical and contemporary PBL efforts is the lack of formal student evaluation (Reis & Renzulli, 1991). Indeed, assessment of student progress is often haphazard or non-existent. When assessment is formally planned, it often does not align well with the objectives of the problem-based learning that preceded it. Although recent work on the evaluation of products that emerge from students? PBL efforts is impressive (Reis & Renzulli, 1991; Treffinger, 1989; Westberg, 1991), it is infrequently applied to PBL units.

In the remainder of this paper, we describe the various types of misalignment, provide suggestions for aligning instructional activities and assessment, and describe three examples of PBL instruction-assessment alignment.

To illustrate the problems caused by ineffective use of assessment during PBL activities, consider the following three scenarios: In the first, students are told that the year is 1876, and Alexander Graham Bell is about to patent his telephone. Provided with Bell?s notebooks, several phone patents, and various electrical supplies and other materials, students design a variation of the telephone, build a working prototype, write a patent application, and present and defend their design and prototype to their peers and a person acting the role of a patent examiner. In this situation, both instruction and assessment are problem-based.

In the second scenario, the teacher guides students through the same PBL invention unit, but she then administers traditional multiple-choice and short-answer questions to evaluate student achievement. This lack of consistency will eventually be self-defeating for the teacher. If a purpose of PBL is to allow students to learn authentically (i.e., as people learn when they are outside of school), reverting to less authentic assessments defeats the purpose of using PBL in the first place. This misalignment is quite common when teachers first attempt to implement PBL units, and it results from teacher concern that students are not learning as much content as they would during a traditional lecture. However, research suggests that this concern is overstated (see Dods, 1997; Gallagher, 1997; Gallagher & Stepien, 1996).

In the third scenario, the teacher covers some of the same material that students in the first two scenarios explored (i.e., circuit design), but in a lecture-discussion format. He assesses student achievement at the end of the unit with an elaborate problem-based activity that requires students to apply the lecture material to a real-world situation. As in the case of the second scenario, students are confused and perform poorly, this time because they are not prepared to apply the material in such a complex, authentic way. Many teachers who enthusiastically embrace PBL decide to ?start small? and only implement PBL principles in part of their teaching, rather than promote consistent student expectations by thoroughly implementing PBL both to instruction and assessment.

Suggestions for Aligning PBL Instruction and Assessment

  Suggestion One: Stress that students are professionals in the field in which the ill-structured problem exists and assess them as if you were their supervisor

One of the greatest challenges to teachers is to keep their students motivated and engaged in classroom activities. Having students operate as professionals in the field in which the ill-structured problem arises increases student enthusiasm and ownership for learning. For instance, if studying a unit on solar home design, the students could play the role of architects. As part of the role-playing, teachers acting as facilitators and real-world supervisors hold positions that make assessment both appropriate and realistic. Also in this context, the activity is no longer unrelated to anything outside of the classroom. Students in this scenario can now see that their efforts relate to issues that society has faced or is facing.


Suggestion Two: If instruction is problem-based, assessment should be similarly structured

If students are asked in the course of a unit to solve ill-structured problems through hands-on activities, the assessment should include how well they complete that task. That is not to say an evaluation of their ability to learn factual and foundational information important to solving the problem should not be completed. Rather, an interdisciplinary, real world and hands-on approach to learning should be evaluated largely in the same manner it is taught. To have students design and build a model of a better solar home and then have the assessment be based solely on a true and false, multiple choice test undermines the creative process and sends mixed messages to students about the importance of the PBL activity. The instructor should instead act as a building inspector and qualitatively and quantitatively evaluate the students' work.

Suggestion Three: Provide reasonable guidelines regarding your expectations for the students

A single path to the solution of a real world ill-structured problem rarely exists, whether it relates to what scientists face in the laboratory or professionals encounter in the field. Teachers engaged in PBL should present student expectations before the unit begins so the students will understand their goals and how their progress will be assessed. Of course, ridiculously detailed goals and solution criteria are antithetical to PBL, so expectations should be flexible enough to allow for student exploration. Providing students with this allowance for creativity while maintaining a realistic timeline fosters growth and inventiveness that is not easily achieved within a cookbook lab or worksheet based curriculum. For example, in the case of designing a better solar home, create a list of open-ended goals or "building code specifications" with the students at the beginning of the PBL activity.

Suggestion Four: Don?t hold off on assessment until the end of the activity or unit; model real-world behavior, in which ongoing assessment occurs

In traditional classroom teaching, assessment of student learning is often relegated to the end of a given unit. This assessment tends to stress student recollection of factual knowledge, in direct opposition to current beliefs that significant amounts of learning take place during the process of solving a problem. The emphasis on the use of factual knowledge in conjunction with real world problem solving skills makes PBL an advantageous approach to teaching and assessment. As shown in the following three examples, instructors need to assess students continuously during the course of their problem solving, much as real-world supervisors would oversee a project. For instance, the instructor as building inspector is free to examine how well the students address the established goals while also having the freedom to suggest modifications or ?sign off? on student developments. This role allows the teacher to act as a facilitator, asking guiding questions that allow students to approach a solution or solutions to the problem at hand.

Three Examples

  The following examples are taken from our work with problem-based learning. Although all have interdisciplinary elements, the examples focus primarily on earth science, life science, and physical science, respectively.

Earth Science: Identifying Minerals

  Traditional methods for teaching K-12 students how to identify minerals involve providing students with several tools (e.g., mineral key, pictures of crystal shapes, a streak plate, glass plate, nail, penny, weak acid, hand lens), a step-by-step demonstration about how to use the tools, and an activity in which students identify the minerals based on the demonstrated procedure.

In a PBL unit designed to cover the same curriculum, students were told that they are playing the roles of geologists. Their task was to identify the minerals at a couple of local sites in order to facilitate the modification of local zoning ordinances. All of the above mentioned tools and various minerals were made available, and the students worked in small groups to identify the minerals. As student curiosity became engaged, the freedom to explore eliminated student fears of not using the ?correct? method. After preliminary attempts to identify the materials, the instructor encouraged students to discuss the various methods they employed (i.e., comparing similarities and differences, classifying the various characteristics). At this point, the teacher introduced the standard method of mineral identification and had students probe the method?s strengths and weaknesses. Students were then given the opportunity to apply and expand their knowledge by identifying a new group of minerals.

At the end of the unit, after several similar PBL activities, student achievement was assessed by bringing students to a local quarry and a wilderness area and requiring them to identify as many different minerals as possible. In this setting, the students - who were addressed as geologists throughout the unit - now had the opportunity to work in the field as real geologists. In addition to the teacher?s field-based observations, the students were also asked to log in their field notebooks what they found, where it was found, and describe how and why they identified it as they did. Finally, a debriefing discussion during the next class meeting in which individual students and small groups reflect on and share their experiences with the rest of the class helps students to compare and contrast the creative skills employed. The authenticity of applying skills to a real world problem is what makes this PBL assessment valid and appealing to students.

Life Sciences: Virtual Pet Dissection

  One trend in science education is to find alternatives to animal dissection. However, using computer software, computer generated graphics, or Web-based simulations leaves most teachers and students with the sense that they missed doing a hands-on activity proven to keep students engaged and learning.

In this activity, fourth to sixth grade students were told they were scientists with the important job of determining whether or not the virtual pet constituted a robot. Was it a benign software program or a real robot worthy to be called a pet? In this quest, it was important for students to draw analogies between the form and function of the components existent in this virtual pet with the anatomy of a real vertebrate pet. Ethical issues were also raised within the context of popular movie plots: Was this a harmless software game? Or could a robot take its artificial intelligence and turn against humans in the process of seeking self-preservation? The beauty of the context of this PBL activity is that there is no obvious single correct process for addressing these complex dilemmas.

Groups of four students were given a virtual pet, a small screwdriver, a petri dish to hold small parts so that they did not get lost, and a journal to record perceived analogies between the anatomy of real vertebrate pets and the virtual pet. Whenever possible, the instructors answered student questions with leading questions without directly answering the student question. The instructors also had texts on robots and human anatomy available for students to use as references. The students quickly took ownership of the task at hand and soon realized that their solutions to the ill-structured question were dependent on their ability to perform as good scientists.

As an assessment of this PBL activity, the instructors were able to supervise student engagement and interaction during the activity, examine what was written in each student's scientific journal, and monitor large group discussion at the close of the activity. Through these various assessment techniques, the instructors observed considerable student growth in critical thinking and research skills. Collaboration encouraged students to reflect on peers' and their own plausible solutions to the question of whether or not the virtual pet constituted a robot , and interpersonal skills were also honed as each member enjoined, or sought to discuss why not to enjoin, other members' insights into their eventual solutions to the question. Ultimately, different groups came up with different solutions to the problem. The students also said they really loved the activity and voiced a greater appreciation for the scientific endeavor!

Physical Science: Energy in Chemical and Physical Changes

  In this PBL activity, undergraduate pre-service teachers were asked to address the problem of heating and cooling a house at a point in the future when fossil fuels have been exhausted. The house was situated in a region where cloud cover and a lack of consistent winds and fast moving water prevented the use of solar, wind, and water power. As a result, students had to design a house that was heated and cooled solely by chemical means.

The chemicals made available to the students were road or melting salt (CaCl2), baking soda, and water. The students were not given any information about the exothermic or endothermic properties of these chemicals. The students had access to the popular Texas Instruments TI-83 calculators, calculator-based laboratories with temperature probes, desktop computers, chemical indexes, and physics texts.

In the course of the lab, students determined the solubility of the chemicals and amount of heat produced and absorbed by varying amounts of the chemicals in varying amounts of water, as well as the solubility of these chemicals. If the students were not using conventions such as specific heat or calories they were encouraged to do so through the use of guiding questions from the instructor. Students also contemplated different approaches for dissipating energy to different areas of the house, as well as how to dispose of the chemical waste, which are both inherent dilemmas to this problem.

As an assessment of this PBL activity, in addition to observing students' progress, the instructor was able to examine the equations recorded in student journals. Since this activity was performed in a pre-service teacher course, the instructor asked the students to evaluate the activity and describe what they thought were positive and negative outcomes of this approach. Students mentioned that this approach seemed frustrating when they were uncertain of whether their attempts to arrive at solutions were adequate, but they thought the process was much more realistic and true to scientific inquiry. They appreciated the freedom to design and test different possible solutions to problems. During subsequent class discussion, the students also mentioned how they were better able to understand and recall the material from the lab because they had created the testing procedure as opposed to having it provided to them at the outset.

Discussion

  In these PBL examples, the alignment of instruction and assessment is paramount. Most people would be puzzled by an assessment of mountain climbing ability that involved a test of the prospective mountaineer's ability to paint a watercolor of the mountain. Teachers should find it just as questionable to evaluate a student's ability to solve a real-world problems with a pencil-and-paper, true and false or multiple choice examination, especially after introducing the targeted skills and content with PBL activities. In order to help students gain the social and intellectual skills needed to solve real-life problems, they need to be trained and assessed in real-world scenarios.

Consider if, for example, students in the Identifying Minerals example were never given the opportunity to classify minerals first in their own way. Student understanding of the difficulty experienced by the first geologists to classify minerals would be nonexistent. They could reasonably conclude that all existing minerals have been found and identified. They may even leave with the misconception that traditional conventions in science have achieved perfection and that no further evaluation of these processes is necessary. By allowing the students to function as geologists with the role of classifying minerals for the first time, however, an appreciation for these realities can be achieved.

Providing students with opportunities to demonstrate their ability to identify minerals both during the unit and at the end of the unit is also important. If the hands-on process of identifying minerals was abandoned at the end of the mini-unit and again at the end of the entire unit and was replaced solely with a summative paper and pencil test, most educators would agree that little assessment of what was really learned would take place. Yet, whenever a PBL activity is performed without both formative (ongoing) and summative (end-of-activity) assessment this is precisely what occurs!

Having both summative and formative problem-based assessments is crucial to the success of a PBL activity. For example, consider how the physical science activity would have been different if students were never asked to solve the heating/cooling problem and were instead taught the physical and chemical concepts via a definitionally-based, step-by-step lab. Student ownership of the activity and relative engagement in the creative process would be poor. And what if the unit were changed to include PBL but with only summative assessment? Student ownership and creative engagement would be reintroduced, but the narrow assessment focus would provide little guidance for students and may even result in confusion over teachers' expectations. By not conducting an ongoing evaluation, students could even be discouraged from investing the creative energy it takes to be real-world problem solvers.

Conclusion

  Problem-based learning, when used as a curricular approach to developing scientific talent, provides students with an authentic learning experience that also enhances collaborative and problem-solving skills. Evaluation of student achievement is an important aspect of education, and the skills inherent in the process of solving real-world problems must also be included in that assessment. The alignment of instruction and assessment during PBL is essential if this approach is to be used. If not, the situation is one in which students are taught to value authentic approaches to learning, but only through our admonitions and not our actions -- Do as I say, not as I do. As learning theory suggests (e.g., Bandura, 1986), problem-based learning will be more effective if students are surrounded by models that are practicing what they preach.

References

  Abeles, S. (1988). Avenues to opportunity for self-identified, high-ability secondary school students. In P. F. Brandwein & A. H. Passow (Eds.), Gifted young in science: Potential through performance (pp. 189-200). Washington, DC: National Science Teachers Association.

Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.

Barron, B. J. S., Schwartz, D. L., Vye, N. J., Moore, A., Petrosino, A., Zech, L., Bransford, J. D., & The Cognition and Technology Group at Vanderbilt. (1998). Doing with understanding: Lessons from research on problem- and project-based learning. The Journal of the Learning Sciences, 7, 271-312.

Brandwein, P. F., & Passow, A. H. (Eds.). (1988). Gifted young in science: Potential through performance. Washington, DC: National Science Teachers Association.

Dods, R. F. (1997). An action research study of the effectiveness of problem-based learning in promoting the acquisition and retention of knowledge. Journal for the Education of the Gifted, 20, 423-437.

Eilber, C. R. (1987). Phi Delta Kappan, 68(10), 773-777.

Gallagher, S. A. (1997). Problem-based learning: Where did it come from, what does it do, and where is it going? Journal for the Education of the Gifted, 20, 332-362.

Gallagher, S. A., & Stepien, W. J. (1996). Content acquisition in problem-based learning: Depth versus breadth in American studies. Journal for the Education of the Gifted, 19, 257-275.

Gorman, M. E., & Plucker, J. (in press). Teaching invention as critical creative processes: A course on technoscientific creativity. In M. A. Runco (Ed.), Critical creative processes. Norwood, NJ: Ablex.

Gorman, M. E., Plucker, J., & Callahan, C. M. (1998). Turning students into inventors: Active learning modules for secondary students. Phi Delta Kappan, 79(7), 530-535.

Kopelman, M., Galasso, V. G., & Schmuckler, M. (1988). A program for stimulating creativity in a citywide high school: The Bronx High School of Science. In P. F. Brandwein & A. H. Passow (Eds.), Gifted young in science: Potential through performance (pp. 209-216). Washington, DC: National Science Teachers Association.

Krajcik, J., Blumenfeld, P. C., Marx, R. W., Bass, K. M., Fredricks, J., & Soloway, E. (1998). Inquiry in project-based science classrooms: Initial attempts by middle school students. The Journal of the Learning Sciences, 7, 313-350.

Maker, C. J. (Ed.). (1986). Critical issues in gifted education: Defensible programs for the gifted. Austin, TX: Pro-Ed.

Passow, A. H. (1988). School, university, laboratory, and museum cooperation in identifying and nurturing potential scientists. In P. F. Brandwein & A. H. Passow (Eds.), Gifted young in science: Potential through performance (pp. 245-253). Washington, DC: National Science Teachers Association.

Plucker, J. A., & Gorman, M. E. (1995). Group interaction during a summer course on invention and design for high ability secondary students. The Journal of Secondary Gifted Education, 6, 258-272.

Plucker, J. A., & Gorman, M. E. (in press). Invention is in the mind of the adolescent: Evaluation of a summer course one year later. Creativity Research Journal.

Plucker, J. A., & Nowak, J. A. (in press). Creativity in science for K-8 practitioners: problem-based approaches to discovery and invention. In M. Lynch & C. R. Harris (Eds.), Teaching the creative child, K-8. Boston, MA: Allyn & Bacon.

Reis, S. M., & Renzulli, J. S. (1991). The assessment of creative products in programs for gifted and talented students. Gifted Child Quarterly, 35, 128-134.

Renzulli, J. S. (Ed.). (1986). Systems and models for developing programs for the gifted and talented. Mansfield, Center, CT: Creative Learning Press.

Renzulli, J. S. (1994). Schools for talent development: A practical plan for total school improvement. Mansfield Center, CT: Creative Learning Press.

Renzulli, J. S., & Reis, S. M. (1985). The schoolwide enrichment model: A comprehensive plan for educational excellence. Mansfield Center, CT: Creative Learning Press.

Renzulli, J. S., & Reis, S. M. (1986). The enrichment triad/revolving door model: A schoolwide plan for the development of creative productivity. In J. S. Renzulli (Ed.), Systems and models for developing programs for the gifted and talented (pp. 216-266). Mansfield Center, CT: Creative Learning Press.

Renzulli, J. S., & Reis, S. M. (1994). Research related to the schoolwide enrichment triad model. Gifted Child Quarterly, 38, 7-20.

Savery, J. R., & Duffy, T. M. (1995). Problem based learning: An instructional model and its constructivist framework. Educational Technology, 35(5), 31-38.

Savoie, J. M., & Hughes, A. S. (1994). Problem-based learning as classroom solutions. Educational Leadership, 52(3), 54-57.

Sitkoff, S. (1988). Science centers - An essential support system for teaching science. In P. F. Brandwein & A. H. Passow (Eds.), Gifted young in science: Potential through performance (pp. 179-187). Washington, DC: National Science Teachers Association.

Stepien, W. J., & Pyke, S. L. (1997). Designing problem-based learning units. Journal for the Education of the Gifted, 20, 380-400.

Treffinger, D. J. (1989). Student invention evaluation kit: Field test edition. Sarasota, FL: Center for Creative Learning.

Westberg, K. L. (1991). The effects of instruction in the inventing process on students' development of inventions. Dissertation Abstracts International, 51. (University Microfilms No. 9107625)

Authors' Note

  The authors appreciate the constructive feedback of Tom Keating and several anonymous reviewers. The scenarios described early in this paper are based on a problem-based invention module developed by Michael Gorman and his colleagues at the University of Virginia (see Gorman, Plucker, & Callahan, 1998). Interested educators are encouraged to visit the related Web site http://jefferson.village.virginia.edu/~meg3c/id/id_sep/id_sep.html. However, all but Scenario 3 are fictitious but based on similar real occurrences (all identifying information has been deleted or altered).

PBL Assessment

jplucker@indiana.edu


This page last modified on Friday, September 10, 1999 at 1:14:14 PM
Copyright 1999,2000 where applicable Matt Jadud
This page is neither sponsored nor endorsed in any way by the LEGO(R) Group. LEGO(R), LEGO(R) Mindstorms(TM) Robotics Invention System, and the LEGO(R) logo are registered trademarks of the LEGO(R) Group.