Despite years of teaching one of Duke's largest undergraduate sections in chemistry, Richard MacPhail had never been especially aware of educational assessment until he attended a 2005 National Science Foundation conference on teaching chemistry.
The conference wasn't supposed to be about assessment, but the topic permeated the discussion. MacPhail decided then to make changes when he returned to Duke.
"I had been dissatisfied with the lab recitations in general chemistry classes," he said. "Direction from the teaching assistants and student involvement in the recitations were very uneven. Our solution was to design the recitations to present students with problems and then have them work together on slight twists and manipulations of the core problem to get them to see things more clearly and to focus on the underlying concept rather than just learn one way to solve it."
MacPhail suspected this new method would help students, but he wanted documentation. He turned to a chemistry teaching assistant for help. With assistance from the Arts & Sciences Office of Assessment, the graduate student analyzed various data. She found test scores unchanged but a dramatic improvement in student reviews of the recitations.
And something else: "Students were coming to the university's academic resource center and asking far more sophisticated questions about where they were having problems," MacPhail said. "They had developed a language of learning that helped them better identify and articulate their difficulties."
"That's one of the interesting aspects of assessment: Sometimes you can get at aspects of learning that are not measured on final exams. That got me excited."
MacPhail's experience illustrates what A&S seeks from assessment more broadly: that it be driven by faculty needs, not require extensive time or resources and produce information that is genuinely valuable.
Other examples show how this is happening across the campus:
- Julie Reynolds, associate director of undergraduate studies in biology, worked with Serra while developing a system to test the quality of writing in honors papers. The tool showed that increasing the number of honors papers did not lead to a decline in quality. Other departments have since used the tool for their own projects.
- Many departments now require students to complete an e-portfolio, which provides an electric archive of their work that can be used to measure student development within fields.
- The Sanford School has developed assessment measures to improve the internships that are a graduation requirement for its majors.
- Duke's service-learning program, which includes more than a dozen A&S departments, used surveys and an analysis of student writing to track the impact of service-learning courses.
- The chemistry department used different measures to place incoming students more accurately into gateway courses.
- The Thompson Writing Program developed measurements to assess how well students engaged with scholarly arguments in their writing.
"This only works if there is faculty buy-in," said David Malone, chair of the A&S Faculty Assessment Committee and professor of education. "One thing we're working on is getting junior faculty and faculty other than the department leadership involved in the department assessment committee. The chair is ultimately responsible for the department's efforts, but we want as many faculty involved as possible."
Malone sees assessment reversing the way departments traditionally build their curriculums. His and several other departments now review course offerings to ensure they're guiding students toward desired learning outcomes. "We like to use the term 'backwards design,'" Malone said. "You start at the end point of what we want our students to know when they graduate and work backwards to ensure they have a path to get there."
Kristen Neuschel, who directs the Thompson Writing Program, said assessment has helped her review student work more effectively than she could through the normal grading process. Articulating specific goals and skills has made her "become a better reader of student work," she said.
"When I am dealing with a paper that is a mess, having those articulated goals in my mind made me more appreciative of where they were showing incipient thinking, and then I could reward the student for it. I could tell them there is serious thinking going on right here, and encourage them to develop that.
"In Thompson Writing, we always try to appraise writing rather than grade it. Grading comes at the end, but it's very easy to slip into the default position of punitive reading. Assessment changed the way I read."