Thomas Nechyba has no problem with assessment. As the former head of economics, one of the largest undergraduate departments on campus, he's done it for years. But he does think Duke's recent efforts are aimed more at satisfying external demands than at producing results of value.
Nechyba and some other faculty members have spoken up against Arts & Sciences' widening assessment initiative. He says Duke faculty have done effective assessment in various ways for many years and should resist pressure from SACS and other external forces to pursue practices he doesn't consider legitimate.
"My standard is if the assessment protocol could withstand serious peer review then I'm willing to think about it," said Nechyba, a professor of economics and director of the department's EcoTeach academic center. "The problem is that many of the suggested assessment protocols don't come close to meeting that standard, and those that do are sometimes extremely costly. And we're not getting the kind of resources that could fund new assessments that are truly informative. We're being asked to do assessment on the cheap and then raise the flag and declare victory. A lot of it looks like it is strictly for show."
As department chair for seven years, Nechyba was involved in several initiatives reviewing undergraduate education.
"What we did wasn't a matter of checking boxes, the way SACS wants us to check boxes. As a social scientist, genuine assessment should appeal to me, but instead I think we're doing too much checking of these top-down boxes. If it was for real, we'd put more protocols involving things like serious double-blind studies in place. Or we would be allowed to work with data the university already has but refuses to allow us to analyze. We'd do the kinds of things that we do when we do serious social science research.
"Instead, what we're doing is putting stuff in place that looks like assessment but can easily be manipulated to get the 'right' answer. And I am afraid that there will be a lot of gaming the system to keep SACS satisfied rather than serious assessment to improve teaching and learning."
Nechyba said he is not alone in his criticism, but others involved in assessment say SACS is not dictating any method but does want evidence Duke is following through on its own stated goals.
"If a faculty member has made a change to a course or to a program, I'm sure that there was some source of information that led them to make that change," said Jessica Thornton, manager of institutional assessment and accreditation in the provost's office. "That is assessment. We now are asking faculty to take the next step and to document what happened, and that can feel burdensome. However, that documentation can be beneficial, too. It creates a space where faculty can share processes, or best practices, for teaching and learning."
"Are you doing what you say you're going to do? That seems to be [SACS'] bottom line," said Dean of Arts & Sciences Laurie Patton.
Patton said she shares some of Nechyba's concerns and wants to "find a way to include both the practice of assessment and the ongoing critique of assessment in our everyday academic lives."
"I don't think assessment practices are going to change for the better, and move beyond the 'checking the boxes' approach, unless there is sustained critique of what is currently in place," Patton said. "I would like to create an environment where what we do at Duke is a model for others who want to take the results of their pedagogy seriously. In my view, the only way to do that is to move beyond the 'accreditation' conversation and find better ways to read the results of our pedagogical work."
The economics department continues to ask questions about undergraduate education, such as different styles of teaching large lecture classes or tracking prerequisites. Both are ripe for discussions about effective learning, Nechyba said.
"A university is always facing questions that require one to ask whether something is working," he said. "How do you come to a conclusion that a faculty member is good enough to earn tenure at Duke? We don't count words, we don't count papers, but a group of faculty gets together, does an assessment and comes to a decision. We need to ask these questions, but I'm just skeptical that SACS will be ever be satisfied with the way we do it -- or that serious resource commitments will fund innovative new assessments that stand up to scrutiny."