4 Tips for Making Your Online Surveys Shine

Since she arrived at Duke in January, Shanon Jacobs has leaned on the online survey program, Qualtrics, to help organize virtual events.

From soliciting information from potential participants to gathering RSVPs and post-event feedback, the tool helps in many ways.

“I just created a really simple one the other day for our math awards ceremony,” said Jacobs, program coordinator for Duke’s Math Department and the Department of Statistical Science’s TRIPODS Grant. “I asked what dates everyone could meet. I gave them four options, had them rank them and then we took it from there. It was super simple.”

Jacobs knows that Qualtrics, which Duke students, staff and faculty can use at no charge, has a deep well of features and capabilities that make it valuable for researchers who build elaborate surveys. But it works beautifully for relatively straightforward needs, too. 

“It has a lot of horsepower, but it’s great for both simple polls and heavy-duty data collection,” she said.

Jacobs among few dozen Duke staff and faculty who attended the recent webinar, “Make Informed Decisions with Qualtrics” offered by Duke’s Office of Information Technology. 

“Like so many software systems and tools today, it’s got a hugely rich set of functions and capabilities,” said Alexandra Cooper, associate director for education and training at the Social Science Research Institute and associate director of the Duke Initiative on Survey Methodology housed within SSRI.  “So it can be hard to know which of these are most useful and most important.”

Whether you’re a seasoned user of Qualtrics, or a beginner, here are a few useful tips for making surveys shine.

Save Your Participants Time

If your survey is going to the Duke community, you can automatically collect information such as people’s names and where they work or study by having them authenticate with their NetID.

In addition to no longer needing to ask respondents for identifying data, enabling the NetID login feature – by following these online instructions – also adds a layer of security to your results by ensuring the survey is only filled out by members of your intended audience.

Erin Gauldin, an academic advisor at the Duke Global Health Institute, said that after learning about the time-saving feature, she’ll suggest using it for the many surveys she and her colleagues send to students.

“I do think it would be a nice thing to take off of people’s plates,” Gauldin said

Avoid Agreement Bias

A common way to structure a survey question is to put forth a statement and ask respondents if they agree or disagree. 

Survey experts caution against the question structure due to a phenomenon called “acquiescence bias” or “agreement bias.” That’s when survey takers, due to various societal factors, are more likely to choose the agree side of the question even if they don’t actually agree.

“Instead we ask questions like ‘How well does this describe you?’” Cooper said.

The alternative structure of asking respondents how well a statement describes them, and giving them a range of choices such as “extremely well” or “not well at all,” should yield more accurate data. 

And Qualtrics recognizes the effectiveness of this approach by auto-filling answer choices when you create a question using the “how well does this describe you?” format.

Keep Mobile Devices in Mind

When building a survey, to keep your questions and answers options as short and concise as possible. That results in questions that are easier for users to understand and makes for an uncluttered look on mobile devices.

You can get a feel for how your survey will look on a mobile device by clicking the “Preview” option before publishing the survey.

“Something you often want to check when you’re setting up a survey is what it’s going to look like on different devices,” Cooper said. “Particularly, if you’re creating it on a computer, see what it’s going to look like for somebody who’s taking this survey on a phone. And that’s a reason why keeping questions short, and not too wide, can be really useful.”

Make Your Survey Pop

When you create a survey, the default logo at the top may not be that of your part of Duke. If you want a different logo at the top, click on “Look and Feel” and go to the “Theme” option to pick from ready-to-use logos from a wide range of Duke schools, departments and units, including a basic Duke University logo.

Of you can select a blank survey option and build one with no logo at all, which, as Cooper can attest, can be useful in certain situations.

“One time I was working with a group that was trying to survey nursing students at both Duke and UNC in March, which of course can be a controversial month between the two schools as the basketball season unfolds,” she said. “They didn’t want the Duke label on the survey because they didn’t want the UNC students to refuse to answer, so in that case, you may want to go with something that doesn’t have a label at all and that’s another option.”

Send story ideas, shout-outs and photographs to Working@Duke through our story idea form or write working@duke.edu.