Monday, September 10, 2012
One of the biggest challenges when doing surveys is to establish confidence that your results are representative of the population as a whole.
Often the assumption is that the results will only be representative of the population if the whole population is surveyed and enough people respond! But that's not actually the case. In fact, a sample is just as valid. For example, a presidential poll that surveys 500 people can accurately represent the views of millions of people within a confidence interval (sometimes referred to as the margin of error), assuming that the group surveyed is representative of the population as a whole. By using a sample instead of the whole population, you will help the institution ease the problem with survey fatigue that researchers face more and more.
There's a cool calculator online that can help you with all of this-- http://www.surveysystem.com/sscalc.htm. As stated on the website, the calculator can tell you how many responses you need in order to get a statistically valid result. Immediately below the calculator is an explanation of the numbers you need to calculate either a confidence interval, a confidence level or both.
Of course, we in Institutional Research are here to help out if you need assistance in constructing a reliable sample for your institutional survey research work. You can contact us at 4-3164 or IRoffice@ithaca.edu.
Friday, February 10, 2012
It's often difficult to find trustworthy sources of information on how to do good surveys. That's why we're here of course. You can always contact us at firstname.lastname@example.org for help with your survey.
Friday, December 2, 2011
On a scale from 1 to 5, with 1 being 'leave me alone' to 5 being 'make this stop', rate how tired are students with being asked to surveys constantly?
A recent article in the Chronicle of Higher Education (subscription required) and various academic studies have shown that decreased response rates of students is highly correlated with the number of surveys students are being asked to do. Ithaca College is no exception, with students regularly bombarded with requests to do surveys through links on Intercom, by email and even in person. The result is often poor data for the researcher and survey fatigue on the part of students.
Researchers need to assess and may feel that surveys are a way to provide one piece of their assessment puzzle. But surveys aren't always needed and may in fact not be of benefit. There are alternatives that would provide anecdotal data that most surveys are used for and actually add much more value to the evaluators.
Here are a few ways researchers at IC could help to combat survey fatigue and enrich their own assessments:
- Loosely structured interviews: These can be in-person or on the phone through a focus group or one-on-one interaction. This is not as difficult as it seems because a smaller pool of respondents would be required. The richness of the qualitative data cannot be underestimated.
- Don't ask questions that aren't going to lead to action: Evaluations should not include questions that won't be used for action or follow through. In other words, don’t ask questions that aren’t going to lead to some result.
- Make it relevant: Students should have a stake in the results by providing a summary of results or having a small group of respondents lead discussions to disseminate the results. There are a variety of ways to do this, but studies have shown that students will provide better responses if they are engaged in the evaluation process.
- Pilot test the survey: Make sure all of the wrinkles are ironed out before launching the survey. Pilot testers will catch things that even the most expert survey researchers miss because they are coming to the survey fresh.
- Add questions to another survey: By fostering communication between units on campus, we can limit the number of surveys sent to students. One longer survey is much better than 5 short surveys.
The Office of Institutional Research is here to help for all your assessment needs, including providing suggestions on the best approach for your unit. We also have a survey clearinghouse that will provide a centralized place for your survey to foster sharing across campus. Contact us at email@example.com or 4-3164.
Tuesday, October 25, 2011
We often throw around two important terms: anonymous or confidential to indicate the way in which a survey is to be conducted. It's a pretty big difference however and what you select at the outset of the survey design will have substantial implications for your results. Generally, you will want to guarantee confidentiality and not anonymity when doing surveys, but here is some information about each:
- May solicit more candid responses
- May result in the submission of multiple responses by individuals who feel strongly about the topic
- Do not allow basic demographic information to be matched in from other sources, so it will have to be requested on the survey
- Statistically speaking you cannot draw inferences about a population when you do not know exactly who took the survey.
- May cause respondents to be less candid
- Can be set up so that multiple responses by individuals are not possible
- Allow basic demographic information to be matched in from other sources, so it does not have to be requested on the survey
- Make it possible to send survey reminders only to non-respondents
- Since you know the demographics, you can make statistical inferences of the survey results.
This is a pretty simplified version of an oft-studied realm of survey research. We in the Office of Institutional Research are here to help you. Just write to firstname.lastname@example.org or 274-3164.