|07-19-2002, 02:26 PM||#1|
Join Date: Apr 2002
As polls are a new feature here at topmudsites, a lot of us are still feeling out how to write good ones. So far we've been suggesting ways to better polls one poll at a time. Doing it that way, a person will need to have read all the polls to learn how to write one. I figured it might be better to get all the suggestions in one place. If I'm missing something important, or if you have a better suggestion to replace one of mine, please reply to this thread.
1) Above all, avoid ambiguity. Don't ask "How many characters do you play?" when you can ask "How many different characters have you played this month?" or something like that. Watch out for questions that could be interpreted in different ways.
2) Make your question as obvious as possible. Don't assume the respondant will know what you are asking from the answers provided. If possible, put your question in the subject line of the post, or at least in the description of your post. But beware, if you repeat the question somewhere, such as in the body of your post, repeat it word for word. Elaborate if that's what is necessary to get the point of the question across, but don't use another question to do so. A respondant may not know which question to answer, and his or her answer may differ somewhat for each one.
3) Don't hint at your opinions, or what might be your opinion, or even anything that might be incorrectly inferred as such. If people know what you want to hear, then many people will tell you just that, despite not believing it themselves; some will tell you the opposite, just to be contrary; and disproportionately few will take a moderate stance.
4) Don't ask a question that could require more than one choice to answer. For example, don't ask, "Which of these do you like?" It is highly likely that they like more than one of whatever they are. Instead, ask, "Which of these do you like best?" or "Which of these do you choose most often?" or "Which of these are attracts you the most at this moment?"
5) Don't use humor. At all. In regular writing, good wit can complement the real content. In polls, humor will only make your poll less accurate. People will often choose the most humorous answer instead of the answer that is true for them. Even if reader of the poll will know the answer is not meant to be serious - such as in the case of "My leg fell off!" - in order to choose that silly answer, people have to ignore the answer that is true. Even if the people who choose to do pick the silly answer instead of the true answer so are random, there are so few voters in the polls that this will make a huge difference for the worse in the poll's accuracy.
6) Don't let two answers cover the same possibilities. For example, say the question is "How many muds have you played at for more than a year?" In that case, don't write the answers as "0-1," "1-2," "2-3," and "3 or more." If they've played exactly 1 mud for more than a year, they won't know whether to pick the first answer or the second, and so on. Rather, use such answers as "0-1," "2-3," "4-5," etc. If fractions are real possibilities, tell them to round to the nearest integer. (And if fractions are very important, then maybe you should consider changing to a smaller unit of measurement.)
Here's another example: never put "all of the above" as one of the answers. If that is logically a choice at all, then so is "this one and that one, but not this one," and so on. Theoretically, you could list all the combinations, and add the qualification "only" before all the simpler answers. But practically speaking, this would get so bulky that even if you didn't make a mistake in listing all the combinations, you'd end up with an inaccurate poll simply because people wouldn't take the time to follow the complex instructions.
7) Use a consistent scale. If you use "1-10" on the first answer, then use "11-20" on the second, not "11-35." ?
The only exception I can think of are infinitely large extremes or large extremes of unknown size. You see this exception most often on the very last answer (e.g., "more than 60" or "100 or greater"). This is because, though a very large number may be improbable, it is nonetheless possible, therefore we shouldn't exclude it. The people who analyze the survey just have to take into account that particular answer is not using the same scale as the other answers.
8) If feasible, include all possibilities in your answers. Just as you wouldn't skip from "1-20" to "500-600," leaving out everything between, don't skip non-numerical possibilities in such a way. If you can't feasibly provide every existing character class for them to pick from, then don't ask "What class is your favorite?" Instead, ask "Which of these classes is your favorite?" On the other hand, if think you can feasibly provide, say, every existing WoT mud when asking for the respondants' favorite WoT mud, then do so... But still frame your question carefully, because maybe you are wrong, and there is a WoT mud out there that you are unaware of. Or if you want, using "Other" as the last non-numerical answer is almost always a good idea.
9) Try to use the least number of answer choices necessary to learn what it is you really want to learn. If you have followed all the above guidelines, then polls with few answer choices will be more statistically accurate than polls with many answer choices. On the other hand, they may be less useful as well, depending on the purpose of your poll. If the purpose of your poll is to find out how old mud players are, you will get the most statistically accurate measurement if your only two answer choices are "younger than the universe as a whole" and "older than the universe as a whole," because obviously everyone would fall into the first category; but the poll wouldn't be very useful to you.