Joy Chiu

0 followers · 283 views

on the atlas · 19

highlights · 16

For closed-ended opinion questions, there are two main types of order effects: contrast effects ( where the order results in greater differences in responses), and assimilation effects (where responses are more similar as a result of their order).
Writing Survey Questions | Pew Research Center
Research has also shown that social desirability bias can be greater when an interviewer is present (e.g., telephone and face-to-face surveys) than when respondents complete the survey themselves (e.g., paper and web surveys).
Writing Survey Questions | Pew Research Center
One other challenge in developing questionnaires is what is called “social desirability bias.” People have a natural tendency to want to be accepted and liked
Writing Survey Questions | Pew Research Center
Research has shown that, compared with the better educated and better informed, less educated and less informed respondents have a greater tendency to agree with such statements. This is sometimes called an “acquiescence bias” (since some kinds of respondents are more likely to acquiesce to the assertion than are others). This behavior is even more pronounced when there’s an interviewer present, rather than when the survey is self-administered.
Writing Survey Questions | Pew Research Center
We often write two versions of a question and ask half of the survey sample one version of the question and the other half the second version. Thus, we say we have two forms of the questionnaire. Respondents are assigned randomly to receive either form, so we can assume that the two groups of respondents are essentially identical.
Writing Survey Questions | Pew Research Center
A 2019 Center study found that forced-choice questions tend to yield more accurate responses, especially for sensitive questions. Based on that research, the Center generally avoids using select-all-that-apply questions.
Writing Survey Questions | Pew Research Center
If any PII has been featured in the findings (e.g. participant names, faces, videos, etc.), is there a clear rationale and consent for doing so?
Demystifying the art of giving user research feedback | by Dalia El-Shimy | Shopify UX
Has personally identifiable information (PII) for participants been removed from the findings?
Demystifying the art of giving user research feedback | by Dalia El-Shimy | Shopify UX
If so, have those conflicts been sufficiently addressed or explored?
Demystifying the art of giving user research feedback | by Dalia El-Shimy | Shopify UX
Are there any findings that are in conflict with findings from other research/data/support explorations?
Demystifying the art of giving user research feedback | by Dalia El-Shimy | Shopify UX
Are there any findings in the document that are in conflict with one another?
Demystifying the art of giving user research feedback | by Dalia El-Shimy | Shopify UX
Is it an anecdote, or a fully synthesized insight?
Demystifying the art of giving user research feedback | by Dalia El-Shimy | Shopify UX
On selecting participants: How were the participants chosen? Have all types of users who will be impacted been considered? What participant characteristics did the researcher specify, and why? To what extent have baseline parameters that define participants been considered? How relevant are the research questions to the group of chosen participants? How diverse should the participants be? Should a range of expertise be considered (e.g. novice vs. advanced users), and why? Is a screener question needed to further qualify potential participants? Does the screener complement what the researcher m…
Demystifying the art of giving user research feedback | by Dalia El-Shimy | Shopify UX
Questions to consider when giving feedback on research plans On the rationale behind the work: Why is this research being done? Did the researcher offer a strong rationale? Have research or data explorations been done on this topic within your organization? What about outside your organization? If there is already existing research, why is this study necessary? What will we learn that hasn’t already been uncovered with past research? Which project(s)/initiatives benefit from this research being done? Is this research actionable? What decisions are the team trying to make based on the research?…
Demystifying the art of giving user research feedback | by Dalia El-Shimy | Shopify UX
On the research questions: Are the research questions well-formulated? Do they attempt to understand or uncover a process/need/challenge (e.g. “What are the biggest challenges people experience when it comes to doing X?”)? Or are they questions the team should be asking itself after the research has been completed (e.g. “What can we do to improve X?”)? Do the questions include leading/non-neutral language that might bias the research? Are the research questions well-scoped? As they’re written, is it possible to truly answer the question through research with the timeframe in mind? When are the…
Demystifying the art of giving user research feedback | by Dalia El-Shimy | Shopify UX
On the research method: Why and how was the research method chosen? To what extent is the method suitable to the research question at hand? What other methods could have been considered, and how well would they have answered the research questions? What advantages does the chosen method offer? What disadvantages or biases might it present?
Demystifying the art of giving user research feedback | by Dalia El-Shimy | Shopify UX