How do small sample sizes help us understand a broader population?
“In addition to executives who prefer the authoritative appearance of controlled experimentation, you may run into sample-size queens who dispute the validity or utility of qualitative research. These people are often well-intentioned quants who are just applying the wrong standard of confidence because it’s what they know. Avoid arguments about statistical significance; you will not win. Instead, focus on gathering useful insights in order to meet real-world goals.”
So, as one delegate from last week’s Australian International Education Conference asked me, how can a small (qualitative) sample size help us understand a broader population? As Erika points out above, qual research is not about statistical significance - but it is a great way to observe and better understand what people think, do, and say, and to use that authentic detail in useful ways.
This might mean:
Sitting down with an individual and asking them to show you how they navigate a website, use their device or understand certain information;
Observing people as they interact in certain spaces and places over a period of hours of days, and asking them about what they're doing;
Bringing a small group of people together to talk about a topic, and listening carefully to the language they use, the behaviours they describe, and how they respond to each other.
But just because we're not aiming for statistical significance doesn't mean we can invite just anyone. Designing a qualitative sample asks difficult but important questions, such as:
Who do we really need to hear from to understand this issue better?
Who do we NOT usually hear from via other means ('quieter' voices)?
What demographics are important to understand for this project?
What attitudinal or behavioural traits do we want to explore?
It's amazing what you can learn from some thoughtful, long-form conversations with as few as 6 individuals. Drop me a line if you'd like to learn more!
Up next in the post-conference FAQs: "How much do you pay students to participate in your research?"