Deutsch
English
Français
Are you planning a study?
Our team is happy to assist you with planning and implementation—to ensure robust results based on reliable data.
Why Sample Quality Is the Foundation of Reliable Insights
Interview with Dr. Marcus Roller
Companies make important decisions based on data and insights. This makes it all the more crucial that these insights are grounded in a methodologically sound data foundation—and this is precisely where sample quality plays a central role. In this interview, Dr. Marcus Roller, Head of Social Research at intervista, explains why sample quality is far more than just a methodological detail.
Marcus, why is the quality of the sample so important in the first place?
Because the sample is the foundation of every study. Everything we later present as an insight is based on the people who actually participate in a survey. If this group does not adequately represent the target population, distortions are highly likely to occur—thereby increasing the risk of drawing incorrect conclusions from data that appears plausible.
What characterizes a good sample?
A good sample represents the target population as accurately as possible in terms of the characteristics relevant to the research question. The methodological ideal is random selection: every person in the target population has an equal chance of being included in the sample. In practice, this ideal cannot always be fully achieved, but it remains the central quality benchmark.
Why is a sample almost never perfect in practice?
There are several hurdles between methodological standards and field reality. Often, there is no complete access to the entire target population, and participation in surveys is voluntary. As a result, some groups of people are more likely to participate than others. This leads to over- and under-representation. Quota sampling can partially mitigate such effects, but it does not completely solve the underlying problem—especially when relevant segments of the target population are entirely missing from the sample.
Changing Methods
In the past, telephone surveys were considered particularly reliable. Why?
Because for a long time, landlines provided good access to large segments of the population. Combined with proper sampling and professional fieldwork, this approach allowed for the creation of highly reliable samples. That is why the telephone was a central method of empirical research for many years.
Why has this become more difficult with telephone surveys?
As landline coverage has declined, this method has lost its reach. At the same time, willingness to participate in telephone surveys has also dropped, meaning that methods relying on randomly generated cell phone numbers cannot compensate for this reduced coverage. Furthermore, fieldwork has become more complex and expensive. The risk of systematic bias has therefore increased—and in many cases, so has the economic barrier to using this method.
What challenge has long existed with online surveys?
For many years, online surveys suffered from under-coverage. Certain demographic groups—particularly older adults—were less represented online and therefore appeared less frequently in samples. This gap has narrowed significantly today. As a result, online interviews are now methodologically well-suited for many research questions—provided the sample is properly recruited and controlled.
Does that mean online surveys are automatically representative?
No. The survey method alone does not guarantee a good sample. What matters most is who is eligible to be included in the sample, how participants are recruited, and how well the final sample actually reflects the target population.
What role do recruitment sources play in sample quality?
A central one. The recruitment source determines who is even reachable and can therefore be included in the sample. If certain groups are systematically missing or others are overrepresented, this directly affects the quality of the results. Recruitment approaches that rely heavily on self-selection—such as river sampling methods or recruitment through specific channels like social media—are particularly problematic.
Why isn’t a large sample size enough on its own?
Because size alone does not correct for systematic bias. A large but biased sample remains biased. Thousands of interviews are of little use if the wrong people were interviewed.
Isn’t it possible to simply solve such problems through weighting?
Weighting is an important tool, but it is not a panacea. It can correct for known variations, such as those based on age, gender, or region. What it cannot address are differences that have not been measured or for which we do not know the actual distribution in the population. Likewise, weighting fails when certain groups—which we must assume will respond differently than the rest of the sample—are entirely missing.
What does this mean for interpreting research findings?
Companies should never view findings in isolation from their methodological foundation. Good research isn’t just about compelling charts; those come only at the very end of the research process. Just as important are a well-designed and properly recruited sample, transparent quality controls, and an open acknowledgment of limitations. After all, sound decisions require sound data.
Marcus Roller
More information about data and panel quality
To learn how we ensure sample quality in practice, visit our page on the intervista online panel.