Why sample size changes everything in the results

Nowadays, companies and researchers increasingly rely on data to make decisions, and rightly so. However, amidst this fever for numbers, there is a recurring error: collecting without planning.

In this context, underestimating the sample size is like building a house on sand. The foundation becomes unstable, and everything can collapse later. Therefore, before going into the field, whether for market research, a satisfaction analysis, or an academic study, the first step should be to clearly answer: how many people do we need to hear from for the data to truly make sense?

The truth is that the right answer depends on the context. Still, there is a basic rule: the greater the desired precision, the larger (and better) the sample should be.

Shall we illustrate?

Imagine you manage the marketing for a cosmetics company. The team decides to test a new product and wants to understand if it is appealing. You interview 20 people, all from your team. Most approve.

Would you launch the product based on this data?

Probably not.

This group is small, homogeneous, and biased. The data is not representative and cannot be used as a basis for strategic decisions. Now, imagine you listen to 400 people from different regions, profiles, and age groups, with a margin of error of 3%. The scenario changes completely.

Calculate here: Sample Size Calculator

What does good sampling prevent?

• Noise in decisions: when data does not reflect reality, decisions go off track.

• Unnecessary cost: poorly planned research leads to rework.

• Loss of credibility: stakeholders and clients notice when data does not match the facts.

• Wasted opportunities: poorly collected data does not generate actionable insights.

Understanding the impact of the margin of error

You may have seen graphs or reports saying: “Candidate X has 35% of voting intentions, with a margin of error of 2 percentage points.”

This confidence interval — from 33% to 37% — shows that reality can vary, but it is within a reliable limit. The smaller the sample, the larger this margin will be. And the larger the margin, the more imprecise the data.

Therefore, in more sensitive research, such as public opinion polls or acceptance tests, it is common to work with large samples and small margins (2% to 3%). This provides security for predicting behaviors and making assertive decisions.

Read more: What is data collection for? Understand its importance

And what changes according to the research objective?

The truth is that there is no single formula for defining the sample size. Everything depends on the type of study, the objectives, and how the data will be used. One of the biggest errors in data collection is applying the same reasoning to completely different situations.

See below how the purpose of the research changes the sampling logic:

Exploratory research (initial)

When the objective is to investigate a new topic, identify patterns, or raise hypotheses, smaller samples are acceptable, especially if they are qualitative. In these cases, the focus is not to statistically represent an entire population, but rather to observe behaviors, emotions, and motivations in depth.

For example, before launching a new app, you can talk to 12 potential users to understand their pain points. This type of deep listening can reveal valuable insights, which will later be tested in quantitative research.

Still, even with few respondents, diversity within the sample is essential. In other words, a small but varied group can be much more productive than a large and homogeneous group.

Product tests or satisfaction surveys

On the other hand, when the objective is to evaluate the acceptance of something already created, such as a new product, service, or campaign, the ideal is to work with broader and more heterogeneous samples. This is because this data will support important decisions, such as launches, adjustments, or marketing investments.

Therefore, the greater the impact of the decision, the more robust the sample needs to be. And, even more importantly: it must reflect the plurality of the public that will be impacted by the action.

See this example: testing packaging with 100 consumers from the same neighborhood may seem efficient, but it will not be enough to reflect the reaction of people from other regions, socioeconomic profiles, or age groups.

Segmented research

In this case, extra care is needed. If you intend to cross-reference data by profile, age, gender, region, purchasing behavior, each segment needs to have sufficient representativeness within the total sample.

As a result, the number of respondents grows proportionally to the number of cuts you want to make.

Let's take a practical example: if your research needs to represent men and women from three different age groups, it will be necessary to guarantee a minimum number of responses for each of these six subgroups. Otherwise, comparisons lose statistical power.

Big Data: yes, it also requires criteria

It is common to think that when working with large volumes of data, the sampling problem disappears. However, even in the Big Data universe, selection bias and lack of representativeness remain real risks.

A database can contain millions of records, but if they come from a single source or are concentrated in a very specific profile, the insights can end up being severely distorted.

In summary, more data does not automatically mean better data. What really matters is the quality of the sampling structure, even in massive environments.

Checklist for successful sampling

Before launching your data collection, clearly answer:

✅ Who do I want to hear from?
Precisely define your target population. Your data will only be valuable if it is relevant to those who matter.

✅ What decisions will be made based on this research?
The more strategic the decision, the greater the rigor of the sampling should be.

✅ What level of precision do I need?
Define the acceptable margin of error and the necessary confidence level (usually 95%).

✅ Do I want to segment the data later?
If so, each subgroup needs to be represented with sufficient volume.

✅ Do I have diversity within my sample?
Excessive homogeneity distorts. Make sure to include different profiles.

✅ Are my resources (time, team, and budget) compatible with the scope?
It's not always possible to do everything. Adjust the plan to reality, without compromising quality.

Continue reading: How to Get a Sample of Consumers for Online Research?