Sampling Methods: Selection, Representation, and Bias

Define the population and inference

Sampling methods determine which people, events or records enter a study and which conclusions can reasonably extend beyond them. Begin with the target population and unit. “Students” may mean every enrolled student, first-year students at one campus or those using a particular service. Define the period and eligibility criteria. The sampling frame, such as an enrollment list, may omit people in the intended population; explain that gap before presenting a sample size.

Match selection to the question. A study estimating the proportion of students with a particular experience needs a route toward representation. A qualitative study exploring how a process works may deliberately select participants with varied roles and experiences. Neither design is universally superior. The method is judged by whether its selection logic supports the claim the paper makes.

Identify the practical constraints. Access, privacy, recruitment time and language can shape who participates. A technically random invitation can still yield a biased respondent group if some people have less time or reason to reply. Plan how to record response, compare known characteristics of respondents and nonrespondents where ethical and feasible, and limit the resulting inference.

Understand probability selection

In a simple random sample, each unit in a complete frame has a known selection chance. Describe how the random selection was implemented, not merely that people were “chosen randomly.” Stratified sampling selects within defined groups to ensure their inclusion and can improve precision for important comparisons. Weighting may be needed if selection probabilities differ; the paper should explain the weights and their effect.

Cluster sampling selects groups such as schools or neighborhoods before selecting or observing units within them. It can reduce fieldwork costs but introduces similarity among members of a cluster. An estimate that treats clustered observations as fully independent may understate uncertainty. Multistage designs require clear reporting of each selection step, inclusion probabilities where relevant and the method used to calculate uncertainty.

A large sample does not repair a poor frame. An online poll open only to people who saw a link can collect thousands of answers while excluding others and attracting people with strong opinions. More responses reduce random variation among respondents, but not necessarily selection bias. Explain the population actually reachable through the recruitment method.

Use purposive selection transparently

Qualitative research may select cases because they can illuminate a process. Maximum-variation sampling seeks differences in context or experience; a critical case can test whether a proposed explanation holds under favorable or difficult conditions. Snowball recruitment may reach people outside official lists, though networks can overrepresent connected groups. State the rationale and how it shapes the knowledge gained.

Describe the recruitment path and power relationships. A manager inviting employees to discuss workplace safety may create pressure to participate and discourage candid accounts. Use an independent route where appropriate, explain voluntary participation and avoid exposing who declined. Selection is an ethical as well as a methodological decision.

Do not claim “saturation” solely because a planned interview quota was reached. Explain what was being assessed: whether new interviews introduced relevant concepts, whether variation among groups was examined and what limits time or access imposed. A small, focused project can be valuable without claiming that every possible perspective has been exhausted.

Plan sample size for the actual analysis

For a quantitative estimate, precision depends on variability, design and response, not just a familiar rule of thumb. A study comparing groups needs enough observations in each relevant group, not only in total. State the assumptions behind a sample-size calculation, including the effect size or desired precision, and revise expectations if recruitment differs. Avoid promising a precise subgroup result from a handful of cases.

For qualitative work, justify the amount of material by the scope, complexity, diversity and depth of analysis. Thirty short interviews do not necessarily offer more insight than fewer detailed observations if the question concerns a complex process. Explain what the collected sample permits a reader to understand and which experiences may be absent.

Handle missingness after selection. Some invited people never respond; others skip a sensitive question or leave the study. Report denominators at each stage. Assess whether missingness could depend on the outcome under study. Deleting incomplete cases without examining the pattern can change the analyzed population. Describe imputation or sensitivity analyses if used, including their assumptions.

Report the limits of transfer

When the target population is difficult to enumerate, distinguish coverage from response. A registry may omit people who moved recently; an invitation sent to all names in that registry still misses them. Document any known coverage gap and consider a second recruitment route. If this route changes selection probabilities in an unknown way, explain why a simple weighted population estimate may not be justified. Collecting more observations through the same incomplete frame will not resolve that problem.

Present a clear recruitment flow: eligible units, invitations, responses, exclusions and analyzed cases. Give important characteristics and compare them with the target population when relevant data exist. Do not imply that a confidence interval covers a population to which the design provides no defensible access. Nonprobability samples can support useful descriptions and hypotheses, but their uncertainty cannot be reduced to sampling error alone.

In qualitative reporting, provide contextual detail so a reader can judge whether a process might apply elsewhere. Transfer depends on similarities and differences in setting, not a claim that interviewed participants statistically represent everyone. Sampling quality is the alignment of selection, evidence and inference. State that alignment plainly so a reader knows what the study can and cannot say.

Review the final sample against the plan before drawing conclusions. If a planned subgroup produced very few responses, combine categories only when doing so is substantively defensible and protects confidentiality. Do not erase the subgroup silently. A transparent report can still answer a narrower question than originally intended and identify the evidence required to address the broader one.