Sampling uncertainty is only one part of total survey error
In a probability sample, a conventional margin of sampling error describes how estimates would vary across repeated samples under the design. Weighting, clustering, stratification, and subgroup analysis can change that uncertainty.
Coverage error, nonresponse, question wording, interviewer or mode effects, processing mistakes, and inaccurate answers are not automatically included in the conventional margin of error.
Nonprobability studies need different language
Opt-in panels and open-link studies do not give every member of the target population a known chance of selection. Their intervals depend on adjustment and modeling assumptions, so reporting should identify them as model-based rather than borrowing the language of a simple random sample.
Demosight distinguishes sampling intervals from model-based intervals and pairs either with fieldwork, weighting, effective-sample-size, and limitation disclosures.
How to read a close result
Two point estimates that differ slightly are not automatically meaningfully different. Review the uncertainty around the difference, the subgroup sample sizes, the study design, and whether the comparison was planned before the data were examined.
- Look beyond the topline percentage.
- Check whether the interval is sampling-based or model-based.
- Inspect effective sample sizes for weighted subgroups.
- Read the questionnaire and limitations before drawing conclusions.
Sources and further reading
Demosight publishes these guides for research literacy. They do not turn a founding-pilot response into a population estimate.