Effect Size - Overview - Population and Sample Effect Sizes

Population and Sample Effect Sizes

The term effect size can refer to a statistic calculated from a sample of data, or to a parameter of a hypothetical statistical population. Conventions for distinguishing sample from population effect sizes follow standard statistical practices — one common approach is to use Greek letters like ρ to denote population parameters and Latin letters like r to denote the corresponding statistic; alternatively, a "hat" can be placed over the population parameter to denote the statistic, e.g. with being the estimate of the parameter .

As in any statistical setting, effect sizes are estimated with error, and may be biased unless the effect size estimator that is used is appropriate for the manner in which the data were sampled and the manner in which the measurements were made. An example of this is publication bias, which occurs when scientists only report results when the estimated effect sizes are large or are statistically significant. As a result, if many researchers are carrying out studies under low statistical power, the reported results are biased to be stronger than true effects, if any. Another example where effect sizes may be distorted is in a multiple trial experiment, where the effect size calculation is based on the averaged or aggregated response across the trials.