|
|
|
Sample Size & Power What is power? Power is the probability of a statistical test to detect a meaningful effect. The more powerful a test, the greater the likelihood that it will produce statistically significant results if there is truly a meaningful effect. There are three components that determine the power of a test:
What is an adequate sample size? Is bigger always better? We sample because we cannot obtain data from every possible data point or participant (the entire population). For example, when your doctor tries to diagnose a problem, aren't you glad he takes a sample of your blood, not all of it? Just as a sample of your blood indicates conditions in your entire bloodstream, survey sampling allows valid inferences about a population. An important mathematical concept, the Central Limit Theorem, states that as sample size increases, the sample distribution approaches a normal distribution. This means that larger samples are better estimators of population effects than smaller samples. And yes, there is a "magic" number at which sample size becomes pretty good for estimating population parameters, especially when determining significant differences between group means. To help you determine sample size, we've prepared an Excel® spreadsheet to perform the calculations. What's sampling error? What's measurement error? What's the difference? Sampling error refers to the understanding that the sample we select is not exactly the same as the population, no matter how large the sample. If we were to take repeated samples from the same population, the sample statistics would vary a little from sample to sample. For example, if the true mean height of everyone in the world is, say, 5'8" and we sample people's heights, one sample might have a mean height of 5'9", whereas another sample might have a mean height of 5'7". This deviation from the true population mean is the sampling error, and the larger the sample, the smaller the sampling error. Measurement error refers to the understanding that measurement is never 100% accurate; there is always some lack of precision even in the most carefully made measurement. Ever wonder why you weigh 151 pounds the first time you step on the scale, 152 pounds the second time, and 150 pounds the third? While it is possible that your weight actually changed from measurement to measurement, it is much more likely that your bathroom scale is slightly "off" from your true weight each time. Similarly, how tall are you? 5'8 1/2"? Exactly? If you had a very precise ruler, would you perhaps be 5' 8 5/8" tall? How about 5' 8 21/32"? What would your height be if the only tool you had was a wooden yardstick? In each case, your measured height would be slightly different from your true height. This lack of complete measurement precision is referred to as measurement error and, in general, the greater the reliability of the measurement, the smaller the measurement error. Copyright © 2006 nVision Research. All Rights Reserved.
|
