Power Failure
Say you're poring over a brain imaging study that's making a hot new claim. If it has the power of an average neuroscience study, the odds that its findings are actually true are 50-50 or lower, according to a report in the MayNature Reviews Neuroscience.A study sample, by definition, is a slice of reality. With the proper statistical number crunching, researchers can quantify the likelihood that a pattern seen in a sample occurs in the real world.
Statistical power — the probability that a study will find an effect if one actually exists — depends on two factors: sample size and effect size.
In a brain imaging study that's looking for a telltale autism signature, for example, the sample size is the total number of participants, and the effect size is the difference in brain activity between the autism and control groups.
If a study has a small sample size, it needs to find a moderate-to-large effect in order to surpass the power threshold for statistical significance.
What surprised me, however, is that even when a small study finds a statistically significant effect, it's more likely to be false than a finding from a large study.
Read more at...