(a) Misuse of statistics

We begin this chapter by examining some of the problems with the medical literature. A common response, particularly from those beginning to work in science and medicine, is that everything written in a book or journal article is 'true'. This is not the case, and is something that will become apparent as you begin to examine the literature with a critical eye.

As an example of this, a common problem for scientists, statisticians, clinicians and the media is the misuse of statistical analysis. The table below is taken from a paper examining the number of errors in use of statistics in papers (published in the Arthritis and Rheumatism journal) for two different time periods:

Error
1967-68 (n = 47)
1982 (n = 74)
Undefined method
14 (30%)
7 (9%)
Inadequate description of measures of location or dispersion
6 (13%)
7 (9%)
Repeated observations treated as independent
1 (2%)
4 (5%)
Two groups compared on more than 10 variables at 5% level
3 (6%)
28 (38%)
Multiple t tests instead of analysis of variance
2 (4%)
18 (24%)
x2 tests used when expected frequencies too small
3 (6%)
4 (5%)
At least one of the above errors
28 (60%)
49 (66%)

(source: Felson, DT, Cupples LA, Meenan RF. Misuse of statistical methods in Arthritis and Rheumatism. 1982 versus 1967-68, Arthritis and Rheumatism 1984;27:1018-22).

This table shows how papers are published with common statistical errors. Were you to be interested in these papers for informing your clinical practice, you must exclude the papers with errors from your reading. Papers that revealed statistical flaws in published research led to improvements in peer review of papers and ultimately to the inclusion of critical appraisal into the medical curriculum so that doctors have had the opportunity to practise appraising studies for themselves.

Ethical considerations

Statistical errors can have serious effects on patients and on future research. This should always be borne in mind when appraising the literature. It is imperative that scientists, clinicians, and statisticians are aware that the misuse of statistical analysis, either intentional or accidental, can have serious consequences for patients. For example: