Subgroup analysis

The answer to a randomized controlled trial that does not confirm one’s beliefs is not the conduct of several subanalyses until one can see what one believes. Rather, the answer is to re-examine one’s beliefs carefully.

Oei et al (BMJ 1999) quoted by Schulz and Grimes in a review of the problems inherent in subgroup analysis (in May 7 Lancet, subscribers only)

They also provide the following illuminating example:

The Lancet published an illustrative example. Aspirin displayed a strongly beneficial effect in preventing death after myocardial infarction (p< 0·00001, with a narrow confidence interval). The editors urged the researchers to include nearly 40 subgroup analyses. The investigators reluctantly agreed under the condition that they could provide a subgroup analysis of their own to illustrate their unreliability. They showed that participants born under the astrological signs Gemini or Libra had a slightly adverse effect on death from aspirin (9% increase, SD 13; not significant) whereas participants born under all other astrological signs reaped a strikingly beneficial effect (28% reduction, SD 5; p 0·00001).

Anecdotal reports of support from astrologers to the contrary, this chance zodiac finding has generated little interest from the medical community.

For non-medico’s subgroup analysis is when you breakdown the data from a trial by patient characteristics like age, sex, underlying risk factors, or severity of disease. The problem is that as you increase the number of comparisons, the risk of finding a spuriously significant result increases.

In their defense, subgroup analysis can reasonably, IMHO, be used as the basis for further research.