The answer to a randomized controlled trial that does not confirm one’s beliefs is not the conduct of several subanalyses until one can see what one believes. Rather, the answer is to re-examine one’s beliefs carefully.Oei et al (BMJ 1999) quoted by Schulz and Grimes in a review of the problems inherent in subgroup analysis (in May 7 Lancet, subscribers only)
They also provide the following illuminating example:
The Lancet published an illustrative example. Aspirin displayed a strongly beneficial effect in preventing death after myocardial infarction (p< 0·00001, with a narrow confidence interval). The editors urged the researchers to include nearly 40 subgroup analyses. The investigators reluctantly agreed under the condition that they could provide a subgroup analysis of their own to illustrate their unreliability. They showed that participants born under the astrological signs Gemini or Libra had a slightly adverse effect on death from aspirin (9% increase, SD 13; not significant) whereas participants born under all other astrological signs reaped a strikingly beneficial effect (28% reduction, SD 5; p 0·00001).For non-medico’s subgroup analysis is when you breakdown the data from a trial by patient characteristics like age, sex, underlying risk factors, or severity of disease. The problem is that as you increase the number of comparisons, the risk of finding a spuriously significant result increases.
Anecdotal reports of support from astrologers to the contrary, this chance zodiac finding has generated little interest from the medical community.
In their defense, subgroup analysis can reasonably, IMHO, be used as the basis for further research.