Methods Commentary: Framing the Question and A Priori Hypotheses

The following commentary has been contributed by the CLARITY Group at McMaster University.

G.Guyatt, J. Busse

This discussion will provide guidance about the framing of the question in systematic reviews addressing patient management issues and the importance of a priori hypotheses to explain heterogeneity.

Structured Questions of Patient Management: Narrow or Broad

A well accepted methodology associated with framing of questions addressing alternative management strategies in systematic reviews mandates carefully specifying the patient population (P), the intervention of interest (I), the comparator (C), and the outcomes of interest (O). This is known as the “PICO” approach to formulating a research question.

The most challenging decision in framing a PICO question is how broadly to define the patients and intervention. For example, in addressing the effects of antiplatelet agents on vascular disease, one might include only patients with transient ischemic attacks; those with ischemic attacks and strokes; or those with any vascular disease (cerebro-, cardio-, or peripheral vascular disease). The intervention might be a relatively narrow range of doses of aspirin; all doses of aspirin; or all anti-platelet agents.

On what basis should systematic review authors make this decision? The underlying biology must suggest that, across the range of patients and interventions, it is plausible that the magnitude of effect on the key outcomes is more or less the same. If that is not the case, the review or guideline will generate misleading estimates for at least some subpopulations of patients and interventions.

For instance, if antiplatelet agents differ in effectiveness in those with peripheral vascular disease versus those with myocardial infarction (as one study of clopidogrel versus aspirin that enrolled patients from both populations suggested[1]) a single estimate across the range of patients will not best inform the decision-making needs of patients and clinicians. The same will be true if different antiplatelet agents have differing magnitudes of effect.

The same logic applies to choice of the intervention, the comparator, and measure of outcome. A key choice with respect to intervention will often be the dose, intensity, or setting of the intervention. When considering respiratory rehabilitation for instance, should one include both in- and outpatient rehabilitation? With respect to the comparator, is it appropriate to plan generating pooled estimates from medication trials when some studies use placebo and others use unblinded standard care as their control arm? With respect to outcomes, should one pool studies with short-term follow-up along with those with long-term follow-up? If one anticipates a more or less similar effect across disparities in patients, interventions and outcomes aggregating is appropriate; if one anticipates substantial differences, it is not.

Merits of narrow versus broad questions

We have established that systematic review authors should be able to answer “yes” to the following question: Across the range of patients, interventions, comparators, and ways of measuring the outcome, will the magnitude of effect will be similar? Unfortunately, when considering this question, the answer is usually neither yes nor no, but rather “maybe”.

One is then left with a dilemma. Pooling broadly has important advantages. First, more study participants (and specifically more outcome events) will increase precision and confidence intervals associated with the pooled estimate of effect will narrow: the fundamental rationale for conducting a meta-analysis in the first place. Second, broader criteria will enhance the applicability of the result. The down side of broad criteria, however, is that one risks aggregating across patients, interventions, comparators or outcomes with true substantially different effects. The results of such pooling will be misleading estimates for each of the relevant sub-groups.

The Solution to the Narrow Versus Broad Eligibility Dilemma

Often, and appropriately, systematic reviews deal with the potentially vexing question of what breadth of population, intervention, comparator or outcome to choose by starting with a broad question, but including a priori specification of sub-group effects that may explain any heterogeneity they find. These hypotheses may apply to patients (e.g., effects differ in those with transient ischemic attacks and strokes, versus those with coronary or peripheral vascular disease) or interventions (e.g., high versus low doses of aspirin or aspirin versus other antiplatelet agents). A priori hypotheses may also relate to the choice of comparator (e.g., effects of amiodarone on conversion to sinus rhythm in patients with atrial fibrillation differ depending on whether the comparator is placebo or an active agent unlikely to influence return to sinus rhythm [2]), the outcome (e.g., the effect of an antihypertensive agent differs on vascular events in the cerebral or myocardial circulation), or methodology (e.g., low risk of bias studies yield different effects than high risk of bias studies).


Authors of systematic reviews should include specific eligibility criteria related to patients, interventions, comparators and outcomes. In doing so, they should be guided by the premise that it is plausible that the magnitude of effect is similar across the range of patients, interventions, comparators and outcomes included. Nevertheless, we advocate broad eligibility criteria with thoughtful a priori specification of possible explanations of variability in study results considering each of the four PICO elements as well as differences in risk of bias across studies.


1. CAPRIE Steering Committee. A randomised, blinded, trial of clopidogrel versus aspirin in patients at risk of ischaemic events (CAPRIE). Lancet, 1997.
2. Letelier, L.M., et al., Effectiveness of amiodarone for conversion of atrial fibrillation to sinus rhythm: a meta-analysis. Arch Intern Med, 2003. 163(7): p. 777-85.

G.Guyatt, J. Busse

Copyright The Clarity Group and Evidence Partners 2011