Is it sensible to calculate a pooled estimate?

With the right software, it's very easy to do a meta-analysis. In fact it's almost too easy. Before pressing the button to calculate a meta-analysis, it's important to ask whether it is sensible to do so.

There are two parts to making this decision - clinical and statistical. The first, clinical, part involves asking yourself whether the studies you have found really do all address the same question so that an average of their results would be sensible. There might be differences in the participants, interventions or outcomes that lead you to think that the treatment effect is really very different in the different studies. Here are some examples to think about:

• Would you expect pregnant women to respond differently from teenage boys to interventions designed to help them stop smoking?
• Would you expect the effect of acupuncture to vary depending on who did it?
• Would you expect the outcome of treatments for depression to be similar at 3 days and 3 months?

There is always an element of judgement in these decisions. But if you think there are good reasons why you would expect the effects of an intervention to differ substantially between studies, you should not pool the results.

The second, statistical, way of thinking about the consistency of the results in the studies included in a review is to look for big differences between the results of the trials. This will be covered later, in the module on Heterogeneity. For now, it's enough to know that you should look for studies where the results don't seem to fit, and then investigate possible reasons for this.