Systematic reviews and meta-analyses

In Module 1 we summarised the process of preparing a systematic review. Part of that process is to calculate the results of each study identified by the reviewer, and then to calculate an average of those results - if appropriate - in a meta-analysis. Systematic reviews do not have to have a meta-analysis - there are times when it is not appropriate or possible

To represent this visually, the figure below shows that a meta-analysis may be part of a systematic review. A meta-analysis is also possible without doing a systematic review - you could just find a few studies and calculate a result, with no attempt to be systematic about how the particular studies were chosen.

One slight complication is that these two terms are often used interchangeably, particularly in North America. In this learning material, the term 'systematic review' will refer to the entire process of collecting, reviewing and presenting all available evidence, while the term 'meta-analysis' will refer to the statistical technique involved in extracting and combining data to produce a summary result.

What is a meta-analysis?

A meta-analysis is a two-stage process. The first stage is the extraction of data from each individual study and the calculation of a result for that study (the 'point estimate' or 'summary statistic'), with an estimate of the chance variation we would expect with studies like that (the 'confidence interval').

The second stage involves deciding whether it is appropriate to calculate a pooled average result across studies and, if so, calculating and presenting such a result. Part of this process is to give greater weight to the results from studies which give us more information, because these are likely to be closer to the truth we are trying to estimate. We'll come back to these topics in later modules.

The results of meta-analyses are often presented in a forest plot. Run through this PDF file, which explains the parts of these plots.

A meta-analysis does not just add up the numbers from the trials

One common criticism of meta-analysis is that it somehow simply adds together the results from quite different studies and calculates a summary statistic as if it is one big study. It would be wrong to do this, and this is not what a meta-analysis does. A meta-analysis looks at the results within each study, and then calculates a weighted average.

The reasons for this are explained in a later module. For now, it's enough to realise that if we just add up the numbers of people and events (such as deaths) from a number of trials, we effectively treat it as one big trial. In effect we will be comparing people in the treatment group of one trial with people in the control group of another trial. This comparison is not randomised, and it is likely that there will be some differences in the way the trials were carried out. This doesn't make sense when we have gone to a lot of trouble to find randomised comparisons, and it does not make logical sense to do this.