Deciding on a change (from baseline)

Another problem in meta-analysis of continuous data is change-from-baseline outcomes. As an example, consider the following results from the Hypertension Optimal Treatment (HOT) trial. This trial was published in The Lancet in 1998. Two of the treatment groups presented were attempts to reduce diastolic blood pressure in hypertensive participants to targets of less than 90 mmHg and less than 85 mmHg respectively.

Should the analysis focus on the final BP or the change-from-baseline? Does it matter?

We can work out that the average (mean) change-from-baseline is 85.2 - 105.4 = -20.2 for the first group and 83.2 - 105.4 = -22.2 for the second group. The difference between these is the same as the difference between the final means, that is 2.0. As a general rule, the two estimates of treatment effect (i.e. differences between the two groups) should not be too different in properly conducted randomized trials where the two groups are similar at baseline. Indeed in this example they are identical because the trial is so large that the average baseline BPs were identical in the two groups. In most randomized trials, this won't quite be the case. In some trials, especially small or poorly conducted trials, the difference can appear quite profound. The choice of whether to use final value or change score in your meta-analysis is a difficult one. There are two issues to consider.

First, there is a statistical argument to prefer change-from-baseline outcomes. This is closely related to the arguments in favour of crossover trials. Repeated measurements made on the same participants (at baseline and after treatment) tend to be correlated. This leads to smaller standard errors, and hence smaller confidence intervals, for the estimate of treatment effect when using change-from-baseline.

Second, there is a very real practical problem that can make the use of change-from-baseline very difficult. In order to use change-from-baseline outcomes in a meta-analysis we need their standard deviations. Notice in the table above that we have given standard deviations for the baseline measures and the final measures, but not the changes. What are the standard deviations for these changes? The answer is that we can't possibly know from the information in the table. It could be that every participant in the <90 mmHg group reduced their BP by exactly 20.2 and every participant in the <85 mmHg group reduced theirs by exactly 22.2. In this case the standard deviations of the changes will both be 0. That would be very strong evidence of a difference between the groups. Or it could be that BP reductions in the two groups were highly variable (some increase, some decrease), with a large standard deviation, and the difference in means of 2.0 would then look quite unimportant.

So how do we find out the standard deviations of the changes? If you are lucky you will find them explicitly presented in the trial report. In fact, the report of the HOT trial does give them. The BP reductions are, 20.3 (SD 5.6) in the <90 mmHg group and 22.3 (SD 5.4) in the <85 mmHg group. Note that in this case the standard deviations of change are actually larger than the standard deviations of final values - so there is no benefit in this study in terms of power in using change scores.

Many studies however will not give you the standard deviation of the change, and often reviewers face the situation of several included studies, some presenting final value mean and standard deviation, and some reporting mean and standard deviation of the change. In this situation, you can follow one of two alternatives:

(a) You can derive the standard deviation of change and estimates of mean change

If initial and final mean values are given, the mean change in each group is the difference between these values. The standard deviation of the change depends on the correlation between initial and final values, which is unlikely to be reported. If the correlation can be obtained, or perhaps imputed, methods for calculating the standard deviation are given in Section 16.1.3 of the Cochrane Handbook for Systematic Reviews of Interventions. If data are imputed, the effect of uncertainty in the correlation should be investigated in a sensitivity analysis. If initial values aren’t given, this approach cannot be used.

(b) Combine final values and change scores in the same analysis

The data we quoted from the HOT trial demonstrated that both the difference in mean final values and the difference in mean changes both estimate the same treatment effect. Because of this we can combine trials reporting mean changes with trials reporting mean final values in the same meta-analysis. Often the change scores will be less variable than the final values – combining the data in a mean difference analysis will give appropriate weights both to change scores and final values, as study weights are related to the standard deviations of the outcomes. So, in many circumstances it is not necessary to get very concerned about having a mixture of final values and change scores from your trials.

However, there are two points of concern. The first is the confusion you may cause in a reader by mixing change scores and final values in a review. For example, the final values in the data from the HOT trial were around 85mmHg, the change scores were around -20mmHg. It will be clearer to a reader if you present the change scores as one subgroup, and the final values as another subgroup in RevMan, and then combine the two in an overall analysis.

The second concern is that this approach will not work when you have different measurement scales, when you would want to use the standardised mean difference - this method cannot mix change and final values.

Summary

To perform meta-analysis of continuous data you will need to extract or calculate means and standard deviations from the reports of your included trials. This is often more difficult to do than extracting event rates for dichotomous outcomes as the information you need is not always present, or in a standard form.Some things to check are:

• Are these data symmetrically distributed or skewed? If skewed, you may need to present the results in the Additional Tables and not perform a meta-analysis.
• Is the presented measure of variation a standard deviation? It may be a standard error (check if it looks too small), or something else. If so, convert it before you enter it in RevMan.
• Do your included studies all measure outcome using the same scale? If not, you will need to convert to standard units (if you can) or use a standardised mean difference.
• Should you use a random effects or fixed effect meta-analysis? Whether this makes a difference will depend on the amount of heterogeneity present.
• Should you enter final value or change scores? This will be partly determined by what is reported in your included studies, and it is possible to mix the two in the same analysis. If you have to impute a standard deviation, you should perform a sensitivity analysis and see how it affects your results. If they change, draw your conclusions with care!