Correcting for publication bias.

From what we have seen in this module so far, we know that the methods we have for detecting the possibility of publication bias in systematic reviews are not very good. Any methods for attempting to correct for this perceived bias are therefore also not ideal, but the following methods have been suggested. These are rarely used in Cochrane reviews but are included here for completeness.

"Trim and fill method"

In this method the tail of the side of the funnel plot with the smaller studies is chopped off to make the funnel plot symmetrical, and it is then replicated and added back to both sides so the plot becomes symmetrical. The centre and variability of the filled funnel plot are then estimated (there are complicated statistical methods to do this formally).

Fail safe N

Here, the number of null studies (of similar size) which would be required to remove an observed significant effect is estimated. This method may give you an idea of the likely importance of any publication bias present. For example, if it tells you that several large negative trials would need to exist to overturn your positive result, you may decide it is quite unlikely that these were missed. This remains, however, a judgement.

Modelling

Models for the probability that studies with particular results do or do not get published can be designed and used to investigate possible publication bias.

Summary

There is quite a lot of work being undertaken, both in the form of trials registers and more intensive searching to try to help reviewers identify all trials, and methodological research to advance the ways we measure and account for publication bias in systematic reviews. Currently however, the only thing we know for certain about publication bias is that it exists. Our methods for assessing its presence can only provide suggestions, not definite answers.

The main purpose of including issues to do with publication bias in your review is to ensure that you, and readers of your review, are aware of the fact that publication bias is possible, and to attempt, at least in part, to estimate how big an impact it might have on the results of your review.