Why bother with protocols?

People starting reviews are usually keen to get on with finding studies, reading them and trying to make sense of it all. When you feel like this, having to try to write down exactly what you want to do can seem like a good way to kill that enthusiasm. So we must think protocols are important to expect everyone to produce one. Here are the reasons why.

Planning

An even better way to kill enthusiasm than pausing to write a protocol is to start doing something, only to find out you've gone about it the wrong way. Preparing a protocol for your review makes you stop and think about what you're doing. If you get the protocol right, the rest of the review really does follow from it.

Most reviews have a team of people working on them, and the protocol stage is a good opportunity to focus everyone's thoughts on the task in hand, debate areas of uncertainty, and make sure everyone knows just what is involved in doing the systematic review.

Once you've written down exactly what you plan to do, the protocol can act as a working document for the reviewers. You can use it to assign tasks to people, identify the resources you'll need, and keep track of progress by setting target dates for parts of the review.

People also use protocols to apply for funding - the information required for a grant application is often similar to what you will put in the protocol.

Tell everyone else what you're doing

Within the Cochrane Collaboration, protocols for reviews go through some form of peer review (the exact arrangements vary between review groups). Some people also send their protocols to others they know are interested in the review question, as they may have helpful suggestions. Once your review group has approved your protocol, it is published in The Cochrane Library. As well as enabling people to send you comments, this is one way we can try to make sure different people don't start the same review without realising it, and so avoid duplication of effort.

Minimising bias

Writing down exactly what you plan to do, in advance, is one of the ways you can minimise bias. A systematic review is retrospective - you're looking back at a set of studies that have already been done. It's easy to imagine that if you are familiar with the results of these studies, it might change the way you define the question, set the selection criteria, decide which interventions to compare, and choose which outcomes to look for.

However, most people starting a review know at least some of the studies relevant to their question, simply because they are interested in the topic. So are we being unrealistic to think we can rid ourselves of our biases by writing a protocol? It's probably impossible to get rid of all bias, which is why we talk about minimising it. Writing a protocol is one way we should be able to reduce bias, even if we can't eliminate it, by being transparent about what we plan to do in advance.