Defining the question

Getting the question right is the most important step in doing your review. As well as telling others what the review is about, it will guide how you collect studies, how you check whether studies are eligible and how you do the analysis. So take your time over it, discuss it with your co-reviewers and show it to others.

Chapter 5 of the Cochrane Handbook for Systematic Reviews of Interventions covers this in some detail, so this section just summarises what’s in there and adds a few comments. Defining the question is also the focus of ‘Developing a protocol for a review’ workshops – these can be a good place to try your ideas out on other people and learn from the review questions being thought about by other people. For details of these, and other, workshops contact your review group or look at the dates of workshops on www.cochrane.org.

It's also worth having a look on The Cochrane Library, or checking with your review group, for reviews of similar topics to your's to see how others have defined their questions.

Types of participants

Define the health problem. Be careful about using definitions that depend on

• time
• place
• technology

For example, diagnostic criteria that were first developed in the UK in 1994 couldn't have been used before then, and may not be used by people in other countries at all. Expensive or recent tests may not be available in many countries. So be careful. Don't risk setting conditions that will force you to exclude studies that would otherwise be relevant to your objective.

Define the population and setting. Think about

• age
• sex
• race
• other factors that may make some people respond very differently to the intervention
• where the participants are (hospital, community, etc)

Again, take care with these, as you may find that setting rigid criteria means you have to throw studies out. It is often better to be over inclusive at this stage. If, for example, you restrict the age to people aged eighteen or older, what will you do with a study of hundreds of participants that has three sixteen-year-olds in it?

Types of interventions

Define the interventions. For drugs, think about

• drug preparation
• route of administration
• dose
• duration
• frequency

Be careful not to limit the question too much. For example, think about when a drug regimen is so different that it would have a totally different effect, rather than just specifying the commonly used dose.

For non-drug interventions, defining the intervention can be a bit more difficult. You'll need to think about describing similar bits of interventions or concepts - exactly what was done, how often it was done, who did it, were they trained, etc. It can be particularly tricky where the intervention is complex, for example where a set of interventions is performed by one or more people. You may need to end up with a set of core parts to the intervention, and then some other non-core parts.

A review of assertive community treatment for people with severe mental disorders is a good example of a complex intervention clearly defined, and the reasons why it was defined in that way justified. (Marshall M, Lockwood A. Assertive community treatment for people with severe mental disorders. Cochrane Database of Systematic Reviews 1998, Issue 2. Art. No.: CD001089. DOI: 10.1002/14651858.CD001089)

"For an intervention to be accepted as ACT it must have been described in the trial report as: Assertive Community Treatment, Assertive Case Management or PACT; or as being based on the Madison, Treatment in Community Living, Assertive Community Treatment or Stein and Test models. Trials of case management that did not meet the criteria for ACT are considered in (a separate review). The review did not consider the use of ACT as an alternative to acute hospital admission. The review also excluded studies of 'Home-Based Care' (which involves a multi-disciplinary team assessing and treating urgent psychiatric referrals at home).

Home-based care is a form of crisis intervention which deals with those who are usually acutely ill, and should not be classified with either ACT or case management as these are long-term interventions for severely and persistently ill people."

Define the comparisons. Here, you need to decide whether the group you will be comparing the intervention group with have

• a placebo
• nothing
• some other treatment

To help you understand this it may be helpful to look at an example, a Cochrane Review of magnesium sulphate and other anticonvulsants for women with pre-eclampsia (Duley L, Gülmezoglu AM, Henderson-Smart DJ. Magnesium sulphate and other anticonvulsants for women with pre-eclampsia. Cochrane Database of Systematic Reviews 2003, Issue 2. Art. No.: CD000025. DOI: 10.1002/14651858.CD000025).

In this review the part of the question related to types of intervention and comparisons is defined as “comparisons of an anticonvulsant with placebo (or no anticonvulsant), and of one anticonvulsant with another”. From the analysis section of this review (below) you can see how the reviewers have organised some of their comparisons.

Types of outcomes

The general rules are to list the outcomes of interest to decision-makers, particularly those who have the health problem of interest. Think about

• how the outcomes might be measured
• when the outcomes should be measured
• which are the most important outcomes
• bad outcomes as well as good outcomes

As with defining the participants, be careful about choosing outcomes where the definition changes over time and place.

Even if you believe certain outcomes are unlikely to be reported, you should list them if they are important. Then when you are writing up the review, you can comment that you looked for information on those outcomes but the studies didn't report it. This will define a gap in the research and hopefully encourage researchers to include that outcome in future trials.

There is more in later modules about the types of data you may encounter, and how you might handle them.

Types of studies

Most Cochrane reviews focus on randomised controlled trials both because of concerns about bias in other types of study design, and bias in retrieving other study types, as explained in module 1.

There are occasions when randomised controlled trials are unlikely ever to be done. For example, if an intervention saves people with a condition where previously everyone died, it would not be appropriate to do a randomised trial in which some of the participants did not get treated. Other examples would be where the intervention is applied at a level that makes randomisation impractical, such as tobacco taxes at a national level. If you think your topic fits into categories like these, you'll need to discuss it with your review group.

You should not choose to look for other study types, where randomised controlled trials are feasible, simply because you don't believe trials have been done. There are reviews with no trials - if the question is important enough for a review, then the lack of trials is an important finding.