Drawing up a balance sheet

A useful tip is to draw up a balance sheet where you list all the "beneficial" (or positive) effects associated with the intervention on one side, and all the potential "harms" (or negative) outcomes on the other side.

Take the example of whether or not to treat a sore throat with antibiotics (when we don't know for sure what is causing it). The potential positive effects of treatment include:

• reduction in severity of illness
• reduction in pain
• reduction in the duration of illness
• reduction in infections following on (such as sinusitis, and acute otitis media)
• reduction in subsequent noninfective complications (such as rheumatic fever and acute glomerulonephritis (an acute inflammation of the kidney)).

The potential negative effects of treatment include:

• adverse effects of antibiotics (such as diarrhoea, thrush etc)
• the cost to society in terms of antibiotic prescription and overuse

Sometimes the labeling of outcomes can be subjective, depending on the perspective you are coming from. Take the example of an intervention for Alzheimer's disease. If the intervention improves mobility, this might be seen as a positive effect from a patient's perspective, but a negative effect from a carer's perspective.

What if there are no data available on some outcomes?

It may be that there are no data available on some of the outcomes. For example, some of the slow onset or more rare beneficial or adverse effects associated with treatments are unlikely to be seen in trials with relatively short-term follow-up or with small numbers of participants. It is often necessary to consider other types of data (such as from long term cohort studies or post marketing surveillance studies) in order to obtain reliable data on potential harms. To do this thoroughly, however, adds a whole new dimension to a systematic review, as we would have to search for these studies and then appraise and combine them. At the moment few Cochrane reviews do this.

If you have insufficient information about particular outcomes, this should be clearly stated in your review. If you feel these outcomes are important you may suggest that they are included in future trials by discussing them in the "Recommendations for Future Research" section of your review.

What do users of reviews need to know about these outcomes?

We have seen, in Module 11, the importance of knowing the absolute change (not just the relative change) in the probability of each outcome as a result of the intervention. However, in order for users in different settings to decide how to apply this information, they also need to know how common the outcome is in their particular setting. To come back to the example we used for the balance sheet, in most developed countries the risk of rheumatic fever or acute glomerulonephritis as a complication of sore throat is extremely low, however, in many developing countries the risk is much greater to begin with, and even a small reduction in the absolute probability of this outcome may be very important.

In other instances, it may be important for the user to know the natural history of the condition. For example, with many self-limiting illnesses, a one day reduction in symptom duration, while it may be statistically significant, needs to be set against an illness which may only last for four or five days at most.

Try to avoid value judgments

Value judgments are any statements where you make assumptions about the value placed on particular outcomes. An easy way to spot them is to ask yourself "Would everyone agree with this statement?" If the answer is probably not, then it may be best to not make that judgment in your review. It's better to present the data in a way that will allow the user to make a balanced objective 'trade-off' given their own personal circumstances. A one day reduction in the duration of an illness may mean a lot to a particular user, where others will be willing to accept that extra day's illness in return for reducing their chance of some other outcome (for example, the side effects of the intervention).