Too much information, too little time

This is the essence of an important problem facing people who make decisions about health care. There is simply too much information around for people to keep up to date. On top of this, high quality information is often not easy to find.

The rationale for systematic reviews is explained in the article by Cynthia Mulrow, which you should now read. If you don't have access to it, the main points are summarised below for you.

The full text of the article is available at:
http://www.bmj.com/cgi/content/full/309/6954/597

The rationale for systematic reviews can be summarised like this:

• people making decisions about treatment choices, or other health care interventions, need reliable information
• there is too much information around for decision makers to keep up to date
• therefore, decision makers need reviews of existing information
• reviews can be unscientific and biased in the way they collect, appraise and summarise information
• systematic reviews attempt to minimise these biases to provide a reliable basis for making decisions

Are reviews science?

Systematic reviews aren't new, and have been used in the natural sciences, such as physics, for some time. The idea behind them is simple. Science is cumulative, with new ideas being based on previous knowledge and observation, and new advances in science should help us make sense of what we already know and have observed. But if we don't collect previous knowledge and observation in a systematic way, we are unlikely to make progress as quickly as we could. For example, if we are not aware of relevant research done by someone in another country, we cannot use that to help formulate our own research. We might even choose to do exactly the same research, without realising it has already been done. This is wasteful.

You might imagine that researchers do this routinely, but a study of controlled trials in leading medical journals showed that these studies are often published without reference to what we know already, and without discussion of what the new study adds to our knowledge. The reader is expected to go and find out all that for herself. Try the activity on the left.

The need to be more scientific about how science progresses is increasingly being recognised in health care. For instance, the UK Medical Research Council now requires evidence that a systematic review has been prepared before it will commission a new trial. This ensures that the question has not already been answered, and that the results of previous research are used in designing the new trial.

What is bias?

You've probably already noticed that the word 'bias' has been used in this module. This word has a meaning here which is similar to its meaning in common usage. Bias means something that will cause a consistent deviation from the truth. This is different from the play of chance.

For example, if we took a random sample of five people on a shopping street, four of them might be men. Would we then predict that four out of every five people are men? No, the play of chance meant that the sample we took was not representative of the whole population. The next sample of five people may well have had four, or even five, women

If, however, we tried to estimate the proportion of men in the whole population by taking samples from a football crowd, we would probably find that, even if we took a very large sample, there would be more men than women. This is simply because we are taking a sample from a place which is not typical of the world at large - we have introduced a bias into our sampling which will cause us always to overestimate the proportion of men in the population.