Searching far and wide

How do you go about finding studies that meet the inclusion criteria for your review? At one extreme you could do a very quick search on one electronic database and find a couple of relevant articles. At the other extreme you could try to find every study that's ever been done addressing your review's question.

As you might expect, there are problems with both these approaches. If you don't look very hard, the studies you do find are not likely to be representative of all the studies done. The reasons for this are explained in detail in the module on 'Publication bias'. For now, you just need to know that studies with dramatic results are much easier to find than studies that don't have dramatic findings. The other problem with only looking for a few studies is that you end up with less information. This can limit the precision of the results of your review, and restrict the conclusions you can make.

However, is it feasible to find absolutely every relevant study that has ever been done? It's certainly not easy and might not be possible in most reviews. Many studies are never published, and those that are may not be indexed in places, such as MEDLINE, you'd normally look. At some point, the effort required to find more studies becomes too much, but there is relatively little evidence on exactly when we need to stop. So, for now, most people adopt a pragmatic approach - look as far and as wide as possible, taking care to look in such a way that we take account of what we know about the biases in finding studies.

In the meantime, one of the major, ongoing efforts of the Cochrane Collaboration is to make the reports of relevant studies easier to find.

Developing a logical approach to searching

In developing your search strategy, there are a few principles. Your search should:

• be sensitive (trying to find as many studies as possible)
• minimise bias
• be efficient

To make your search sensitive, you'll need to look in a number of different places - no single database, journal or book will contain all relevant records. To minimise bias, you will need to think about finding studies that aren't in the major sources like MEDLINE. For an efficient search, it is usual to start looking in the place you expect to have the highest yield.

To do your search well, you'll need access to help from an information specialist/librarian, who has a good knowledge of helping people with systematic reviews. It is only recently that people have started going to libraries and asking for help with finding everything on a given topic, rather than asking for only a few bits of relevant information, so these skills are fairly new!