Needles in Haystacks: Part 2: which haystacks should we look in?

Having decided what we are going to search for, the next important aspect of the searching is deciding where to look. In consultation with the review team we have decided which bibliographic databases and other resources should be searched and the words and phrases that will be used. We need to search different databases because they each include different literature.  Sometimes the content overlaps so we have to make sure we remove duplicate records from the search results, but it’s better to find something twice than miss it from a search.

A good systematic review must search at least Medline and Embase because these databases are the most comprehensive and important ones in health sciences.  Our search includes a number of other databases, for example CINAHL which covers a wide range of topics including nursing and allied health disciplines. We have also considered specialist databases, and are including the database PEDro which indexes the physiotherapy literature in our list of databases to search. We are also searching clinical trials registers to identify ongoing or unpublished studies. This should help make sure we find the most up to date research available.

Since starting this blog post I have now complete the searches.  I have identified 4233 bibliographic references for the reviewers to look through. Hidden amongst these will be the studies which might just help us find out which orthotics are most helpful for people with knee instability in neuromuscular disease and central nervous system disorders.

Blogger: Rocio Rodriguez-Lopez

Needles in Haystacks: Part 1: what sort of needles are we looking for?

For a good many of us, searching for information on a topic involves typing a word or phrase into Google and scrolling through the millions of hits! Searching for studies to include in a systematic review is a rather more rigorous process!  This two part blog will give you a flavour of what is involved.

Searches in a systematic review must aim to retrieve all the relevant evidence about the research question and follow a specific methodology, so that, if necessary, the search could be reproduced.  However, they also need to be conducted within the budget, time and resources available for the project. As Information Specialist for the project this is my responsibility.

My first job was to devise a search strategy that would be capable of identifying all the relevant studies. The medical literature is vast and indexed across a considerable number of bibliographic databases so it is important to be clear what you are searching for. Searches in a systematic review are based on the research question which is composed of four elements. For this review:

• Population is people with knee instability related to a neuromuscular disease or central nervous system disorder
• Intervention is orthotic devices
• Comparator is any used
• Outcomes  are any investigated

The main challenges of this project are the number of diseases and conditions of interest and the number of possible orthotic devices. Studies can be indexed very differently on different databases so it is important to think of all the terms that could be used to describe the devices. The clinical experts on our Advisory Group provided valuable input on the names of all the possible relevant orthotic devices, so we are confident that we will locate the relevant studies.

Because there are many, diverse neuromuscular diseases and central nervous system conditions, we could possibly miss some if we were to list them in our searches, even with clinical advice. It would make searches very long, unwieldy and time-consuming. Instead we plan to concentrate on finding studies that describe using an orthotic device to deal with knee instability. For the same reasons, the comparators and outcomes have not been included in the search.

Blogger: Rocio Rodriguez-Lopez

A complex problem

In an earlier post I described what makes a review 'systematic'. In considering all the relevant research, we have to use transparent and replicable methods. Ideally, a reader of our review should be able to reproduce the review and get the same results. An important part of our methods is to clearly specify the nature of the research to be included in the review, our ‘inclusion criteria’.

Inclusion criteria are set out using an acronym called PICOS which stands for Participants, Interventions, Comparators, Outcomes and Study designs. In a review looking at a new drug to improve survival in breast cancer, for example, the participants are those with breast cancer, the intervention is the new drug, the comparator the currently prescribed drug, the outcome survival and usually the study design will be randomised controlled trials (often abbreviated to RCTs) as these are the most rigorous form of medical research.

Rarely, however are systematic reviews so straightforward.  The OKIS review is particularly complex as our inclusion criteria are very broad.  The participants are people with neuromuscular disorders of which there are many types. Neuromuscular disorders can involve the structure of the muscle cells in conditions such as Muscular Dystrophy the neuromuscular junction where the nerves and muscles meet such as Myasthenia Gravis and the motor nerves in the arms, legs, neck and face as in Charcot-Marie Tooth Disease to give just a few examples. This review also includes neuromuscular disorders arising from central nervous system causes such as Stroke, Multiple Sclerosis and Post-Polio Syndrome. The common element in all these disorders for this review is knee instability.

Another complexity in this review is that there is more than one type of orthotic intervention that may be used to help with knee instability. Some orthoses are fixed below the knee supporting the ankle and foot. Others are longer and cover the knee, ankle and foot. For severe weakness they can also include a hip band. Each one has its own acronym! The range of possible comparators is equally diverse. And, you’ve guessed it, the range of outcomes is wide and varies according to the conditions investigated. Pain, walking ability, usage of the orthotic and patient satisfaction are just some of the outcomes of interest. Furthermore, this topic area is at an early stage of research so we cannot restrict our study design inclusion criteria to RCTs as these have not been conducted in great numbers. We will, therefore, consider a range of study designs.

The complexity of this topic will be challenging at every stage but we now know the terms we need to use as we start to search for all the relevant research...but that will have to be the subject of another blog post!

Blogger: Debra Fayter