OKIS: What our research found!

Our research report has been peer reviewed, edited and published by our funders the NIHR HTA programme, so we can now share our findings with you. This blog is to give you a flavour of what we did, what we found and our recommendations going forward. Links to the full report and a variety of summaries catering for every level of interest are provided at the end.

What were the views of users of orthotic devices?

We talked directly to users of orthoses for knee instability resulting from a neuromuscular disease (NMD) or central nervous system (CNS) condition. We wanted to know what makes a device acceptable, how much they used their devices and what influences use. In-depth interviews with a diverse range of users revealed that orthotic devices are vital for users to live as normal a life as possible, engage in daily activities and have independence for as long as possible. Malfunctions of a device can therefore have serious consequences for the wearer. To ensure devices are used and not left lying in cupboards, they need to be effective, reliable, comfortable and durable. Many of those interviewed were frustrated with what they saw as deficiencies in service provision relating to appointment and administrative systems and referral pathways.

It’s important for us as researchers to make sure that when we carry out research into whether something works, that what we take as an indication of success or failure (the outcome) can be measured, but also is relevant and important to patients. In our interviews with users of orthotics we got consistent feedback that a reduction in pain, falls or trips, with improved balance and stability were of primary importance. These outcomes tended not to be measured in the research we found assessing the effectiveness of devices. They were also only seen to a limited extent in the responses from health professionals to our survey of practice.

We recommend that future research should include more qualitative studies to build on our work, and investigate in further depth some of the issues raised in our interviews with users of orthotic devices.

What did we learn from the healthcare professionals?

Orthotists, physiotherapists and rehabilitation medicine physicians were among the 238 healthcare professionals who responded to our on-line survey. They told us they prescribe a range of orthoses for knee instability related to NMD or CNS conditions. These included knee-ankle-foot orthoses (KAFOs) ankle-foot-orthoses (AFOs) and knee braces; with shoe adaptations and insoles also prescribed by many. About half the devices prescribed or fitted were reported to be custom-made.

The majority of healthcare professionals thought comfort and confidence in mobility were extremely important outcomes from treatment, matching in part the priorities for patients. When it came to outcomes for assessing the effectiveness of the devices provided, about a quarter of the health professionals said that no formal outcome measure was used. No single outcome measure was used by the rest and only a third said they formally assessed patient satisfaction. The good news is that having identified this gap in service delivery, our interview work with users of orthotic devices can help provide an understanding of what a core set of patient reported outcome measures in the clinical setting should look like. We recommend development of a core outcome measure set, and that reduction in pain, falls and trips, improved balance and stability as well as participation in paid employment, outdoor activities (such as gardening), family visits and social events should be considered. This should ensure a more patient relevant assessment of the impact of any change in device or management strategy on individual patients.

We also suggest that future research should explore different models of delivery of orthotic service for people with NMD and CNS conditions to identify best practice in terms of greatest benefit to patients and value for money.

What does the existing evidence tell us?

We took a systematic look at the existing evidence to assess how well the various orthotic options for knee instability actually help patients in every-day life. Unfortunately the few studies we found had small numbers of participants, were generally poorly reported and all were at risk of bias, meaning the findings cannot be relied upon. Disappointingly, we also found that the studies we did identify, assessed mostly mechanical outcomes such as gait analysis and energy consumption. While these are important aspects of assessing a device, they were not supplemented by assessment of outcomes important to patients such as daily activities. Given all of this, it wouldn’t be right for us to draw any specific conclusions about individual devices. What we have been able to do is make specific recommendations for future research in this area.

Research on the effectiveness of devices for knee instability in NMD and CNS conditions is needed using outcome measures relevant to patients’ everyday lives. Given the challenges we have identified in our work, future trials should start with a feasibility study. Because of the relative rarity of some NMD and CNS conditions and the personalised nature of the orthotic device or devices needed, a national registry may be a more useful approach than trials.

Where can you read more?

This blog should give you a flavour of what we did, what we found and the recommendations we have made. Depending on how much detail you would like, there are various options for reading more in the NIHR Journals Library:

The full report is 296 pages long and obviously includes full details of the background, methods, results and implications. A plain English summary provides an overview in lay language in just 249 words; or there is a scientific summary in 2,730 words.

A four page short report can be downloaded from here: Orthotics for Knee Instability We also have a limited number of paper copies available: to receive one, send your name and address to kneeorthotics@gmail.com

Did our engagement strategy work?

We would really like to know your thoughts on our blog for this project; what you have found useful, interesting, helpful, and what was missing or unhelpful. Please post a comment in response to this blog or email me, Alison Booth, at kneeorthotics@gmail.com

Orthotics services: who is delivering what?

One of the aims of our research is to identify the types of orthotic devices currently being used by the NHS in the management of instability of the knee related to neuromuscular and CNS conditions. We also want to find the answers to other questions such as what are the pathways that patients have to follow once they are referred for an orthotic device. By finding out what resources are required to assess and fit patients with a device, we should then be able to estimate the costs for providing services. We are also interested in what healthcare professionals think are important outcomes for patients and how those outcomes should be measured. This will complement the information we are gathering from patients.

To collect the information we need, we are carrying out a survey of orthotists, physiotherapists and rehabilitation medicine clinicians who are involved in the provision of orthotic devices for patients with knee instability. The content of the questionnaire has been informed by discussions with healthcare professionals in a focus group, one-to-one interviews and discussions with our project Steering Group, which includes members of the different professional groups involved in provision of orthotic devices.

Our problem is that there is so little research available on this topic that we need to ask for a lot of information to address these really important questions. We expect that the survey will take 20-30 minutes to complete which is longer than we would like, but we know that many health professionals working in this area are as keen as we are to address this knowledge gap.

The findings from the survey will be made widely available through healthcare professional networks, patient groups and research journals, as well as forming part of the formal report to our funders, the NIHR HTA programme.

Blogger: Catriona McDaid

Needles in Haystacks: Part 2: which haystacks should we look in?

Having decided what we are going to search for, the next important aspect of the searching is deciding where to look. In consultation with the review team we have decided which bibliographic databases and other resources should be searched and the words and phrases that will be used. We need to search different databases because they each include different literature.  Sometimes the content overlaps so we have to make sure we remove duplicate records from the search results, but it’s better to find something twice than miss it from a search.

A good systematic review must search at least Medline and Embase because these databases are the most comprehensive and important ones in health sciences.  Our search includes a number of other databases, for example CINAHL which covers a wide range of topics including nursing and allied health disciplines. We have also considered specialist databases, and are including the database PEDro which indexes the physiotherapy literature in our list of databases to search. We are also searching clinical trials registers to identify ongoing or unpublished studies. This should help make sure we find the most up to date research available.

Since starting this blog post I have now complete the searches.  I have identified 4233 bibliographic references for the reviewers to look through. Hidden amongst these will be the studies which might just help us find out which orthotics are most helpful for people with knee instability in neuromuscular disease and central nervous system disorders.

Blogger: Rocio Rodriguez-Lopez

A complex problem

In an earlier post I described what makes a review 'systematic'. In considering all the relevant research, we have to use transparent and replicable methods. Ideally, a reader of our review should be able to reproduce the review and get the same results. An important part of our methods is to clearly specify the nature of the research to be included in the review, our ‘inclusion criteria’.

Inclusion criteria are set out using an acronym called PICOS which stands for Participants, Interventions, Comparators, Outcomes and Study designs. In a review looking at a new drug to improve survival in breast cancer, for example, the participants are those with breast cancer, the intervention is the new drug, the comparator the currently prescribed drug, the outcome survival and usually the study design will be randomised controlled trials (often abbreviated to RCTs) as these are the most rigorous form of medical research.

Rarely, however are systematic reviews so straightforward.  The OKIS review is particularly complex as our inclusion criteria are very broad.  The participants are people with neuromuscular disorders of which there are many types. Neuromuscular disorders can involve the structure of the muscle cells in conditions such as Muscular Dystrophy the neuromuscular junction where the nerves and muscles meet such as Myasthenia Gravis and the motor nerves in the arms, legs, neck and face as in Charcot-Marie Tooth Disease to give just a few examples. This review also includes neuromuscular disorders arising from central nervous system causes such as Stroke, Multiple Sclerosis and Post-Polio Syndrome. The common element in all these disorders for this review is knee instability.

Another complexity in this review is that there is more than one type of orthotic intervention that may be used to help with knee instability. Some orthoses are fixed below the knee supporting the ankle and foot. Others are longer and cover the knee, ankle and foot. For severe weakness they can also include a hip band. Each one has its own acronym! The range of possible comparators is equally diverse. And, you’ve guessed it, the range of outcomes is wide and varies according to the conditions investigated. Pain, walking ability, usage of the orthotic and patient satisfaction are just some of the outcomes of interest. Furthermore, this topic area is at an early stage of research so we cannot restrict our study design inclusion criteria to RCTs as these have not been conducted in great numbers. We will, therefore, consider a range of study designs.

The complexity of this topic will be challenging at every stage but we now know the terms we need to use as we start to search for all the relevant research...but that will have to be the subject of another blog post!

Blogger: Debra Fayter

What is qualitative research?

In addition to seeing what research evidence already exists about the use of knee orthotics by people with neuromuscular diseases, we are undertaking a qualitative study.

Qualitative research is used when we are interested in understanding how people make sense of their world and the experiences they have. Most qualitative research studies are small scale, focusing on a relatively small number of cases. However they provide depth and detail in the context of the chosen setting; this is sometimes referred to as ‘rich description’. Qualitative studies tend to focus on words rather than numbers. Qualitative research includes a range of methods, such as interviews, focus groups, direct observation, and analysis of texts and documents, or of audio/video recorded speech or behaviour.

For this project we will be undertaking interviews with people who have a neuromuscular disease and an orthotic device for knee instability. We are especially interested in finding out what people think are the most important measures of the success of an orthotic device – what is important to them as individuals. We also plan to run some focus groups with a small number of people who have the same neuromuscular condition. This will provide additional detail on outcomes and factors of interest across a number of common neuromuscular diseases.

Blogger: Dorothy McCaughan