OKIS: What our research found!

Our research report has been peer reviewed, edited and published by our funders the NIHR HTA programme, so we can now share our findings with you. This blog is to give you a flavour of what we did, what we found and our recommendations going forward. Links to the full report and a variety of summaries catering for every level of interest are provided at the end.

What were the views of users of orthotic devices?

We talked directly to users of orthoses for knee instability resulting from a neuromuscular disease (NMD) or central nervous system (CNS) condition. We wanted to know what makes a device acceptable, how much they used their devices and what influences use. In-depth interviews with a diverse range of users revealed that orthotic devices are vital for users to live as normal a life as possible, engage in daily activities and have independence for as long as possible. Malfunctions of a device can therefore have serious consequences for the wearer. To ensure devices are used and not left lying in cupboards, they need to be effective, reliable, comfortable and durable. Many of those interviewed were frustrated with what they saw as deficiencies in service provision relating to appointment and administrative systems and referral pathways.

It’s important for us as researchers to make sure that when we carry out research into whether something works, that what we take as an indication of success or failure (the outcome) can be measured, but also is relevant and important to patients. In our interviews with users of orthotics we got consistent feedback that a reduction in pain, falls or trips, with improved balance and stability were of primary importance. These outcomes tended not to be measured in the research we found assessing the effectiveness of devices. They were also only seen to a limited extent in the responses from health professionals to our survey of practice.

We recommend that future research should include more qualitative studies to build on our work, and investigate in further depth some of the issues raised in our interviews with users of orthotic devices.

What did we learn from the healthcare professionals?

Orthotists, physiotherapists and rehabilitation medicine physicians were among the 238 healthcare professionals who responded to our on-line survey. They told us they prescribe a range of orthoses for knee instability related to NMD or CNS conditions. These included knee-ankle-foot orthoses (KAFOs) ankle-foot-orthoses (AFOs) and knee braces; with shoe adaptations and insoles also prescribed by many. About half the devices prescribed or fitted were reported to be custom-made.

The majority of healthcare professionals thought comfort and confidence in mobility were extremely important outcomes from treatment, matching in part the priorities for patients. When it came to outcomes for assessing the effectiveness of the devices provided, about a quarter of the health professionals said that no formal outcome measure was used. No single outcome measure was used by the rest and only a third said they formally assessed patient satisfaction. The good news is that having identified this gap in service delivery, our interview work with users of orthotic devices can help provide an understanding of what a core set of patient reported outcome measures in the clinical setting should look like. We recommend development of a core outcome measure set, and that reduction in pain, falls and trips, improved balance and stability as well as participation in paid employment, outdoor activities (such as gardening), family visits and social events should be considered. This should ensure a more patient relevant assessment of the impact of any change in device or management strategy on individual patients.

We also suggest that future research should explore different models of delivery of orthotic service for people with NMD and CNS conditions to identify best practice in terms of greatest benefit to patients and value for money.

What does the existing evidence tell us?

We took a systematic look at the existing evidence to assess how well the various orthotic options for knee instability actually help patients in every-day life. Unfortunately the few studies we found had small numbers of participants, were generally poorly reported and all were at risk of bias, meaning the findings cannot be relied upon. Disappointingly, we also found that the studies we did identify, assessed mostly mechanical outcomes such as gait analysis and energy consumption. While these are important aspects of assessing a device, they were not supplemented by assessment of outcomes important to patients such as daily activities. Given all of this, it wouldn’t be right for us to draw any specific conclusions about individual devices. What we have been able to do is make specific recommendations for future research in this area.

Research on the effectiveness of devices for knee instability in NMD and CNS conditions is needed using outcome measures relevant to patients’ everyday lives. Given the challenges we have identified in our work, future trials should start with a feasibility study. Because of the relative rarity of some NMD and CNS conditions and the personalised nature of the orthotic device or devices needed, a national registry may be a more useful approach than trials.

Where can you read more?

This blog should give you a flavour of what we did, what we found and the recommendations we have made. Depending on how much detail you would like, there are various options for reading more in the NIHR Journals Library:

The full report is 296 pages long and obviously includes full details of the background, methods, results and implications. A plain English summary provides an overview in lay language in just 249 words; or there is a scientific summary in 2,730 words.

A four page short report can be downloaded from here: Orthotics for Knee Instability We also have a limited number of paper copies available: to receive one, send your name and address to kneeorthotics@gmail.com

Did our engagement strategy work?

We would really like to know your thoughts on our blog for this project; what you have found useful, interesting, helpful, and what was missing or unhelpful. Please post a comment in response to this blog or email me, Alison Booth, at kneeorthotics@gmail.com

Survey of health professionals: analysis begins.

Our survey of health professionals is now closed and we are starting on the analysis of the data collected in the online questionnaire. This will involve a relatively straightforward descriptions of the data collected for presentation in the final report. For example, we will look at how many members of the different health professions filled in our questionnaire; how many of them see patients with neuromuscular diseases such as polio; what the different ways of working, are such as working in stand-alone clinics or as part of a multidisciplinary facility. This may sound a bit basic, but currently this information is unknown. We hope to provide a much better picture of what and how orthotic services are delivered in the UK. 

Now we have closed the survey so we can download all the responses collected into a spreadsheet. Each response has a unique identifier so responses to each question by each individual can be tracked without being able to identify the person themselves. We will then ‘code’ the responses so the statistical package can perform the analysis. This simply means all the answer choices have a number assigned to them. For example, the answers to the question “What is your gender?” would be coded as 1 for male and 2 for female. We will use simple statistics such as means (the average value) and modes (the more frequent value); and frequency tables and cross-tabulations to present our results in the final report. For the free text responses we will be using qualitative analysis techniques to identify all the issues raised and will present these findings in a descriptive narrative.

Many thanks again to all our respondents, your time is much appreciated and will form an important part of the findings of our project.

Blogger: Joanne O’Connor

Orthotics provision: Survey of healthcare professionals launched

We have now opened our survey of healthcare professionals (HCPs) who are involved in providing orthotic devices for knee instability in people with neuromuscular disease or central nervous system conditions. The link to the online survey is being sent out through three professional organisations:

• Association of Chartered Physiotherapists Interested in Neurology (ACPIN)
• British Association of Prosthetists and Orthotists (BAPO)
• British Society for Rehabilitation Medicine (BSRM)

If you are a health professional working in this area and have not received an invitiation to take part, or you have any other queries please get in touch with us at kneeorthotics@gmail.com

Blogger: Catriona McDaid

Orthotics services: who is delivering what?

One of the aims of our research is to identify the types of orthotic devices currently being used by the NHS in the management of instability of the knee related to neuromuscular and CNS conditions. We also want to find the answers to other questions such as what are the pathways that patients have to follow once they are referred for an orthotic device. By finding out what resources are required to assess and fit patients with a device, we should then be able to estimate the costs for providing services. We are also interested in what healthcare professionals think are important outcomes for patients and how those outcomes should be measured. This will complement the information we are gathering from patients.

To collect the information we need, we are carrying out a survey of orthotists, physiotherapists and rehabilitation medicine clinicians who are involved in the provision of orthotic devices for patients with knee instability. The content of the questionnaire has been informed by discussions with healthcare professionals in a focus group, one-to-one interviews and discussions with our project Steering Group, which includes members of the different professional groups involved in provision of orthotic devices.

Our problem is that there is so little research available on this topic that we need to ask for a lot of information to address these really important questions. We expect that the survey will take 20-30 minutes to complete which is longer than we would like, but we know that many health professionals working in this area are as keen as we are to address this knowledge gap.

The findings from the survey will be made widely available through healthcare professional networks, patient groups and research journals, as well as forming part of the formal report to our funders, the NIHR HTA programme.

Blogger: Catriona McDaid

Being picky: study selection for the systematic review

The OKIS systematic review will bring together the results of all the previous research on the benefits and any harms from using orthotic devices for knee instability in adults with neuromuscular disease. This will tell us what is currently known about the usefulness of the devices. In previous posts, Rocio described search terms and where to search how she went about identifying all the existing studies. In this post I describe what we’re doing with what she found.

Rocio’s searches of a wide range of databases gave us 4233 references to potentially relevant studies.  She put these references into a reference manager program called ‘Endnote’. Alison, Joanne and I sifted through the references to determine which were obviously irrelevant from the information given and which needed to be read as full papers to determine their relevance. To ensure nothing was missed, two of us looked at every reference independently. As a result we identified 485 papers that we need to read in full.

The next job is to evaluate each of the 485 studies against our predetermined review inclusion criteria for participants, interventions, outcomes and study designs as mentioned in a previous post. We will document our decision to include or exclude each study with a reason. Again, two of us will do this and where there is disagreement on the relevance of a study we will discuss it and either arrive at a consensus or seek advice from a third member of the project team. This process of study selection is one of the aspects that makes a review systematic and ensures transparency.

It is too early to say how many studies will ultimately be included in the OKIS review. The above process is time-consuming, but as systematic reviewers we have to be patient! We’ll have our results soon enough!

Blogger: Debra Fayter

Needles in Haystacks: Part 2: which haystacks should we look in?

Having decided what we are going to search for, the next important aspect of the searching is deciding where to look. In consultation with the review team we have decided which bibliographic databases and other resources should be searched and the words and phrases that will be used. We need to search different databases because they each include different literature.  Sometimes the content overlaps so we have to make sure we remove duplicate records from the search results, but it’s better to find something twice than miss it from a search.

A good systematic review must search at least Medline and Embase because these databases are the most comprehensive and important ones in health sciences.  Our search includes a number of other databases, for example CINAHL which covers a wide range of topics including nursing and allied health disciplines. We have also considered specialist databases, and are including the database PEDro which indexes the physiotherapy literature in our list of databases to search. We are also searching clinical trials registers to identify ongoing or unpublished studies. This should help make sure we find the most up to date research available.

Since starting this blog post I have now complete the searches.  I have identified 4233 bibliographic references for the reviewers to look through. Hidden amongst these will be the studies which might just help us find out which orthotics are most helpful for people with knee instability in neuromuscular disease and central nervous system disorders.

Blogger: Rocio Rodriguez-Lopez

Needles in Haystacks: Part 1: what sort of needles are we looking for?

For a good many of us, searching for information on a topic involves typing a word or phrase into Google and scrolling through the millions of hits! Searching for studies to include in a systematic review is a rather more rigorous process!  This two part blog will give you a flavour of what is involved.

Searches in a systematic review must aim to retrieve all the relevant evidence about the research question and follow a specific methodology, so that, if necessary, the search could be reproduced.  However, they also need to be conducted within the budget, time and resources available for the project. As Information Specialist for the project this is my responsibility.

My first job was to devise a search strategy that would be capable of identifying all the relevant studies. The medical literature is vast and indexed across a considerable number of bibliographic databases so it is important to be clear what you are searching for. Searches in a systematic review are based on the research question which is composed of four elements. For this review:

• Population is people with knee instability related to a neuromuscular disease or central nervous system disorder
• Intervention is orthotic devices
• Comparator is any used
• Outcomes  are any investigated

The main challenges of this project are the number of diseases and conditions of interest and the number of possible orthotic devices. Studies can be indexed very differently on different databases so it is important to think of all the terms that could be used to describe the devices. The clinical experts on our Advisory Group provided valuable input on the names of all the possible relevant orthotic devices, so we are confident that we will locate the relevant studies.

Because there are many, diverse neuromuscular diseases and central nervous system conditions, we could possibly miss some if we were to list them in our searches, even with clinical advice. It would make searches very long, unwieldy and time-consuming. Instead we plan to concentrate on finding studies that describe using an orthotic device to deal with knee instability. For the same reasons, the comparators and outcomes have not been included in the search.

Blogger: Rocio Rodriguez-Lopez

A complex problem

In an earlier post I described what makes a review 'systematic'. In considering all the relevant research, we have to use transparent and replicable methods. Ideally, a reader of our review should be able to reproduce the review and get the same results. An important part of our methods is to clearly specify the nature of the research to be included in the review, our ‘inclusion criteria’.

Inclusion criteria are set out using an acronym called PICOS which stands for Participants, Interventions, Comparators, Outcomes and Study designs. In a review looking at a new drug to improve survival in breast cancer, for example, the participants are those with breast cancer, the intervention is the new drug, the comparator the currently prescribed drug, the outcome survival and usually the study design will be randomised controlled trials (often abbreviated to RCTs) as these are the most rigorous form of medical research.

Rarely, however are systematic reviews so straightforward.  The OKIS review is particularly complex as our inclusion criteria are very broad.  The participants are people with neuromuscular disorders of which there are many types. Neuromuscular disorders can involve the structure of the muscle cells in conditions such as Muscular Dystrophy the neuromuscular junction where the nerves and muscles meet such as Myasthenia Gravis and the motor nerves in the arms, legs, neck and face as in Charcot-Marie Tooth Disease to give just a few examples. This review also includes neuromuscular disorders arising from central nervous system causes such as Stroke, Multiple Sclerosis and Post-Polio Syndrome. The common element in all these disorders for this review is knee instability.

Another complexity in this review is that there is more than one type of orthotic intervention that may be used to help with knee instability. Some orthoses are fixed below the knee supporting the ankle and foot. Others are longer and cover the knee, ankle and foot. For severe weakness they can also include a hip band. Each one has its own acronym! The range of possible comparators is equally diverse. And, you’ve guessed it, the range of outcomes is wide and varies according to the conditions investigated. Pain, walking ability, usage of the orthotic and patient satisfaction are just some of the outcomes of interest. Furthermore, this topic area is at an early stage of research so we cannot restrict our study design inclusion criteria to RCTs as these have not been conducted in great numbers. We will, therefore, consider a range of study designs.

The complexity of this topic will be challenging at every stage but we now know the terms we need to use as we start to search for all the relevant research...but that will have to be the subject of another blog post!

Blogger: Debra Fayter

Designing the survey

One of elements of this project is to conduct a survey of healthcare professionals. This means that we will write a list of questions that we want the relevant health professionals to answer. The survey will ask questions about topics such as how patients get referred to them, how often they see patients and how they would judge if the orthotic is working for the patient.

Surveys can be conducted over the phone, face to face, be sent by post or electronically via the internet. We have chosen to use the internet and create a web based survey because it is a fast, simple and cost efficient way of reaching large numbers of people. We believe that there may be differences across the country in terms of what services are available to patients and we want to make sure that we find out what these differences are. By using a web survey, we can hopefully reach a range of health professionals all over the country.

The same questions are sent to everyone we want to join in the survey. The questions are formulated to try and make sure that everyone will understand them in the same way so the answers are consistent and provide the information we want to collect. This is the big challenge for any survey. To help us make sure we’ve got it right we will test the survey on a small number of health professionals. This will tell us what it’s like for participants to complete, including how long it takes, and allow us to make sure the questions are being answered in the way we anticipate.

The results of the survey will be the collective answers that we get. They should give us an idea of what orthotic services are provided to patients and how much NHS resources are used to provide these services. This will help us to estimate the cost of these services to the NHS.

Blogger: Joanne O’Connor

Keeping things in perspective

As researchers we spend a great deal of time initially reading around a topic and examining the published research evidence. In order to be answerable, research questions are usually very specific, as in this project where we are focussed on orthotic devices for knee instability in people who have a neuromuscular disease or central nervous system disorder. So when we get the opportunity to meet and talk with people our research is aimed at helping whenever possible we take it!

Such an opportunity was taken when we attended a really interesting meeting of the Yorkshire and Humber Muscle Group in York last Friday. When we got in touch with the group to circulate our advert for patient members to join our Steering Group they invited us along to the meeting. As well as having an opportunity to tell the group about our research we heard about the work that the group is doing in the Yorkshire and Humber region. This included reports from those providing support to people with neuromuscular conditions in the region, fund raising successes and possibilities, and advice on new self-managed funding arrangements. The meeting was hosted by the Muscular Dystrophy Campaign, who provide details about this group and similar ones all over the country on their website.

Some of the discussions at the meeting were a very strong reminder for us that in the context of complex neuromuscular conditions walking problems can be just one of many challenges people are dealing with. We are aware that our research is asking very focused and specific questions but the meeting has made us very mindful of the wider context in which our research question fits. This will be reflected in the discussion section of our final report.

Bloggers: Catriona McDaid and Alison Booth

Seeking patient representatives in Yorkshire region

Can you help us in our research?

We are a group of researchers at the University of York carrying out research into the use of orthotic devices to help adults with neuromuscular conditions who have an unstable knee.

We are seeking two people who can bring the perspective of patients to our Project Advisory Group.

The role of the Project Advisory Group is to make sure that the research takes into consideration the views of patients and healthcare professionals and to ensure the research is of a high standard. The project will last 12 months and started on 1^st April 2014. The group will meet three times over the year. We might also contact you occasionally in between meetings. We are seeking two people who

·                Have been offered an orthotic device to help with knee problems related to a neuromuscular condition

·                Are comfortable working as part of a small team

·                Are willing to share their opinions at a meeting and listen to the views of others

·                Are able to attend three Advisory Group meetings at the University of York– so live within easy travelling distance of York

You do not need to have any research experience just an enthusiasm and interest in the project. Information about the research and the team is available in our other blogs and pages.

If you would like to join us, please send an email to Catriona McDaid at kneeorthotics@gmail.com explaining why you would like to join the Project Advisory Group by Friday 25^th April.  You can also email Catriona if you have any queries about this role or would like further information.                       

Thank you for your interest.

And we've started!

It was great to get the project started - on the auspicious day of 1st April(!). Although it is the beginning it has felt like quite a journey getting here. The project was commissioned by the NIHR HTA programme back in Spring 2013 when they advertised an outline of the question they wanted addressed and invited proposals from researchers. But the HTA will have started exploring it as a possible topic for commissioning research on well before then. If you are interested in how HTA prioritises topics for research you can find out more information by following the links on the HTA Programme website.  Members of the public, health care professionals and policy makers can make suggestions to NIHR about any ideas they would like them to consider for funding. You can find details of how they identify research questions and access the on line suggestion form at http://www.nets.nihr.ac.uk/identifying-research.

The overall questions to be addressed by this research project are: what orthotic devices are currently in use in the NHS for knee instability for people with neuromuscular conditions, what are the costs involved and what further research is needed. An important aspect of our work is to find out what the most important issues are for patients. Our research project is intended as preparatory work to inform a different piece of research sometime in the future which will look at the clinical and cost-effectiveness of different types of orthotic management of the knee in people with neuromuscular disease.

We are starting with the systematic review of previous evidence. However we have also started planning the qualitative study of the views of orthotic users as we will need to apply for research ethics approval before we can start recruiting people to take part. We have also started planning exactly what we need to ask healthcare professionals delivering orthotic services and how we can make sure we get a good response rate to the survey. So please feel free to tell anyone relevant about this blog and our research project and help us build up a network.

Blogger: Catriona McDaid

Getting the perspective of patients

A priority for the OKIS project is to recruit two people to bring the perspective of patients to our Project Advisory Group. We have prepared an advert and a description of the role to send out to relevant patient organisations.

In summary:

• The role of the Project Advisory Group is to make sure that the research takes into consideration the views of patients and healthcare professionals and to ensure the research is of a high standard.
• The project will last 12 months, starting on 1st April 2014
• The group will meet three times over the year and we might also contact you occasionally in between meetings.
• We were seeking two people who:
• Have been offered an orthotic device to help with knee problems related to a neuromuscular condition
• Are comfortable working as part of a small team
• Communicate effectively and are willing to share their opinions at a meeting and listen to the views of others
• Are able to attend meetings at the University of York– so live within easy travelling distance of York

The people who express an interest in joining the project will take part in a brief telephone interview and the successful applicants appointed. We will be offering an induction session and mentor from the team should they wish some support in their role.