OKIS: What our research found!

Our research report has been peer reviewed, edited and published by our funders the NIHR HTA programme, so we can now share our findings with you. This blog is to give you a flavour of what we did, what we found and our recommendations going forward. Links to the full report and a variety of summaries catering for every level of interest are provided at the end.

What were the views of users of orthotic devices?

We talked directly to users of orthoses for knee instability resulting from a neuromuscular disease (NMD) or central nervous system (CNS) condition. We wanted to know what makes a device acceptable, how much they used their devices and what influences use. In-depth interviews with a diverse range of users revealed that orthotic devices are vital for users to live as normal a life as possible, engage in daily activities and have independence for as long as possible. Malfunctions of a device can therefore have serious consequences for the wearer. To ensure devices are used and not left lying in cupboards, they need to be effective, reliable, comfortable and durable. Many of those interviewed were frustrated with what they saw as deficiencies in service provision relating to appointment and administrative systems and referral pathways.

It’s important for us as researchers to make sure that when we carry out research into whether something works, that what we take as an indication of success or failure (the outcome) can be measured, but also is relevant and important to patients. In our interviews with users of orthotics we got consistent feedback that a reduction in pain, falls or trips, with improved balance and stability were of primary importance. These outcomes tended not to be measured in the research we found assessing the effectiveness of devices. They were also only seen to a limited extent in the responses from health professionals to our survey of practice.

We recommend that future research should include more qualitative studies to build on our work, and investigate in further depth some of the issues raised in our interviews with users of orthotic devices.

What did we learn from the healthcare professionals?

Orthotists, physiotherapists and rehabilitation medicine physicians were among the 238 healthcare professionals who responded to our on-line survey. They told us they prescribe a range of orthoses for knee instability related to NMD or CNS conditions. These included knee-ankle-foot orthoses (KAFOs) ankle-foot-orthoses (AFOs) and knee braces; with shoe adaptations and insoles also prescribed by many. About half the devices prescribed or fitted were reported to be custom-made.

The majority of healthcare professionals thought comfort and confidence in mobility were extremely important outcomes from treatment, matching in part the priorities for patients. When it came to outcomes for assessing the effectiveness of the devices provided, about a quarter of the health professionals said that no formal outcome measure was used. No single outcome measure was used by the rest and only a third said they formally assessed patient satisfaction. The good news is that having identified this gap in service delivery, our interview work with users of orthotic devices can help provide an understanding of what a core set of patient reported outcome measures in the clinical setting should look like. We recommend development of a core outcome measure set, and that reduction in pain, falls and trips, improved balance and stability as well as participation in paid employment, outdoor activities (such as gardening), family visits and social events should be considered. This should ensure a more patient relevant assessment of the impact of any change in device or management strategy on individual patients.

We also suggest that future research should explore different models of delivery of orthotic service for people with NMD and CNS conditions to identify best practice in terms of greatest benefit to patients and value for money.

What does the existing evidence tell us?

We took a systematic look at the existing evidence to assess how well the various orthotic options for knee instability actually help patients in every-day life. Unfortunately the few studies we found had small numbers of participants, were generally poorly reported and all were at risk of bias, meaning the findings cannot be relied upon. Disappointingly, we also found that the studies we did identify, assessed mostly mechanical outcomes such as gait analysis and energy consumption. While these are important aspects of assessing a device, they were not supplemented by assessment of outcomes important to patients such as daily activities. Given all of this, it wouldn’t be right for us to draw any specific conclusions about individual devices. What we have been able to do is make specific recommendations for future research in this area.

Research on the effectiveness of devices for knee instability in NMD and CNS conditions is needed using outcome measures relevant to patients’ everyday lives. Given the challenges we have identified in our work, future trials should start with a feasibility study. Because of the relative rarity of some NMD and CNS conditions and the personalised nature of the orthotic device or devices needed, a national registry may be a more useful approach than trials.

Where can you read more?

This blog should give you a flavour of what we did, what we found and the recommendations we have made. Depending on how much detail you would like, there are various options for reading more in the NIHR Journals Library:

The full report is 296 pages long and obviously includes full details of the background, methods, results and implications. A plain English summary provides an overview in lay language in just 249 words; or there is a scientific summary in 2,730 words.

A four page short report can be downloaded from here: Orthotics for Knee Instability We also have a limited number of paper copies available: to receive one, send your name and address to kneeorthotics@gmail.com

Did our engagement strategy work?

We would really like to know your thoughts on our blog for this project; what you have found useful, interesting, helpful, and what was missing or unhelpful. Please post a comment in response to this blog or email me, Alison Booth, at kneeorthotics@gmail.com

Pulling everything together

We are now at the stage of reviewing the evidence and information collected in the various elements of this project and starting to interpret the findings. To assist in this process we are tapping into health professional and research methods expertise from the project Advisory Group.

At our third Advisory Group meeting the research team gave an update on each of the elements of this project. The systematic review has been completed and the discussion and recommendations sections are being compiled. The survey of health professionals has closed and the responses collated for analysis. The third element, the qualitative study of patients’ perspectives is almost complete with just a few more interviews to be held. In the meantime we have started coding and the data emerging are looking very informative in relation to the questions we set out to answer.

Meeting with the Advisory Group was a chance for the research team to ask the health professionals for their thoughts and advice. For example there was a discussion about how often patients are reviewed to assess how they are managing with an orthotic, a question in the healthcare professional survey. The Advisory Group suggested comparing this information with BAPO guidance on frequency of patient review. Some of the survey respondents dealt with in-patient groups and after discussion with the Advisory Group we will try to separate out the in-patient and out-patient results.

The next stage now is to complete the analysis of the qualitative study and write the full report for submission to our funders, the NIHR HTA programme. Once they received the report they will check it and send it out to external peer review. We will be asked to address any comments from the peer reviewers and then the report will be published in the NIHR journal library. We will also be disseminating the findings in a variety of other ways – which we will tell you about in another blog.

Blogger: Catriona McDaid

Survey of health professionals: analysis begins.

Our survey of health professionals is now closed and we are starting on the analysis of the data collected in the online questionnaire. This will involve a relatively straightforward descriptions of the data collected for presentation in the final report. For example, we will look at how many members of the different health professions filled in our questionnaire; how many of them see patients with neuromuscular diseases such as polio; what the different ways of working, are such as working in stand-alone clinics or as part of a multidisciplinary facility. This may sound a bit basic, but currently this information is unknown. We hope to provide a much better picture of what and how orthotic services are delivered in the UK. 

Now we have closed the survey so we can download all the responses collected into a spreadsheet. Each response has a unique identifier so responses to each question by each individual can be tracked without being able to identify the person themselves. We will then ‘code’ the responses so the statistical package can perform the analysis. This simply means all the answer choices have a number assigned to them. For example, the answers to the question “What is your gender?” would be coded as 1 for male and 2 for female. We will use simple statistics such as means (the average value) and modes (the more frequent value); and frequency tables and cross-tabulations to present our results in the final report. For the free text responses we will be using qualitative analysis techniques to identify all the issues raised and will present these findings in a descriptive narrative.

Many thanks again to all our respondents, your time is much appreciated and will form an important part of the findings of our project.

Blogger: Joanne O’Connor

Survey of healthcare professionals: one week left!

Our survey of healthcare professionals (HCPs) who are involved in providing orthotic devices for knee instability in people with neuromuscular disease or central nervous system conditions is well underway. The aim of the survey is to identify the types of orthotic devices currently being used by the NHS in the management of instability of the knee related to neuromuscular and CNS conditions. We also want to find out about patient pathways after referral for an orthotic device and what healthcare professionals think are important outcomes for patients.

The link to the online survey has been sent out through three professional organisations, (ACPIN, BAPO and BSRM) and many thanks to all those who have already responded. The survey will be closed on Monday 2^nd February so health professionals working in this area still have time to contribute to this under researched area. For an invitation to take part, or you have any other queries please get in touch with us at kneeorthotics@gmail.com

Blogger: Joanne O’Connor

Orthotics provision: Survey of healthcare professionals launched

We have now opened our survey of healthcare professionals (HCPs) who are involved in providing orthotic devices for knee instability in people with neuromuscular disease or central nervous system conditions. The link to the online survey is being sent out through three professional organisations:

• Association of Chartered Physiotherapists Interested in Neurology (ACPIN)
• British Association of Prosthetists and Orthotists (BAPO)
• British Society for Rehabilitation Medicine (BSRM)

If you are a health professional working in this area and have not received an invitiation to take part, or you have any other queries please get in touch with us at kneeorthotics@gmail.com

Blogger: Catriona McDaid

Orthotics services: who is delivering what?

One of the aims of our research is to identify the types of orthotic devices currently being used by the NHS in the management of instability of the knee related to neuromuscular and CNS conditions. We also want to find the answers to other questions such as what are the pathways that patients have to follow once they are referred for an orthotic device. By finding out what resources are required to assess and fit patients with a device, we should then be able to estimate the costs for providing services. We are also interested in what healthcare professionals think are important outcomes for patients and how those outcomes should be measured. This will complement the information we are gathering from patients.

To collect the information we need, we are carrying out a survey of orthotists, physiotherapists and rehabilitation medicine clinicians who are involved in the provision of orthotic devices for patients with knee instability. The content of the questionnaire has been informed by discussions with healthcare professionals in a focus group, one-to-one interviews and discussions with our project Steering Group, which includes members of the different professional groups involved in provision of orthotic devices.

Our problem is that there is so little research available on this topic that we need to ask for a lot of information to address these really important questions. We expect that the survey will take 20-30 minutes to complete which is longer than we would like, but we know that many health professionals working in this area are as keen as we are to address this knowledge gap.

The findings from the survey will be made widely available through healthcare professional networks, patient groups and research journals, as well as forming part of the formal report to our funders, the NIHR HTA programme.

Blogger: Catriona McDaid

AHPs: essential but under valued?

There are over 64,000 allied health professionals (AHPs) in the NHS from a range of different disciplines. AHPs play vital roles within a range of care teams, in hospitals and the community: often working with older people with more complex conditions. They are increasingly expanding their skills to include prescribing and consultant practitioner status, with patients able to self-refer.

QualityWatch, an independent scruitineer of quality in health and social care, has stated that, “despite the size and importance of the AHP workforce, AHPs are rarely the subject of major policy debates and there is a concern that their contribution to care is often hidden, overlooked or potentially undervalued.”  In their report Focus on: Allied Health professionals. Can we measure quality of care?, the authors say this is primarily because of a lack of consistent data nationally on the work undertaken by AHPs.

The report highlights the need to have systems in place that can capture information on all aspects of the quality of AHP care in all settings. The authors also identify the need for continued development of AHP research. The OKIS team are pleased to be helping progress this agenda for physiotherapists, prosthetists and orthotists involved with prescribing and fitting orthotic devices for knee instability. By identifying current pathways of care and finding/understanding where and why variation exists it should be possible to ensure the best use is made of AHP skills for the benefit of patients.

Focus on focus groups

To help us decide what questions to ask in the survey of health professionals, we are organising two focus groups of healthcare professionals.  A focus group is a form of qualitative research where a group of people are asked about their perceptions, beliefs or opinions on a particular topic. For our focus groups we have invited orthotists, physiotherapists, rehabilitation consultants and gait analysts to come along and give their opinions and personal experiences of treating patients with neuromuscular diseases (NMD) and/or central nervous system (CNS) disorders.

For these groups we developed a topic guide, a predefined list of questions to promote discussion. Participants were asked to give their experience of the care pathways of patients with NMD or CNS disorders with knee instability, what types of orthotic devices they prescribe and what outcomes they perceive as most important. These questions are put to the group who have been asked to take it in turns to give their answers so everyone gets a chance to speak. Because of the group setting, participants can build on answers given by others, and agree with or challenge what is being said. Often one person’s comments can trigger ideas, recollections or thoughts in another person – one of the values of the focus group setting.

In order to undertake these focus groups, I submitted an application to a research ethics committee, which included the topic guide and other documents detailing what we planned to do, the recruitment process and how we would obtain written consent from participants. This research ethics process is described in Dorothy’s post: Gaining research ethics approval.

Focus groups are intended to allow an informal environment and relaxed conversation, in contrast to a more formal one to one interview. Given the conversational style of focus groups, the optimal number of participants is about 6-8 people. This gives everyone a chance to speak with ease without having to elaborate too much or be too concise.

The research ethics committee approved our submission in June and the first focus group has now taken place. As hoped, it brought up some interesting points, such as the use of patient vignettes (short stories), to help focus the questions in the survey of health professionals.

Blogger: Joanne O’Connor

Is everyone able to give informed consent to take part in a research study?

This is an interesting question which raises a number of issues. For consent to be considered both legal and ethical, it must be: given by a person with capacity; voluntarily given, with no undue influence; given by someone who has been adequately informed; and a fair choice.

What is capacity?

Capacity refers to the everyday ability that individuals possess to make decisions or to take actions that affect them, from simple decisions such as what to have for breakfast to far-reaching decisions about serious medical treatment or financial affairs. A person lacks capacity if he or she is unable to make or communicate a decision about a particular matter because of an impairment of, or a disturbance in, the mind or the brain. This may be the result of a variety of conditions, including dementia, mental illness, learning disability, brain damage, intoxication or any other condition causing confusion, drowsiness or loss of consciousness (e.g. concussion, stroke, heart attack, epileptic fit, serious accident, delirium).

Vulnerable groups

Some groups of people are regarded as ‘vulnerable’ in terms of giving informed consent to take part in research studies. Children (under 16 years of age), people with learning and difficulties and others who may lack capacity to give informed consent (such as people with dementia or enduring and severe mental health problems, or those who are unconscious) fall into this category. The interests of these groups of people are protected in a variety of ways, including by acts of legislation, an important one being The Mental Capacity Act (2005).

Lack of capacity to give consent

The Mental Capacity Act (2005) provides a comprehensive framework for decision making on behalf of adults aged 16 and over who are unable to make decisions for themselves, i.e. they lack capacity. The Act applies to all decisions taken on behalf of people who permanently or temporarily lack capacity to make such decisions themselves, including decisions to include such people in research. All researchers working with research participants who lack, or may lack, capacity need to be aware of its underlying principles and the provisions relating to research. The Act is accompanied by a statutory Code of Practice providing guidance on how it should be used. Researchers and others making decisions involving people lacking capacity have a legal duty to have regard to the guidance in the Code of Practice. The Act applies in England and Wales only.

Children and young people

There are many ethical and legal issues to consider when involving children and young people in research, depending on the type of study being carried out (for example, whether or not it is a clinical trial of a medicinal product (CTIMP), and whereabouts in the UK the study is taking place. In particular, it is important to understand the key concepts of consent and assent.

Consent is a legally defined decision given by someone who is competent, who has been adequately informed (and has adequate understanding), and who is free from undue influence enabling them to make a voluntary decision. The person can provide consent, provided they are competent. Otherwise someone who is empowered by law can provide it (e.g. a parent in the case of children). A child who is not capable of giving consent alone can still be involved in the decision making process with others who are able in law to provide consent. There is no legislative statute in England, Wales or Northern Ireland governing a child’s right to consent to take part in studies other than CTIMPs. Common law presumes that young people aged between 16 and 18 years old are usually competent to give consent to treatment.

Assent is difficult to define and is used in diverse ways e.g. compliance by a child as young as three years old, to active agreement of a young teenager. Assent is agreement given by a child/young person or others who are not legally empowered to give consent. It is important to provide children/young people with information that matches their capacity when seeking assent.

Gillick competence

Case law suggests that if a young person has sufficient understanding and intelligence to fully understand what is proposed, and can use and weigh this information in reaching a decision, they are considered ‘Gillick competent’ and can give consent to treatment. ‘Gillick competence’ is a term originating in medical law to decide whether a child (16 years or younger) is able to consent to his or her own medical treatment, without the need for parental permission or knowledge. The standard is based on a decision of the House of Lords in the case Gillick versus West Norfolk and Wisbech Area Health Authority (1985). The case is binding in England and Wales, and has been approved in Australia, Canada and New Zealand. When a young person is believed to be competent, consent from those with parental responsibility is not legally necessary, although the involvement of parents in decision making is encouraged in most circumstances.

In the absence of law relating specifically to research, it is commonly assumed that the principle of ‘Gillick competence’ can be applied not only to consent for treatment but also to consent for research.

As you can see from the above, researchers hoping to gain consent from individuals to take part in a study must give careful consideration to a range of sometimes complex ethical and legal issues.

Blogger: Dorothy McCaughan

Designing the survey

One of elements of this project is to conduct a survey of healthcare professionals. This means that we will write a list of questions that we want the relevant health professionals to answer. The survey will ask questions about topics such as how patients get referred to them, how often they see patients and how they would judge if the orthotic is working for the patient.

Surveys can be conducted over the phone, face to face, be sent by post or electronically via the internet. We have chosen to use the internet and create a web based survey because it is a fast, simple and cost efficient way of reaching large numbers of people. We believe that there may be differences across the country in terms of what services are available to patients and we want to make sure that we find out what these differences are. By using a web survey, we can hopefully reach a range of health professionals all over the country.

The same questions are sent to everyone we want to join in the survey. The questions are formulated to try and make sure that everyone will understand them in the same way so the answers are consistent and provide the information we want to collect. This is the big challenge for any survey. To help us make sure we’ve got it right we will test the survey on a small number of health professionals. This will tell us what it’s like for participants to complete, including how long it takes, and allow us to make sure the questions are being answered in the way we anticipate.

The results of the survey will be the collective answers that we get. They should give us an idea of what orthotic services are provided to patients and how much NHS resources are used to provide these services. This will help us to estimate the cost of these services to the NHS.

Blogger: Joanne O’Connor

Checking we’re on the right course

Wednesday saw our first full Advisory Group meeting, where everyone involved could sit down, check on progress and discuss the next stages of the project. Logistically getting everyone together is never easy though we have found that at least initially it’s good to meet face to face so we can get to know each other. After that it’s easier to make the most of technology and have people join meetings with video conferencing.  As the project runs over 12 months there are only three such meetings planned and budgeted for, so it’s really important that we make the most of the time.

Plans for raising awareness of the project via this blog were agreed, with everyone getting involved in cascading links via their networks. These include key professional bodies, charities and organisations that provide support to people with neuromuscular diseases and central nervous system disorders.

Refinements were made to the details of the studies that will be included in the systematic review.  For example we had originally intended to only include studies with 20 participants or more. But as our clinical experts advised that few if any will include this many participants, we have removed the limit. This means we will capture as much of the evidence as possible in this under researched area.

We discussed the focus groups that will inform the survey of health professionals. Joanne our health economist presented the draft topic guide to the group. Through discussion she was able to add in some additional questions, for example, it will be important to find out whether patients are issued with a spare device or different devices at the same time.

We also discussed the best way of identifying patients to take part in interviews and the issues in recruiting from clinics and non-NHS settings. The routes we use will have implications for gaining both research ethics and research governance approvals, which can be lengthy processes. In order to make sure we include as many patients with a range of conditions we will be looking at using a variety of recruitment approaches so are exploring all the options further.

The structured discussion and resulting advice from group members was invaluable. The ‘Advisory Group’ lived up to its name and function: from offers of practical assistance in recruiting, to highlighting issues that might have been missed, to monitoring project progress.

Everyone agreed that it had been an interesting and positive meeting and certainly the advice from the health professionals was really helpful in informing the next stages. Now all we need to do is follow up on the agreed action points!

Blogger: Alison Booth 

What makes a review systematic?

The first of the three elements of the OKIS project is to carry out a systematic review of the existing evidence on orthotics for knee instability in adults. But what exactly is a systematic review? The short answer is that it’s a rigorous way of identifying, evaluating and summarising all the relevant research papers on a given topic. So far so good, but what makes a review ‘systematic’? It’s because it follows a strict protocol or plan where the research methods are set out in advance and are stuck to throughout the review.

The first step of a systematic review is to specify the research question in terms of the study participants, the intervention, the outcomes and the types of study to be included. This is essential information in the protocol. We then conduct a thorough search of electronic databases and other sources for all the relevant literature to answer the research question. We extract data from the relevant studies and assess their quality and finally synthesise the results. We’ll post information about what’s involved and what we find at each of these stages as we reach them.

At the end of this systematic review we should have an unbiased assessment of the existing evidence on the effectiveness of orthotic devices for knee instability. This will, we anticipate, show us what best practice is and where further research is needed to improve knowledge in this area.

Over the course of these blogs we’ll tell you more about the systematic review process and keep you up to date with how we’re getting on. I have conducted reviews on many topics but I’m particularly excited about this one as the results of it will be combined with the views and experiences of health professionals fitting orthotics and the views and experiences of those who have been fitted with an orthotic device for knee instability. This should make sure the project overall provides robust information useful to practitioners, patients and those commissioning and providing services.

Blogger: Debra Fayter

P.S. If you're keen, you can see the detailed protocol for this project on our funders' website: http://www.nets.nihr.ac.uk/projects/hta/133002 

And we've started!

It was great to get the project started - on the auspicious day of 1st April(!). Although it is the beginning it has felt like quite a journey getting here. The project was commissioned by the NIHR HTA programme back in Spring 2013 when they advertised an outline of the question they wanted addressed and invited proposals from researchers. But the HTA will have started exploring it as a possible topic for commissioning research on well before then. If you are interested in how HTA prioritises topics for research you can find out more information by following the links on the HTA Programme website.  Members of the public, health care professionals and policy makers can make suggestions to NIHR about any ideas they would like them to consider for funding. You can find details of how they identify research questions and access the on line suggestion form at http://www.nets.nihr.ac.uk/identifying-research.

The overall questions to be addressed by this research project are: what orthotic devices are currently in use in the NHS for knee instability for people with neuromuscular conditions, what are the costs involved and what further research is needed. An important aspect of our work is to find out what the most important issues are for patients. Our research project is intended as preparatory work to inform a different piece of research sometime in the future which will look at the clinical and cost-effectiveness of different types of orthotic management of the knee in people with neuromuscular disease.

We are starting with the systematic review of previous evidence. However we have also started planning the qualitative study of the views of orthotic users as we will need to apply for research ethics approval before we can start recruiting people to take part. We have also started planning exactly what we need to ask healthcare professionals delivering orthotic services and how we can make sure we get a good response rate to the survey. So please feel free to tell anyone relevant about this blog and our research project and help us build up a network.

Blogger: Catriona McDaid