Advertisement

Exploring the implications of different approaches to estimate centre-level adherence using objective adherence data in an adult cystic fibrosis centre – a retrospective observational study

  • Zhe Hui Hoo
    Correspondence
    Corresponding author: Zhe Hui Hoo, Room 1.03, Innovation Centre, 217 Portobello, Sheffield S1 4DP, UK.
    Affiliations
    School of Health and Related Research (ScHARR), University of Sheffield, Room 1.03, Innovation Centre, 217 Portobello, S1 4DP Sheffield, UK

    Sheffield Adult CF Centre, Northern General Hospital, Brearley Outpatient, Northern General Hospital, Herries Road, S5 7AU Sheffield, UK
    Search for articles by this author
  • Rachael Curley
    Affiliations
    School of Health and Related Research (ScHARR), University of Sheffield, Room 1.03, Innovation Centre, 217 Portobello, S1 4DP Sheffield, UK

    Sheffield Adult CF Centre, Northern General Hospital, Brearley Outpatient, Northern General Hospital, Herries Road, S5 7AU Sheffield, UK
    Search for articles by this author
  • Stephen J Walters
    Affiliations
    School of Health and Related Research (ScHARR), University of Sheffield, Room 1.03, Innovation Centre, 217 Portobello, S1 4DP Sheffield, UK
    Search for articles by this author
  • Michael J Campbell
    Affiliations
    School of Health and Related Research (ScHARR), University of Sheffield, Room 1.03, Innovation Centre, 217 Portobello, S1 4DP Sheffield, UK
    Search for articles by this author
  • Martin J Wildman
    Affiliations
    School of Health and Related Research (ScHARR), University of Sheffield, Room 1.03, Innovation Centre, 217 Portobello, S1 4DP Sheffield, UK

    Sheffield Adult CF Centre, Northern General Hospital, Brearley Outpatient, Northern General Hospital, Herries Road, S5 7AU Sheffield, UK
    Search for articles by this author
Open ArchivePublished:October 31, 2019DOI:https://doi.org/10.1016/j.jcf.2019.10.008

      Highlights

      • Centre-level adherence measurement can allow system-wide collaborative learning.
      • Current approaches to calculating adherence may over-estimate effective adherence.
      • Accurate understanding of adherence requires precisely defined prescriptions.
      • Quantification of missing data is crucial to understand centre-level adherence.
      • We propose a standardised method of calculating centre-level adherence.

      Abstract

      Background

      Accurate centre-level medication adherence measurement allows identification of highly performing CF centres, drives shared learning and informs quality improvement. Self-reported adherence is unreliable but data-logging nebulisers can capture objective data. However, adherence levels in current literature are limited by the use of agreed prescriptions and convenience sampling. In this single-centre retrospective study, we quantified the differences in centre-level adherence with different methods of calculating adherence (unadjusted vs normative adherence) and different data sampling frames (convenience sampling vs including difficult to obtain data).

      Methods

      Adherence data were objectively captured using I-neb® from 2013-2016 in Sheffield Adult CF Centre. Adults on non data-logging devices, on ivacaftor or with previous lung transplantation were excluded. Adherence was calculated based on agreed regimen (‘unadjusted adherence’) or minimum required regimen (‘normative adherence’). I-nebs® not brought to clinic were downloaded during home visits. Adults not on any inhaled therapy but with chronic Pseudomonas aeruginosa infection were included by counting their adherence as “0”.

      Results

      Of the 131 included adults, 126 provided I-neb® data. Calculating unadjusted adherence from I-nebs® brought to clinics resulted in the highest centre-level adherence (median 41.8% in 2013). Median adherence reduced after sequentially accounting for minimum required regimen (40.0% in 2013), I-nebs® not brought to clinics (32.9% in 2013) and adults not on any inhaled therapy (31.0% in 2013).

      Conclusions

      Different approaches of calculating adherence produced different adherence levels. Adherence levels based only on agreed regimen among adults who readily brought their nebulisers to clinics can over-estimate the effective adherence of CF centres.

      Graphical abstract

      Keywords

      1. Introduction

      Cystic fibrosis is a life-limiting genetic condition in which mortality is predominantly due to progressive lung damage driven by recurrent pulmonary exacerbations [
      • Elborn J.S.
      Cystic fibrosis.
      ]. Inhaled therapies e.g. antibiotics and mucolytics are efficacious in preventing exacerbations and maintaining lung health [
      • Yang C.
      • Montgomery M.
      Dornase alfa for cystic fibrosis.
      ]; but real-world median medication adherence of 35-50% among adults with CF is low [
      • Daniels T.
      • Goodacre L.
      • Sutton C.
      • Pollard K.
      • Conway S.
      • Peckham D.
      Accurate assessment of adherence: self-report and clinician report vs electronic monitoring of nebulizers.
      ,
      • Narayanan S.
      • Mainz J.G.
      • Gala S.
      • Tabori H.
      • Grossoehme D.
      Adherence to therapies in cystic fibrosis: a targeted literature review.
      ] especially in comparison to adherence of 80-100% in clinical trials [
      • Pugatsch T.
      • Shoseyov D.
      • Cohen-Cymberknoh M.
      • et al.
      Adherence pattern to study drugs in clinical trials by patients with cystic fibrosis.
      ]. The CF community is therefore unlikely to derive the optimal health benefits from inhaled therapies that were observed in clinical trials. Medication possession ratio of ∼65% [
      • White H.
      • Shaw N.
      • Denman S.
      • Pollard K.
      • Wynne S.
      • Peckham D.G.
      Variation in lung function as a marker of adherence to oral and inhaled medication in cystic fibrosis.
      ] in the presence of objectively measured adherence of ∼35% [
      • Daniels T.
      • Goodacre L.
      • Sutton C.
      • Pollard K.
      • Conway S.
      • Peckham D.
      Accurate assessment of adherence: self-report and clinician report vs electronic monitoring of nebulizers.
      ] also highlights the possibility of significant waste. There are currently no effective adherence interventions for routine CF clinical use and the development of such interventions is a research priority [
      • Rowbotham N.J.
      • Smith S.
      • Leighton P.A.
      • et al.
      The top 10 research priorities in cystic fibrosis developed by a partnership between people with CF and healthcare providers.
      ].
      Various quality improvement (QI) initiatives have transformed the delivery of CF healthcare [
      • Rault G.
      • Lombrail P.
      Strategies for care quality improvement in Cystic Fibrosis.
      ]. The expertise of CF community in QI can potentially be harnessed to support adherence by creating a learning health system [
      • Friedman C.
      • Rubin J.
      • Brown J.
      • et al.
      Toward a science of learning systems: a research agenda for the high-functioning Learning Health System.
      ] in which objectively measured adherence using data-logging nebulisers such as I-neb® and eTrack® informs benchmarking and shared learning [
      • Hind D.
      • Drabble S.J.
      • Arden M.A.
      • et al.
      Supporting medication adherence for adults with cystic fibrosis: a randomised feasibility study.
      ]. Benchmarking allows centre comparisons to highlight variation in care and the identification of highly performing centres to learn from [
      • Schechter M.S.
      Benchmarking to improve the quality of cystic fibrosis care.
      ]. Though data-logging devices are more costly than standard nebulisers, a health economics evaluation suggest these devices will more than justify their costs if they are used effectively to provide feedback and improve adherence [
      • Tappenden P.
      • Sadler S.
      • Wildman M.
      An early health economic analysis of the potential cost effectiveness of an adherence intervention to improve outcomes for patients with cystic fibrosis.
      ]. In the UK, NHS England specialised commissioning have agreed to fund objective adherence data capture across most adult English CF centres via the national Commissioning for Quality and Innovation (CQUIN) programme. A learning health system among these centres has been established (ISRCTN14464661) and is currently recruiting adults with CF in 19 out of the 28 UK centres. Therefore, understanding how to use objective adherence data as a quality indicator is an important and timely issue.
      Centre comparisons using adherence as a quality indicator rely on centres being confident that adherence measures are equally standardised, thus comparable between centres so that they accurately reflect the effective use of inhaled therapies within a centre. Though objective adherence data are accurate and reliable, robust comparison between centres also depends on how data are processed, analysed and reported. The ABC taxonomy for medication adherence recommends that adherence definitions should be clinically relevant and account for deviation that adversely influence the intended effects of medication regimen [
      • Vrijens B.
      • De Geest S.
      • Hughes D.A.
      • et al.
      A new taxonomy for describing and defining adherence to medications.
      ]. The Respiratory Effectiveness Group emphasises the importance of considering “successful medication adherence” holistically so that adherence levels reflect treatment effectiveness [
      • Braido F.
      • Chrystyn H.
      • Baiardini I.
      • et al.
      "Trying, but failing" – the role of inhaler technique and mode of delivery in respiratory medication adherence.
      ]. In general, inadequate prescription of efficacious treatment (“therapeutic inertia”) is the second biggest cause of ineffective treatment after low adherence [
      • Allen J.D.
      • Curtiss F.R.
      • Fairman K.A.
      Nonadherence, clinical inertia, or therapeutic inertia?.
      ]. Therapeutic inertia is pertinent in CF because “treatment burden” is widely perceived to be a major barrier to adherence [
      • Narayanan S.
      • Mainz J.G.
      • Gala S.
      • Tabori H.
      • Grossoehme D.
      Adherence to therapies in cystic fibrosis: a targeted literature review.
      ]; with the result that treatment rationalisation or non-initiation of treatment can be one strategy that is adopted in the hope that a lowered treatment burden might promote greater engagement [
      • Geller D.E.
      • Madge S.
      Technological and behavioral strategies to reduce treatment burden and improve adherence to inhaled antibiotics in cystic fibrosis.
      ]. Another aspect of therapeutic inertia is appropriate inaction; such as treatment modifications due to medication intolerance, costs of medication, therapy being ineffective in the past and patient values/preference. Regardless of the reason(s) for excluding efficacious treatment(s) from an agreed prescription, there is the potential to reduce treatment effectiveness whilst inflating calculated adherence level unless there is standardisation against a normative adherence metric. For example, not initiating long-term inhaled antibiotics in a person with chronic Pseudomonas aeruginosa infection due to medication cost will result in a lowered denominator for calculating percent adherence and the resultant percent adherence is unlikely to reflect the effective utilisation of inhaled therapies (i.e. the denominator will be one daily dose instead of three daily doses assuming the person was already using dornase alfa and a twice daily inhaled antibiotic was not initiated). Adherence measures that incorporate treatment effectiveness are largely neglected in extant CF literature which tends to only report agreed adherence. Whilst agreed adherence is a valuable patient centred measure, its personalised nature undermines the standardisation required for centre comparison.
      Another limitation in extant literature is the use of convenience samples that ignores the group of people who are unwilling to share their data or who are not using any medication. People with the lowest adherence levels may be least willing to share their data [
      • Morton R.
      I'm sorry, doctor, my dog ate my inhaler” – the trials and tribulations of a clinical researcher.
      ]; hence centre comparisons would be confounded by inconsistent sampling frames and differential missing data unless the entire cohort is accounted for, such that the denominator used to calculate centre-level adherence is defined by all the appropriate patients making up the population of interest. In many studies, difficult-to-reach patients would simply be missing and the missingness might well be unavoidably invisible for long-term conditions such as asthma. Yet CF is unique in that almost all people with CF in the UK are identified within the CF registry [

      Cystic Fibrosis Trust. Standards for the clinical care of children and adults with cystic fibrosis in the UK. https://www.cysticfibrosis.org.uk/the-work-we-do/clinical-care/consensus-documents. Date last updated: 1 December 2011. Date last accessed: 3 May 2019.

      ]. Registry data can identify the number of people who should actually feature as the centre denominator in centre-level adherence measurement and quantify the number of missing people in adherence levels calculated by each centre.
      Understanding the properties of different approaches to calculate centre-level adherence data is an important first step towards robust comparison using adherence as a quality indicator. We therefore set out to quantify the differences in centre-level adherence with different methods of calculating adherence and different data sampling frames.

      2. Methods

      This single-centre retrospective analysis included all eligible adults with CF in Sheffield aged ≥16 years diagnosed with the UK CF Trust criteria. Adults with lung transplantation or on ivacaftor were excluded because this dataset was originally compiled to evaluate the impact of adherence on health outcomes, and both treatments have transformative effects on lung health [
      • Hoo Z.H.
      • Bramley N.R.
      • Curley R.
      • et al.
      Intravenous antibiotic use and exacerbation events in an adult cystic fibrosis centre: a prospective observational study.
      ]. Adults using inhaled therapies via devices without data-logging capabilities were also excluded because only objective adherence levels were evaluated (the clinical characteristics of these adults were tabulated in Appendix A). However, adults with chronic Pseudomonas aeruginosa infection not on any CF inhaled therapies were included since they should have been on inhaled therapies. This study was approved by NHS Health Research Authority (IRAS number 210313).
      Inhaled therapy prescriptions and relevant demographic data (age, gender, genotype status [
      • Castellani C.
      • Cuppens H.
      • Macek Jr., M.
      • et al.
      Consensus on the use and interpretation of cystic fibrosis mutation analysis in clinical practice.
      ], pancreatic status, CF related diabetes, Pseudomonas aeruginosa status as defined by the Leeds criteria [
      • Lee T.W.
      • Brownlee K.G.
      • Conway S.P.
      • Denton M.
      • Littlewood J.M.
      Evaluation of a new definition for chronic Pseudomonas aeruginosa infection in cystic fibrosis patients.
      ], body mass index, %FEV1, intravenous antibiotics use) were extracted by two investigators independently reviewing paper notes and electronic records. Where extracted data differed, both investigators re-reviewed original data to arrive at a consensus. Nebuliser adherence data were downloaded from I-nebs®, which typically store 3,000-4,000 datapoints and provide date- and time-stamped data for every dose of nebulised medication [
      • Geller D.E.
      • Madge S.
      Technological and behavioral strategies to reduce treatment burden and improve adherence to inhaled antibiotics in cystic fibrosis.
      ].
      ‘Unadjusted adherence’ was calculated as the percentage of total nebulisers taken against the dose agreed between clinicians and adults with CF (i.e. the denominator is personalised rather than standardised). ‘Normative adherence’ was also calculated as a percentage, but included denominator adjustment (standardised to define the minimum required treatment regimen) according to a person's Pseudomonas aeruginosa status and numerator adjustments (capping daily maximum nebuliser use at 100%, accounting for doses taken after midnight and accounting for dose spacing for inhaled antibiotics) as previously described [
      • Hoo Z.H.
      • Curley R.
      • Campbell M.J.
      • Walters S.J.
      • Hind D.
      • Wildman M.J.
      Accurate reporting of adherence to inhaled therapies in adults with cystic fibrosis: methods to calculate “normative adherence”.
      ]. For example, those with chronic Pseudomonas aeruginosa infection should take at least a nebulised mucolytic and an antibiotic i.e. the denominator will be at least “3” (1x dornase alfa, 2x antibiotic). If that person only agreed to use dornase alfa once daily (i.e. 1 nebuliser/day), even if every dose was taken (giving 100% unadjusted adherence), the normative adherence would only be 33%. A detailed description for the calculation of normative adherence in a range of clinical scenarios is provided in Appendix B.
      For each adherence calculation method, centre-level adherence was determined as the median for all adults (i.e. adherence was calculated for every adult and the median for all adults was determined to avoid potential bias by differences in data duration between individuals). For each adherence calculation method, three sampling frames were applied to determine the centre-level adherence. First, adherence was calculated using only data from I-nebs® that were downloaded in each calendar year during clinical reviews, i.e. among adults who readily handed in their devices. Second, all available I-neb® data were used, i.e. including elusive I-nebs® that were “difficult to obtain” which had to be chased via repeated requests and home visits. This refers to I-nebs® that were not downloaded within the calendar year but data became retrospectively available when downloaded in subsequent year(s). These I-nebs® would have been a source of missing data without concerted efforts to retrieve them and would have remained missing in many settings. Therefore, the difficult to obtain I-nebs® were used to approximate the effect of ‘missing I-nebs® data’. Third, adherence levels also included adults with chronic Pseudomonas aeruginosa infection but not using any CF inhaled therapies throughout the calendar year by assigning their adherence as “0”.
      Analysis were performed using SPSS v25 (IBM Corp) and R v3.3.0 (www.r-project.org). Data for 2013, 2014, 2015 and 2016 were analysed separately to determine the consistency of any observations. Appropriate descriptive statistics were generated. Where relevant, between-group comparisons were performed using non-parametric tests [
      • Campbell M.J.
      • Gardner M.J.
      Calculating confidence intervals for some non-parametric analyses.
      ] (due to non-normal adherence data distribution and presence of outliers) whilst agreement between paired measurements were assessed with ‘difference vs average’ plots [
      • Bland J.M.
      • Altman D.G.
      Statistical methods for assessing agreement between two methods of clinical measurement.
      ]. P-value <0.05 was considered statistically significant. The sample size was pragmatic, and all available data were included in this analysis.

      3. Results

      This analysis included 131 adults, with 126 adults providing I-neb® data and five adults with chronic Pseudomonas aeruginosa infection but not on CF inhaled therapies. Year-by-year demographic data were stratified in Table 1 according to whether the adults used I-neb® or not. The number of adults with chronic Pseudomonas aeruginosa infection not on inhaled therapies was small. Around 1/3 of the adults in Sheffield used only non-data logging devices (31% in 2013, 28% in 2014, 30% in 2015 and 37% in 2016) and were excluded from this analysis. Results in Appendix A suggest that these adults shared broadly similar clinical characteristics compared to adults included in this analysis.
      Table 1Clinical characteristics of study subjects
      Complete clinical characteristics data were available for all study subjects.
      for 2013 to 2016
      Clinical characteristics2013201420152016
      Used I-neb® (n = 89)No nebuliser
      These are adults who should be on preventative inhaled therapies since they have chronic Pseudomonas aeruginosa infection but were not on any preventative inhaled therapies.
      (n = 4)
      Used I-neb® (n = 97)No nebuliser
      These are adults who should be on preventative inhaled therapies since they have chronic Pseudomonas aeruginosa infection but were not on any preventative inhaled therapies.
      (n = 3)
      Used I-neb® (n = 104)No nebuliser
      These are adults who should be on preventative inhaled therapies since they have chronic Pseudomonas aeruginosa infection but were not on any preventative inhaled therapies.
      (n = 3)
      Used I-neb® (n = 102)No nebuliser
      These are adults who should be on preventative inhaled therapies since they have chronic Pseudomonas aeruginosa infection but were not on any preventative inhaled therapies.
      (n = 0)
      Age in years, median (IQR)25 (19 – 30)28 (26 – 31)25 (19 – 31)29 (29 – 31)26 (20 – 32)43 (39 – 46)26 (19 – 32)N/A
      Female, n (%)37 (41.6)2 (50.0)39 (40.2)2 (66.7)43 (41.3)2 (66.7)42 (41.2)N/A
      Homozygous class I-III,
      Genotype status was defined by international consensus [20]. Homozygous class I-III mutations indicate ‘severe genotype’.
      n (%)
      80 (89.9)4 (100.0)88 (90.7)3 (100.0)91 (87.5)2 (66.7)91 (89.2)N/A
      Pancreatic insufficient, n (%)85 (95.5)4 (100.0)91 (93.8)3 (100.0)96 (92.3)2 (66.7)95 (93.1)N/A
      CF related diabetes, n (%)23 (25.8)1 (25.0)25 (25.8)2 (66.7)25 (24.0)1 (33.3)32 (31.4)N/A
      Chronic P. aeruginosa,
      The Leeds criteria were used to define Pseudomonas aeruginosa status [21].
      n (%)
      47 (52.8)4 (100.0)52 (53.6)3 (100.0)51 (49.0)3 (100.0)49 (48.0)N/A
      BMI, median (IQR)21.5 (19.7 – 24.3)9.9 (16.7 – 20.8)22.2 (20.1 – 24.3)19.4 (17.1 – 19.9)23.0 (20.7 – 24.9)30.2 (25.7 – 30.5)23.2 (20.6 – 25.4)N/A
      Best %FEV1,
      This represents the highest %FEV1 reading (calculated with GLI equations) in the calendar year period.
      median (IQR)
      75.9 (52.9 – 90.0)60.8 (20.4 – 92.0)74.0 (55.0 – 87.5)37.8 (25.8 – 64.5)76.0 (58.5 – 87.6)89.9 (85.1 – 92.7)76.4 (62.1 – 87.0)N/A
      IV antibiotic days, median (IQR)14 (0 – 41)63 (21 – 95)14 (0 – 31)28 (21 – 56)20 (2 – 36)21 (11 – 46)18 (0 – 42)N/A
      On inadequate prescription,
      A person with chronic Pseudomonas aeruginosa infection was deemed to be on inadequate prescription if he/she was not on at least once daily dose of mucolytic and at least twice daily doses of inhaled antibiotic (taking into account on/off long-term antibiotic regimens). In this group of people, denominator adjustment was required to calculate normative adherence. Everyone with chronic Pseudomonas aeruginosa infection not on any inhaled therapies was deemed to be on inadequate prescription.
      n (%)
      12 (13.5)4 (100.0)12 (12.4)3 (100.0)16 (15.4)3 (100.0)16 (15.7)N/A
      ϕ Complete clinical characteristics data were available for all study subjects.
      These are adults who should be on preventative inhaled therapies since they have chronic Pseudomonas aeruginosa infection but were not on any preventative inhaled therapies.
      Genotype status was defined by international consensus
      • Castellani C.
      • Cuppens H.
      • Macek Jr., M.
      • et al.
      Consensus on the use and interpretation of cystic fibrosis mutation analysis in clinical practice.
      . Homozygous class I-III mutations indicate ‘severe genotype’.
      The Leeds criteria were used to define Pseudomonas aeruginosa status
      • Lee T.W.
      • Brownlee K.G.
      • Conway S.P.
      • Denton M.
      • Littlewood J.M.
      Evaluation of a new definition for chronic Pseudomonas aeruginosa infection in cystic fibrosis patients.
      .
      § This represents the highest %FEV1 reading (calculated with GLI equations) in the calendar year period.
      A person with chronic Pseudomonas aeruginosa infection was deemed to be on inadequate prescription if he/she was not on at least once daily dose of mucolytic and at least twice daily doses of inhaled antibiotic (taking into account on/off long-term antibiotic regimens). In this group of people, denominator adjustment was required to calculate normative adherence. Everyone with chronic Pseudomonas aeruginosa infection not on any inhaled therapies was deemed to be on inadequate prescription.
      Although many of the unadjusted adherence values were similar to normative adherence, unadjusted values generally over-estimated adherence (see Figure 1 and Table 2). At a centre-level, the median paired differences were 2.6-5.1%. In those with low adherence (<5%), unadjusted and normative adherence differed little due to the floor effect of adherence (adherence level cannot be negative). However, in some adults with higher unadjusted adherence levels, the difference could be up to 40-100% especially in those with nebuliser over-use (unadjusted adherence >100%) because daily adherence was capped at 100% for normative adherence.
      Figure 1
      Figure 1‘Difference vs average’ plots for % normative adherence and % unadjusted adherence from 2013 to 2016 among adults who readily handed in their I-nebs®
      Table 2The impact of different data analysis and processing methods on adherence levels for 2013 to 2016
      Year2013201420152016
      Method 1: % unadjusted adherence, median (IQR)
      Sampling frame 1
      Including adults who readily handed in their I-neb®
      41.8 (25.8 – 70.0) n = 7452.6 (29.1 – 82.8) n = 8457.7 (28.4 – 85.5) n = 9459.1 (28.6 – 88.9) n = 94
      Sampling frame 2
      Including all adults using I-neb® (i.e. including I-neb® that were difficult to obtain)
      36.9 (19.1 – 64.7) n = 8944.9 (19.3 – 77.1) n = 9751.4 (23.8 – 80.7) n = 10452.8 (24.3 – 88.0) n = 102
      Sampling frame 3
      Including all adults using I-neb® and assigning the value of “0” for adults with chronic Pseudomonas aeruginosa infection but not using any CF inhaled therapies
      35.1 (16.9 – 64.3) n = 9344.7 (16.3 – 76.6) n = 10050.8 (22.0 – 80.2) n = 10752.8 (24.3 – 88.0) n = 102
      Method 2: % normative adherence, median (IQR)
      Sampling frame 1
      Including adults who readily handed in their I-neb®
      40.0 (20.4 – 67.2) n = 7445.2 (28.4 – 75.2) n = 8449.6 (23.1 – 80.1) n = 9453.4 (27.7 – 77.4) n = 94
      Sampling frame 2
      Including all adults using I-neb® (i.e. including I-neb® that were difficult to obtain)
      32.9 (16.4 – 59.9) n = 8941.0 (17.2 – 65.4) n = 9745.4 (19.4 – 73.8) n = 10450.8 (23.8 – 71.6) n = 102
      Sampling frame 3
      Including all adults using I-neb® and assigning the value of “0” for adults with chronic Pseudomonas aeruginosa infection but not using any CF inhaled therapies
      31.0 (15.1 – 58.1) n = 9339.7 (16.1 – 64.8) n = 10044.2 (18.8 – 69.6) n = 10750.8 (23.8 – 71.6) n = 102
      Including adults who readily handed in their I-neb®
      Including all adults using I-neb® (i.e. including I-neb® that were difficult to obtain)
      Including all adults using I-neb® and assigning the value of “0” for adults with chronic Pseudomonas aeruginosa infection but not using any CF inhaled therapies
      Despite the modest sample size, adherence levels for difficult to obtain I-nebs® (median normative adherence 8.4% in 2013, n=15; 9.8% in 2014, n=13; 6.1% in 2015, n=10; and 10.2% in 2016, n=8) were significantly lower than readily obtained I-nebs® (median normative adherence 40.0% in 2013, n=74; 45.2% in 2014, n=84; 49.6% in 2015, n=94; and 53.4% in 2016, n=94), Mann-Whitney p-value <0.001 for all four years. Thus measuring adherence using only readily obtained I-neb® over-estimated centre-level adherence. Since difficult to obtain I-nebs® typically had such low adherence levels, adopting the convention of assigning their adherence levels as “0” would only result in very slight under-estimation of centre-level adherence (the resultant median normative adherence would be 30.6% in 2013, 41.0% in 2014, 44.6% in 2015 and 50.8% in 2016). Despite the small number of adults with chronic Pseudomonas aeruginosa infection who were not on any CF inhaled therapy, centre-level median adherence fell further if they were included in the overall estimate of adherence (see Table 2).

      4. Discussion

      Once randomised controlled trials have demonstrated the efficacy of a treatment, it is reasonable to consider the proportion of eligible patients in a centre who are prescribed that treatment by clinicians and objectively captured adherence of patients to the prescription as an indicator of care quality. That is to say that once an RCT establishes a relationship between a process and an outcome, it is reasonable to use a process measure as an indicator of care quality. Our analysis empirically demonstrates that different approaches to processing and analysing objective data are likely to influence centre-level medication adherence. Medians are typically more robust to the impact of outliers [
      • Driscoll P.
      • Lecky F.
      • Crosby M.
      An introduction to everyday statistics – 2.
      ], yet there was sequential reduction in centre-level median adherence as the analysis methodology moved from unadjusted adherence among readily obtained I-nebs® to normative adherence which included difficult to obtain I-nebs® and adults not on inhaled therapies when they should. Therefore, the considerations of both treatment appropriateness (normative adherence) and missing data (sampled via difficult-to-reach I-nebs®) are important in accurately understanding objective adherence as a centre-level quality indicator.
      In many long-term conditions, centre performance based on objective adherence and clinician prescribing patterns can be difficult to interpret, as the best centres might appear to have the worst adherence if therapeutic inertia is least prevalent and if they retain some engagement with difficult-to-reach patients with the lowest medication adherence. An adolescent diabetic clinic where the most rebellious teenagers refuse to attend might have excellent glycosylated haemoglobin data; but that data is potentially misleading as an indicator of care quality without an understanding of the population that should have been reached. CF is unique in that the UK CF registry has data on almost all people with CF, thus the number of patients within a centre eligible for care (the centre denominator) is known, allowing CF to provide a unique setting for the study of intervention reach in long-term conditions.
      The high prevalence of missing data in studies investigating adherence has been reported previously [
      • Morton R.
      I'm sorry, doctor, my dog ate my inhaler” – the trials and tribulations of a clinical researcher.
      ]. Data are seldom missing at random and analyses in other long-term conditions also demonstrate the potential for missing data to introduce bias [
      • Shivasabesan G.
      • Mitra B.
      • O'Reilly G.M.
      Missing data in trauma registries: a systematic review.
      ]. In our analysis, we have chosen to account for people not on any CF inhaled therapies by assigning “0” as their adherence level. This adjustment may be crude, but people not using any CF inhaled therapies when they should (since long-term inhaled antibiotics are recommended for chronic Pseudomonas aeruginosa infection) do have zero adherence. We also justify our approach by exploring the data of people on data-logging nebulisers who did not readily share their data. If a centre was not engaged in chasing down these data and simply omit difficult-to-reach I-nebs® from the centre denominator, an inappropriately high estimate of centre performance would be obtained. Assigning “transiently missing” adherence levels as “0” for the calculation of centre-level adherence in real time would only result in very slight under-estimation since difficult-to-reach I-nebs® tended to have very low adherence levels.In addition to completely missing data, there is substantial variation in the prescription of inhaled therapies which is too large to be explained by just case-mix and chance. UK data showed an almost 3-fold difference (86.8% vs 30.2%) in the prescription of dornase alfa between the adult CF centres with highest and lowest use [

      Cystic Fibrosis Trust. Standards for the clinical care of children and adults with cystic fibrosis in the UK. https://www.cysticfibrosis.org.uk/the-work-we-do/clinical-care/consensus-documents. Date last updated: 1 December 2011. Date last accessed: 3 May 2019.

      ]. US data showed that only two-thirds of people with CF were prescribed the recommended inhaled therapies [
      • Glauser T.A.
      • Nevins P.H.
      • Williamson J.C.
      • et al.
      Adherence to the 2007 cystic fibrosis pulmonary guidelines: a national survey of CF care centers.
      ]. Since some of this variation reflects therapeutic inertia, it is important that treatment appropriateness is captured by standardisation according to patient characteristics via normative adherence.
      The extant CF literature tend to report unadjusted adherence levels from convenience samples without accounting for the appropriateness of treatment prescription or people with missing adherence data. This is likely to over-estimate centre-level adherence. In other words, although reported adherence levels among adults with CF in the literature are low at 35-50% [
      • Daniels T.
      • Goodacre L.
      • Sutton C.
      • Pollard K.
      • Conway S.
      • Peckham D.
      Accurate assessment of adherence: self-report and clinician report vs electronic monitoring of nebulizers.
      ,
      • Narayanan S.
      • Mainz J.G.
      • Gala S.
      • Tabori H.
      • Grossoehme D.
      Adherence to therapies in cystic fibrosis: a targeted literature review.
      ], the actual total cohort effective adherence levels are almost certainly even lower. This perspective is important in highlighting that the challenges of medication adherence may be even worse than published data suggest. With median centre-level objective adherence as low as 30%, centre comparisons and benchmarking within a learning health system that starts to drive improvement has the potential to make a major impact on the quality of care.
      Our findings have implications for benchmarking in justifying a standardised approach that allows objective adherence data to be used as a centre-level process measure suitable for quality measurement. The use of standardised process measure is important since using health outcomes as a quality measure for benchmarking in CF is particularly problematic because a relatively small UK CF population is spread across many centres. FEV1 is an important outcome measure in CF, but a recent sample size estimation suggests that 273 adults per centre are needed to detect a 5% FEV1 difference at the 5% statistical significance level [
      • Nightingale J.A.
      • Osmond C.
      Does current reporting of lung function by the UK cystic fibrosis registry allow a fair comparison of adult centres?.
      ]. Yet only 6/28 (21.4%) of all UK adult CF centres have ≥273 adults [

      Cystic Fibrosis Trust. Standards for the clinical care of children and adults with cystic fibrosis in the UK. https://www.cysticfibrosis.org.uk/the-work-we-do/clinical-care/consensus-documents. Date last updated: 1 December 2011. Date last accessed: 3 May 2019.

      ]. CF QI initiatives focusing on process measures that allow rapid feedback to prompt improvement and subsequent reassessment have been reported, [
      • Rault G.
      • Lombrail P.
      Strategies for care quality improvement in Cystic Fibrosis.
      ] but to date real time adherence data have not been exploited for this purpose. We hope that this study will lay the groundwork for such studies in the future. For any benchmarking exercises using adherence as a metric to be reliable, it is crucial to determine whether differences in the metric represents a genuine difference in effective adherence or whether it is merely an artefact of data issues (e.g. different prescribing practices between centres or differential missing data). The patient-centred unadjusted adherence measure based on personalised prescriptions reflecting individualised concordance is important when discussing adherence with individual patients but it is not a suitably standardised indicator for centre benchmarking. By using a standardised metric free from the vagaries of prescribing practices (normative adherence which is standardised in light of patients’ clinical characteristics) and missing data (standardised using registry data to define the centre denominator) to reflect treatment effectiveness, centres involved in the benchmarking exercise can be more confident that apples are being compared to apples.
      We acknowledge the uncertainty involved in deciding what inhaled therapies an adult with CF should be using based on their clinical characteristics. There are differing levels of evidence for inhaled therapies among people with differing lung disease severity and also for different treatment options. In our previous publication [
      • Hoo Z.H.
      • Curley R.
      • Campbell M.J.
      • Walters S.J.
      • Hind D.
      • Wildman M.J.
      Accurate reporting of adherence to inhaled therapies in adults with cystic fibrosis: methods to calculate “normative adherence”.
      ], we have taken the approach that perfect should not be the enemy of the good in attempting to specify an a priori method of processing adherence data which might be expected to ensure that a higher percentage adherence to the specified treatment regimen is associated with greater treatment effectiveness. The approach we have used will not capture all the subtleties involved in matching treatment regimens to complex patients; nevertheless the approach is pragmatic, can be applied in busy clinical setting, goes some way towards dealing with the issue of treatment effectiveness and goes much further in resolving issues around missing data. The Sheffield dataset was not large enough to definitively elucidate the relationship between health outcomes and different approaches of calculating adherence. Therefore, different adherence indices and methods of processing adherence data should be empirically tested in a suitably large dataset with objective adherence data and carefully measured key outcomes to determine the optimum method of calculating adherence levels. Nonetheless, different methods of adjusting for inadequate prescriptions would still find lower levels of adherence following the adjustments if efficacious treatments are under-prescribed. It is important to understand the direction of any bias. In this study, by only implementing denominator adjustments for adults with chronic Pseudomonas aeruginosa infection as defined by the Leeds criteria (which is known to lack sensitivity [
      • Hoo Z.H.
      • Edenborough F.P.
      • Curley R.
      • et al.
      Understanding Pseudomonas status among adults with cystic fibrosis: a real-world comparison of the Leeds criteria against clinicians' decision.
      ]), we are likely to under-adjust for required treatment prescription and hence our estimates of effective adherence is likely to be an over-estimation. Thus when adjustment lowers centre adherence, we can be confident that the revised figure is appropriately lowered and if anything, elimination of measurement bias (if technically possible) would merely lower it further. More thorough adjustments for other factors which influence treatment effectiveness would reveal even greater discrepancies between unadjusted and normative adherence. Another factor to consider in adjusting for treatment prescription is that not everyone would be able to tolerate inhaled therapies. It is uncertain whether the proportion of people with CF genuinely unable to tolerate any inhaled therapies will vary substantially from centre-to-centre. Accounting for this group consistently across all centres should help to improve the reliability of centre-comparison with adherence as the metric for improvement.
      This study has other limitations. Objective adherence to dry powder inhalers (used by <10 adults each year in this single-centre cohort) and non-data logging nebulisers (e.g. eFlow®) could not be measured and were therefore excluded. However, the characteristics associated with adherence e.g. age and socioeconomic deprivation were broadly similar between the adults excluded because they were solely using devices without data-logging capabilities and adults using I-nebs®. Hence this exclusion may not necessarily bias the results and our analysis could still provide an insight into what data might be available if adherence were measured across the whole centre. Our sample size is pragmatic and single-centre studies may lack generalisability. Nonetheless, our dataset is currently the largest electronic data capture adherence dataset in CF with 18,303 weeks of adherence data from 126 adults. The consistency of our findings from 2013-2016 also provide some reassurance that the results are unlikely to occur just by chance.

      5. Conclusions

      We have demonstrated that objective adherence levels are influenced by the different approaches of sampling, processing and analysing adherence data. We have also proposed pragmatic methods to account for between-centre variation in treatment prescriptions and potential differential adherence data missingness, so that the resultant adherence metric better reflects the centre-level effectiveness of medication use. Standardising the approach of calculating adherence is an important first step towards robust centre-comparison to identify the relevant differences in structure and care processes that can stimulate improvement. After ensuring that adherence data between centres are comparable, understanding the case-mix factors which influence centre-level adherence is the next important step to make sense of the variation in adherence according to the ‘pyramid of investigation’ model [
      • Mohammed M.A.
      • Rathbone A.
      • Myers P.
      • Patel D.
      • Onions H.
      • Stevens A.
      An investigation into general practitioners associated with high patient mortality flagged up through the Shipman inquiry: retrospective analysis of routine data.
      ]. Case-mix factors that influence adherence level will be the subject of our next paper. It has also not escaped our notice that the results in Table 2 suggest a consistent improvement in our centre's adherence levels from 2013 to 2016, regardless of the metric used to report objective adherence. More detailed longitudinal analyses of our adherence data will be the subject of another paper.

      Funding

      Zhe Hui Hoo is funded by a National Institute for Health Research (NIHR) Doctoral Research Fellowship for this research project (DRF-2014-07-092). This publication presents independent research. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

      Contributors

      Each author had full access to the data and takes responsibility for the integrity and accuracy of this study. All authors contributed to and approved of the final submitted manuscript. HZH and MJW were responsible for the study design, data analysis and interpretation, and the writing of the manuscript. HZH and RC were responsible for acquisition of data. RC, SJW and MJC contributed substantially to the data analysis and interpretation, and the writing of the manuscript.

      Ethics Approval

      Regulatory approval for this study was obtained from NHS Health Research Authority (IRAS number 210313).

      Declaration of Competing Interest

      None declared.

      Acknowledgements

      We would like to thank Shona Simmons (Sheffield Adult CF Centre, Northern General Hospital, Sheffield, UK), Nicole R Bramley (Faculty of Medicine, Dentistry & Health, University of Sheffield, Sheffield, UK) and Muhaned SA El-Gheryani (Faculty of Medicine, Dentistry & Health, University of Sheffield, Sheffield, UK) for their help with data acquisition.

      Appendix. Supplementary materials

      References

        • Elborn J.S.
        Cystic fibrosis.
        Lancet. 2016; 388: 2519-2531
        • Yang C.
        • Montgomery M.
        Dornase alfa for cystic fibrosis.
        Cochrane Database Syst Rev. 2018; 9CD001127
        • Daniels T.
        • Goodacre L.
        • Sutton C.
        • Pollard K.
        • Conway S.
        • Peckham D.
        Accurate assessment of adherence: self-report and clinician report vs electronic monitoring of nebulizers.
        Chest. 2011; 140: 425-432
        • Narayanan S.
        • Mainz J.G.
        • Gala S.
        • Tabori H.
        • Grossoehme D.
        Adherence to therapies in cystic fibrosis: a targeted literature review.
        Expert Rev Respir Med. 2017; 11: 129-145
        • Pugatsch T.
        • Shoseyov D.
        • Cohen-Cymberknoh M.
        • et al.
        Adherence pattern to study drugs in clinical trials by patients with cystic fibrosis.
        Pediatr Pulmonol. 2016; 51: 143-146
        • White H.
        • Shaw N.
        • Denman S.
        • Pollard K.
        • Wynne S.
        • Peckham D.G.
        Variation in lung function as a marker of adherence to oral and inhaled medication in cystic fibrosis.
        Eur Respir J. 2017; 491600987
        • Rowbotham N.J.
        • Smith S.
        • Leighton P.A.
        • et al.
        The top 10 research priorities in cystic fibrosis developed by a partnership between people with CF and healthcare providers.
        Thorax. 2018; 73: 388-390
        • Rault G.
        • Lombrail P.
        Strategies for care quality improvement in Cystic Fibrosis.
        Orphanet J Rare Dis. 2018; 13: 26
        • Friedman C.
        • Rubin J.
        • Brown J.
        • et al.
        Toward a science of learning systems: a research agenda for the high-functioning Learning Health System.
        J Am Med Inform Assoc. 2015; 22: 43-50
        • Hind D.
        • Drabble S.J.
        • Arden M.A.
        • et al.
        Supporting medication adherence for adults with cystic fibrosis: a randomised feasibility study.
        BMC Pulm Med. 2019; 19: 77
        • Schechter M.S.
        Benchmarking to improve the quality of cystic fibrosis care.
        Curr Opin Pulm Med. 2012; 18: 596-601
        • Tappenden P.
        • Sadler S.
        • Wildman M.
        An early health economic analysis of the potential cost effectiveness of an adherence intervention to improve outcomes for patients with cystic fibrosis.
        Pharmacoeconomics. 2017; 35: 647-659
        • Vrijens B.
        • De Geest S.
        • Hughes D.A.
        • et al.
        A new taxonomy for describing and defining adherence to medications.
        Br J Clin Pharmacol. 2012; 73: 691-705
        • Braido F.
        • Chrystyn H.
        • Baiardini I.
        • et al.
        "Trying, but failing" – the role of inhaler technique and mode of delivery in respiratory medication adherence.
        J Allergy Clin Immunol Pract. 2016; 4: 823-832
        • Allen J.D.
        • Curtiss F.R.
        • Fairman K.A.
        Nonadherence, clinical inertia, or therapeutic inertia?.
        J Manag Care Pharm. 2009; 15: 690-695
        • Geller D.E.
        • Madge S.
        Technological and behavioral strategies to reduce treatment burden and improve adherence to inhaled antibiotics in cystic fibrosis.
        Respir Med. 2011; 105: S24-S31
        • Morton R.
        I'm sorry, doctor, my dog ate my inhaler” – the trials and tribulations of a clinical researcher.
        BMJ. 2015; 350: h2097
      1. Cystic Fibrosis Trust. Standards for the clinical care of children and adults with cystic fibrosis in the UK. https://www.cysticfibrosis.org.uk/the-work-we-do/clinical-care/consensus-documents. Date last updated: 1 December 2011. Date last accessed: 3 May 2019.

        • Hoo Z.H.
        • Bramley N.R.
        • Curley R.
        • et al.
        Intravenous antibiotic use and exacerbation events in an adult cystic fibrosis centre: a prospective observational study.
        Respir Med. 2019; 154: 109-115
        • Castellani C.
        • Cuppens H.
        • Macek Jr., M.
        • et al.
        Consensus on the use and interpretation of cystic fibrosis mutation analysis in clinical practice.
        J Cyst Fibros. 2008; 7: 179-196
        • Lee T.W.
        • Brownlee K.G.
        • Conway S.P.
        • Denton M.
        • Littlewood J.M.
        Evaluation of a new definition for chronic Pseudomonas aeruginosa infection in cystic fibrosis patients.
        J Cyst Fibros. 2003; 2: 29-34
        • Hoo Z.H.
        • Curley R.
        • Campbell M.J.
        • Walters S.J.
        • Hind D.
        • Wildman M.J.
        Accurate reporting of adherence to inhaled therapies in adults with cystic fibrosis: methods to calculate “normative adherence”.
        Patient Prefer Adherence. 2016; 10: 887-900
        • Campbell M.J.
        • Gardner M.J.
        Calculating confidence intervals for some non-parametric analyses.
        Br Med J (Clin Res Ed). 1988; 296: 1454-1456
        • Bland J.M.
        • Altman D.G.
        Statistical methods for assessing agreement between two methods of clinical measurement.
        Lancet. 1986; 1: 307-310
        • Driscoll P.
        • Lecky F.
        • Crosby M.
        An introduction to everyday statistics – 2.
        J Accid Emerg Med. 2000; 17: 274-281
        • Shivasabesan G.
        • Mitra B.
        • O'Reilly G.M.
        Missing data in trauma registries: a systematic review.
        Injury. 2018; 49: 1641-1647
        • Glauser T.A.
        • Nevins P.H.
        • Williamson J.C.
        • et al.
        Adherence to the 2007 cystic fibrosis pulmonary guidelines: a national survey of CF care centers.
        Pediatr Pulmonol. 2012; 47: 434-440
        • Nightingale J.A.
        • Osmond C.
        Does current reporting of lung function by the UK cystic fibrosis registry allow a fair comparison of adult centres?.
        J Cyst Fibros. 2017; 16: 585-591
        • Hoo Z.H.
        • Edenborough F.P.
        • Curley R.
        • et al.
        Understanding Pseudomonas status among adults with cystic fibrosis: a real-world comparison of the Leeds criteria against clinicians' decision.
        Eur J Clin Microbiol Infect Dis. 2018; 37: 735-743
        • Mohammed M.A.
        • Rathbone A.
        • Myers P.
        • Patel D.
        • Onions H.
        • Stevens A.
        An investigation into general practitioners associated with high patient mortality flagged up through the Shipman inquiry: retrospective analysis of routine data.
        BMJ. 2004; 328: 1474-1477