THE STUDY

• Abstract: Need to state the perspective within the abstract and state the comparator as part of the objective. The abstract would be improved by stating the cost per QALY gained and cost per CRQ gained (either as well as or instead of the probability that treatment is cost-effective).

• page 5, paragraph 2: it would be useful to state whether the SEPRP education sessions were done on groups of patients or 1:1 with individuals.

• Page 7, lines 52-55 and page 8, lines 41-43: substantially more details are needed on the imputation. What method of imputation was conducted and why, was uncertainty around imputed values propagated into the analysis (and if so how), what software or command was used and was the imputation conducted on total costs or on the quantities of individual resources? These statements also imply that imputation of costs and QALYs was conducted separately and therefore makes no allowance for the correlations between costs and outcomes. It is also not clear whether data were imputed on CRQ and EQ-5D or just EQ-5D. It is also unclear whether values missing after patients withdrew from the study were imputed in the same way as missing questionnaires, or how deaths were taken into account in the analysis of CRQ.

• It is unclear when participants completed resource use questionnaires and how the results were used to generate the quantities and costs in Table 2 and Appendix Table 3. It is unclear how resource use data were collected for patients who had died. • Page 6, line 56: it would be useful to say “evidence on resource use, EQ-5D and CRQ was collected”, rather than simply “health status”

• Page 7, line 55-57 and page 8, lines 43-6: these statements imply that costs, CRQ and QALYs were estimated separately and therefore make no allowance for the correlations between costs and effects. If this is the case it is a major shortcoming of the analysis. It might be clearer to describe the analysis and imputation of both costs and outcomes within the same section, rather than repeating almost identical statements in the cross-section and the outcomes section. Although the distribution used is stated for costs, there is no indication of what distribution was assumed for outcomes. At a minimum, the approach used needs to be clearly described and justified.

• Page 8, lines 10-23: it would be very useful to state what the range of scores is for CRQ to give some context to the magnitude of the differences observed in the trial

• Page 9, line 11: The i, j and k terms in this equation and the sentence that follows should presumably be put in subscript.

• Page 9, lines 29-31: It is unclear from the methods whether this analysis of incremental net benefit is part of an analysis of the joint distribution of costs and effects, or whether it was conducted separately as a univariate analysis. It is also necessary to state what distribution was used. It is also not entirely clear which of the results are informed by this analysis (presumably just the probability that treatment is cost-effective?)

• Table 1: full references are needed for all sources of unit costs. The majority of drugs included in this table appear to be given as brand names; in the case of combination inhalers this may be appropriate, although it would be useful to also state the generic names. It would also be useful to state whether “prednisone” is given orally or inhaled.

• Appendix table 1: is this a duplicate of information published in an earlier paper? If so, this should be clearly stated and permission is likely to be needed to replicate the table.

• Appendix Table 2: it would be helpful to separate out patient expenses from healthcare service expenses in all sections of this table. At present, it is difficult to follow the calculations: particularly when the figure of €564 appears in two places.

RESULTS & CONCLUSIONS

• The results (most notably Table 2 and Appendix Table 3) are currently not clearly presented.

• Table 2 and Appendix Table 3 appear to show a concerningly large difference in baseline CRQ and baseline EQ-5D utility, with the intervention group having substantially higher baseline quality of life than the control group. Given the magnitude of these differences, it is highly likely that the analysis would have been sensitive to the methods used to adjust for baseline. It is unclear which results have been adjusted for baseline. Assuming that the results in table 2 include no adjustment for baseline while the results in table 3 are adjusted suggests that adjusting for baseline using GEE eliminates the difference in QALYs, but not the difference in CRQ, which is surprising.

• The study includes no sensitivity analyses. It is generally recommended that all important assumptions and methodological approaches are varied in sensitivity analysis. In particular, in this case it would have been particularly useful to explore the impact of different approaches to missing data or baseline imbalance, as well as varying the number of patients per practice and the number of practices by which the intervention costs are divided.

• As with most economic evaluations, the conclusions of the analysis hinge upon the ceiling ratio used to interpret the results. The authors avoid specifying a ceiling ratio a priori since the ceiling ratio is “unknown for Ireland”. However, the authors nonetheless draw conclusions that are implicitly based on strong judgements about the ceiling ratio: for example in the abstract they state that “strongly favourable results exist when health status was measured using the disease-specific CRQ instrument”, as well as similar statements in the discussion. It is unclear what ceiling ratio these statements are based on, although it would appear that the authors only consider ceiling ratios in excess of €5000 per one-point increase in CRQ. This ceiling ratio seems extremely high: based on the UK ceiling ratio of £20,000/QALY gained, a ceiling ratio of €5000 per one-point increase in CRQ would suggest that a 1 point increase in CRQ is equivalent to around 0.167 QALYs, which seems unlikely. It is therefore very unclear whether the conclusions are justified. The range of ceiling ratios presented in the abstract, Table 3, page 10, lines 30-35 is likely to be too high for CRQ and it may be better to present a different range of ceiling ratios for each of the two outcomes and present the CEACs on different graphs. It may also not be necessary to present the probability of being cost-effective at five ceiling ratios in four different places in the paper. The shape of the CEAC for CRQ could also be quite different from that shown in Figure 1 if additional points between 0 and €5000 were evaluated.

• Table 2 and Appendix Table 3: It is not entirely clear exactly what results are presented in these table or how they were estimated. It would be useful to explicitly state whether the resource use and cost data presented in Table 2 represents the total used over the 22- week study period (rather than any other time period). It is unclear why physiotherapy, public health nurse, dietician, home help and social worker visits are recorded as percentages, rather than quantities used over the 22-week study period. Presenting such figures as percentages raises the question of how the total cost over the 22-week period was estimated from the percentage at baseline and percentage at 22 weeks: for example, does the analysis assume that anyone who stopped having physiotherapy between baseline and 22 weeks stopped halfway through and what assumptions are made about how frequently patients saw a physiotherapist? It is also unclear why Appendix table 3 is labelled “baseline (26 weeks)”: does it represent the resource use on the day of randomisation, resource use 26 weeks before the study intervention, resource use 26 weeks after the start of study intervention or the resource use that patients reported at baseline when recalling their resource use over the past 26 weeks? How are the baseline resource use quantities taken into account in the analysis? It is also unclear how the standard deviations were calculated (particularly if multiple imputation was conducted) and how (or whether) the data presented in this table include imputation of missing data. It is also unclear whether these figures were estimated in the GEE models that adjusted for baseline differences, or whether they are unadjusted figures.

• Table 2 last line and page 9, line 3: it would be clearer to describe the QALYs as “unadjusted QALYs”, rather than “QALYs gained” if this figure represents the total unadjusted QALYs over the 22-week period.

• Page 9, line 56: it is unclear which time point these CRQ scores represent: are they based on the measurements at 22 weeks, the average between baseline and 22 weeks or something else?

• It would be useful to state the cost-effectiveness ratios from a healthcare perspective as well as from a patient+ healthcare perspective (either in Table 3 or in the text)

• It would be useful to comment on the statistical significance of the adjusted differences in QALYs, CRQ, total costs, healthcare service costs and patient costs between treatment arms and give p-values.

• Page 10, lines 48-51: the double negative in this sentence makes it very difficult to understand and possibly inaccurate.

• Page 11, lines 21-31 and page 12 lines 11-13: I would anticipate that a great many studies have looked at the sensitivity of EQ-5D in moderate COPD and it would be useful to cite these here. In particular, if there is evidence finding EQ-5D to be sensitive and responsive in COPD, it would be useful to discuss this in relation to the results of the current study.

• Page 11, 31-35: additional justification may be needed to conclude that other studies should use disease-specific health status measures to assess cost-effectiveness and resource allocation. This study also highlights the difficulty of identifying an appropriate ceiling ratio and drawing conclusions based on ICERs using non- preference-based measures.

• The study used 2009 prices. It would be useful if the paper discussed whether (or how) prices have changed and whether the conclusions are still valid in 2013

• The 22-week time horizon is a big weakness (as the authors acknowledge). It would be useful to have additional discussion on what the CUA results might have been if results were extrapolated beyond the 22-week time horizon and on whether there is any evidence on how long the effect of interventions such as this tends to last.

• Page 11, lines 55-8: the wording of this sentence implies that further follow-up is planned, although if it is the case, it is unclear why this follow-up hasn't already been finished if the main trial finished in 2011.

REPORTING & ETHICS

The following points on the EVEREST checklist have not been followed: o page 6, lines 53-54: perspective isn't justified. The perspective used in the analysis (“the perspective of the health care provider and the patient”) isn‟t a commonly used one and it is particularly useful to justify why this perspective was used. This sentence could also be made clearer and it may be better to have one sentence stating the perspective and another sentence stating the units of health outcomes. o Page 8, lines 26-40: it is important to state which UK EQ-5D tariff was used (presumably the time trade-off tariff) and give a reference. This paragraph should also state at what time points EQ-5D was measured and briefly describe what assumptions were used to interpolate between the measurements (e.g. assuming linear changes) and between a patient‟s last measurement and death. It would also be useful to explicitly state that EQ-5D-3L was used. o it would be useful to explicitly describe the study as a cost utility analysis and cost-effectiveness analysis within the text o The paper should explicitly state that costs and QALYs were not discounted o No sensitivity analysis was conducted. Allowance for sampling variation does not constitute sensitivity analysis

Looking at the latest checklist (CHEERS), the study also doesn‟t fulfil the following points: o [Explain] why the single study was a sufficient source of clinical effectiveness data o Discussion of generalisability and equity

GENERAL COMMENTS

Comments to authors

The key messages and conclusions of the abstract are phrased in an unclear way: e.g. “favourable cost-effectiveness results exist when health status was measured using the disease-specific CRQ instrument”, rather than a more precise statement such as “Analyses using the CRQ suggested that SEPRP was cost-effective if society is willing to pay at least €850 per one- point increase in CRP”. In particular, page 12, line 10, page 14 line 21, page 2 lines 19 & 25 and page 4 line 9 should state what ceiling ratio is being used: e.g. by adding "if society is willing to pay at least 850 Euros per 1-unit increase in CRQ" after "health status". In practice, it is highly likely that society‟s willingness to pay, or the amount that the healthcare system can afford is far lower than 850 Euros per 1 point increase in CRQ, which could mean that the conclusions are the same for CRQ as for QALYs.

Page 7, line 20: The paper should explicitly state what follow-up point(s) CRQ, EQ-5D and resource use are measured? (measured only at baseline and 22 weeks?) It would also be useful to explicitly state that the follow-up is at 22 weeks on page 11 (line 3) and page 10, line 45.

Brief information on the methods for imputation needs to be added into page 8, lines 32-40 and page 9 lines 28-30. At a minimum, it is necessary to say that uvis was used and that utilities and each type of resource use were imputed separately, although it would also be useful to state the variables included as predictors of missing data. How many imputed datasets were generated for each variable? If more than one, how were these combined?

Page 10, line 53 & page 11, lines 3 & 8: it would be useful to state that these figures are unadjusted; similarly, it would be useful to state on page 11, line 13 that these figures are adjusted for baseline utility, etc. In particular, this will help explain to readers why the incremental QALY gain (0.002) is not equal to the difference in QALYs between arms (0.337-0.305).

Table 2: it would be useful to explicitly state what is meant by "raw data": i.e. to explicitly state whether the data shown here include no imputation and no adjustment for imbalance in baseline utility The study excludes patients who died, which could introduce bias (particularly as three quarters of the deaths were in the control group). The paper should justify excluding such patients and discuss what impact this could have had on conclusions.

Table 2, lines 39-45: it would be useful to state whether these are at follow-up (rather than baseline), for consistency with EQ-5D and to avoid confusion.

Table 3: footnotes should be added to make it clear which of the values shown in results include imputed values and which include adjustment for which covariates. For example, do all results shown in this table include imputed values, while all differences and all of the results below the heading of "COST EFFECTIVENESS ANALYSIS" include adjustment for baseline CRQ or EQ5D score, baseline, healthcare cost and clustering? It would also be very useful to state that the cost-effectiveness analysis results are based only on the healthcare costs (not the patient costs).

Table 2 and Appendix Table 3: it would be useful to expand these table headers or add a footnote to make it 100% clear what data are being presented: for example, in Appendix table 3, it would be much clearer to say "raw data estimates at baseline (including costs in the 26 weeks leading up to randomisation)" rather than "raw data estimates at baseline (26 weeks pre-randomisation)", since the latter incorrectly implies that patients were surveyed 26 weeks before they were randomised. Similarly, in Table 2, it would be clearer to have a footnote that explicitly stated that resource use captures the period from randomisation to week 22.