Preview

PDF, English - main document Download (106MB) | Terms of use

Abstract

In the process of the development of a new treatment, many requirements for market authorization must be met through various stages. After approval, the additional benefit of a new treatment is compared to already established treatments. This assessment can decide on the amount of reimbursement of the new treatment on the market and create transparency for patients regarding the treatments’ medical effectiveness and toxicity.

In case of non-curative or advanced diseases like cancer, three different additional benefit assessment methods have been developed. The European Society for Medical Oncology (ESMO) and Institute for Quality and Efficiency in Health Care (IQWiG) constructed methods with an ordinal outcome. For the main classification, IQWiG compares the upper limit of the 95% hazard ratio (HR) confidence interval (CI) (HR+) against relative risk (RR) based thresholds and ESMO uses mainly the lower limit of the 95% HR-CI (HR–). The American Society of Clinical Oncology (ASCO) defined a continuous outcome using HR point estimate (PE). Hence, the main difference of the three methods is the used statistical quantity. There are several points of criticism to each of the assessment methods. For example, the use of the HR-PE for the assessment of the clinical benefit could penalize studies of substantial benefit by ignoring the precision of the estimate. In contrast, the upper or lower limit of the HR-CI considers the variability of the estimate and hence should provide more information.

The aim of this thesis is to obtain a better understanding of the differences between the methods and to answer the question which statistical quantity has the best properties to assess additional benefit. Furthermore, it is investigated which category of ESMO and IQWiG corresponds to which ASCO score in order to achieve an easier comparison between all three methods. To achieve these objectives, this thesis evaluates and compares the above described methods by means of simulation studies comprising different failure time distributions, treatment effects, power, allocation ratios, censoring types, and censoring rates. Furthermore, scenarios with non-proportional hazards, underpowered trials, and overpowered trials are investigated. The original IQWiG method (IQWiGRR) and ESMO show a high positive association for moderate treatment effects. ASCO and IQWiGRR as well as ASCO and Mod-IQWiGHR (proposed modification of IQWiGRR using HR based thresholds instead of RR based once) show a high positive association over the whole range of treatment effects. Moreover, ESMO excessively awards its maximal category over the whole range of treatment effects (in most scenarios over 80%) and hence cannot distinguish between small and large treatment effects. ASCO and IQWiGRR have a more conservative behaviour. Different violated assumptions such as non-proportional hazards, over-/underpowered studies, and informative censoring, do not lead to penalization in ESMOs grading; e.g. the maximal category rate is not reduced. Overall, the used thresholds of ESMO for categorization are chosen too liberal, which lead to high false positive rates and easily achievable maximal category grading. ASCO and IQWiGRR have a more desirable behaviour. In most cases Mod-IQWiGHR does provide a better solution in case of violated assumptions, i.e. no excessive increase of the maximal category rate. Nevertheless, in most other scenarios where no assumptions are violated it might be too conservative, i.e. low true positive rate and low maximal category rate. Furthermore, ESMO shows an even increased rate of maximal category in case of different failure time distributions, which still adhere to the proportional hazard assumptions and hence no changes in the category distribution should be expected. The other methods (ASCO, IQWiGRR, and Mod-IQWiGHR) do not show this undesired behaviour. Hence, ESMO is the most liberal method. Nonetheless, under the condition that appropriate thresholds are chosen, HR–, which ESMO uses among other things, is the best statistical quantity to assess additional benefit. Even HR-PE provides better properties than HR+.

To improve practical comparison between the methods, this thesis proposed ASCO cutoff values which correspond to the respective categories of the current methods: An ASCO score larger than 17, 20, and 23 corresponds to ESMO categories 2, 3, and 4, respectively. ASCO cutoff values of 20 (23) and 36 (45) separate the score of ASCO into the three IQWiGRR (Mod-IQWiGHR) categories "minor", "considerable", and "major added benefit".

Overall, this thesis demonstrates that HR– instead of HR+ should be used or the current thresholds should be at least adjusted to optimize the true positive and false positive rate. Thus, in future this research can be used as a guide for improvements of the methods and contributes to the enhancement of additional benefit assessment.