Developing a clinical trial for a small population of patients, such as those for rare diseases, is a complicated undertaking. Treatments may improve the patient’s health but it is seldom a large, robust change that is easy to document. Adding to that, our understanding of a rare disease’s natural history is constantly evolving as is the standard of care for any given disease. As such, an outcome measure in a small clinical trial can be more of a ‘moving target’ than a fixed parameter. But that outcome measure is the one that will be looked at to determine if the treatment works. Therefore, a poorly designed clinical trial that results in an outcome measure that can be questioned by doctors, insurance companies, patients, caregivers, and regulatory agencies is a clinical trial that is a waste of everyone’s time.
Recently, a group of clinical researchers in Europe critically looked at a number of small clinical trials and based on their assessment, listed 33 recommendations for researchers wanting to develop a clinical trial that will reduce the possibilities of the trial being questioned by its critics. Their recommendations are listed below and were recently published in the Orphanet Journal of Rare Diseases. The recommendations cover a wide range of topics, beginning with questions related to whether a trial should be undertaken or not, to different design parameters based on genetics, surrogate endpoints, randomization, etc.
Level of evidence – decision theory
Formulate decision rules in a formal Bayesian decision-theoretic framework.
Societal decision rules (regulation, reimbursement) should be determined based on explicit modelling of how they will inter-depend with commercial drug developing decisions.
Increase transparency in regulatory and payer decisions.
The well-being of the individual trial patient must have priority.
Pharmacological consideration – simulation
If fast computations of power curves are needed from a non-linear mixed effects model, we recommend using the parametric power estimation algorithm as implemented in the stochastic simulation and estimation tool of PsN (potentially with a type-I correction based on the “randtest” tool in PsN).
We recommend the use of Sampling Importance Resampling to characterize the uncertainty of non-linear mixed effects model parameter estimates in small sample size studies. Non-estimability of parameters may be assessed using preconditioning. The use of the bootstrap model averaging method is recommended when conducting model-based decision-making after a trial. Robust model-based adaptive optimal designs may be used to improve model certainty in clinical trials.
Pharmacological consideration – optimal design
For evaluation of designs of studies with longitudinal discrete or time-to-event data, evaluation of the Fisher Information matrix should be done without linearization. Using the new approach MC-HMC (in the R package MIXFIM) will provide adequate prediction of standard errors and allow to compare several designs.
When there is little information on the value of the parameters at the design stage, adaptive designs can be used. Two-stage balanced designs are a good compromise. The new version of in the R functions PFIM can be used for adaptive design with continuous longitudinal data.
When there is uncertainty in the model regarding the parameters, a robust approach across candidate models should be used to design studies with longitudinal data.
Pharmacological consideration – genetic factors
It is recommended to use “varclust” for clustering of gene expression or metabolomics data and extraction of a small number of potential predictors of patients’ response to the treatment based on highly dimensional “omics” . Also, it is recommended to use PESEL for estimation of the number of important principal components.
It is recommended to use both regular and group SLOPE for identification of biomarkers based on the genotype data, since regular SLOPE has a higher power of detection of additive gene effects, while group SLOPE allows for identification of rare recessive variants.
It is recommended to use the modified Bayesian Information Criterion for efficient aggregation of genotype and ancestry of genetic markers and identifying biomarkers in admixed populations.
Choice of endpoint – biomarkers
In case of small trials, which are in particular variable in size, we recommend the use of the causal inference framework, combined with efficient computational methods.
In case of the evaluation of surrogate endpoints in small trials subject to missingness, we recommend the use of pseudo-likelihood estimation with proper inverse probability weighted and doubly robust corrections.
In case of hierarchical and otherwise complex designs, we recommend using principled, yet fast and stable, two-stage approaches.
In case of genetic and otherwise high-dimensional markers, we recommend the use the methodology expressly developed for this context, in conjunction with the software tools made available (R package IntegratedJM).
In case of a surrogate with dose-response or otherwise multivariate information present, we recommend to use the Quantitative Structure Transcription Assay Relationship framework results.
In case of the evaluation of surrogate endpoints in small studies, we recommend using weighting-based methods, because the methodology has been shown to work well theoretically, because it has been implemented in user-friendly SAS and R software, and because its practical performance is fast and stable.
Methodological considerations – randomisation
Do not select a randomisation procedure by arbitrary arguments, use scientific arguments based on the impact of randomisation on the study endpoint taking into account the expected magnitude of bias.
Tailor the randomisation procedure used in small-population randomized clinical trial by following ERDO using randomizeR.
In case of a randomized clinical trial, we recommend to conduct a sensitivity analysis to examine the impact of bias on the type-I-error probability.
Methodological considerations – adaptive design
In the case of confirmatory testing, we recommend adapting the significance level by incorporating other information, e.g. using information from drug development programs in adults for designing and analyzing pediatric trials.
Where randomized control clinical trials are infeasible, we propose “threshold-crossing” designs within an adaptive development program as a way forward to enable comparison between different treatment options.
In the case of design modification during the conduct of a confirmatory clinical trial, we recommend using adaptive methods to ensure that the type-I-error is sufficiently controlled not to endanger confirmatory conclusions. Especially in clinical trial with multiple objectives special care has to be taken to address several sources of multiplicity.
Methodological considerations – pharmacogenetics
For the analysis of N-of-1 trials, we recommend using an approach that is a modified fixed-effects meta-analysis for the case where establishing that the treatment works is the objective, and an approach through mixed models if variation in response to treatment is to be studied.
When conducting a series of N-of-1 trials we recommend paying close attention to the purpose of the study and calculating the sample size accordingly using the approach provided in detail by Dr Stephen Senn.
We recommend that response should not be defined using arbitrary and naïve dichotomies but that it should be analysed carefully paying due attention to components of variance and where possible using designs to identify them.
When analyzing between-patient studies, we recommend avoiding information-destroying transformations (such as dichotomies) and exploiting the explanatory power of covariates, which may be identified from ancillary studies and patient databases.
The comparison of dose response curves should be done by the bootstrap approach.
If the aim of the study is the extrapolation of efficacy and safety information, we recommend considering and comparing the MEDs of two given populations.
The derived methodology shows a very robust performance and can be used also in cases where no precise information about the functional form of the regression curves is available.
In case of planning a dose-finding study comparing two populations, we recommend to use optimal designs in order to achieve substantially more precise results.
Hilgers R-D, Bogdan M, Burman C-F, et al. Lessons learned from IDeAl — 33 recommendations from the IDeAl-net about design and analysis of small population clinical trials Orphanet J Rare Dis. 2018;13:77 https://doi.org/10.1186/s13023-018-0820-8