Using Stochastic Simulation-Estimation and Automated Model Development to Assess Power and Accuracy for Covariate Identification

Publication

When study designs are evaluated using clinical trial simulations for their ability to identify covariate effects in population pharmacokinetic (PopPK) modeling, it is typically assumed that the true model will be known at the data analysis stage. In this study, this was compared with the more realistic assumption that the PopPK model needs to be built on the data generated by the planned study. Three approaches were compared: (i) stochastic simulation and re-estimation (SSE) with the simulation model, (ii) automated model development (AMD) with exploratory covariate search (AMD-exploratory), and (iii) AMD forcing the covariate effect into the model from the start and reevaluating it in the end (AMD-structural). With a simulated covariate effect (a hypothetical pregnancy effect on clearance), we assessed (i) the type 1 error (T1E) and the power of covariate identification and (ii) covariate parameter accuracy. The T1E rate was controlled in SSE and AMD-exploratory but 20% inflated for AMD-structural. The power of covariate identification in rich, medium, and sparse designs was (i) 99%, 100%, and 79% in SSE, (ii) 74%, 72%, and 41% in AMD-exploratory, and (iii) 92%, 93%, and 80% in AMD-structural. Sparse designs amplified power differences between strategies, with AMD-exploratory often selecting alternative or no covariates. The rRMSE of covariate parameter estimates was lowest in SSE (27%, 22%, and 42%), followed by AMD-exploratory (34%, 26%, and 49%) and then AMD-structural (42%, 47%, and 60%). SSE provides optimistic power estimates as model building is data-driven, while AMD-based approaches incorporate model uncertainty and reflect real-world analysis conditions.