A Binary Classification Made to Partition the Drug Safety Test Cases

All content on this webpage originated from other webpages,Please check the medicilon's website for details. Medicilon offers fully integrated pharmaceutical services for the global scientific community. We focus on providing an exceptional client-centered experience and advancing the drug discovery process.

Email: Marketing@medicilon.com.cn Website: www.medicilon.com

Drug safety testing effect estimates generated from all methods were compared to a binary classification made to partition the test cases into ‘positive controls’ and ‘negative controls’. The classification was performed by OMOP through systematic review of structured product labels available on the FDA website before December 19, 2009, using the occurrence of a condition in the adverse event section of the majority of labels within a class as a surrogate for a ‘positive control’, and selecting conditions unrelated to any labeled events as ‘negative controls’. For ACE inhibitors, 84 ‘positive controls’ and 2780 ‘negative controls’ were identified and used for experimentation. The ‘positive controls’ include labeled events known to be related to ACE inhibitor exposure, such as cough, hypotension, hyperkalemia, and renal impairment. ‘Negative controls’ include a wide range of conditions observed in the database that are unrelated to any known effect of exposure, such as uterine leiomyoma, osteomyelitis, ankle fracture, incisional hernia, malignant neoplasm of brain, and hammer toe. The full set of test cases is available for download at. Sensitivity was measured as the proportion of the 84 labeled events identified at statistically significant levels, based on alpha = 0.05 and 0.001. Specificity was measured as the fraction of the 2780 negative controls that failed to meet statistical significance. Positive predictive value was estimated as the proportion of the outcomes meeting statistical significance that were classified as positive controls.

For each method, a receiver operating characteristic (ROC) curve was produced. All drug safety testing pairs were rank-ordered by the effect size point estimate, and the sensitivity and specificity was estimated at all observed threshold values. Five complementary measures of performance were estimated based on these ROC curves. The c statistic, or the area under the ROC curve, provides a predictive probability that two random drug safety testing pairs, one positive control and one negative control, would be properly rank-ordered. The c statistic ranges from 0 to 1, with 1 indicating perfect prediction and 0.5 a random prediction. Partial area under ROC curve at 10% false positive (PAUC10) is used to focus on the highest scores and eliminate the range of the ROC curve with unacceptable low specificity. The value of PAUC10 ranges from 0 to 0.10, with random prediction scoring 0.005. Recall at 5% false positive (RECALL5) estimates what fraction of the positive controls is identified at a threshold of 95% specificity. Precision at 100 (P100) provides a measure of what proportion of the drug safety testing pairs amongst the 100 highest estimates are positive controls. ‘Mean average precision’ (MAP) is a metric commonly used in information retrieval that provides the average precision at each threshold value that represents a ‘true positive’ association.

Each method has multiple parameter settings, based on design decisions around surveillance windows to define time-at-risk, covariates to include, and metrics to calculate. Parameter settings for all methods were selected by choosing the configuration that maximizes PAUC10. Sensitivity analysis was performed to assess impact of different design decisions of the all performance measures.