Both authors read and approved the final manuscript. National Center for Biotechnology Information , U. Journal List Basic Clin Androl v. Basic Clin Androl. Published online Nov Mohamed Macki 1 and Ali A. Dabaja 2. Ali A. Author information Article notes Copyright and License information Disclaimer. Dabaja, Phone: , Email: gro. Corresponding author.
Received Apr 17; Accepted Sep This article has been cited by other articles in PMC. Conclusion Because the statistically significantly higher incidence of AE in the HPV vaccine group was primarily limited to injection-site reactions, the vaccinations are safe preventative measures in both males and females.
Background The human papillomavirus HPV is an important preventable cause of sexually-transmitted disease and squamous cell carcinomas. Open in a separate window. Table 1 Literature review of vaccine-related adverse events reported from HPV vaccination in randomized controlled trials. These adverse events included malaria, headache, dysmenorrhea, abdominal pain, vertigo, cough, nasopharyngitis Amorphous aluminum hydroxyphosphate sulfate AAHS adjuvant; aluminum hydroxide [Al OH 3 ] a Bronchospasm 1 day after the third dose b Serious adverse events in the vaccine group were gastroenteritis, headache, hypertension, injection-site pain, and decrease in joint movement at the injection site c Serious adverse events in both the vaccinated and control groups were likely due to malaria infection, unrelated to the vaccine d Elective discontinuation in the vaccine group was due to spontaneous abortion, unrelated to the vaccine e Elective discontinuation in the vaccine group was due to vaccine-related malaise and headache f Elective discontinuation in the vaccine group was due to vaccine-related pyrexia.
Statistical analysis Demographic information was described using summary statistics. Discussion In , an English physician, Edward Jenner, performed the first vaccination by inoculating an 8-year-old boy with pus from a cowpox lesion [ 40 ].
Acknowledgements Not applicable. Funding No sources of funding were involved in this research: the design of the study and collection, analysis, interpretation of data, or in writing the manuscript. Availability of data and materials The data supporting the findings in this manuscript can be obtained by e-mail the corresponding author, Ali Dabaja, MD. Competing interests The authors declare that they have no competing interests.
References 1. Quadrivalent vaccine against human papillomavirus to prevent anogenital diseases. N Engl J Med. Group FIS. Quadrivalent vaccine against human papillomavirus to prevent high-grade cervical lesions. Open Med. Efficacy of a bivalent L1 virus-like particle vaccine in prevention of infection with human papillomavirus types 16 and 18 in young women: a randomised controlled trial.
Prophylactic quadrivalent human papillomavirus types 6, 11, 16, and 18 L1 virus-like particle vaccine in young women: a randomised double-blind placebo-controlled multicentre phase II efficacy trial.
Lancet Oncol. Safety, immunogenicity, and efficacy of quadrivalent human papillomavirus types 6, 11, 16, 18 recombinant vaccine in women aged years: a randomised, double-blind trial. J Obstet Gynaecol Res.
Hong Kong Med J. Safety and immunogenicity of a quadrivalent human papillomavirus types 6, 11, 16, and 18 vaccine in HIV-infected children 7 to 12 years old. J Acquir Immune Defic Syndr.
J Infect Dis. Cancer Sci. Safety and persistent immunogenicity of a quadrivalent human papillomavirus types 6, 11, 16, 18 L1 virus-like particle vaccine in preadolescents and adolescents: a randomized controlled trial. Pediatr Infect Dis J.
J Adolesc Health. Int J Cancer. Clin Vaccine Immunol. J Korean Med Sci. Int J Gynecol Cancer. Risk of miscarriage with bivalent vaccine against human papillomavirus HPV types 16 and pooled analysis of two randomised controlled trials. Efficacy of a prophylactic adjuvanted bivalent L1 virus-like-particle vaccine against infection with human papillomavirus types 16 and 18 in young women: an interim analysis of a phase III double-blind, randomised controlled trial.
Jpn J Clin Oncol. The long-term immune response after HPV16 peptide vaccination in women with low-grade pre-malignant disorders of the uterine cervix: a placebo-controlled phase II study. Cancer Immunol Immunother. Occurrence of vaccine and non-vaccine human papillomavirus types in adolescent Finnish females 4 years post-vaccination.
Effect of the human papillomavirus HPV quadrivalent vaccine in a subgroup of women with cervical and vulvar disease: retrospective pooled analysis of trial data. Hum Vaccin Immunother. Int J Gynaecol Obstet. Prophylactic efficacy of a quadrivalent human papillomavirus HPV vaccine in women with virological evidence of HPV infection. Safety and reactogenicity of a quadrivalent human papillomavirus types 6, 11, 16, 18 L1 viral-like-particle vaccine in older adolescents and young adults.
Hum Vaccin. Stern AM, Markel H. The history of vaccines and immunization: familiar patterns, new challenges. Health Aff Millwood ; 24 3 — In order for the results of such trials to be clinically useful, they must also be relevant to a definable patient population in a specific healthcare setting, a concept that is termed external validity or generalizability note, these terms are used interchangeably [ 3 ] in this review and describe the applicability of the study results outside of the trial environment [ 5 — 7 ].
As it is challenging to simultaneously optimize internal and external validity, efficacy data from traditional explanatory RCTs are often complemented by evidence from pragmatic trials including pragmatic RCTs or observational studies that determine the performance of an intervention under conditions more closely resembling routine clinical practice, and include more heterogeneous patient populations and less stringent treatment and delivery protocols [ 4 ]. While some pragmatic trials have good internal validity and some observational studies may lack external validity, generally explanatory RCTs tend to maximize internal validity at the expense of external validity, while studies conducted in a setting more closely resembling real-world practice may do the opposite.
As such, evidence from all these sources can be complementary in understanding the effect of an intervention and furthering clinical research [ 8 ]. In recent years, the need to better understand the external validity of RCT results has been identified across numerous therapeutic areas [ 9 — 13 ].
However, a comprehensive literature review of studies that have assessed the representativeness of RCT populations has not been undertaken in recent years note, the term representativeness has been used throughout this review to describe the similarities between RCT samples and real-world populations.
To examine this issue, we conducted a literature review of studies that have attempted to evaluate external validity in one of two ways: i by comparing the clinical characteristics of an RCT sample with those of everyday clinical practice patients, or ii by assessing what proportion of a real-world population would satisfy the criteria for RCT inclusion.
In the context of the current review, real-world populations are defined as those patients encountered in routine clinical practice settings for example, patients included in observational cohorts or patients identified from medical chart review, registries, or insurance databases. The primary objective of the review was to assess the extent to which RCT samples are representative of real-world populations which may or may not affect the external validity of the trial findings.
Other objectives were to identify key issues that may impact the external validity of trial findings with reference to included studies and also to outline recommendations from the identified studies for improving external validity. The present review was limited to RCTs in oncology, mental health, and cardiology as, when the review was undertaken, these were identified as the main therapeutic areas in which RCT and real-world populations had been compared.
It should be noted that the focus of the current review was explanatory and not pragmatic RCTs. The methodological framework of this literature review was employed to examine the extent, range, and nature of research activity regarding the representativeness of RCT patient samples and the implications of this to the external validity of the findings.
The review involved a five-stage process [ 14 ]: identification of the research question; identification of studies relevant to the research question; selection of studies to include in the review; charting of information and data within the included studies; collating, summarizing, and reporting results of the review. A search protocol was written that outlined the objectives, search methods, and the process for study selection and data extraction.
When combined with citation searching, these sources presented a reasonable basis for a targeted search of the published literature. The searches were run on 30 September and included published studies conducted from to in order to reflect contemporary clinical trial practice.
Studies could have quantitatively assessed how many patients in a real-world population would satisfy the eligibility requirements of an RCT, or compared the clinical characteristics of an RCT sample with a real-world population. Only those studies reporting on pharmaceutical interventions studied as part of an RCT placebo-controlled or active comparator were included. This analysis was limited to studies in cardiology, mental health, and oncology, as the larger numbers of publications identified in these therapeutic areas allowed for a higher level synthesis of their findings.
Search results were assessed for relevance by two independent researchers by reviewing the title and abstract of all identified studies. Studies meeting or potentially meeting the review eligibility criteria were assessed in more detail using the full text. A third reviewer TKM resolved disagreements on study selection. A data extraction table was developed and tested on a sample of studies before further refinement. All data included in the final manuscript were quality checked.
The following data were extracted from each included publication: i generalizability objectives; ii patient populations and country of study; iii methods; iv description of RCT and real-world data sources; v listing of comparisons made and key results; vi overall conclusions; vii recommendations addressing identified issues and best practices. Following a detailed review, a framework for the narrative analysis of the data was developed that included categorization of the identified studies by two methods.
Method A involved a formal statistical comparison for example, use of Wilcoxon rank sum test and chi-square test for continuous and categorical variables, respectively of baseline characteristics between a real-world patient population and a patient sample enrolled in an RCT in the same specific disease area. Patients were compared for baseline characteristics such as demographics, clinical and disease data, and treatments and procedures. A range of different statistical methodologies were employed in the included studies, and it is outside of the scope of this review to detail them all; the reader is referred to the individual studies for more information.
Method B involved a determination of the proportion of patients in a real-world population that would have been trial eligible or ineligible by review of individual patient medical records followed by the application of explicit eligibility criteria derived from specific RCTs or common criteria derived from a review of multiple RCTs in the same disease to individual patient data.
The ineligibility rates as calculated in each individual Method B study were tabulated and the distribution by quartiles examined. A minority of studies employed a mixture of methods A and B and presentation of the findings from such studies was split by method Tables 2 and 3. In order to interpret the main findings of the literature review as they related to external validity, a qualitative synthesis of individual study results was undertaken.
A second researcher checked the grouping of each study by category; in the event of any disagreements, the findings of each paper were discussed until resolution was reached. The study selection is shown in Fig. The original search returned 5, studies of which 46 in the areas of cardiology, mental health, and oncology were identified as relevant after abstract review. An additional six studies were identified through citation searching. Study selection for a literature review assessing the external validity of randomized controlled trials.
Of the 52 studies included, 18 An additional seven studies The highest number of studies was conducted in the USA Table 1. The populations studied using Method A were compared for demographics, clinical characteristics, baseline treatments and procedures, and other variables Table 1. Additional analyses were conducted in some Method B studies as detailed in Table 1. The sources and settings from which RCT samples and real-world patient populations were drawn are listed in Tables 2 and 3 a more detailed summary of sources is provided in Additional file 2.
In 37 The remaining 15 studies [ 52 — 66 ] did not reach an explicit conclusion regarding external validity or concluded that populations were broadly similar, although we note that in some cases the authors still reported differences between RCT samples and real-world populations Tables 2 and 3 [ 53 , 57 , 62 , 64 , 65 ]. Studies included in the review generally demonstrated that, compared with patients enrolled in major cardiology RCTs, patients encountered in everyday practice were more likely to have higher risk characteristics as they were older, more likely to be female and to have clinical impairment and co-morbid disease, were treated less frequently with guideline-recommended therapy, and received fewer in-hospital procedures Table 2.
In 11 studies employing Method B, 18 different sets of eligibility criteria were applied to real-world populations and ineligibility rates reported; in eight cases The reasons for ineligibility varied considerably by study depending on the specific condition under assessment.
Method B studies. Some individual studies reported multiple ineligibility rates derived from the application of selection criteria from a number of different RCTs to a single real-world population. Hence, in the 34 studies that employed Method B, 54 different ineligibility rates were calculated. In general, the identified studies reported that real-world patients with mental health disorders tended to be more severely ill than patients enrolled in RCTs. They also appeared to have more co-morbidities and, in some cases, lower overall functioning and socioeconomic status Table 2.
Common reasons for RCT exclusion across studies included current or history of substance abuse, suicide risk, presence of co-morbidities such as other Axis I disorder, co-morbid anxiety, and other central nervous system [CNS] or neuromuscular disorder , insufficient symptom duration or low disease severity in studies of major depressive disease , contraindicated medication, and significant medical condition see Additional files 3 and 4.
Compared with RCT-enrolled patients, real-world patients with cancer were often older, and more likely to be female, have a poor performance status, and worse disease prognosis Table 2 in the studies selected in this review. Reasons for trial exclusion included poor performance status, previous history of cancer, co-morbidities, reduced life expectancy, CNS or brain metastases, and older age see Additional files 3 and 4.
In the majority of included studies, the authors made some attempt to identify factors influencing the external validity of RCTs. The influence of implicit factors on external validity could only be hypothesized in the included studies and are outlined below. Explicit factors were identified as a key driver for differences in RCT samples and real-world populations, as demonstrated by the often high rates of trial ineligibility Fig.
For example, in cardiology studies, patients often appeared to be excluded on the basis of older age and presence of co-morbid disease. The authors of these studies suggested that cardiovascular disease may represent a more complicated syndrome in such patients [ 15 ] and that they are more likely to experience adverse events [ 16 , 19 ].
As such, the results from these studies may not provide a complete picture of anticipated drug efficacy and safety in clinical practice. Female patients were also under-represented in the cardiology trials identified in this review [ 15 , 17 , 24 , 29 , 37 ]; one of the reasons for this may be due to cardiovascular disease affecting women later in life, meaning that upper age limit restrictions may disproportionately limit their inclusion in RCTs relative to men [ 37 ].
In mental health studies, high proportions of patients were excluded on the basis of substance abuse, which is a particular issue for the external validity of trials in bipolar patients where rates are high [ 41 ]. Patient samples in oncology trials were often found to have better disease prognosis and better performance status compared with real-world patients with cancer [ 23 , 25 , 31 , 38 , 45 ].
Implicit factors that may have affected the external validity of RCTs were also identified in some of the studies reviewed. Two cardiology studies noted that issues with informed consent, whereby the most severely ill patients are less likely to give informed consent or it is harder to gain informed consent, may lead to the selection of lower risk patients for trial participation [ 16 , 17 ].
In addition, one study indicated that psychiatric patients with more severe aggression were also less likely to consent to enter an RCT [ 26 ]. In oncology patients and their physicians , one of the biggest barriers to trial participation was noted to be fear of randomization to the placebo arm [ 43 ].
A number of other patient-related factors were also identified, including logistical issues related to study participation, beliefs and attitudes regarding the safety of trial medications, cultural factors, level of satisfaction with current treatment, and willingness to participate [ 39 , 43 , 48 , 49 ].
Finally, one study demonstrated that patients who participate in trials may have different personality traits than those who do not; patients with depression who were enrolled in an antidepressant medication RCT were found to score more highly on a personality scale that assessed preferences for novel experiences compared with non-participants [ 57 ]. Many of the studies included in the present review made recommendations to improve the external validity of RCTs.
These recommendations are outlined in Table 4 and include modifying RCT design to improve external validity directly, and generating complementary evidence from alternative study types to address the limited external validity of the RCT post hoc. The present analysis utilized a robust literature review methodology to identify studies that compared the clinical characteristics of an RCT sample and patients from a real-world source Method A or assessed the proportion of a real-world population that would satisfy criteria for RCT inclusion Method B.
Publications identified by this methodology indicated that RCT samples in cardiology, mental health, and oncology studies that assessed pharmaceutical interventions in adult patients were often not broadly representative of patients treated in everyday clinical practice and that caution should be exercised when extrapolating data from trials to patients treated in usual care settings.
Note that, with the exception of a single study [ 40 ], none of the RCTs described in the included studies were documented as being of a pragmatic design. In this Method B study, the RCTs in acute coronary syndrome from which eligibility criteria were extracted were described as having pragmatic enrollment strategies; however, the analysis still suggested that there were important differences in risk profile between RCT eligible and ineligible patients [ 40 ]. Differences in demographics, clinical characteristics, and treatments and procedures were reported between RCT and real-world patients by studies that employed Method A in their analyses [ 15 , 17 , 21 — 25 , 27 , 29 — 31 , 37 , 38 , 42 , 44 , 45 , 48 , 49 ].
Furthermore, it was observed that large proportions of the general disease population were often excluded from trial participation. We note that some differences in generalizability were observed between the different therapeutic areas studied in the present review. In only a minority of studies did the authors conclude that RCT samples were broadly representative of real-world populations and that external validity was not impacted, or failed to reach an explicit conclusion regarding external validity despite demonstrating some differences in baseline characteristics between groups [ 52 — 66 ].
While involving the review of older studies and use of more restrictive search criteria than the present review, this earlier study also demonstrated that RCTs often exclude large proportions of the general disease population and specific patient groups from trial participation. In agreement with the present review, it was reported that the elderly, women, and patients with co-morbidities were frequently ineligible for trial inclusion [ 2 ].
However, note that RCT findings may still be externally valid even in circumstances where the patient sample is not broadly representative of the real-world population. For example, one study included in the present review concluded that patients with unstable angina or non-ST-segment elevation myocardial infarction who would have been excluded from enoxaparin RCTs could be safety treated in clinical practice [ 53 ].
That the external validity of RCT results is often limited is widely acknowledged by clinicians as a problem when it comes to extrapolating data to the patients seen in everyday practice [ 3 , 7 ]. Indeed, it is an often-cited reason for the frequent underuse of guideline-recommended therapies [ 67 ]. Where there is no evidence of efficacy in specific patient groups, clinicians may well be right in withholding treatment so as to prevent unanticipated harm [ 35 ].
This situation could, however, mean that patients at highest baseline risk who might be expected to receive the most benefit from a particular therapy are undertreated. Authors reported that frequently excluded patients were the elderly, females, or those with co-morbidities in cardiology studies [ 15 — 17 , 19 , 24 , 29 , 34 , 35 , 40 , 44 , 53 , 55 ], patients with evidence of substance abuse or co-morbid psychological disorders in mental health studies [ 18 , 28 , 32 , 33 , 41 , 42 , 47 , 49 , 50 , 61 , 64 ], and patients with poor disease prognosis in oncology studies [ 20 , 25 , 31 , 38 , 39 , 45 , 46 ].
These RCT populations were, therefore, often highly selected and represented a patient sample at much lower risk of adverse events and complications compared with patients in clinical practice. While the use of highly selected populations does not necessarily imply that a given treatment under study would fail to have equivalent efficacy and safety in under-represented patient groups, it does create uncertainty that can only be dispelled through the generation of additional evidence.
However, it is pertinent to also consider how inclusion of high-risk patients may affect the outcomes of traditional trials. Patients with more co-morbidities or co-interventions may be more likely to prematurely discontinue study participation, which could lead to high attrition rates and a negative impact on trial validity and outcomes.
The studies reviewed herein made several recommendations to either improve the external validity of RCTs or compensate for limitations thereof. These included adaptation of trial designs to include a more heterogeneous patient sample that better represents different subgroups such as the elderly or patients with co-morbidities [ 19 , 20 , 28 — 33 , 46 ].
Some studies suggested that adoption of pragmatic trial designs may be a way forward [ 48 , 49 ]. In addition to its application as an aid to trial design, PRECIS-2 has the potential for use in the assessment of completed trials for methodological quality and the likelihood of outcome bias in much the same way as the current Grading of Research, Assessment, Development and Evaluation GRADE system is used to assist guideline developers. There is growing interest in different analytical methods that utilize data from multiple studies to extend and complement the evidence provided by a single clinical trial.
Meta-analysis [ 72 , 73 ] can be used to combine evidence from multiple clinical trials to provide a more valid estimate of treatment effect, assuming the studies being combined are similar enough to permit synthesis. Cross-design synthesis is a type of meta-analysis in which evidence from studies with complementary designs are combined in an effort to leverage complementary strengths such as internal validity of RCTs and external validity of observational studies and minimize the weaknesses of each [ 74 ].
Another approach that leverages real-world data to extend findings from a traditional trial involves development of propensity scores that predict, for each trial subject, membership in a corresponding real-world population [ 75 , 76 ].
Subjects over-represented in the clinical trial relative to the target real-world population receive lower weights while those under-represented receive higher weights. Finally, simple descriptive analysis of real-world data can also be employed in the trial planning stages to better understand the impact of specific design decisions for example, potential exclusion criteria on the anticipated generalizability of the trial results and so improve design.
Adaptation of statistical analysis plans was recommended by two of the studies reviewed here as a method to facilitate analysis of important patient subgroups [ 20 , 37 ]. Several of the reviewed studies highlighted incomplete reporting as a potential issue for the external validity of RCTs [ 24 , 28 , 38 , 51 , 63 ]. Improvements in trial reporting to provide a more detailed description of RCT samples would enable clinicians to better assess the external validity of RCTs and so more accurately extrapolate trial findings to their own patients.
Following reporting guidelines such as CONSORT, which is a requirement for publication in many peer-reviewed journals [ 1 ], may go some way to address issues of inconsistent reporting and may provide greater transparency with respect to trial eligibility. Trials should follow the need for evidence but be part of a broader strategy for evidence generation. As such, complementary data obtained from other appropriately designed alternatives conducted in Phase IV of the development lifecycle are required to address limitations in the external validity of RCTs post hoc.
As recommended by some of the studies included in this review [ 15 , 23 , 36 ], the use of non-randomized observational studies that utilize large healthcare databases can support RCT findings by determining treatment effectiveness in routine clinical practice [ 6 , 77 ]. Such studies include a wide range of different designs including prospective and retrospective cohort studies, case—control studies, and cross-sectional studies in which any intervention studied is determined by clinical practice and not a rigid protocol [ 78 ].
Taken together, RCT and observational study data should provide a complementary body of evidence that optimizes both internal and external validity. The findings presented in this review must be viewed within the limitations of the methodology employed. Ten RCTs reported positive results post-intervention. Conclusions and implications: Nutrition counseling interventions involving active participation and collaboration showed the most promise in affecting positive nutrition-related outcomes in community-dwelling older adults.
Future research should determine which nutrition interventions will benefit community-dwelling older adults with less active participation and health perceptions, and lower educational attainment.
Published by Elsevier Inc. All rights reserved.
0コメント