China Real-World Evidence Research Design and Statistical Analyses for medical device guideline has been released by the Center for Medical Device Evaluation (CMDE).
This guideline aims to standardize and guide the application of real-world data in the clinical evaluation of medical devices. It provides technical guidance for applicants conducting real-world studies and for regulatory authorities in their technical evaluations.
Scope:
The “Guidance Principles for the Design and Statistical Analysis of Real-World Studies of Medical Devices for Regulatory Review” aims to provide clear and comprehensive guidance for the design and analysis of real-world studies of medical devices. They cover:
- various types of real-world studies,
- design considerations,
- data quality,
- bias risk assessment,
- statistical analysis,
- safety risk assessment, and
- real-world study report.
The guidelines emphasize the importance of scientific rigor, transparency, and ethical considerations in the development and assessment of medical devices to ensure patient safety and device effectiveness.
The guidance applies to real-world studies of medical devices and does not cover RWE studies of in-vitro diagnostic reagents managed as medical devices. It builds upon the “Technical Guidance for the Use of Real-World Data in the Clinical Evaluation of Medical Devices (Trial)” and further refines the general requirements for the design and statistical analysis of real-world studies of medical devices. At this stage of development, real-world evidence primarily serves as a supplement to existing clinical evidence in the clinical evaluation of medical devices.
Below you will find a summary of the key points:
A) Common Types of Real-World Evidence Studies and their Applications:
- Experimental Real-World Studies:
- Pragmatic Randomized Controlled Trials (pRCTs) are a common type of experimental real-world study. pRCTs assess the treatment outcomes of different interventions in real or near-real medical settings, using randomization and controls. They aim to evaluate the effectiveness of interventions in routine clinical practice.
- Observational Real-World Studies:
- Descriptive Study Designs: Include cross-sectional, case reports, and case series designs. These are not suitable for causal inference.
- Cohort Designs: In the assessment of medical device safety and effectiveness, cohort studies involve tracking and comparing outcomes in different groups based on device usage. This is an observational research method to assess the association between the device and the outcomes’ occurrence and magnitude.
- Case-Control and Derived Designs: Case-control studies compare the proportion of patients using the device among those who experienced an outcome event (cases) and those who did not (controls) to study the association between the device and the outcome.
- Case-crossover design: A case-crossover design is an observational study design commonly used to examine the relationship between short-term exposures and acute outcome events.
- Real-World Data as External Controls in Single-Arm Trials:
- External control involves selecting a group of study subjects with similar characteristics from other trials or historical cases. This type of design falls outside the scope of this guidance, and specific guidance on its use and design will be developed separately.
B) Design Considerations for Real-World Studies for Medical Devices:
- Study Background and Objectives: Clarify the safety and effectiveness issues that the real-world study aims to address based on product scope and technical characteristics, taking existing evidence into account.
- Feasibility Assessment: Evaluate the objective conditions for conducting real-world studies after defining the study objectives, considering factors like available experience, knowledge, confounding variables’ predictability, data availability, and data quality.
- Selection of Appropriate Real-World Study Designs: Choose the appropriate study design type based on the study objectives, referencing the guidance’s content. Different real-world study design types, such as pRCTs, cohort studies, and case-control studies, have distinct characteristics suitable for different scenarios.
- Study Flowchart: Present the study process in a flowchart format, showing specific activities in chronological order, such as ethical review, population selection, intervention, data collection (e.g., tests, examinations, scorecard completion), and quality control measures.
- Study Population: Define the target study population clearly with precise inclusion and exclusion criteria. Ensure clarity and avoid ambiguity in defining the criteria.
- Device Exposure: While pRCTs and traditional RCTs randomize device exposure, observational real-world studies determine device exposure based on real-world circumstances, introducing selection bias.
- Control Group: In pragmatic randomized control designs, control groups are formed through randomization. In observational real-world designs like cohort and case-control studies, appropriate methods are used to establish control groups, striving for balance in the distribution of confounding variables between groups.
- Evaluation Criteria: Describe the rationale for selecting evaluation criteria, specify the observation objectives, definitions, observation time windows, types of indicators, measurement methods, calculation formulas (where applicable), and criteria (applicable to qualitative and ordinal indicators). Clearly define primary, secondary, and safety evaluation criteria.
- Follow-Up Duration: Determine the starting point and duration of follow-up based on the study objectives and design. For implantable devices, the starting point is typically the day of implantation. For devices used in multiple treatments, it is the day the last treatment is completed, with safety events during the treatment period also observed. In retrospective real-world studies, the length and starting point of follow-up may be limited by the availability of existing data. In cases where there is a latent period for outcome symptoms or a delay in the intervention’s effect, consider setting a time window between exposure and outcome to prevent reverse causality.
- Sample Size Calculation and Power Analysis: For retrospective real-world studies, estimate sample size based on available data. For prospective real-world studies, calculate sample size based on estimated parameter values. Different study designs require different sample size estimation methods, such as cross-sectional studies estimating sample size based on expected precision or studies with control groups estimating sample size based on inter-group differences, relative risks, odds ratios, etc.
- Quality Control:
- Data Quality：
- Data Collection
- Quality Evaluation
- Bias Risk:
- Selection Bias
- Information Bias
- Confounding Bias
- Data Quality：
C) Statistical Analysis and Reporting:
- Statistical Analysis Plan (SAP): Real-world studies need to include a detailed and specific plan for statistical analysis, identifying the specific statistical methods and parameter settings to be used, as well as the rationale and justification for the statistical methods and parameter settings. Real-world studies more often involve stratified analyses, regression analyses, adjusted statistical analyses based on inclination scores, and different analyses using the same data.
- Analyzing data sets: Pre-define different datasets according to different analysis purposes, such as validity dataset and safety dataset, subgroup analysis dataset, and so on.
- Identification of confounding variables to be adjusted: Real-world study designs that do not use randomized groupings need to identify in advance the confounding variables to be adjusted for, and all confounding variables need to be identified as much as possible in order to control for confounding bias during the design and statistical analysis phases.
- Statistical analysis of mixing adjustments: Adjustment methods used to control for confounding include stratification, multivariate regression analysis, adjustment methods based on propensity scores, more sophisticated statistical methods such as marginal structural modeling, instrumental variables, and structural equation modeling.
- Handling of Missing Data: Clearly specify the rules for handling missing data and explain the rationale behind them. Appropriate handling of missing data is essential for minimizing bias and achieving valid results.
- Subgroup analysis: Subgroup analyses are required if there is heterogeneity in the population included in the study and the heterogeneity may lead to different effect values.
- Sensitivity analysis: Sensitivity analyses are used to assess the robustness of research findings and may be required in a number of different contexts, particularly for observational real-world research.
D): Real-World Study Report:
- The guidance provides detailed requirements for the content and format of the real-world study report. Appendices include templates for the real-world study report, statistical analysis plan, and protocol for pragmatic randomized controlled trials.
- Study reports need to follow the general principles of completeness, accuracy and standardization. Differences exist in the content of different types of real-world study reports. pRCT study report content can refer to the CONSORT guidelines for effectiveness trials, observational studies such as cohort designs and case-control designs can refer to the STROBE guidelines, and other applicable documents such as the STaRT-RWE checklist can be referenced to help improve the completeness of clinical reporting elements.
Further information:
Read the original CMDE article on China real world evidence medical device.
Read our previous blog post on China medical device real-world evidence increasingly important.
If you require advice on China medical device real-world evidence policies and regulations to support medical device marketing authorization applications, contact your Cisema consultant today. Discover our services for medical device registration, renewals and NMPA Legal Agent.
