What is a derivation cohort

gms | German Medical Science

short version

1. Scientific background

With a total of 358,684 deaths in Germany in 2007, cardiovascular diseases are among the clinical pictures of enormous epidemiological importance. Cardiovascular diseases are also of the utmost importance from an economic perspective. In 2006 the costs of diseases of the cardiovascular system amount to approx. 35 billion euros.

It is assumed that cardiovascular morbidity and mortality can be changed through preventive measures. In addition to the use of preventive measures aimed at the population, individual-related (possibly medicinal) interventions are often indicated for people with an increased overall risk. For the selection of persons with an increased overall cardiovascular risk, so-called risk prognosis instruments are constructed and used.

A statistical evaluation of data from a population creates equations, point scores or table diagrams (risk charts), according to which a risk for a cardiovascular event or a probability of survival without this event can be estimated depending on the severity of the risk factors. Risk forecasting tools can also be used graphically, e.g. B. be shown as nomograms.

A number of different risk forecasting tools exist. However, these are based on various primary studies or databases, which mostly do not include the German population. The transferability of these forecasting instruments to the populations not examined in these data sources as well as the comparability of the validity of these forecasting instruments is called into question.

2. Question

The evaluation should answer the following questions:

  • What instruments are available for risk prediction for cardiovascular diseases?
  • What is the evidence for the transferability of the existing risk prediction tools for cardiovascular disease to populations that were not involved in the prognostic study?
  • To what extent are the existing methods of risk prediction for cardiovascular diseases comparable?

3. Methodology

Sources of information and research strategy

The literature search will be carried out in the most important medical electronic databases (MEDLINE, EMBASE, etc.) in April 2008. The research strategy is limited to the years from 2004 and the languages ​​German and English. An extensive hand search for publications on prognostic instruments for cardiovascular diseases and the external validity of various prognostic instruments is also carried out.

Inclusion and exclusion criteria

The assessment includes publications on prognostic instruments for cardiovascular diseases in patients without any prior cardiovascular disease, as well as publications with information on external validation or comparing such prognostic instruments with one another. The instruments that focus on specific patient risk groups are not taken into account. Discrimination and calibration are used as quality criteria.

Data evaluation and information synthesis

Both systematic overviews and primary publications on prognostic instruments as well as to assess the validity and comparability of various prognostic instruments are included as sources of information. The information synthesis is qualitative.

4. Results

Results of the literature research

The systematic literature search yields 734 hits. 116 publications are selected for full-text review and checked for inclusion in the assessment. The literature search identified three systematic reviews, eight publications with descriptions of prognostic instruments and 13 publications with information on the validity of the prognostic instruments. Using the hand search in the reference lists of the relevant articles, 30 further publications with descriptions of prognostic instruments and 16 further publications with information on the validity of the prognostic instruments are identified.

Risk forecasting tools

Most risk forecasting tools are based on the US Framingham cohort, almost all of the others are based on European, mainly British and Italian cohorts. The PROCAM study is based entirely on the German reference population. The SCORE charts for Germany and the WHO / ISH charts for the European risk region EUR-A only partially use German data. Both population-based as well as patient and professional cohorts are included as basic populations for deriving a prognostic instrument, in some studies only men or women.

Almost all prognostic instruments use sex, age, smoking, one or more information on the lipid status and blood pressure values ​​as variables. In many prognostic instruments, diabetes mellitus or blood sugar is also used as a variable for risk calculation, in several instruments the variables left ventricular hypertrophy according to ECG, body mass index, antihypertensive therapy and in some prognostic instruments other different variables. The multinational studies also stratify their forecasting instruments regionally. Usually only five to six prognostic variables are used in the prognostic instruments.

The most important end parameters are death from coronary heart disease, death from cardiovascular disease, coronary heart disease or coronary event (death, myocardial infarction, possibly angina pectoris and / or coronary revascularization) and cerebrovascular event (stroke, possibly also transient ischemic attack), cardiovascular disease or Cardiovascular event (coronary event, cerebrovascular event, possibly intermittent claudication and / or heart failure). The time span for forecast events is usually ten years.

Three different statistical methods are used to evaluate the data of the reference population in order to create a score, logistic, Weibull and Cox regression models, whereby a step-by-step regression analysis is selected for all methods.

External validity of instruments for risk prediction for cardiovascular diseases

Information on the calibration of the prognostic instruments (quotient of predicted to observed risk) is presented in about half of the studies. Only in individual studies is the calibration in the range from 0.9 to 1.1. All three studies from Germany lack information on the calibration of the forecasting instruments.

Many studies on the transferability of prognostic instruments show an AUC value for discrimination (value for the correct assignment of people with different risk levels; AUC = area under the curve; best value: 1.0) between 0.7 and 0.8 for various prognostic instruments (adequate discrimination), few studies an AUC value between 0.8 and 0.9 (good discrimination) and no study an AUC value above 0.9 (excellent discrimination).

In the studies with information on the discrimination of the prognostic instruments (various Framingham equations) for the German population, with one exception, all AUC values ​​are between 0.73 and 0.78 (sufficient discrimination). Studies to check the external validity of the new prognostic instruments developed in the German population, such as PROCAM (2007) and SCORE-Germany, are still pending.

Comparison of the validity of different instruments for risk prediction for cardiovascular diseases

When comparing the validity of different risk forecasting instruments by applying one of these forecasting instruments to the derivation cohort (correctness), a trend towards better calibration and better discrimination for the prognostic instruments calculated on the basis of the respective derivation cohort can be seen in all studies.

When comparing the validity of different risk forecasting instruments by applying one of these forecasting instruments to the validation cohort (reproducibility), there is a trend towards better calibration and better discrimination for the forecasting instruments calculated on the basis of the corresponding derivation cohort.

When comparing prognostic instruments by applying them to other populations (transferability), the newly derived Framingham prognostic instruments provide a somewhat better discrimination compared to previously calculated instruments. The significance of the reviewed German prognostic instrument PROCAM from 2002 in comparison to Framingham instruments for the European population is not clear. To date, there are no studies comparing different prognostic tools in the German population.

5. Discussion

Literature research

Despite the very broad search strategy in the most important medical databases, it is possible that relevant articles on the topic of the report may be overlooked due to the problem of the complexity of the literature search in prognostic studies.

Risk forecasting tools

The representativeness of the study participants for the respective total population is questionable in many studies for the derivation of risk prognosis instruments and the basic populations in the studies with regard to the disease stage are not homogeneous.

The high number of variables that are seldom used in risk forecasting instruments speaks among other things. that the relevance of these variables for the risk prognosis has not been clearly established.

The consideration of endpoints with clinical events is more subjective than the exclusive consideration of mortality, but it has significantly higher clinical and social significance for the individual.

The Cox regression should be preferred as the evaluation method, since this regression analysis, among other things, can calculate the risk for several points in time and allows a relatively simple adaptation of the model for the other populations.

Despite a reduction in the precision in the transition from a risk equation to the point score and to the risk chart, a risk chart allows a clearer representation of the actual risk and the risk to be aimed for for the patient than a value determined directly from the risk equation.

External validity of instruments for risk prediction for cardiovascular diseases

In the available studies on external validation, mostly geographical as well as historical, methodological and person spectrum-related components of transferability are checked. Due to the considerable differences in cardiovascular morbidity and mortality between different countries or regions, the geographical transferability appears to be the most important component.

Since the populations on which the forecasting instruments are based were recruited many years ago in most studies, the forecasting instruments derived on the basis of these populations may not be transferable to current populations.

It is not to be expected that the somewhat different measurement methods and spectra of people in different studies will significantly limit the transferability of the prognostic instruments.

A precise threshold value for a good or bad calibration has not yet been clearly defined in the literature. In order to avoid the problem of poor calibration, the mean values ​​for risk factors of the basic population of a forecasting instrument and the mean event rates of this population are replaced by parameters of the population to be forecasted in the equations (recalibration).

An exact and plausible threshold value for good or bad discrimination of the prognostic instruments is also not mentioned in the literature. The division into excellent, good, sufficient, weak and very weak discrimination is subjective. It is also recommended that the discrimination check only be carried out after recalibration for the relevant population.

Comparison of the validity of different instruments for risk prediction for cardiovascular diseases

The better validity of risk prognostic instruments through application to the derivation as well as to the validation cohort of this prognostic instrument and in particular to other populations can be explained by the considerable geographical variance of the cardiovascular morbidity and mortality.

The lack of studies comparing different prognostic instruments in the German population does not allow any statements about their comparability.

6. Conclusions

The identified instruments for risk prediction of cardiovascular diseases are not sufficiently validated in the German population. Their use can lead to a misjudgment of the risk of individual patients. For this reason, the prognostic instruments available in Germany for informed decision-making and the choice of therapy should only be used with critical caution. The implementation of studies for the external validation of the prognostic instruments and for the comparison of different prognostic instruments in the German population (if possible after previous recalibration) as well as randomized studies on the therapeutic consequences and the clinical benefit of the use of prognostic instruments are recommended.