1. health and fitness
2. disease
3. heart disease

European Heart Journal (1996) 17, 1181-1191
Early diagnosis of acute myocardial infarction
using clinical and electrocardiographic data at
presentation: derivation and evaluation of logistic
regression models
R. L. Kennedy*, A. M. Burton*, H. S. Fraser*, L. N. McStay* and R. F. Harrisonf
*Department of Medicine, Western General Hospital, Edinburgh, U.K. and jDepartment of Automatic Control &
Systems Engineering, University of Sheffield
was the most effective on test data, yielding accuracies of
84-3 and 83-6% on the two test sets. A model constructed
solely of electrocardiographic data performed nearly as well
as those incorporating clinical data. Previously published
logistic regression models did not perform so well as the
models derived from data collected for this study.
(Eur Heart J 1996; 17: 1181-1191)
Key Words: Myocardial infarction, diagnosis, logistic
regression, electrocardiograph, case history.
networks'17 191. It is not clear whether any of these
methods is clearly superior to the others. Also, the place
The early diagnosis and management of acute chest pain of these tools in clinical practice remains to be estabremains one of the greatest challenges in emergency lished, although early observations have suggested
medicine. Identification of patients with unstable cor- that their use, in a study situation, improves diagnosonary disease is essential so that appropriate investi- tic performance and the use of high dependency
12 20 211
gation, therapy and monitoring can be delivered, beds' ' ' . The algorithms which have been described
particularly to high risk groups. Application of an recently are complex, making use of a relatively large
10 191
appropriate algorithm might improve diagnostic per- number (typically around 50) of data input items' ' .
formance, optimize therapy and use of resources, and The more complex a model is, the less likely it is to be
may also be helpful in healthcare planning. Appropriate used in routine practice.
interpretation of the electrocardiogram at presentation
Logistic regression is a non-linear classification
yields the most discriminatory information regarding technique which uses binary data to derive a series of
diagnosis and prognosis'1"41, but the power of these data coefficients which, when applied to unseen vectors,
are increased by combining them with clinical infor- yield a probability of a single output (e.g. the presence
mation'51. A variety of techniques has been used to of myocardial infarction). The performance of logistic
combine clinical and electrocardiographic data into de- regression models for early diagnosis of acute myocision support algorithms'61. Methods used include linear cardial infarction is almost certainly superior to those
discriminant analysis'7'81, Bayesian inference'9101, recur- derived using decision trees'221. The other major advansive partition analysis'"'121, logistic regression'13"151, tages are that the models are easy to apply and can
knowledge-based systems'161, and artificial neural be derived easily using widely available statistical
packages. Thus, it should be easy, given the appropriate
data, to optimise models for use in any given
Revision submitted and accepted 5 September 1995.
situation, overcoming the problems with portability
Correspondence: Professor R. L. Kennedy, Department of Medi- which have been encountered previously in this area of
cine, District General Hospital, Kayll Road, Sunderland SR4 7TP, work.
U.K.
Introduction
0195-668X/96/08I181 + 11 $18.00/0
© 1996 The European Society of Cardiology
Downloaded from by guest on October 15, 2014
The aim of this study was to determine which, and how
many, data items are required to construct a decision
support algorithm for early diagnosis of acute myocardial
infarction using clinical and electrocardiographic data
available at presentation. Logistic regression models were
derived using data items from 600 consecutive patients at
one centre (Edinburgh), then tested prospectively on 510
cases from the same centre and 662 consecutive cases from
another centre (Sheffield). Although performance of the
models increased with progressive addition of data inputs
when applied to training data, a simple six-factor model
1182 R. L. Kennedy et al.
The aims of this study were: (1) to determine the
optimal inputs for a logistic regression model to assist
with early diagnosis of acute myocardial infarction; (2)
to examine the applicability of such models to data
collected prospectively in different centres; (3) to compare the performance of models derived only from
electrocardiographic data with those which also include
clinical data — for retrospective use, electrocardiographic data is easier to collect and validate; (4) to
compare the performance of models optimized for use in
our centres with that of previously published logistic
regression models.
Patients and methods
Patients and clinical data
The logistic regression models derived from the
above data were independently tested on data collected
from 662 patients attending the Northern General
Eur Heart J, Vol. 17, August 1996
Logistic regression models
These were derived using the maximum likelihood
method to calculate coefficients on the Advanced Statistics Module of the SPSS for Windows Program (SPSS
Inc., Chicago, Illinois, U.S.A.). Logically redundant
inputs were eliminated. Thus, the inputs corresponding
to the fourth age input, and the second input bands for
duration of pain and total duration of symptoms were
not included. The final probability of acute myocardial
infarction was calculated as described previously1'51
using the equation:
Probability (%)= 100 x [1 - 1/(1 + exp(b0 + Zb r /J]
where b0 is the constant term, br is the coefficient for
any given input and yx is the numerical value for that
input.
Data analysis
The likelihood ratio was defined as sensitivity/
100 — specificity for an item positively associated with
the diagnosis and 100 — sensitivity/specificity for an item
with negative correlation. Sensitivity was defined as true
positives/true-positives + false-negatives, specificity as
true negatives/true negatives + false-positives, positive
predictive accuracy as true positives/true positives +
false-positives and accuracy as true positives + true
negatives/total number of patients. We have made
extensive use of the receiver operating characteristic
curve. For a full review of the use of this tool see1231.
Basically, as used here, the plots were of sensitivity
versus 100 — specificity at different diagnostic thresholds. The plots allowed us to estimate the optimum
diagnostic thresholds for the two models. The area
under each of the curves, and their standard error was
calculated according to the method described by Hanley
and McNeill1241. This area gives a measure of the ability
of each test to correctly rank normal and abnormal
cases. It is related to, but not equivalent to, the diagnostic accuracy of the test and can be used to compare
statistically two curves. Such comparison was achieved
using the method described in a further paper by Hanley
and McNeill1251. Kendell Tau correlation of the
paired ratings for this calculation was performed using
the SPSS program. One way analysis of variance
(ANOVA) was used where indicated. Comparisons of
diagnostic accuracy between models was achieved using
McNemar's test in 2 x 2 contingency tables.
Downloaded from by guest on October 15, 2014
The study included consecutive patients attending the
Accident and Emergency Department of the Royal
Infirmary, Edinburgh, Scotland, with a principal complaint of non-traumatic chest pain. The relevant clinical
and electrocardiographic data (see below) was entered
onto a purpose-designed proforma at, or very soon after,
the patient's presentation. The study included both
patients who were admitted and those who were discharged. A total of 1110 patients were recruited during
the study period (September 1993 to January 1994). The
final diagnosis for these patients was assigned independently by three of us — Consultant Physician (R. L.
Kennedy), Research Nurse (A. M. Burton) and Cardiology Registrar (H. S. Fraser). This diagnosis made use of
follow-up electrocardiograms, cardiac enzyme studies
and other investigations as well as clinical history obtained from a review of the patient's notes. For patients
directly discharged from Accident and Emergency, we
contacted patients regarding further symptoms and,
where necessary, we contacted their General Practitioners and reviewed the notes of any further hospital
follow-up. Acute myocardial infarction was diagnosed
on the basis of clinical, electrocardiographic and cardiac
enzyme (total creatine phosphokinase, CK-MB and
lactate dehydrogenase). Cardiac enzyme measurements
were carried out using standard enzyme activity assays
in the Department of Clinical Biochemistry, the Royal
Infirmary, Edinburgh. Unstable angina was defined as
either more than two episodes of pain lasting more than
10 min in a 24 h period or more than three episodes in a
48 h period or angina which was associated with the
development of new electrocardiographic changes of
ischaemia (either at diagnosis or in the subsequent 3
days). The input data items for the logistic regression
models were all derived from data available at the time
of the patient's presentation. We used up to 54 items
which were coded as binary inputs as shown in the
Appendix.
Hospital, Sheffield. The data were collected and the
final diagnosis determined as described above for the
Edinburgh data.
Informed consent was obtained from all patients
participating in the study which was approved by the
Medical Ethics Committees in the two participating
centres.
Logistic regression models for AMI diagnosis 1183
Results
Patient data and selection of inputs for
logistic regression models
The logistic regression models were derived, unless
otherwise stated, from 600 consecutive Edinburgh
patients. These were 383 men and 217 women with a
mean age of 57-4 years (SD 17-1, range 14-92). The final
diagnosis in these patients was Q wave myocardial
infarction in 86, non-Q wave myocardial infarction in
27, unstable angina in 41, stable angina in 159 and
non-cardiac in 287. The models were tested on data from
510 Edinburgh patients — 345 men and 165 women with
a mean age of 58-7 years (SD 15-9, range 17-90). Their
final diagnosis was Q wave myocardial infarction in 79,
non-Q wave myocardial infarction in 28, unstable angina in 44, stable angina in 160 and non-cardiac in 199.
The Sheffield patients, which acted as a further test set
were 389 men and 273 women with a mean age of 59-9
years (SD 14-5, range 17-92). Their final diagnosis was
Q wave myocardial infarction in 147 cases, non-Q wave
myocardial infarction in 50, unstable angina in 103,
stable angina in 133 and non-cardiac in 229.
The likelihood ratios for data items in the
Edinburgh data varied widely, as shown in Fig. 1 (items
are arranged in descending order of their likelihood
ratio, as detailed in Table 1). Figure 2 shows likelihood
ratios for the Sheffield cohort with the items arranged in
the same order as in Fig. 1. There are obvious differences
between the two data sets which may influence the
performance of models derived from one data set and
tested on the other. The five most discriminatory items
for the two data sets were identical: ST elevation, new Q
waves, hypoperfusion, ST depression and vomiting. The
difference between the two data sets is most obvious
with regard to coronary risk factors: smoking (1-23 vs
104) was associated more with a diagnosis of acute
myocardial infarction in the Edinburgh patients, while
hypertension (1 -49 vs 163), diabetes (111 vs 1 -63) and
hyperlipidaemia (101 vs 1-87) were more discriminatory
in the Sheffield cohort.
Logistic regression models based on
likelihood ratio data
Using data from 600 Edinburgh patients, a series of
logistic regression models was developed by progressively adding data items in decreasing order of their
likelihood ratio for acute myocardial infarction diagnosis. These models are described in Table 1. On the 600
training sets, there was a progressive increase in the
performance of the models with the addition of data
items: thus, areas under the receiver operating characteristic curves were 89-6 ± 1-98, 91 2 ± 1-70, 92-8 ± 1-43,
Eur Heart J, Vol. 17, August 1996
Downloaded from by guest on October 15, 2014
Figure 1 The 54 candidate data items (see Appendix) are arranged in decreasing order of likelihood ratio with
respect to diagnosis of acute myocardial infarction. The actual items represented by the bars on this figure are
given in Table 1.
1184 R. L. Kennedy et al.
31.1 -;
•=
1 1 . 1 —;
3
93-5 ±1-31, 94-6 ±1-20, 94-9 ± 1-28, 96-4 ±0-96, respectively for 6, 9, 12, 20, 30, 40 and 53 data items
(P<000\, ANOVA). However, when these models were
applied to unseen data (510 Edinburgh patients), there
was no difference in their overall performance: Areas
under the receiver operating characteristic curves were
891 ± 2 1 8 , 89-5 ± 2 1 5 , 89-4 ±2-12 and 88-9 ±2-12,
respectively for 6, 12, 30 and 53 data item models. At the
optimum diagnostic threshold (15%), the six-factor
model had sensitivity, specificity and accuracy of 80-4%,
85-3% and 84-3%, respectively and a sensitivity for
diagnosis of Q-wave acute myocardial infarction of
89-9%. The equivalent figures for the 53 factor model
were 79-4%, 83-1%, 82-4% and 89-9%, respectively (not
significant).
Testing of logistic regression models on
Sheffield data
The performance of four models on Sheffield data is
shown in Fig. 3. The areas under the receiver operating
characteristic curves were 92-3 ±1-30, 93-6 ± 1 0 5 ,
92-8 ± 118 and 87-7 ± 1-70, respectively for 6, 12, 30 and
53 factor models. The difference between the first three
models was not significant but there was a significant
decline in performance when all 53 factors were used
(P<0-0\). At a diagnostic threshold of 15%, the sixEur Heart J, Vol. 17, August 1996
factor model had sensitivity of 91-9%, specificity of
80-2%, accuracy of 83-6% and positive predictive value
of 66-3%. The vast majority of patients who were
mis-diagnosed by this model were patients with unstable cardiac disease but without acute myocardial
infarction, as shown in Fig. 4. When logistic regression
models were derived using Sheffield data there was
again an increase in performance of the models with
progressive addition of data items as shown in Fig. 5.
The areas under the receiver operating characteristic
curves were 92-8 ± 1-24, 947 ±0-93, 96-3 ± 0-69 and
97-5 ±0-62, respectively for 6, 12, 30 and 53 factor
models. We did not test these models on an independent set of Sheffield data.
Comparison with a model derived from only
electrocardiogram data
A model derived only from electrocardiographic data
would be more readily applied to retrospective cases
than one which incorporates clinical factors, which
cannot always be gathered reliably from case notes. The
equation for a model which used only electrocardiographic data is shown in Table 2. The area under the
curve for this model tested on the 510 Edinburgh cases
was 88-7 ± 2-24 which was not significantly different for
the six-factor model (including items from the clinical
Downloaded from by guest on October 15, 2014
Figure 2 The 54 data items for the Sheffield data set. The items are arranged in exactly the same order as those
shown in Fig. 1 for the Edinburgh data.
Logistic regression models for AMI diagnosis
1185
Table 1 Constants and coefficients for logistic regression models
Number of inputs
Input
-307
316
1-37
1 95
1 54
0-47
0-68
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
-3-83
312
109
1-78
1 38
Oil
0-63
0-97
0-86
0-84
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
12
-4-50
312
0-96
1-48
1-38
009
0-65
0-95
0-85
0-54
0-95
0-79
0-69
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
20
-504
3-45
0-98
1 61
1-37
-006
0-59
0-84
0-8
0-63
0-88
0-96
0-87
-0-90
-0-31
-0-38
0-86
0-53
007
0-48
-006
—
—
—
—
30
40
-5-42
3-59
1-27
203
1-78
007
0-55
1-34
0-76
0-29
107
0-90
002
- 1-57
-0-27
-013
0-80
0-27
0-24
0-87
— 0-21
-004
-1-38
0-42
-007
216
103
0-80
005
-9-45
-0-47
-0-47
-0-25
-6-30
-011
0-50
-1-03
-106
0-96
-013
807
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
— 5-71
3-67
0-90
1-70
1-67
0005
0-25
106
0-65
0-39
0-93
0-94
0 31
- 114
-0-33
-0-08
0-91
0-32
016
0-74
011
011
-119
0-22
-0-26
1-94
0-90
0-96
0-06
-216
-0-28
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
—
history) described above. At the optimum threshold,
sensitivity, specificity, accuracy and positive predictive
value of the electrocardiographic data model were
720%, 93-5%, 890% and 74-8%, respectively. For
Sheffield data, the area under the receiver operating
characteristic curve for the electrocardiographic data
—
53
- 8 12
4-00
1 49
2-27
210
003
0-52
1-39
0-80
0-43
1 35
1 37
-015
- 1-70
- 0 49
-015
0-89
0-43
0-29
0-75
-0-49
-0-32
- 109
0-26
-0003
2-50
113
018
011
- 9 10
-018
-009
-0-33
-6-42
-0-37
0-40
-1-18
-0-75
1-33
011
7-72
- 1-22
-1-72
0-39
0-70
0-72
006
-0-52
- 1 99
2-35
-0-46
-0-22
- 1-27
-0-16
Downloaded from by guest on October 15, 2014
Constant
ST elevation
New Q waves
Hypoperfusion
ST depression
Vomiting
LVF
T wave inversion
Pain in right arm
Nausea
Sweating
Age 5
Pain 2
Syncope
Pain in neck/jaw
Age 7
Age 6
Pain in back
Pain in left arm
Tight pain
Hypertension
Sharp pain
Breathing
Retrosternal pain
Pain in left chest
BBB
Dull pain
Smoker
Family history
Duration
Pain 1
Old ischaemia
Short of breath
Age 1
Worse angina
Episodic
Previous Ml
Old Q waves
Posture
Pain in right chest
Pain 5
Tender
Age 2
Previous angina
Sex
Age 3
Pain 3
Not sinus rhythm
Palpitations
CP major
Diabetes
Rate
Ex-smoker
Hyperlipidaemia
6
9
model was 90-1 ± 1 50 compared with 92-2 ± 1 30 for the
six-factor model incorporating clinical data (not significant). The sensitivity, specificity, accuracy and positive
predictive value of the electrocardiographic data model
on Sheffield data were 87-3%, 83-8%, 84-8% and 69-6%,
respectively.
Eur Heart J, Vol. 17, August 1996
1186
R. L. Kennedy et al.
100
20
40
60
100-specificity
80
100
Figure 3 Receiver operating characteristic curves for the performance of
four models on Sheffield data (
, 6;
, 12; • • • •, 30;
,
53). The models were derived from 600 Edinburgh patients. There was no
difference between the performance of models derived using 6, 12 and 30
data items. However, the use of 53 data items was associated with a
significant decline in performance (P<0-01, see text).
Downloaded from by guest on October 15, 2014
100
I 80
3
o
e
60
S J
3
I
•a 4 0 ^
bjj
is
&
20
t
Q wave
Non-Q
Unstable
Stable
Other
Figure 4 Performance of a six-factor model on Sheffield data. The y-axis shows
the percentage of patients in each category diagnosed as having acute myocardial
infarction by the model. Patients are divided into groups according to whether
they have Q wave or non-Q wave acute myocardial infarction, unstable or stable
angina or other diagnoses.
Comparison with published models
In this part of the study, the performance of two
previously published models was examined. We have
compared the performance of the models using the
published coefficients with new models derived using
Eur Heart J, Vol. 17, August 1996
the same data inputs but with coefficient calculated from
our training set of 600 cases. The performance of these
models was evaluated using the 662 cases from Sheffield.
The model described by Tierney et al.[l3] makes use of
two electrocardiographic and two clinical factors, while
the model described by Selker et alJl5] is more complex
Logistic regression models for AMI diagnosis
1187
100 r
20
40
60
100-specificity
80
100
Figure 5 Receiver operating characteristic curves for logistic regression
models derived from, and tested on, the Sheffield data. There was a
significant increase in diagnostic performance with increasing numbers of
data inputs (P<001, ANOVA). (
, 6;
, 12; • • •, 31;
, 53).
Inputs
Table 3 Logistic regression models based on previously
published models
Coefficients
Coefficient
Input
Constant
BBB
New Q waves
Not sinus rhythm
ST depression
ST elevation
T wave inversion
Tierney
Selker
-418
2-33
2-19
-3-93
0-31
0-62
—
-3-37
1-73
0-90
-0-80
1-70
3-52
117
with 12 inputs. The coefficients for these models are to
be found in the relevant papers, while the coefficients
derived from our cases are shown in Table 3.
Areas under the receiver operating characteristic
curves for the Tierney model were 85-1 ± 169 using the
published coefficients and 885 ± 1 -53 for the calculated
coefficients (not significant). Sensitivity for diagnosis of
acute myocardial infarction (using the calculated coefficients) was only 54-9% compared with 91-9% for
the six-factor model derived from Edinburgh data
(/><0-001). The specificity, accuracy and positive predictive value for the model were 97-6%, 84-9% and 908%,
respectively. For the Selker model, the areas under the
receiver operating characteristic curves were 85-2 ± 1-52
and 89-4 ± 1-39, respectively using published and calculated coefficients(P<005). With a sensitivity, specificity,
accuracy and positive predictive value of 87-3%, 804%,
82-5% and 65-3%, respectively, the performance of this
model was very similar to that of the models which we
have derived de novo for this study.
Constant
ST elevation
New Q waves
Sweating
Previous Ml
Presence of chest pain
Pain major symptom
Sex
Age 40 or less
Age >50
Male over 50 years
ST depression
T waves elevated
T waves inverted
T wave + ST changes
1-4?
lO:^_
.—
—.
—.
—
1-23
0-88
0-71
- 1 44
0-67
-0-43
0-99
009
1-13
- 0-314
Discussion
We have described the derivation and testing of logistic
regression models for the early diagnosis of acute myocardial infarction. The present study confirms the findings of others that this technique is appropriate for the
analysis of clinical and electrocardiographic data from
patients presenting with acute chest pain/suspected
acute myocardial infarction'13~15*221. Logistic regression
models have also been described for estimating probability of ischaemic heart disease in patients referred
for coronary angiography126' and for prognostic
stratification of patients following acute myocardial
Eur Heart J, Vol. 17, August 1996
Downloaded from by guest on October 15, 2014
Table 2 Logistic regression model derived using only
electrocardiographic data
1188 R. L. Kennedy et al.
infarction'271. In this study, we have examined in detail
the influence of varying the data input items on the
performance and portability of models. When tested on
the training data, the performance of logistic regression
models increased with increasing number of data input
items. This relationship was demonstrated with data
from both centres involved in the study. However,
simple models comprising only six data items performed
at least as well as more complex models on unseen, test
data. The optimum diagnostic thresholds for the models
were determined using receiver operating characteristic
curve analysis, and were identical for the two populations of patients included in the study. Optimal performance of simple models may only be possible if the
models are specifically derived for use in a given centre,
using the most predictive factors for patients presenting
to that centre.
Eur Heart J, Vol. 17, August 1996
The time-insensitive predictive instrument (TIPI)
developed by Selker et a/.(15] made use of seven data
items available in the emergency room. Like the models
described here, the time-insensitive predictive instrument
included both clinical and electrocardiographic data but
the model was slightly more complex than our six-factor
model because a number of the items were combined,
giving rise to a total of 12 inputs. The model described
by Selker et al. was developed using data from 3453
patients in six different hospitals and was tested on a
further 2320 cases. The area under the receiver operating
characteristic curve in Selker's study was 088, which is
similar to the values obtained for our models. Patients
could be divided into four groups using the timeinsensitive predictive instrument and the incidence of
acute ischaemic heart disease increased progressively
Downloaded from by guest on October 15, 2014
The potential inputs for our models were ranked
in order of their likelihood ratio (with respect to diagnosis of acute myocardial infarction) for the cohorts of
patients from two different centres. Although there was
a strong correlation between the likelihood ratios for the
data items in the two cohorts, there were also important
differences which account for the impaired performance
of more complex models on unseen data from a centre
other than that in which the models were derived. The
difference in likelihood ratios was most apparent with
cardiovascular risk factors such as smoking, diabetes
and hyperlipidaemia. It is not clear whether these variations reflect differences in the pathogenesis of coronary
artery disease in the two populations. Conclusions on
the comparative epidemiology of IHD would be impossible to draw from the numbers involved in this study.
The most discriminatory items were, however, identical
between the two data sets, suggesting that simple diagnostic models may be more portable than more complex
models.
Although the admission electrocardiogram is
highly predictive of acute myocardial infarction, it
is well known that diagnostic changes are only apparent
at presentation in about half of the patients who ultimately prove to have acute myocardial infarction'1 •2-28\
It is possible, but not documented, that this proportion
may be diminishing as public awareness and advances in
pre-hospital care are leading to patients presenting to
hospital earlier after the onset of symptoms. The 12-lead
electrocardiogram at presentation, and in the hours
following admission, is also important prognostically —
a normal electrocardiogram being associated with a
very low risk of death or other major cardiovascular
event[4'5>28). There was very little difference, on either test
set of data, between the performance of our model
which relied solely on electrocardiogram inputs and the
models which also incorporated clinical data items. This
is not surprising since three of our four most discriminatory data times were derived from the admission
electrocardiogram. The electrocardiogram data model
was much more sensitive, but had lower specificity and
overall accuracy, on Sheffield data compared with
Edinburgh data. This probably reflects the higher inci-
dence of electrocardiographs abnormalities at presentation in the Sheffield cohort. The reason for this is not
clear. It is not due to a difference in the timing of
presentation — the median time from start of symptoms
to presentation was 3 h in both cohorts. In our previous
studies'17181, we have investigated the performance of
models derived using only clinical data. Such models
perform much less well than models which make use of
electrocardiogram data: diagnostic performance typically declines by 10-15%, and the models are less
portable than those presented here. In each of the two
patient populations studied here, three of the five most
predictive factors were derived from the electrocardiogram at presentation.
We compared the performance of our logistic
regression models with previously published models.
The model described by Tierney et a/.'l3) is a very simple
one comprising two items each of electrocardiographic
and clinical data. The area under the receiver operating
characteristic curve for Tierney's test data was 850,
which is virtually identical to that for our data when the
published coefficients were used. The model performed
slightly, but not significantly, better when the coefficients
were recalculated using our own data. Tierney et al.
found that their model was significantly more specific
and accurate than the physicians involved in their study.
We have not presented data on the diagnostic performance of physicians in the present study. Tierney et al.
reported a positive predictive value of 490% for their
model, which is much less than the 90-8% described with
identical data inputs in our study. It seems reasonable
that a model which relies heavily on electrocardiogram
inputs will be very specific but rather less sensitive than
a model which takes greater account of clinical factors.
The negative predictive value in Tierney's study was
high at 970%, compared with 836% for the equivalent
model in this study. These discrepancies must reflect
differences in the patient population under study. A
recent study showed that the model described by Tierney
et al. and another logistic regression model described
by Pozen et a/.'141, performed poorly on data from
pre-hospital patients in Rotterdam'291. As with the
present study, these authors found that a model derived
from their own data performed markedly better.
Logistic regression models for AMI diagnosis
Logistic regression has a major advantage in that
it is a simple and widely used statistic technique which
can be applied readily to new data using standard
statistical packages. The equation derived could easily
be implemented on a hand-held programmable calculator, and the value obtained interpreted as a probability
of acute myocardial infarction. If models consisting of
only a few data items perform, as suggested here, as well
as more complex models then it would be relatively
simple to optimize a logistic regression equation for use
in individual clinical settings. Many of the more recently
described algorithms for early diagnosis of acute
myocardial infarction are relatively complex, difficult
to adapt and make use of many (up to 50 or more)
data input items. The methods used include decision
trees'12', Bayesian inference12'1 and artificial neural networks'17'19'. While these methods may offer advantages
in terms of operator feedback, and in the case of neural
networks, more sophisticated pattern recognition capability, their advantage over a simple technique such as
logistic regression, needs to be demonstrated before they
are widely applied. Also, it is important to be clear that
the number and nature of the input data items being
used is appropriate.
In conclusion, logistic regression is a simple
method with which to derive predictive models for
patients presenting with suspected acute myocardial
infarction. A relatively small number of data input items
may yield optimal performance. The electrocardiogram
inputs are by far the most important and models with
only electrocardiogram data perform almost as well as
those with clinical data items in addition. Care should be
exercised in using published logistic regression models,
although these models may help in selecting data inputs
for locally-derived models. The achievement of complex,
computer-based algorithms should be viewed with caution and compared carefully with the performance of
simple models of the sort described in the present study.
This project was supported by the Scottish Office Home and
Health Department (Grant No. K/MRS/56/C2066) for collection
of data in Edinburgh and by Altim Medical Systems Limited,
Belper, Derbyshire, U.K. for collection of data in Sheffield. We are
very grateful to the Physicians, Accident and Emergency Consultants and General Practitioners in both centres who allowed us to
include their patients in the study and to Ms Janine Robinson who
helped us with analysis of the data. The study would not have been
possible without the active co-operation of the Accident and
Emergency doctors who helped us by filling out proformas and by
providing follow-up data. We would particularly like to acknowledge the following — Drs D Begg, M. Bell, S. Cowell, A. Dodds,
G Foster, B. Halliday, I. Hay, R. Hill, J. Lennox, J. Poole, M.
Strachan, G. Watt, A. Campbell, G. Campbell-Hewson, A. de
Beaux, U. Guly, R. MacCallum, S. Maurice, R. Mitchell, M.
Mnalane, P. Stuart, J. Watters, R. Casasola, J. Dalgliesh, C. Fan,
E. Forrest, A. McGhee, D. Newby, R. Parris, J. Rigden, R.
Rintoul and S. Robertson.
References
[1] Brush JE, Brand DA, Acampora D, Chalmer B, Wackers FJ.
Use of the initial electrocardiogram to predict in-hospital
complications of acute myocardial infarction. N Engl J Med
1985; 312: 1137-41.
[2] Stark ME, Vacek JL. The initial electrocardiogram during
admission for myocardial infarction. Use as a predictor of
clinical course and facility utilization. Arch Intern Med 1987;
147: 843-6.
[3] Yusuf S, Pearson M, Parish S, Ramsdale D, Rossi P, Sleight
P. The entry ECG in the early diagnosis and prognostic
stratification of patients with suspected acute myocardial
infarction. Eur Heart J 1984; 5: 690-6.
[4] Gheorghiade M, Anderson J, Rosman H et al. Risk identification at the time of admission to coronary care unit in
patients with suspected myocardial infarction. Am Heart J
1988; 116: 1212-17.
[5] Karlson BW, Herlitz J, Wiklund O, Richter A, Hjalmarson A.
Early prediction of acute myocardial infarction from clinical
history, examination and electrocardiogram in the emergency
room. Am J Cardiol 1991; 68: 171-5.
Eur Heart J, Vol. 17, August 1996
Downloaded from by guest on October 15, 2014
from 1-6% in the low probability group to 81-6% in the
high probability group. It is not surprising, that when
tested on our Sheffield data using the published coefficients, the time-insensitive predictive instrument did
not perform quite as it did on Selker's data. However,
the area under the receiver operating characteristic curve
improved (/><005) when the model was re-derived using
our Edinburgh data. This suggests that the selection of
the variables to be included in the time-insensitive
predictive instrument is appropriate. The authors are
not explicit about how these variables were selected from
the 120 candidate variables, but state that this selection
was based on known clinical importance, predictive
power and reliability when abstracted from patient's
case notes.
We have not attempted to test the impact that
the use of logistic regression models might have on
clinical practice. It is important before embarking on
such a study to optimize the model being tested and to
estimate the size of study which will be required to show
a significant impact. This can be done conveniently
using receiver operating characteristic analysis'241. Thus,
we can estimate from data we have previously collected
on the diagnostic performance of Accident and Emergency doctors'17181 that a study of around 1600 patients
would be necessary to demonstrate a significant difference (95% probability) between doctors and an
algorithm of the sort described here. It is, of course,
impossible to estimate the exact impact the use of a
decision support aid would have on clinical practice but
it is important to be clear that an algorithm under test
performs accurately and reliably on the patient population under study as well as showing that it has the
capacity, not only to model, but also to enhance clinical
practice. The potential benefits have to be weighed
against the cost of using a computer-based device, for
example, in our previous studies'17181, entering data into
a computer program could take up to 10 min extra per
patient. It is not clear at present whether the advantages
previously shown to be associated with the use of
diagnostic algorithms'1 li20>211 arise simply by ensuring
that clinical data are analysed in a standard fashion, or
whether additional information is generated allowing
the doctors to identify particular subgroups of patients.
A formal trial of any algorithm should correct for the
benefits associated with standardized history taking,
which is an integral part of using such an algorithm.
1189
1190 R. L. Kennedy et al.
Eur Heart J, Vol. 17, August 1996
[18]
[19]
[20]
[21]
[22]
[23]
[24]
[25]
[26]
[27]
[28]
[29]
Intelligence in Medicine, Maastricht, Netherlands. 1991;
119-28.
Harrison RF, Marshall SJ, Kennedy RL. Neural networks,
heart attack, and Bayesian decisions: an application of the
Boltzmann perceptron network. J Art Neural Networks 1994;
1- 183-202.
Baxt WG. Use of an artificial neural network for the diagnosis of myocardial infarction. Ann Intern Med 1991; 115:
843-8.
Pozen MW, D'Agostino RB, Selker HP, Sytkowski PA, Hood
WB. A predictive instrument to improve coronary-care-unit
admission practices in acute ischaemic heart disease. A prospective multicenter clinical trial. N Engl J Med 1984; 310:
1273-8.
Jonsbu J, Aase O, Rollag A, Liestol K, Eriksen J. Prospective
evaluation of an EDB-based diagnostic program to be used in
patients admitted to hospital with acute chest pain. Eur Heart
J 1993; 14: 441-6.
Long WJ, Griffith JL, Selker HP, D'Agostino RB. A comparison of logistic regression to decision-tree induction in a
medical domain. Comp Biomed Res 1993; 26: 74-97.
Zweig MH, Campbell G. Receiver-Operating Characteristic
Curve (ROC) plots: A fundamental evaluation tool in clinical
medicine. Clin Chem 1993; 39- 561-7
Hanley JA, McNeil BJ. The meaning and use of the area
under a receiver operating characteristic (ROC) curve. Radiol
1982; 143: 29-36.
Hanley JA, McNeil BJ. A method for comparing the areas
under receiver operating characteristic curves derived from the
same cases. Radiol 1983; 148: 839^3.
Detrano R, Bobbio M, Olson H et al. Computer probability
estimates of angiographic coronary artery disease: Transportability and comparison with a cardiologist's estimates. Comp
Biomed Res 1992; 25: 468-85.
Parsons RW, Jamrozik KD, Hobbs MST, Thompson DL.
Early identification of patients at low risk of death after
myocardial infarction and potentially suitable for early hospital discharge. Br Med J 1994; 308: 1006-10.
Lee TH, Cook EF, Weisberg MC, Sargent RK, Wilson C,
Goldman L. Acute chest pain in the emergency room. Identification of low-risk patients. Arch Intern Med 1985; 145: 65-9.
Grijseels EWM, Deckers JW, Hoes AW et al. Pre-hospital
tnage of patients with suspected myocardial infarction. Evaluation of previously developed algorithms and new proposals.
Eur Heart J 1995; 16: 325-32.
Downloaded from by guest on October 15, 2014
[6] Kennedy RL, Harrison RF, Marshall SJ. Do we need
computer-based decision support for the diagnosis of acute
chest pain? J Royal Soc Med 1993; 86: 31-4.
[7] Pipberger HV, Klingeman JD, Cosma J. Computer evaluation
of statistical properties of clinical information in the differential diagnosis of chest pain. Meth Inform Med 1968, 7. 79-93
[8] Lubsen J, Pool T, van der Does E. A practical device for the
application of a diagnostic or prognostic function. Meth
Inform Med 1978; 17: 127-9.
[9] Boissel JP, Vanerie R. Systeme d'aide a la decision a la phase
aigue de l'infarctus du myocarde. Proceedings of the International Symposium on Medical Informatics, Toulouse,
France 1977; 571-83.
[10] Aase O, Jonsbu J, Liestol K, Rollag A, Eriksen J. Decision
support by computer analysis of selected case history variables
in the emergency room among patients with acute chest pain
Eur Heart J 1993, 14: 433^40.
[11] Goldman L, Weinbeg M, Weisberg M C e l al. A computerdenved protocol to aid in the diagnosis of emergency room
patients with acute chest pain. N Engl J Med 1982; 307:
588-96.
[12] Goldman L, Cook EF, Brand DA et al. A computer protocol
to predict myocardial infarction in emergency department patients with chest pain. N Engl J Med 1988; 318:
797-803.
[13] Tierney WM, Roth BJ, Psaty B el al. Predictors of myocardial
infarction in emergency room patients. Crit Care Med 1985,
13: 526-31.
[14] Pozen MW, D'Agostino RB, Mitchell JB el al. The usefulness
of a predictive instrument to reduce inappropriate admissions to the coronary care unit. Ann Intern Med 1980; 92:
238-42
[15] Selker HP, Griffith J, D'Agostino RB. A tool for judging
coronary care unit admission appropriateness, valid for both
real-time and retrospective use. A time-insensitive predictive
instrument (TIPI) for acute cardiac ischaemia: A multicenter
study. Med Care 1991; 29: 610-27
[16] Emerson PA, Wyatt J, Dillistone L, Crichton N, Russell NJ.
The development of ACORN, an expert system enabling
nurses to make decisions about patients with chest pain in
an accident and emergency department. Proceedings of the
Conference on Medical Informatics in Clinical Medicine,
Nottingham, U.K. 1988; 37-40.
[17] Harrison RF, Marshall SJ, Kennedy RL. A connectionist
approach to the early diagnosis of myocardial infarction.
Proceedings of the 3rd European Conference on Artificial
Logistic regression models for AMI diagnosis 1191
Appendix
Binary inputs to the ANN
Input node
Age in years (under 30, 30-39, 40-49, 50-59, 60-69, 70-79, 80 and over)
Smoker
Ex-smoker
Family history of ischaemic heart disease
Diabetes mellitus
Hypertension
Hyperlipidaemia
Is chest pain the major symptom''
Central chest pain
Pain in left side of chest
Pain in right side of chest
Pain radiates to back
Pain radiates to left arm, neck or jaw
Pain radiates to right arm
Worse on inspiration
Pain related to posture
Chest wall tenderness
Pain described as sharp or stabbing
Pain described as tight, heavy, gripping or crushing
Sweating
Short of breath
Nausea
Vomiting
Syncope
Episodic pain
Hours since 1st symptom (0-5, 6-10, 11-20, 21-40, over 40)
Hours of pain at presentation (0-5, 6-10, 11-20, 21-40, 41-80, over 80)
History of angina
Previous myocardial infarction
Worse than usual angina/similar to previous acute myocardial infarction
Fine crackles suggestive of pulmonary oedema
Added heart sounds
Signs of hypoperfusion
New ST-segment elevation
New pathological Q waves
ST segment or T-wave changes suggestive of ischaemia
Bundle branch block
Old electrocardiogram features of myocardial infarction
Electrocardiogram signs of ischaemia known to be old
Downloaded from by guest on October 15, 2014
1-7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32-36
37-42
43
44
45
46
47
48
49
50
51
52
53
54
Parameter
Eur Heart J, Vol. 17, August 1996