Article critique: Binary logistic Regression
ORIGINAL RESEARCH
published: 12 June 2018
doi: 10.3389/fpsyt.2018.00258
Frontiers in Psychiatry | www.frontiersin.org 1 June 2018 | Volume 9 | Article 258
Edited by:
Meichun Mohler-Kuo,
University of Applied Sciences and
Arts of Western Switzerland,
Switzerland
Reviewed by:
Eric Noorthoorn,
GGNet Mental Health Centre,
Netherlands
Raoul Borbé,
Universität Ulm, Germany
*Correspondence:
Florian Hotzy
Specialty section:
This article was submitted to
Public Mental Health,
a section of the journal
Frontiers in Psychiatry
Received: 08 November 2017
Accepted: 24 May 2018
Published: 12 June 2018
Citation:
Hotzy F, Theodoridou A, Hoff P,
Schneeberger AR, Seifritz E, Olbrich S
and Jäger M (2018) Machine
Learning: An Approach in Identifying
Risk Factors for Coercion Compared
to Binary Logistic Regression.
Front. Psychiatry 9:258.
doi: 10.3389/fpsyt.2018.00258
Machine Learning: An Approach in
Identifying Risk Factors for Coercion
Compared to Binary Logistic
Regression
Florian Hotzy1*, Anastasia Theodoridou1, Paul Hoff1, Andres R. Schneeberger2,3,4,
Erich Seifritz1, Sebastian Olbrich1 and Matthias Jäger1
1 Department for Psychiatry, Psychotherapy and Psychosomatics, University Hospital of Psychiatry Zurich, Zurich,
Switzerland, 2 Psychiatrische Dienste Graubuenden, Chur, Switzerland, 3 Universitaere Psychiatrische Kliniken Basel,
Universitaet Basel, Basel, Switzerland, 4 Department of Psychiatry and Behavioral Sciences, Albert Einstein College of
Medicine, New York, NY, United States
Introduction: Although knowledge about negative effects of coercive measures in
psychiatry exists, its prevalence is still high in clinical routine. This study aimed at define
risk factors and test machine learning algorithms for their accuracy in the prediction of
the risk to being subjected to coercive measures.
Methods: In a sample of involuntarily hospitalized patients (n = 393) at the University
Hospital of Psychiatry Zurich, we analyzed risk factors for the experience of coercion
(n = 170 patients) using chi-square tests and Mann Whitney U tests. We trained machine
learning algorithms [logistic regression, Supported Vector Machine (SVM), and decision
trees] with these risk factors and tested obtained models for their accuracy via five-fold
cross validation. To verify the results we compared them to binary logistic regression.
Results: In a model with 8 risk-factors which were available at admission, the SVM
algorithm identified 102 out of 170 patients, which had experienced coercion and 174
out of 223 patients without coercion (69% accuracy with 60% sensitivity and 78%
specificity, AUC 0.74). In a model with 18 risk-factors, available after discharge, the
logistic regression algorithm identified 121 out of 170 with and 176 out of 223 without
coercion (75% accuracy, 71% sensitivity, and 79% specificity, AUC 0.82).
Discussion: Incorporating both clinical and demographic variables can help to estimate
the risk of experiencing coercion for psychiatric patients. This study could show that
trained machine learning algorithms are comparable to binary logistic regression and
can reach a good or even excellent area under the curve (AUC) in the prediction of
the outcome coercion/no coercion when cross validation is used. Due to the better
generalizability machine learning is a promising approach for further studies, especially
when more variables are analyzed. More detailed knowledge about individual risk factors
may help to prevent the occurrence of situations involving coercion.
Keywords: coercion, seclusion, restraint, coercive medication, involuntary hospitalization, machine learning
Hotzy et al. Machine Learning and Coercion
INTRODUCTION
The use of coercive measures (e.g., seclusion, physical and
mechanical restraint, forced medication) in psychiatric patients is
a massive invasion in their integrity and freedom. As a result, the
usage of coercion is controversially discussed since the beginning
of modern psychiatry and certain approaches have tried to reduce
its rates (1). Although some of those approaches were successful,
there are still many patients in which coercion is used. Often the
usage of coercion seems necessary when the patients are a danger
for themselves or for others due to an underlying psychiatric
disorder (2, 3). These situations are always associated with an
ethical dilemma. On one side coercion shall help to protect the
patient’s or other’s integrity (2, 3). On the other hand it restricts
the freedom of the person which is one of the basic human rights
(4). Being a threat to oneself or others may have different reasons
in psychiatric patients. In some situations patients are delusional
and feel threatened by others which leads to the reaction to
protect themselves and can result in threats to other patients or
staff (5). Also in situations where the patients are threatening
themselves or have suicidal ideations caused by the symptoms
of their psychiatric disorder, coercive measures might become
necessary to secure the patients survival.
The use of coercion distinguishes psychiatry from other
medical disciplines where informed patients can decide to accept
or reject a specific measure. Psychiatry at one hand aims to help
the patients to develop a self-determined life without burden of
psychiatric symptoms. On the other hand psychiatry is legally
determined to reject the patients freedom to move (involuntary
hospitalization) but also the freedom to reject a specific measure
(forced medication, physical or mechanical restraint, seclusion)
if harm to self or others has to be disrupted.
It is obvious that such situations are challenging for the
patients but also for the therapeutic team. Those challenges
were topic of previous studies where it was shown that patients
who experienced coercive measures often describe feelings of
helplessness (6, 7), fear (8), anger (9, 10) and humiliation
(11). Due to that, some patients stated to avoid searching for
psychiatric help in a crisis (12, 13). On the other hand there
were some patients who retrospectively agree with the coercive
measure (7, 9) and state that they would like to be forced into
treatment again in the case of a future crisis (14). These contrary
findings underline the controversy of this topic.
It was the goal of earlier studies to understand which patients
experience coercion and to characterize their clinical, but also
their socioeconomic features. Gaining better understanding of
risk factors to experience coercion was thought to be helpful in
the development of therapeutic strategies for patients at risk and
thus, to reduce the prevalence of coercion.
During the last years specialized psychiatric intensive care
units (PICU) had been the center of extensive research and it
could be shown that some patient characteristics are associated
with the transfer from a general psychiatric unit to a PICU
and with the usage of coercion on these specialized wards
(15). Furthermore psychotic disorders were shown to be
frequently associated with coercion (16–24). Also personality
disorders (25, 26), substance-use-related disorders (19) and
mental retardation (25) were found to be associated with
coercion. A history of aggression (16–18, 22, 23, 25, 27–
29) was frequently found to be associated with coercion and
violence/threats were described to be the second most frequent
reasons after agitation/disorientation for the usage of coercion
(30). Patients with a history of former voluntary and/or
involuntary commitments (IC) and frequent hospitalizations
(16–20, 24) and those with longer duration of hospitalizations
(31) were also described to experience coercion more often.
Those factors were described nearly uniformly throughout
literature. Whereas other factors like male (20, 23–25, 32, 33)
and female gender (22, 29) or younger (19, 20, 23, 25, 28, 29,
32, 33) and older age (22, 24) were controversially associated
with coercion in different study sites. These inconsistent findings
impede the definition of risk-factors which are independent of
specific countries. The inconsistencies between study sites were
discussed to be caused by cultural influences, organizational
factors, societal factors, the clinic-culture or a combination (34,
35). Besides that, one has to bear in mind that prior studies
followed different methodological approaches to analyze data
which additionally limits the comparability between different
study sites. Some studies used descriptive approaches (16, 32)
or group comparisons with binominal, non-parametric tests or
ANOVA (17–20, 22–24, 26, 29, 30). To describe risk factors
regression analysis was frequently used (19–21, 23, 26, 28, 29, 31,
33) and some studies extended their findings with an estimation
of the area under the curve (AUC) (23). One study used a
latent class analysis (LCA) which is capable of detecting the
presence of groups in individuals with relatively homogeneous
clinical courses (25). Another study used Multilevel random
effects modeling (27). Only a few studies tried to describe the
potency of specific risk factors to affect the outcome coercion/no
coercion. Furthermore, the description of the specificity and
sensitivity of the statistical models is scarce. One study which
followed this approach described an acceptable AUC for one
model using bivariate analysis (23). Another study found that
with the included parameters only a limited prediction of patients
at risk was possible (31). Thus, besides the analysis of risk factors
at our study site, the second aim of this study was to find statistical
approaches with a good balance in their specificity and sensitivity
and prediction accuracy for the outcome “coercion/no coercion”
in psychiatric inpatients. Furthermore we wanted to analyze the
risk factors for their weights in affecting the outcome coercion/no
coercion.
In today’s psychiatric research machine learning is an
emerging methodology. It is connoted with a great potential
for innovation and paradigm shift as the algorithms facilitate
integration of multiple measurements as well as allow objective
predictions of previously “unseen” observations. We used this
new approach to train and compare models with parameters
available at admission and after discharge. To test for the
hypothesis that machine learning algorithms are effective in the
prediction of the outcome coercion/no coercion in psychiatric
patients we compared binary regression analysis to the machine
learning algorithms according to their sensitivity, specificity,
accuracy, and AUC. Furthermore, we used machine learning to
weight the included predictors for their potency in affecting the
Frontiers in Psychiatry | www.frontiersin.org 2 June 2018 | Volume 9 | Article 258
Hotzy et al. Machine Learning and Coercion
outcome coercion/no coercion. For the comparison of the two
approaches we analyzed clinical data of involuntarily hospitalized
patients at the University Hospital of Psychiatry Zurich and built
two groups depending on the outcome Coercion/No Coercion.
METHODS
Setting
The study was reviewed and approved by the Cantonal Ethics
Commission of Zurich, Switzerland (Ref.-No. EK: 2016-00749,
decision on 01.09.2016). Commitment documents as well as
the medical records of patients involuntarily hospitalized at
the University Hospital of Psychiatry Zurich during a 6-month
period from January first to June 30, 2016 were analyzed.
N = 16 wards of the University Hospital of Psychiatry Zurich
with a total of 252 beds were included. The clinic provides mental
health services for a catchment area of 485,000 inhabitants.
Study Sample
No exclusion criteria were defined. We screened a comprehensive
cohort of all patients admitted voluntarily and involuntarily to
the University Hospital of Psychiatry Zurich during a 6-month
period from January first to June 30, 2016 (n = 1,699 patients).
For the analysis we included involuntarily committed patients
(n = 577) and voluntarily committed patients who were retained
at a later stage during their hospitalization and then changed to
the legal status of involuntary hospitalization (n = 35).
Selection of Predictor Variables
Selection of predictor variables for “training” an algorithm
in machine learning is challenging. We used a recommended
method and searched the literature databases for variables
which were already described to be associated with the usage
of coercion: Psychiatric diagnosis (16–24), aggressive behavior
(16–18, 22, 23, 25, 27–30), former voluntary or involuntary
commitment (IC) and frequent hospitalizations (16–20, 24),
gender (20, 22–25, 29, 32, 33), and age (19, 20, 22, 23,
25, 28, 29, 32, 33) were identified as variables of interest.
We searched the routine documentation in the electronic
medical files of the patients for these variables. The medical
files include documentation about the socio-demographic
parameters, admission circumstances, prescribed medication,
documentation of coercive measures, and treatment planning. As
there was no standardized assessment for aggression we searched
which indirect information could be used and found IC due to
danger to others and involvement of police in the admission
process as indirect markers for aggressive behavior. Furthermore
we included the procedural aspects abscondence, appeal to
the court, duration until day passes, duration of IC, duration
of hospitalization into analysis. When patients are exposed to
coercive medication mostly antipsychotics or benzodiazepines
are used. We were interested if the patients, exposed to coercion
differed from those without coercion according to their regular
prescribed medication during hospitalization. Thus, we searched
the medical files for the prescription of medication classes
(antipsychotics, antidepressants, benzodiazepines, and others).
Analysis and Machine Learning
We conducted analysis with MATLAB (MATLAB and Statistics
Toolbox Release 2012b, The MathWorks, Inc., Natick,
Massachusetts, United States.) and SPSS 23.0 (IBM Corp.
Released 2011. IBM SPSS Statistics for Windows, Version 23.0.
Armonk, NY: IBM Corp.) for Windows.
In a first step we compared patients with/without experience
of coercion. We used cross-tabulation with chi-square tests for
categorical variables. Due to the non-normal distribution we
used Mann–Whitney tests for numeric variables. Variables that
differed between both groups in bivariate analyses were included
as potential risk factors in multivariate analysis. To analyze the
impact of the risk factors on the outcome coercion/no coercion
binary logistic regression analysis was used with coercion/no
coercion as the dependent variable. The goodness of fit of the
binary logistic regression model was assessed by the receiver
operating characteristic (ROC) curve method. The AUC served
as the criterion to determine the level of discrimination.
Discrimination was deemed acceptable at AUC values between
0.7 and 0.79, excellent at values between 0.8 and 0.89, and
outstanding at values over 0.9 (23). The specificity and sensitivity,
positive predictive value (PPV) and negative predictive value
(NPV) were calculated from the results of the different models.
Because of multiple comparisons Bonferroni’s adjustments
were made to prevent Type I error inflation (α = 0.05/5 = 0.01).
In a second step we tested the hypothesis that machine
learning algorithms can be used to predict the outcome.
Again the outcome of coercion/no coercion was used as
dependent variable. Because the outcome was already defined,
supervised learning algorithms [Logistic regression, supported
vector machine (SVM), and bagged trees algorithms] were used.
We used cross-validation to test the trained model. The training
set was divided in 5 equal sized subsets with one part being
used to train a model and the other four subsets to evaluate
the accuracy of the learnt model (five-fold cross validation). The
error rate of each subset was an estimate of the error rate of
the classifier. Cross-validation is used in machine learning to
establish the generalizability of an algorithm to new or previously
“unseen” subjects. The validity of the algorithms in predicting
the outcome coercion from no coercion was evaluated using
prediction accuracy, sensitivity, specificity, positive predictive
value (PPV) and negative predictive value (NPV). In this
study, sensitivity and specificity represented correctly predicted
occurrence of coercion (true positives) and correctly predicted
lack of coercion (true negatives), respectively.
Logistic Regression
The classifier models the class probabilities as a function of the
linear combination of predictors. Logistic regression utilizes a
typical linear regression formulation.
Support Vector Machines (SVM)
This technique separates data by a hyperplane, trying to
maximize the margin and creating the maximum distance
between the hyperplane and the values which lie on each side.
The higher this distance gets the better is the reduction of the
expected generalization error.
Frontiers in Psychiatry | www.frontiersin.org 3 June 2018 | Volume 9 | Article 258
Hotzy et al. Machine Learning and Coercion
SVM are robust in dealing with large numbers of features
included because only those features which lie on the margin
of the hyperplane are included. If data are non-linear and
separation is not possible on one hyperplane, SVM can create
more dimensional hyperplanes in a higher dimensional feature
space. SVM methods are binary. So in the case of this study
where we compared the patient group with/without coercion no
dummy-variables had to be created for the response-feature.
Decision Trees
Decision trees classify instances by sorting them based on feature
values. The nodes represent instances in the feature to be
classified and the branches represent values that the node can
become. The instance which divides the training data in the best
way is selected as the root node. Than the instance which best
divides this feature is chosen and so on. There are many ways to
select the instance which is best at dividing data. It is possible to
train ensembles of regression trees. They combine results from
many weak learners into one high-quality ensemble model and
are potent in the analysis of skewed data.
In generally, methods like SVMs and neural networks perform
well with balanced continuous and multi-dimensional features
whereas logic-based systems like decision trees or rule learners
perform better with discrete/categorical variables.
SVMs are potent in dealing with large data which increase
their prediction accuracy. These techniques can also work in
the case of multi co-linearity and non-linear relationships. Logic
based systems like decision trees are easier to interpret than
SVMs.
Imbalance Problem
Class imbalance where the number of patients in one class
(e.g., no coercion) exceeds the patients in the other class (e.g.,
coercion) is a common problem in machine learning. A typical
machine learning algorithm trained with an imbalanced data set
would assign new observations to the majority class (e.g., no
coercion) (36). In this study we met this problem by creating
an artificial group with balanced distribution of the outcome
(coercion/no coercion). We assigned random numbers to the
cohort of 612 patients which were involuntary hospitalized
during the study period. We selected those patients without
documentation of coercion during their hospitalization and
sorted them by ascending numbers. We then excluded the first
half of this group of patients. Thus, we conducted the analysis
with 393 patients (no coercion: n = 223, coercion: n = 170). In
those patients who experienced coercion, at least one coercive
measure (e.g., seclusion, coercive medication, restraint alone, or
in combination) was used during hospitalization.
RESULTS
Comparison Between Groups of Patients
With/Without Coercion During Involuntary
Hospitalization
Being a threat to others (72%) or self and others (20%)
were the most frequent reasons for the usage of coercion.
Clinical aspects like a higher CGI at admission, psychotic or
personality disorders, the prescription of antipsychotics and
benzodiazepines, harm to others or harm to self and others
before admission, and male gender were significantly associated
with the usage of coercion. From the procedural side being
retained, police involvement at admission, the number of
former admissions, a history of IC, a longer duration until
patients were allowed for day passes, duration until revocation
of involuntary hospitalization and duration of hospitalization,
appeal for prolongation from the clinic but also appeal for early
discharge from the patient were significantly associated with the
use of coercion. We found an association between a secondary
diagnosis of a substance-use-related disorder and coercion which
was not significant (for details see Tables 1, 2).
Age at admission (Mann–Whitney U: 17454.000, Z: −1.346,
p = 0.178, n = 393) and Nationality did not differ significantly
between the groups [χ2
(6)
= 6.466, p = 0.373, n = 393].
Also we found no significant group difference for skills in
German language, which is the official language in the state of
Zurich, [χ2 = 0.384, p = 0.825, n = 393] and educational-level
[χ2
(6)
= 8.285, p = 0.218, n = 393].
Two Models to Predict the Outcome
Coercion/No Coercion
The main question of this study was to find models with a good
accuracy in the prediction of the outcome coercion/no coercion.
With a supervised learning technique a predictive model can be
tested for both, input and output data. We trained and tested
two models for their accuracy in the prediction of the outcome
coercion/no coercion. For comparison we computed the same
two models in binary logistic regression.
The first model included data which were available at hospital
admission. In the second model we included variables which are
available after a whole course of hospitalization. We hypothesized
this second model to have higher prediction accuracy. The
variables included in both models are shown in Table 3.
Binary logistic regression in SPSS and logistic regression in
ML had the same results for B, SE, and p. This is comprehensible
as logistic regression utilizes a typical linear regression
formulation. The calculation of the coefficients/weights is
different between both approaches and led to different results.
Details are shown in Table 4.
The machine learning algorithms (Quadratic SVM, Ensemble
RUSBoosted Trees and Logistic regression) predicted the
outcome parameters (coercion/no coercion) with a balanced
accuracy ranging from 66.5 to 69% (the quadratic SVM algorithm
identified 102 out of 170 patients which experienced coercion)
in the model with 8 parameters and 71.5–76% in the model
with 18 parameters. In contrast the binary logistic regression in
SPSS had a balanced accuracy of 68.5% in the 8 item model and
78.5% in the 18 item model. In the 18 item model the logistic
regression algorithm identified 121 out of 170 patients which
experienced coercion (sensitivity). This resulted in an accuracy of
75%. The binary logistic regression of SPSS identified 124 out of
170 patients which experienced coercion and was more potent in
predicting those who did not experience coercion (187 out of 223
patients). This resulted in an accuracy of 78.5%.The Quadratic
Frontiers in Psychiatry | www.frontiersin.org 4 June 2018 | Volume 9 | Article 258
Hotzy et al. Machine Learning and Coercion
TABLE 1 | Comparison of socio-demographic and clinical aspects in patients with/without coercion.
Total (n = 393) No Coercion Coercion χ2 d.f.* P-value
N % N % N %
Gender 7.858 1 0.003
Male 204 52 102 46 102 60
Female 189 48 121 54 68 40
Reason for IC 50.253 3 <0.001
Harm to self 193 49 143 64 50 29
Harm to others 87 22 29 13 58 34
Harm to self and others 101 26 44 20 57 34
Other 12 3 7 3 5 3
ICD-10 primary diagnosis 59.746 6 <0.001
Organic disorder (F0) 71 18 44 20 27 16
Substance use disorder (F1) 49 13 37 17 12 7
Psychotic disorder (F2) 159 40 70 31 89 52
Affective disorder (F3) 51 13 31 14 20 12
Neurotic disorder (F4) 37 10 36 16 1 1
Personality disorder (F6) 13 3 1 1 12 7
Other 13 3 4 1 9 5
ICD-10 secondary F1 diagnosis 4.695 1 0.021
No 307 78 183 82 124 73
Yes 86 22 40 18 46 27
CGI at admission 28.857 3 <0.001
1–2 5 1 5 2 0 0
3–4 18 5 15 7 3 2
5–6 161 41 108 48 53 31
7–8 209 53 95 43 114 67
Police involved at admission 11.978 1 <0.001
No 257 65 162 73 95 56
Yes 136 35 61 27 75 44
Antipsychotics 50.147 1 <0.001
No 78 20 72 32 6 3
Yes 315 80 151 68 164 97
Benzodiazepines 25.006 1 <0.001
No 92 23 73 33 19 11
Yes 301 77 150 67 151 89
Retainment 19.167 1 <0.001
No 362 92 217 97 145 85
Yes 31 8 6 3 25 15
Former IC 22.197 1 <0.001
No 206 52 140 63 66 39
Yes 187 48 83 37 104 61
Abscondence
No 317 81 195 87 122 72 15.203 1 <0.001
Yes 76 19 28 13 48 28
Appeal for prolongation of IC 17.063 1 <0.001
No 354 90 213 95 141 83
Yes 39 10 10 5 29 17
Appeal for early discharge 14.257 1 <0.001
No 320 81 196 88 124 73
Yes 73 19 27 12 46 27
Rehospitalization during 6 months 12.951 1 <0.001
No 267 68 168 75 99 58
Yes 126 32 55 25 71 42
CGI, Clinical Global Impression; IC, Involuntary Commitment. Chi-square test reveals significant differences between an involuntarily hospitalized cohort of patients which experienced
coercion and those which did not experience coercion.
Frontiers in Psychiatry | www.frontiersin.org 5 June 2018 | Volume 9 | Article 258
Hotzy et al. Machine Learning and Coercion
TABLE 2 | Comparison of socio-demographic and clinical aspects in patients with/without coercion.
Coercion Mann–Whitney U Z Sig
No Yes
Min Mean Median Max Min Mean Median Max
Number of former admissions 0 4 0 69 0 9 2 67 12468.500 −4.831 <0.001
Duration until revocation of IC 0 79 16 10 1 31 25 230 10937.500 −7.189 <0.001
Duration of hospitalization 0 138 22 13 1 37 31 245 11383.000 −6.789 <0.001
Duration until day passes 0 109 10 5 0 18 11 161 12468.500 −5.822 <0.001
IC, Involuntary Commitment. Mann–Whitney U-Test reveals significant differences in procedural aspects of the cohort with compared to the cohort without coercion during hospitalization.
TABLE 3 | Included predictors in both models.
8 item model 18 item model
1. Gender 1. Gender
2. Reason for IC 2. Reason for IC
3. Police involved at admission 3. Police involved at admission
4. ICD-10 primary diagnosis 4. ICD-10 primary diagnosis
5. ICD-10 secondary F1 diagnosis 5. ICD-10 secondary F1 diagnosis
6. Former admissions 6. Former admissions
7. Former IC 7. Former IC
8. CGI at admission 8. CGI at admission
9. Retainment
10. Antipsychotics
11. Benzodieazepines
12. Appeal for early discharge
13. Appeal for prolongation of IC
14. Abscondence
15. Duration until day passes
16. Duration until revocation of IC
17. Duration of hospitalization
18. Rehospitalization during 6 months
IC, Involuntary Commitment, CGI, Clinical Global Impresssion.
SVM was able to predict 185 out of 223 patients without coercion
and had less potency in predicting the outcome coercion (117 out
of 170 patients). For details see Table 5.
Due to inconsistent findings in literature we also created two
models which did not include the variables gender and substance-
use-related disorders as co-diagnosis (which was not significantly
associated in our bivariate analyses). The results were comparable
but not as robust as the 8 and 18 item model. They are shown in
Table 6.
Weighting of Risk Factors to Experience
Coercion
In a next step we analyzed the relevance of each variable in the
prediction of the outcome coercion/no coercion. We compared
the weights of the included variables between logistic regression
in ML and binary logistic regression. We analyzed the relevance
of predictor variables …
