Following the Evidence: Planning for Sustainable Change
The EBP team makes plans to implement an RRT in their hospital.

This is the eighth article in a series from the Arizona State University College of Nursing and Health Innovation’s Cen-
ter for the Advancement of Evidence-Based Practice. Evidence-based practice (EBP) is a problem-solving approach to
the delivery of health care that integrates the best evidence from studies and patient care data with clinician expertise
and patient preferences and values. When delivered in a context of caring and in a supportive organizational cul-
ture, the highest quality of care and best patient outcomes can be achieved.

The purpose of this series is to give nurses the knowledge and skills they need to implement EBP consistently,
one step at a time. Articles will appear every other month to allow you time to incorporate information as you work
toward implementing EBP at your institution. Also, we’ve scheduled “Chat with the Authors” calls every few months to
provide a direct line to the experts to help you resolve questions. Details about how to participate in the next call will
be published with May’s Evidence-Based Practice, Step by Step.

After the evidence-based practice (EBP) team of Rebecca R., Carlos A.,
and Chen M. synthesized and
appraised the evidence they found
to answer their clinical question,
they concluded that rapid re-
sponse teams (RRTs) were effec-
tive in reducing both code rates
outside the ICU (CRO) and non-
ICU mortality (NIM), excluding
patients with do not resuscitate
(DNR) orders (see “Clinical Ap-
praisal of the Evidence: Part III,”
November 2010). They also de-
cided that a reduction in un-
planned ICU admissions (UICUA)
may be a reasonable outcome to
expect. In addition, they chose
the members of their RRT: an
advanced practice nurse, a phy-
sician, an ICU staff nurse, a respi-
ratory therapist, and a chaplain.

The team’s next step is to de-
velop a plan to implement an RRT
in their hospital. They be gin by
planning how to collect baseline
data on their chosen outcomes so
they can evaluate the RRT’s impact
on those outcomes. Carlos explains
to the team that measuring out-
comes, typically before and after
implementing an intervention, is

essential to documenting the im-
pact of the EBP implementation
project on health care quality and/
or patient outcomes.1 Rebecca
adds that they’ll also need to con-
sider cost as an outcome and must
plan for how to capture the costs
of the RRT as well as evaluate the
cost savings for positive changes in
CRO, NIM, and UICUA.

THE IMPLEMENTATION PLAN
Rebecca and Chen are excited
about the plan to implement an
RRT in their hospital and tell
Carlos how much they appreci-
ate his ongoing support. Carlos
checks in often with the team
now that the project is under
way. His experience as an expert
EBP mentor has taught him the
importance of assessing the team’s
progress at frequent intervals to
see how he can support them.

To help the team develop a
detailed plan for implementing
an RRT in their hospital, Car-
los pro vides them with an EBP
Implementation Plan template
that he used in his EBP Gradu –
ate Certificate Program (Figure 1).
This plan was developed using
the Advancing Research and

Clin i cal Practice Through Close
Collaboration (ARCC) model,
in which EBP mentors are key
fa cilitators of sustainable change.
Carlos explains that even though
they now have a template to
guide them in the process, EBP
implementation can be unpre-
dictable. The team cannot antic-
ipate all of the challenges or or-
ganizational nuances they may
encounter in launching an RRT
in their hospital.

Preliminary checkpoint catch-
up. The team reviews the template,
beginning with the Preliminary
Checkpoint, to determine which
steps they’ve already taken and
which they’ll need to prepare
for going forward. They’ve al-
ready completed checkpoints one
through four, but two steps in the
preliminary checkpoint still need to
be addressed: identifying key stake-
holders and acquiring approval
from the internal review board
(IRB; sometimes called the ethics
review board, or the human sub-
jects or ethics committee). The
team members discuss their roles
in the project and agree that these
may evolve as the implementation
plan develops.

54 AJN ▼ January 2011 ▼ Vol. 111, No. 1 ajnonline.com

By Ellen Fineout – O verholt, PhD, RN, FNAP, FAAN,
Kathleen M. Williamson, PhD, RN, Lynn

Gallagher-Ford, RN, MSN, NE-BC, Bernadette
Mazurek Melnyk, PhD, RN, CPNP/PMHNP, FNAP,

FAAN, and Susan B. Stillwell, DNP, RN, CNE

Key stakeholders. Carlos tells
Rebecca and Chen that consider-
ing who would be stakeholders
in a project—in this case, those
individuals or groups that may be
affected by or can influence the
implementation of an RRT—is a
step that’s often overlooked. He
explains that active stakeholders
are those people who have a key
role in making the project happen.
Passive stakeholders are those who
may not be actively involved in
the project but who could promote
or stymie its success. Carlos ad-
vises the team to consider all po-
tential stakeholders, as theirs is
an organization-wide project and
some stakeholders may not be ob-
vious. He asks Rebecca and Chen
to think about the outcomes of
the project and to which stake-
holders throughout the hospital
they’d be important. The team
discusses that, as staff nurses, they
don’t always think about their
work from an organizational
standpoint. Carlos says that
thinking about the project in an
organization-wide context will
help them figure out who needs
to be on the team. He provides
examples of stakeholders who
would not only be critical to the
RRT process but who might also
have connections that could be
important to the project’s success.
For example, connecting with key
councils (practice, quality, criti cal
care) or work groups (education,
communications) may provide ac –
cess to already- established pro-
cesses for introduc ing a policy
into the organization.

The team preliminarily identifies
the members of their RRT, patients,
staff nurses, and administrators as
active stakeholders. They identify
the finance, risk management,
and education departments, mid-
level managers, and the chief ex-
ecutive and chief nursing officers
as potential passive stakeholders.

The team agrees that although
these may not be all of the stake-
holders—more may be identified
as planning continues—they’re
likely key players who need to be
included in the implementation
plan for now. Carlos tells the team
that it’s important to keep thinking
about who will impact the project
and whom the project will impact,
so that everyone who needs to be
on board with the plan is brought
on early.

IRB approval. Carlos explains
that an IRB is charged with mak-
ing sure that subjects involved
in a research study are safe and
that the research is conducted in
such a way that the findings are
applicable to a broader popula-
tion than just those in the study,
which is known as generalizabil­
ity.2 The team discusses whether
they need to submit their imple-
men tation plan to their hospital’s
IRB for approval, since they’re
not conducting research. Al-
though they’ll be collecting out-
comes data to evaluate whether
they’re achiev ing the expected
outcomes cited in the literature,
their evidence-based RRT inter-
vention is a best practice improve-
ment project, not a research study.
Still, Car los stresses that the team
has an obligation to publish how
their evidence-based intervention
works in their hospital. He re minds
them that the seventh step in the
EBP process is to disseminate re-
sults so others can learn how a
project was implemented and eval –
uated (the process) and whether
the out comes identified in the lit-
erature were obtained (the pro­
ject outcomes, or end points) (see
“The Seven Steps of Evidence-
Based Practice,” January 2010).
Car los tells Rebecca and Chen that
if they’re going to publish their
pro ject, they’ll need to submit
their implementation plan for
IRB approval. Moreover, they

cannot collect their baseline data
without prior IRB approval. The
team dis cusses that when they
write up their project, they can
address some of the issues they had
with the reporting of implementa-
tion projects in the literature, such
as how differences in the format-
ting of these reports makes it hard
to synthesize the data (see “Clini-
cal Appraisal of the Evidence: Part
III,” November 2010). For these
reasons, the team feels it’s essen-
tial that they publish their project,
so they’ll pursue IRB approval.

Before the team begins writ-
ing up their implementation plan
(which they will reformulate as
an IRB proposal), they discuss an
essential assumption they hold,
which is that all patients who
enter a hospital sign a “consent
for treatment” expecting clinicians
and others caring for them to pro-
vide the best care possible. Al-
though patients may not re fer to
their care as evidence­based prac­
tice, the EBP team feels strongly
that patients’ expectations reflect
professional practice in which daily
decisions are made based on the
best evidence available. With this
expectation and their decision to
publish the project in mind, the
team discusses that the outcomes
data will be used in a way that
wasn’t covered in the consent for
treatment. Thus, the IRB review
of their proposal should reveal
any ways in which publishing the
outcomes of the project could put
recipients of the practice change
at risk. In effect, the IRB would be
reviewing the plan to make sure
that the data from those patients

Considering who would be
stakeholders in a project is a
step that’s often overlooked.

[email protected] AJN ▼ January 2011 ▼ Vol. 111, No. 1 55

Fi
gu

re
1

. E
BP

Im
pl

em
en

ta
tio

n
Pl

an
T

em
pl

at
e

A
RC

C
E

BP
Im

pl
em

en
ta

tio
n

Pl
an

PI
C

O
T

Q
ue

st
io

n:

Te
am

M
em

be
rs

:

EB
P

M
en

to
r

an
d

C
on

ta
ct

In
fo

:

Pr
el

im
in

ar
y

C
he

ck
po

in
t

•
W

ho
a

re
th

e
st

ak
eh

ol
de

rs
f
or

y
ou

r
pr

oj
ec

t


A

ct
iv

e
(o

n
th

e
im

pl
em

en
ta

tio
n

te
am

)
an

d
Su

pp
or

tiv
e

(n
ot

o
n

th
e

te
am

,
bu

t e
ss

en
tia

l t
o

su
cc

es
s)

•
Id

en
tif

y
pr

oj
ec

t t
ea

m
r

ol
es

a
nd

le
ad

er
sh

ip
•

Be
gi

n
ac

qu
is

iti
on

o
f
an

y
ne

ce
ss

ar
y

ap
pr

ov
al

s
fo

r
pr

oj
ec

t i
m

pl
em

en
ta

tio
n

an
d

di
ss

em
in

at
io

n
(fo

r
ex

am
pl

e,

sy
st

em
a

nd
u

ni
t l

ea
de

rs
hi

p,
in

te
rn

al
r

ev
ie

w
b

oa
rd

[
IR

B]
)

•
Be

gi
n

re
la

tio
ns

hi
p

w
ith

E
BP

M
en

to
r

N
ot

es
:

C
he

ck
po

in
t O

ne
•

H
on

e
PI

C
O

T
qu

es
tio

n
an

d
as

su
re

te
am

is
p

re
pa

re
d

•
Bu

ild
E

BP
k

no
w

le
dg

e
an

d
sk

ill
s

•
Be

gi
n

re
la

tio
ns

hi
p

w
ith

E
BP

M
en

to
r

N
ot

es
:

C
he

ck
po

in
t T

w
o

•
C

on
du

ct
li

te
ra

tu
re

s
ea

rc
h

an
d

re
ta

in
s

tu
di

es
th

at
m

ee
t c

rit
er

ia
f
or

in
cl

us
io

n
•

C
on

ne
ct

w
ith

li
br

ar
ia

n
•

M
ee

t w
ith

im
pl

em
en

ta
tio

n
gr

ou
p

–
TE

A
M

B
U

IL
D

•
Be

gi
n

re
la

tio
ns

hi
p

w
ith

E
BP

M
en

to
r

N
ot

es
:

C
he

ck
po

in
t T

hr
ee

•
C

rit
ic

al
ly

a
pp

ra
is

e
lit

er
at

ur
e

•
M

ee
t w

ith
g

ro
up

to
d

is
cu

ss
h

ow
c

om
pl

et
el

y
ev

id
en

ce
a

ns
w

er
s

qu
es

tio
n;

p
os

e
fo

llo
w

-u
p

qu
es

tio
ns

a
nd

r
e –

re
vi

ew
th

e
lit

er
at

ur
e

as
n

ec
es

sa
ry

•
Be

gi
n

re
la

tio
ns

hi
p

w
ith

E
BP

M
en

to
r

N
ot

es
:

C
he

ck
po

in
t F

ou
r

•
M

ee
t w

ith
g

ro
up

•
Su

m
m

ar
iz

e
ev

id
en

ce
w

ith
f
oc

us
o

n
im

pl
ic

at
io

ns
f
or

p
ra

ct
ic

e
an

d
co

nd
uc

t i
nt

er
vi

ew
s

w
ith

c
on

te
nt

e
xp

er
ts

a
s

ne
ce

ss
ar

y
to

b
en

ch
m

ar
k

•
Be

gi
n

fo
rm

ul
at

in
g

de
ta

ile
d

pl
an

f
or

im
pl

em
en

ta
tio

n
of

e
vi

de
nc

e
•

In
cl

ud
e

w
ho

m
us

t k
no

w
a

bo
ut

th
e

pr
oj

ec
t,

w
he

n
th

ey
w

ill
k

no
w

,
ho

w
th

ey
w

ill
k

no
w

•
Be

gi
n

re
la

tio
ns

hi
p

w
ith

E
BP

M
en

to
r

N
ot

es
:

C
he

ck
po

in
t F

iv
e

•
D

ef
in

e
pr

oj
ec

t p
ur

po
se

—
co

nn
ec

t t
he

e
vi

de
nc

e
an

d
th

e
pr

oj
ec

t
•

D
ef

in
e

ba
se

lin
e

da
ta

c
ol

le
ct

io
n

so
ur

ce
(s

)
(fo

r
ex

am
pl

e,
e

xi
st

in
g

da
ta

se
ts

,
el

ec
tro

ni
c

he
al

th
r

ec
or

d)
,

m
et

ho
ds

,

an
d

m
ea

su
re

s
•

D
ef

in
e

po
st

pr
oj

ec
t o

ut
co

m
e

in
di

ca
to

rs
o

f
a

su
cc

es
sf

ul
p

ro
je

ct
•

G
at

he
r

ou
tc

om
e

m
ea

su
re

s
•

W
rit

e
da

ta
c

ol
le

ct
io

n
pr

ot
oc

ol
•

W
rit

e
th

e
pr

oj
ec

t p
ro

to
co

l (
da

ta
c

ol
le

ct
io

n
fit

s
in

th
is

d
oc

um
en

t)
•

Fi
na

liz
e

an
y

ne
ce

ss
ar

y
ap

pr
ov

al
s

fo
r

pr
oj

ec
t i

m
pl

em
en

ta
tio

n
an

d
di

ss
em

in
at

io
n

(fo
r

ex
am

pl
e,

s
ys

te
m

le
ad

er
–

sh
ip

,
un

it
le

ad
er

sh
ip

,
IR

B)
•

Be
gi

n
re

la
tio

ns
hi

p
w

ith
E

BP
M

en
to

r

N
ot

es
:

56 AJN ▼ January 2011 ▼ Vol. 111, No. 1 ajnonline.com

C
he

ck
po

in
t S

ix

(a
bo

ut
m

id
w

ay
)

•
M

ee
t w

ith
im

pl
em

en
ta

tio
n

gr
ou

p
•

D
is

cu
ss

k
no

w
n

ba
rr

ie
rs

a
nd

f
ac

ili
ta

to
rs

o
f
pr

oj
ec

t
•

D
is

cu
ss

s
tra

te
gi

es
f
or

m
in

im
iz

in
g

ba
rr

ie
rs

a
nd

m
ax

im
iz

in
g

fa
ci

lit
at

or
s

•
Fi

na
liz

e
pr

ot
oc

ol
f
or

im
pl

em
en

ta
tio

n
of

e
vi

de
nc

e
•

Id
en

tif
y

re
so

ur
ce

s
(h

um
an

,
fis

ca
l,

an
d

ot
he

r)
n

ec
es

sa
ry

to
c

om
pl

et
e

pr

oj
ec

t
•

Su
pp

ly
E

BP
M

en
to

r
w

ith
w

rit
te

n
IR

B
ap

pr
ov

al
a

nd
m

an
ag

er
ia

l
su

pp
or

t
•

Be
gi

n
w

or
k

on
p

os
te

r
fo

r
di

ss
em

in
at

io
n

of
in

iti
at

io
n

of
p

ro
je

ct
a

nd
p

ro
gr

es
s

to
d

at
e

to

ed
uc

at
e

st
ak

eh
ol

de
rs

a
bo

ut
p

ro
je

ct
—

ge
t h

el
p

fr
om

s
up

po
rt

st
af

f
•

In
cl

ud
e

sp
ec

ifi
c

pl
an

f
or

h
ow

e
va

lu
at

io
n

w
ill

ta
ke

p
la

ce
:

w
ho

,
w

ha
t,

w
he

n,
w

he
re

,

an
d

ho
w

,
an

d
co

m
m

un
ic

at
io

n
m

ec
ha

ni
sm

s
to

s
ta

ke
ho

ld
er

s
•

Be
gi

n
re

la
tio

ns
hi

p
w

ith
E

BP
M

en
to

r

N
ot

es
:

C
he

ck
po

in
t S

ev
en

•
M

ee
t w

ith
im

pl
em

en
ta

tio
n

gr
ou

p
to

r
ev

ie
w

p
ro

po
se

d
po

st
er

•
M

ak
e

fin
al

a
dj

us
tm

en
t t

o
po

st
er

w
ith

s
up

po
rt

st
af

f
•

In
fo

rm
s

ta
ke

ho
ld

er
s

of
s

ta
rt

da
te

o
f
im

pl
em

en
ta

tio
n

an
d

po
st

er
p

re
se

nt
at

io
n

•
A

dd
re

ss
a

ny
c

on
ce

rn
s

or
q

ue
st

io
ns

o
f
st

ak
eh

ol
de

rs
(

ac
tiv

e
an

d
su

pp
or

tiv
e)

•
Be

gi
n

re
la

tio
ns

hi
p

w
ith

E
BP

M
en

to
r

N
ot

es
:

C
he

ck
po

in
t E

ig
ht

•
Po

st
er

p
re

se
nt

at
io

n
(p

re
fe

rr
ed

e
ve

nt
is

a
s

ys
te

m
-w

id
e

re
co

gn
iti

on
o

f
qu

al
ity

,
re

se
ar

ch
,

or

in
no

va
tio

n)
•

LA
U

N
C

H
E

BP
im

pl
em

en
ta

tio
n

pr
oj

ec
t

•
Be

gi
n

re
la

tio
ns

hi
p

w
ith

E
BP

M
en

to
r

N
ot

es
:

C
he

ck
po

in
t N

in
e

•
M

id
pr

oj
ec

t m
ee

t w
ith

a
ll

ke
y

st
ak

eh
ol

de
rs

to
r

ev
ie

w
p

ro
gr

es
s

an
d

pr
ov

id
e

ou
tc

om
es

to

da
te

•
Re

vi
ew

is
su

es
,

su
cc

es
se

s,
a

ha
’s

,
an

d
tri

um
ph

s
of

p
ro

je
ct

to
d

at
e

•
Be

gi
n

re
la

tio
ns

hi
p

w
ith

E
BP

M
en

to
r

N
ot

es
:

C
he

ck
po

in
t T

en
•

C
om

pl
et

e
fin

al
d

at
a

co
lle

ct
io

n
fo

r
pr

oj
ec

t e
va

lu
at

io
n

•
Pr

es
en

t p
ro

je
ct

r
es

ul
ts

v
ia

p
os

te
r

pr
es

en
ta

tio
n—

lo
ca

lly
a

nd
n

at
io

na
lly

•
Ce

le
br

at
e

w
ith

E
BP

M
en

to
r

an
d

A
ge

nc
y

Le
ad

er
sh

ip

N
ot

es
:

C
he

ck
po

in
t E

le
ve

n
•

Re
vi

ew
p

ro
je

ct
p

ro
gr

es
s,

le
ss

on
s

le
ar

ne
d,

n
ew

q
ue

st
io

ns
g

en
er

at
ed

f
ro

m

pr
oc

es
s

•
Co

ns
ul

t
w

ith
E

BP
M

en
to

r
ab

ou
t

ne
w

q
ue

st
io

ns

N
ot

es
:

©
Fi

ne
ou

t-O
ve

rh
ol

t,
2

0
0
9
.

Th
is

f
or

m
m

ay
b

e
us

ed
f
or

e
du

ca
tio

na
l p

ur
po

se
s

w
ith

ou
t p

er
m

is
si

on
.

If
yo

u
us

e
it

fo
r

yo
ur

p
ra

ct
ic

e
ch

an
ge

,
pl

ea
se

le
t u

s
kn

ow
b

y
e-

m
ai

lin
g

el

le
[email protected]

tra
ns

fo
rm

in
gh

ea
lth

ca
re

w
ith

ar
cc

.c
om

.

[email protected] AJN ▼ January 2011 ▼ Vol. 111, No. 1 57

who receive the intervention will
be treated confidentially.

The team discusses that their
RRT intervention is supported by
studies of RRTs that were sub-
mitted to and approved by their
respective IRBs; that the IRB ap-
prov als of these RRT projects lends
confidence to their intervention.
Rebecca and Chen know it’s im-
portant that their plan be reviewed,
but they express concern about
how to engage the IRB process.
Carlos tells them that the IRB has
several forms available to assist
clinicians and researchers in pin-
pointing those aspects of their

study or project that may increase
risk of any kind to the people in-
volved. The team seeks out more
information on their hospital’s
Web site and finds the appropriate
form for an implementation proj-
ect. They agree to complete the
form together as they develop their
implementation plan.

Checkpoint five and for ward.
As the team moves on to Check-
point Five in the EBP Implemen-
tation Plan template, Carlos talks
to them about the critical impor-
tance of defining the purpose of
the project.

Purpose of the project. A clearly
defined purpose sets the entire
plan ning process in motion, Car-
los says; it’s the touchstone of the
project that the team can return to
periodically to ensure they’re on
course. The team agrees that the
purpose of their project is to im­
plement and evaluate the effective­
ness of an RRT in their hospital.

Baseline data collection. Car-
los tells the team that collecting
data prior to implementation of
the RRT is important because it
will help determine the extent of
any already existing problems
as well as enable the evaluation
of the project outcomes.3 He ex-
plains that various data are gen-
erated within the hospital, which
he calls internal evidence. The
sources for these data are in vari-
ous locations and are referred to
in a variety of ways, such as: qual-
ity management, risk management,
finance, and human resources de-
partments; clinical systems; oper-
ational systems; and electronic
medical records/information tech-
nology (see Table 1). Carlos tells
the team that internal evidence
that’s collected for federal and
state agencies or for regulatory
and specialty organizations, such
as the American Nurses Creden-
tialing Center’s Magnet Recogni-
tion Program, can also be used as
outcomes. As an example, he pro-
vides reports from their hospital’s
quality commit tee that include

data for CRO, UICUA, and over-
all hospital mor tality. Chen asks
what it will require to get data
only for NIM. Carlos replies that
he’ll have to find out which depart-
ment in the hospital creates qual-
ity committee reports and ask if
NIM data can be culled from the
overall hospital mortality data.
He explains that there are many
data repository systems within
the hospital and that each system
may collect different data and may
require a different way of request-
ing those data. Carlos helps the
team understand that obtaining
data may be complicated at times,
but one’s success greatly de pends
on knowing whom to ask.

To help the team capture the
out comes data they’ll need to ob-
tain at baseline and again after the
project, Carlos recommends they
work with the information tech-
nology and finance departments.
Chen asks if putting the outcomes
in a chart would help to clearly
outline the “who, what, when,
where, and how” of baseline data
collection. The team agrees that
this would help them understand
the financial outcomes (sometimes
referred to as the busi ness case),
the process and structure of the
project,4 and the patient outcomes
that will be measured at the end
of the project (see Table 2).

The process. The team discus-
ses how to ensure that the pro-
cess of implementing an RRT in
their hospital goes well. Rebecca
reminds the team about their and
the MERIT trial authors’ obser-
vations on how the MERIT trial
was conducted, particularly on
how the RRT protocol was imple-
mented.5 (The control hospitals’
code teams may have functioned
as RRTs, which could explain
why there was no difference be-
tween the control group and the
intervention group; see “Critical
Appraisal of the Evidence, Part
II,” September 2010). She asks the
group for ideas about how they
can collect data on the process of

Table 1. Potential Sources and
Types of Internal Evidence

Source of Data Type of Data

Quality
Management

Hospital quality indicators
Nursing quality indicators
Patient satisfaction
Regulatory/accreditation requirements

Risk
Management

Incident reporting
Medication errors
Sentinel events
Patient complaints

Finance Admission, transfer, and
discharge data
Billing and coding, capital and
operation budgets
Medicare-severity diagnosis-
related groups (MS-DRGs)
Cost and return on investment
data

Clinical
Systems

Monitoring devices and equipment

Operational
Systems

Patient tracking and flow
Staffing and scheduling

Electronic
Med ical
Records/
Information
Technology

Patient history
Patient assessment
Diagnostic test results
Medication regime
Plan of care

Data collected,
submitted to
and bench-
marked
with outside
sources

National Database of Nursing
Quality Indicators
Centers for Medicare and
Medicaid Services
Patient satisfaction survey
organizations

58 AJN ▼ January 2011 ▼ Vol. 111, No. 1 ajnonline.com

implementing the RRT to dem-
onstrate that they have done it
well. Carlos says that how well
they implement the intervention
is called the fidelity of the inter­
vention. He recommends keeping
good notes on the work being
done. They talk about the need
to develop a project data collec-
tion tool that staff can use when
calling the RRT. Chen volunteers

to develop this form, using simi-
lar forms in the literature they re-
viewed as a basis. Carlos suggests
that maybe Chen should see if
anything new has been published,
since it’s been a few months since
they completed their literature
search.

The team talks about the im-
portance of measuring the costs
and benefits of the RRT, especially

its benefits divided by the costs,
which Carlos notes is called its
return on investment (ROI). Car-
los suggests that the team meet
with the finance department to
dis cuss their plan to measure the
costs and ROI of an RRT. Re-
becca volunteers to be responsi-
ble for ob tain ing the finan cial
data and requests that Carlos be
available for support, if needed,

Table 2: Considerations in Measuring Outcomes for the RRT Implementation Project

Making the Case Data Needed for an RRT Processes/Outcomes to Be Measured

The strategic case: Evaluate project in
relation to its impact (high volume, high
risk, high cost) and the strategic priori-
ties of the organization (business plan,
accreditation, reimbursement, licensing)

Hospital strategic plan; CRO, UICUA, and
NIM data; and expected targets for these
data, if identified

• CRO, UICUA, and NIM before (and after)
implementing a system-wide RRT

The business case (financial outcomes):
Calculate net return on investment—for
example, cost of project minus cost off-
set by reducing identified outcomes

Actual cost assessed for supplies, staff
education, RRT members providing the ser-
vice, other infrastructure for the RRT team
(special process for calling an RRT, for
example), identified outcomes

• Cost savings from prevention of CRO,
UICUA, and NIM before (and after) imple-
menting a system-wide RRT

The resources case (assess/ identify
resources needed to achieve outcomes):

Infrastructure: Policies, procedures,
documentation systems, and data-
reporting processes

Supplies: New equipment or supplies
needed for the project

Human resources: Identify departments
that will be supporting the project
(such as, nursing, respiratory, physi-
cians, information systems, purchas-
ing, education, pastoral care)

Identification of:

Policy for how to activate RRT:
• Define who will write policy
• List committees needed to approve policy
• List processes for rolling out new policy

Equipment required for early intervention
care

Human resources support for hiring per-
sonnel to fill RRT roles or to backfill posi-
tions vacated to fill RRT

• Policies and protocols developed to
facilitate RRT

• Documentation systems adjusted to
accommodate RRT record

• Electronic data reporting available to
capture RRT process and outcome

• Redo code cart to add RRT box contain-
ing supplies/equipment that may expedite
early intervention care

• RRT members evaluation of their role

Process measures to achieve outcomes
(sometimes called process outcomes):
Staff education plan, project data col-
lection, staff and family feedback

Staff education plan
RRT project data collection tool
Staff feedback tool
Family feedback tool

• Staff education completion rates
• Quality of RRT project events, such as how

RRT protocol was followed
• Effectiveness of RRT project events
• Timeliness of project events, such as time

frame from call to RRT arrival
• Family and staff response to how RRT is

delivered (the intervention protocol)
• Outcomes of each RRT call

CRO = code rates outside the ICU; NIM = non-ICU mortality; RRT = rapid response team; UICUA = unplanned ICU admissions.

[email protected] AJN ▼ January 2011 ▼ Vol. 111, No. 1 59

to which he read ily agrees. Chen
agrees to work with Carlos to en-
sure that data on CRO, UICUA,
and NIM are systematically col-
lected and to focus on the process
outcomes (how well the RRT pro-
ject is implemented). For example,
if there was a breach in protocol
implementation—in how well
the RRT protocol was delivered
to the active stakeholders, for in-
stance—that breach could lead
to an outcome that was different
from what was expected. This un-
expected outcome may not be be-
cause the RRT intervention didn’t
work, but because of a glitch in the
process: the RRT pro tocol wasn’t
delivered as planned.

As work on the project is plan-
ned and discussed, the roles of the
team naturally begin to fall into
place. As part of formulating the
implementation plan, they discuss
what questions about data collec-
tion they’ll need to ask in order to
measure their outcomes of CRO,
UICUA, and NIM (see Questions
to Ask in Preparation for Data
Collection). Carlos reflects back on
the definitions and measures the
team discussed in their appraisal
of the evidence and how the dif-
ferent definitions of mortality

(whether it included DNR cases,
for example) led to some confusion
about comparing the impact of an
RRT on that variable (see “Criti-
cal Appraisal of the Evi dence: Part
II,” September 2010). He explains
the importance of how the data
are measured (what mechanisms
are used, for example, and why
and how to know they’re good
methods for measuring the data).
He says that in order to determine
the impact of an EBP project such
as the implementation of an RRT,
the data must be measurable (able
to be counted), accessible (the
team has access to the data), and
user friendly (understandable and
able to be used without difficulty).
Chen and Rebecca decide they
want to create a data collection
plan that meets all of these criteria.
With the questions on data collec-
tion to guide them, they realize
that multiple disciplines within
the hospital (not only nursing) will
be involved in helping to collect
the baseline data for the pro ject.

From the team’s discussion,
Rebecca and Chen put together
a preliminary plan for evaluating
the RRT project, keeping the fol-
lowing key areas in mind: the stra-
tegic case, business case, resources

case, …