1. No category

Washington University in St. Louis
School of Engineering and Applied Science
Electrical and Systems Engineering Department
ESE499
Using Monte Carlo Simulation to Assess
Hospital Operating Room Scheduling
By
Eric Hsu
Gary Savell
Supervisors
Eli Snir
Jason Trobaugh
Submitted in Partial Fulfillment of the Requirement for the BSSSE
Degree,
Electrical and Systems Engineering Department, School of
Engineering and Applied Science,
Washington University in St. Louis
May 2015
ACKNOWLEDGMENTS
We would like to acknowledge the contribution of our advisor, Eli Snir for his assistance
and guidance throughout the undertaking of this project. We would also like to thank our advisor
and course mentor, Jason Trobaugh, for his recommendations of different optimization
techniques. Finally, we would like to thank the Consultant for all of his assistance in helping us
build upon his previous work.
1
ABSTRACT
Operation Room (OR) scheduling is a constant problem that hospitals face. Many
analysts have tackled this problem before and have created solutions that may be good for the
hospital, but oftentimes there is not analysis that can be used to physically assess how beneficial
that solution is prior to its implementation. Thus, there is a need for simulation and evaluation of
different solutions. In this report, we create a new system for analyzing hospital OR schedules.
The system is three-fold: we create new candidate solutions by refining a previously used
objective function, use Monte Carlo simulation to obtain statistics for those solutions such as
mean overtime hours and variation in weekly patient flow, and then evaluate those statistics.
Although we used our own means of evaluation to assess the quality of our solutions, the
evaluation methods may change from user to user. Since this project involves only an initial run
through of our system, assumptions can always be tweaked and further physical metrics and
evaluation methods can be developed in the future.
2
Table of Contents
ACKNOWLEDGMENTS .......................................................................................................................... 1
ABSTRACT ................................................................................................................................................ 2
CONTACT INFORMATION.................................................................................................................... 4
INTRODUCTION AND BACKGROUND .............................................................................................. 5
Overview of the Problem ............................................................................................................................... 5
Method .................................................................................................................................................................. 6
METHODS: FINDING SUBOPTIMAL SOLUTIONS .......................................................................... 7
Defining Decision Variables .......................................................................................................................... 7
Defining Constraints ........................................................................................................................................ 8
Objective Function and Finding Candidate Solutions .......................................................................... 9
METHODS: SIMULATION TECHNIQUES ........................................................................................ 11
Underlying Assumptions ............................................................................................................................. 11
Simulation Method ........................................................................................................................................ 12
Metrics of Evaluation .................................................................................................................................... 13
RESULTS ................................................................................................................................................. 15
Candidate Solutions ...................................................................................................................................... 15
Allocation Score Metric................................................................................................................................ 15
Evaluation of Candidate Solutions ........................................................................................................... 16
A Detailed Look at Our Candidate Solutions ........................................................................................ 18
CONCLUSION ......................................................................................................................................... 21
Opportunities for Future Analysis ........................................................................................................... 22
BIBLIOGRAPHY .................................................................................................................................... 23
APPENDIX A: PROBLEM FORMULATION ..................................................................................... 24
Appendix A.1: Notation of Decision Variables ..................................................................................... 24
Appendix A.2: List of Constraints in Equation Form ......................................................................... 25
APPENDIX B: RESULTS....................................................................................................................... 26
Appendix B.1: Result of Allocation Score Algorithm Calculations ............................................... 26
Appendix B.2: Patient Flow Variation for Selected Candidate Solutions................................... 27
Appendix B.3: Statistics for All Candidate Solutions ......................................................................... 28
3
CONTACT INFORMATION
Group Members:

Eric Hsu, Washington University in St. Louis
[email protected]

Gary Savell, Washington University in St. Louis
[email protected]
Project Supervisors:

Eli Snir, Lecturer in Management, Olin Business School, Washington University in St.
Louis
[email protected]

Jason Trobaugh, Lecturer in Electrical and Systems Engineering, School of Engineering
and Applied Science, Washington University in St. Louis
[email protected]
4
INTRODUCTION AND BACKGROUND
Overview of the Problem
Operating Room (OR) Scheduling is a common problem that hospitals face today.
Hospitals try to utilize the space that they have in the most efficient and cost-effective way to
provide the best services to clients and patients. However, scheduling operating rooms to specific
physician specialties requires the consideration of physician availability, specific operating
hours, patient variation, and medical equipment provided in each operating room. Having an
optimized OR surgical schedule can increase the effective use of each operating room and reduce
the amount of poorly allocated resources used by the hospital.
Since OR scheduling and use has been an issue in hospitals for a long time, many studies
have been performed to maximize OR use[1, 2, 3]. Unlike most finite resource allocation problems,
many different parameters such as physician preferences, hospital specialties (such as General
Surgery, Cardiothoracic, Peripheral Vascular, etc.), patient preferences, and financial return from
patients are considered. Each study provides its own optimal solution for the hospital’s
preference. However, our client, a hospital operations research consultant (referred to as
Consultant), who worked for a major tertiary academic medical center this past year, realized
that implementing the optimal solution in real life was too impractical – many physical metrics
could not be measured reliably by just an objective function.
Thus, our group has decided to obtain and evaluate multiple suboptimal candidate
solutions via simulation with previous hospital patient and OR usage data. This way, the
Consultant will be provided with physical statistics that will allow him to better choose a
candidate solution that best serves his institution and will be more feasible to implement.
5
Method
Our approach to finding quality solutions to this scheduling problem is three-fold. The
first part is that the candidate solutions need to be generated. We chose to use an existing integer
programming model to generate solutions. Secondly, these solutions need to have physical
metrics to allow for evaluation. Third, the solutions need to be evaluated based on their
practicality and efficiency.
Find A
Candidate
Solution
Simulate the
solution and
create metrics
Evaulate the
metrics and
practicality
We chose this method of solving the problem because it is a new way of looking at the
benefits of a particular schedule. Previous work has been done specifically for this hospital based
on a deterministic objective function. We wanted to use an alternative method to test the
accuracy level of the objective function as well as propose other possible options. In addition,
deterministic systems only consider the means, whereas we are considering a distribution with
variation in each given day, which is more realistic.
The simulation also gives the added benefit in that it answers real questions that the
hospital has. A hospital typically cares about patient care safety and quality, physician and
employee contentment, and costs. We addressed the simulation with those items in mind. In
particular, a hospital cares about how much expected time will be spent where they need to pay
overtime because it is often difficult to consider how long patients will be in operating rooms.
Similarly, if an operating room can be freed up because of lack of usage, the hospital also wants
to know that in order to reallocate the OR to a specialty that may need it more.
6
A hospital also cares about patient flow. The three types of patients that we considered
were Admit (A), Inpatient (I), and Outpatient (O). The reason the hospital wants to have data
about patient flow is because of a space constraint of number of beds available. For example, if
all of the Inpatients are scheduled only on one day and not on any of the others, there will be a
shortage of beds available. Thus, the hospital wants low variation of patient types across days of
the week.
METHODS: FINDING SUBOPTIMAL SOLUTIONS
Defining Decision Variables
As part of his analysis for the academic medical center, the Consultant had identified a
model for finding the optimal schedule. The integer programming model optimized an objective
function based on three equally weighted components of high utilization time of ORs: allocation
efficiency and two measures of variation of the time used in the OR, which will be discussed in
more detail below shortly. The ten specialties with their abbreviations are listed below in Table
1. Five different types of ORs exist, each supplied with different resources for different
specialties. Table 2 shows which specialties can use which OR type, giving us a total of 16
Max Daily
Blocks
General Surgery
Gen
3
Orthopedic Surgery
Ortho
4
Transplant Surgery
Tx
2
Otolaryngology
ENT
2
Oral and Maxillofacial Surgery OMFS
2
Urology
Uro
2
Cardiothoracic
CT
2
Peripheral Vascular
PV
2
Neurosurgery
Neuro
2
Plastic Surgery
Plas
2
Table 1: List of services, abbreviations, and maximum daily
number of allocated blocks possible to each service
Service
Abbrev
Specialties
OR
Type
#
ORs
Gen, Ortho,
Tx, ENT,
0
11
OMFS, Uro,
Plas, Neuro
Tx, Uro, CT,
1
1
PV, Neuro
Neuro
2
1
PV
3
1
CT
4
1
Table 2: List of specialties that
can use each OR type and total
number of each OR type
7
different Specialty/OR combinations. Each OR allocation can last 8, 10, or 12 hours. Thus, full
OR schedules require the allocation of blocks to three different criteria: day of the week
(Monday through Friday for 5 days), specialty/OR use (16 combinations), and amount of time
allocated (3 different times), resulting in a total of 240 decision variables. These decision
variables are denoted as 𝑥𝑑,𝑠,𝑡 with the subscripts being defined as follows (see Appendix A.1 for
a more explicit defining of decision variables):

𝑑 = 1, 2, 3, 4, 5 respectively for Monday, Tuesday, Wednesday, Thursday, and Friday

𝑠 = 1, 2, 3, … ,16 based on specialty/OR combination
Corresponding
s in set {S}
Gen
1
Ortho
2
Tx
3, 9
ENT
4
OMFS
5
Uro
6, 10
CT
11, 16
PV
12, 15
Neuro
8, 13, 14
Plas
7
Table 3: Services with their
corresponding values of s in
𝑥𝑑,𝑠,𝑡 . They are labeled to be
subsets in {S}.
Service
shown in Table 2, with 𝑠 = 1 referring to Gen in OR Type
0, 𝑠 = 2 referring to Ortho in OR Type 0, and rightwards
and downwards along Table 2 with increasing 𝑠 up to 𝑠 =
16 referring to CT in OR Type 4. The values of 𝑠 for each
service are also shown in Table 3. Each service consists of
a subset of set S, where the first element of S is Gen,
second is Ortho, etc.

𝑡 = 1, 2, 3 respectively for 8, 10, and 12 hour blocks
Defining Constraints
For practicality, all of the decision variables are nonnegative and integers, as both
negative allocations and partial block allocations are not feasible options in real life. The rest of
the constraints in this model are daily constraints and are denoted as shown below. This means
that all of the following equations are constrained for all values of 𝑑 = 1, 2, 3, 4, 5.
8
The hospital has 15 possible operating rooms that can be used each day; however, two are
allocated for emergencies. This leaves 13 ORs that are to be scheduled per day, all of which are
to be used, resulting in the constraint equation shown below.
16
3
∑ ∑ 𝑥 𝑑,𝑠,𝑡 = 13, 𝑥 𝑑,𝑠,𝑡 ∈ ℕ
𝑠=1 𝑡=1
Other constraints include those listed above in Tables 1 and 2. The maximum number of
blocks per day that each specialty can have is 2, 3, or 4 as shown in Table 1. The maximum
number of ORs allocated to a particular type is also limited by the number of each OR type that
the hospital has. That is, the hospital has 11 ORs of Type 0 and 1 of each other Type, so the daily
total number of blocks allocated to OR Type 0 must be less than or equal to 11 and Types 1 to 4
less than or equal to 1. One last constraint includes the maximum number of 8, 10, and 12 hour
blocks that can be allocated to a given day. For the sake of reasonable hospital organization,
these respective maximum values are 2 (or 3, to be described below), 5, or 8 allocated blocks.
Appendix A.2 provides a full list of the constraints.
Objective Function and Finding Candidate Solutions
As stated above, in the Consultant’s model, the objective function contained three terms:
allocation efficiency and two measures of variation of the time used in the OR. Allocation
efficiency required calculating the number of goal hours for each specialty, which is the number
of hours that each specialty requested divided by a target allocation percentage. As is common
with most hospital OR scheduling problems, this value defaults to 0.8. Table 4 shows such
values at a target allocation percentage of 80%. Allocation efficiency itself is the sum of the
absolute differences between the number of allocated hours and goal hours. The two measures of
variation drew a sample of Admit, Inpatient, and Outpatient values for each specialty from
9
existing patient flow data and combined that data with the candidate solution to obtain total A, I,
and O values for each day of the week. A standard deviation and coefficient of variation of each
group of the 5 A, I, and O values were incorporated into the Consultant’s objective function.
To find our own candidate solutions, we took the
Requested
Goal
Hours
Hours
Consultant’s objective function and changed certain coefficient
Gen
97
121.25
Ortho
148
185.0
values to obtain different meaningful solutions. Since we did
Tx
15
18.75
ENT
40
50.0
not have the software that let us run an integer programming
OMFS
24
30.0
Uro
32
40.0
problem simultaneously with a simulation problem as the
CT
52
65.0
PV
50
62.5
Consultant did, we removed the two measures of variation from
Neuro
75
93.75
Plas
23
28.75
our objective function. From there, we changed the objective
Table 4: Requested and Goal
Hours
for each Specialty at a
function in two ways to obtain two different types of solutions.
target allocation of 0.8
The first way involved changing the importance of a particular specialty’s error from the goal
Service
hours. In the Consultant’s objective function, all of the errors were weighted equally. In these
new objective functions, the importance of particular services is increased to 5 times of all of the
others, thus driving the allocated hours of the particular specialty closer to its goal hours. The
second way involved changing the allocation percentage to 0.75, 0.85, and 0.9 to see what
solutions are obtained from allocating a greater (for 0.75) or lesser (for 0.85 and 0.9) number of
hours. However, at an allocation proportion of 0.9, no solutions are feasible, and the number of 8
hour blocks per day had to be increased from 2 to 3 to accommodate this change.
We denote 𝐴𝑃 as the allocation percentage, 𝑅𝐻𝑆 as the requested number of hours in a
particular service, and 𝑊𝑆 as the importance weight of a particular specialty (where the default
weight of each specialty is 𝑊𝑆 = 1). Since the objective function is calculated based on total
hours, we must multiply each allocated block by the hours allocated to that block. Factoring in
10
preparation time and grand rounds on Wednesdays, the number of hours for the respective 8, 10,
and 12 hour blocks used in calculation are 7.5, 9.5, and 11.5 hours (or 7, 9, and 11 hours on
Wednesday). Thus, the number of hours (with proper subscript notation) multiplied calculates
out to be 5.5 + 2𝑡 (or 5 + 2𝑡 on Wednesdays). For our full objective function, we minimize the
sum of the absolute differences between the goal hours and allocated hours for each specialty
multiplied by that specialty’s importance weight, as shown below.
𝐴𝑃
𝑀𝑖𝑛 𝑍 = ∑ 𝑊𝑆 |1 −
( ∑
𝑅𝐻𝑆
𝑠∈𝑆
3
3
∑(5.5 + 2𝑡) ∗ 𝑥𝑑,𝑠,𝑡 + ∑(5 + 2𝑡) ∗ 𝑥3,𝑠,𝑡 )|
𝑑=1,2,4,5 𝑡=1
𝑡=1
METHODS: SIMULATION TECHNIQUES
Underlying Assumptions
Data of past usage was provided daily, including patient types and minutes used of the
OR. Assuming a normal distribution of the data, we collected sample averages and standard
deviations for each day of the week of usage. If there was not enough data points (less than 8) for
the specific day and specialty combination, we assumed that the sample average and sample
standard deviation of the overall non-daily specific usage would approximate the usage for that
day. For each specialty and block assignment, we pulled data from a normal distribution of the
appropriate mean and standard deviation. We truncated the results so that there would be no
negative hours used, since in all practical applications, negative usage is impossible. As a result,
the 𝑖 𝑡ℎ trial’s usage for day of week 𝑑 for specialty 𝑠 becomes 𝑈𝑖,𝑑,𝑠,𝑡 = 𝑚𝑎𝑥{𝑥̅𝑑,𝑠 +
𝑆𝐷𝑑,𝑠 𝒩(0,1), 0}, where 𝒩(0,1), 𝑥̅𝑑,𝑠 , and 𝑆𝐷𝑑,𝑠 are the standard normal distribution, sample
mean and sample standard deviation, respectively, of specialty 𝑠 on day of the week 𝑑 for block
of time 𝑡. Although 𝑡 is not a parameter of 𝑈𝑖,𝑑,𝑠,𝑡 , we needed one random draw for each different
allocated block of time. Thus, while 𝑡 changes, the calculation of 𝑈𝑖,𝑑,𝑠,𝑡 does not change.
11
Another assumption that was made was how to simulate the data. The two options were
daily and weekly. Data was provided daily, but given that blocks are allocated weekly, both
simulation techniques were feasible. Since daily data was provided, it seemed more appropriate
to complete the simulation daily. A future analysis can include altering the simulation and
viewing how the results would change if the simulation was run weekly.
Since we are also interested in understanding the flow of patients across days, we used
data provided as a sample of different patient flows of Admit, Inpatient, and Outpatient to obtain
a projected patient flow. If we observed fewer than 8 observations of a specific specialty on a
specific day of the week, we sampled from the entire dataset for that specialty. The reason for
using this method as opposed to fitting a distribution to the dataset was that we did not feel
comfortable making an assumption about the characteristics of that distribution, especially for
the data being integer constrained.
Simulation Method
For each candidate solution, we used MATLAB R2013A to run our simulation. The
simulation was run for 10,000 trials (N=10,000). For each run, the random usage vector was
generated using the already specified method. We then calculated various metrics based on how
the given candidate solution would behave if it were implemented with the projected usage. We
used this data in completing different evaluative measurements for each solution. In addition, we
also evaluated patient flow across the days of the week. We randomly selected a set (A, I, O) of
patient flow for a specific day of week and specialty out of the full data set. We then projected
how our schedule would behave given those patient allocations.
12
Metrics of Evaluation
The candidate solutions were evaluated based on a variety of metrics. For the purpose of
defining these metrics, let 𝑈𝑖,𝑑,𝑠,𝑡 be a random usage vector for an individual trial 𝑖, day of week
𝑑, specialty 𝑠, and block time 𝑡. Let 𝐻𝑑,𝑠,𝑡 = (5.5 + 2𝑡) ∗ 𝑥𝑑,𝑠,𝑡 for 𝑑 = 1, 2, 4, 5 and 𝐻3,𝑠,𝑡 =
(5 + 2𝑡) ∗ 𝑥3,𝑠,𝑡 , being the number of hours allocated to each particular block of different day
and specialty. A list of metrics and their definitions are defined below:

̅) allows us to evaluate how much time was used divided by the
Mean Utilization (𝑈
amount of time allocated. More specifically,
𝑁
5
3
1
𝑈
̅ = ∑ ∑ ∑ ∑ 𝑖,𝑑,𝑠,𝑡
𝑈
𝑁
𝐻𝑑,𝑠,𝑡
𝑖=1 𝑑=1 𝑠∈𝑆 𝑡=1

Overtime (𝑉) is the measure of traditional overtime, the sum of overtime hours for each
allocated block for those blocks where more hours were used than were allocated. Let the
set of overtime values be denoted 𝑉 = {𝑉1 , 𝑉2 , … , 𝑉𝑁 }. We evaluated the mean and root
mean square (RMS) of Overtime, 𝑉̅ and 𝑉𝑟𝑚𝑠 , respectively over the N trials. For only
𝑈𝑖,𝑑,𝑠,𝑡 > 𝐻𝑑,𝑠,𝑡 ,
5
3
𝑉𝑖 = ∑ ∑ ∑ 𝑈𝑖,𝑑,𝑠,𝑡 − 𝐻𝑑,𝑠,𝑡
𝑑=1 𝑠∈𝑆 𝑡=1

Excess Allocation (𝐸) measures the amount of time that the ORs were left empty because
nobody needed to use them. Let the set of Excess Allocation values be denoted 𝐸 =
{𝐸1 , 𝐸2 , … , 𝐸𝑁 }. Like with Overtime, we evaluated 𝐸̅ and 𝐸𝑟𝑚𝑠 . For only 𝑈𝑖,𝑑,𝑠,𝑡 <
𝐻𝑑,𝑠,𝑡 , 𝐸𝑖 is defined as:
5
3
𝐸𝑖 = ∑ ∑ ∑ 𝐻𝑑,𝑠,𝑡 − 𝑈𝑖,𝑑,𝑠,𝑡
𝑑=1 𝑠∈𝑆 𝑡=1
13

Patient Flow is a measure of the various types of patients over days. We measure all of
the following only for where a block is actually allocated:
o The mean number of each patient type per day is important to analyze. Consider
the following measurement of the mean patients by type across days, where 𝑖 is
the trial number, 𝑠 is the specialty, and 𝑡 is the block time allocation:
3
𝑁
1
𝐴̅𝑑 = ∑ ∑ ∑ 𝐴𝑖,𝑑,𝑠,𝑡
𝑁
𝑖=1 𝑠∈𝑆 𝑡=1
3
𝑁
1
𝐼𝑑̅ = ∑ ∑ ∑ 𝐼𝑖,𝑑,𝑠,𝑡
𝑁
𝑖=1 𝑠∈𝑆 𝑡=1
3
𝑁
1
𝑂̅𝑑 = ∑ ∑ ∑ 𝑂𝑖,𝑑,𝑠,𝑡
𝑁
𝑖=1 𝑠∈𝑆 𝑡=1
o To get a closer look at the spread of patient types across a day, we look at the
standard deviations of the number of A, I, and O patients across trials each day:
𝑁
𝑆𝐷𝐴𝑑
2
3
3
𝑁
1
1
=
∑ (∑ ∑ 𝐴𝑖,𝑑,𝑠,𝑡 ) −
(∑ ∑ ∑ 𝐴𝑖,𝑑,𝑠,𝑡 )
𝑁−1
𝑁(𝑁 − 1)
𝑖=1
𝑁
𝑠∈𝑆 𝑡=1
𝑖=1 𝑠∈𝑆 𝑡=1
2
3
𝑁
3
1
1
𝑆𝐷𝐼𝑑 =
∑ (∑ ∑ 𝐼𝑖,𝑑,𝑠,𝑡 ) −
(∑ ∑ ∑ 𝐼𝑖,𝑑,𝑠,𝑡 )
𝑁−1
𝑁(𝑁 − 1)
𝑆𝐷𝑂𝑑
𝑖=1
𝑠∈𝑆 𝑡=1
𝑁
3
2
2
𝑖=1 𝑠∈𝑆 𝑡=1
2
𝑁
3
1
1
=
∑ (∑ ∑ 𝑂𝑖,𝑑,𝑠,𝑡 ) −
(∑ ∑ ∑ 𝑂𝑖,𝑑,𝑠,𝑡 )
𝑁−1
𝑁(𝑁 − 1)
𝑖=1
𝑠∈𝑆 𝑡=1
2
𝑖=1 𝑠∈𝑆 𝑡=1
o Weekly patient flow standard deviations are given by
5
5
𝑑=1
𝑑=1
1
1
𝑆𝐷𝐴̅ = ∑ (𝐴̅𝑑 − ∑ 𝐴̅𝑑 )
5
5
2
14
5
5
𝑑=1
𝑑=1
1
1
𝑆𝐷𝐼 ̅ = ∑ (𝐼𝑑̅ − ∑ 𝐼𝑑̅ )
5
5
5
5
𝑑=1
𝑑=1
2
1
1
𝑆𝐷𝑂̅ = ∑ (𝑂̅𝑑 − ∑ 𝑂̅𝑑 )
5
5
2
RESULTS
Candidate Solutions
In our solution set, Candidate Solution 1 is defined as the optimal solution as originally
proposed by the Consultant. We are therefore going to compare all of the metrics of our
candidate solutions to this solution’s metrics. Since the Consultant is interested in seeing what
insights can be gathered about his optimal solution, we want to evaluate his solution in depth.
By changing each weight and each allocation percentage as discussed in the Methods
section, we obtained 13 more candidate solutions by using Microsoft Excel’s GRG Nonlinear
Solver. Candidate solutions 2 through 11 are solutions that drive the allocated hours of particular
specialties towards the goal hours (with allocation percentage 0.8). The order of specialties that
is being focused on is the same order as that shown in Table 3. That is, candidate solution 2
corresponds to Gen, 3 to Ortho, etc. Candidates 12, 13, and 14 correspond to solutions where the
weights of all of the specialties in the objective function are the same, but the allocation
percentages are respectively 0.75, 0.85, and 0.90.
Allocation Score Metric
In order to better comprehend the results over multiple different candidate solutions, we
developed a score called Allocation Score. This score, denoted as Q, combines both Overtime
and Excess Allocation for each day over all specialties. The equations use RMS values in order
15
to have a nonlinear penalty for extreme values, especially for Overtime extreme values. This
makes sense because missing the allocated usage by a small amount is a negligibly costly
compared to missing the allocated usage by large amounts. Similarly, having Excess Allocation
is less costly than Overtime, as not having enough hours (which leads to overtime hours) is
problematic. The overtime penalty is weighted by a value 𝛼, which we varied as a method of
creating robustness in our statistic. We used 𝛼 ∈ {0,1,2,3,4,5,6,7,8,9,10} for evaluation.
𝑁
5
3
1
1
2
𝑄= ∑ √
∑ ∑ ∑(𝑈𝑖,𝑑,𝑠,𝑡 − 𝐻𝑑,𝑠,𝑡 )
𝑁
13 ∙ 5
𝑖=1
𝑑=1 𝑠∈𝑆 𝑡=1
{
5
3
2
1
(𝑈
− 𝐻𝑑,𝑠,𝑡 ) , 𝑈𝑖,𝑑,𝑠,𝑡 > 𝐻𝑑,𝑠,𝑡
+ 𝛼√
∑ ∑ ∑ { 𝑖,𝑑,𝑠,𝑡
13 ∙ 5
0, 𝑈𝑖,𝑑,𝑠,𝑡 ≤ 𝐻𝑑,𝑠,𝑡
𝑑=1 𝑠∈𝑆 𝑡=1
}
Recall that Q can be calculated for each candidate solution, so we have a set {Q} of Q’s.
To then easily compare Q values across solutions at various 𝛼-values, we decided to scale Q by
the minimum Q, or more specifically for each candidate solution 𝑘 ∈ {1, … ,14}:
{𝑄} = {𝑄1 , … , 𝑄14 }
𝑄𝑘′ = 100
𝑄𝑘
min{𝑄}
Thus, we can look at the value of 𝑄𝑘′ where 100 is the best possible solution and each
other 𝑄𝑘′ value is a specific percentage worse than the best 𝑄𝑘′ value based on its relative scale to
the optimal 𝑄𝑘′ . Appendix B.1 contains the full table of 𝑄𝑘′ values calculated at various 𝛼 values.
Evaluation of Candidate Solutions
Deciding different thresholds of the metrics for the removal of different candidate
solutions from consideration was completed by the evaluation of the individual metrics
16
themselves in relation to the other candidate solutions. It is up to the reader and/or analyst to
determine these threshold values for the specific set of candidate solutions based on each
metric’s individual importance for the specific application.
Our initial method of removing candidate solutions was based on Allocation Score at
various values of α. In particular, 𝛼 values of 0 and 1 seemed to be the most accurate measure as
proposed before the Overtime RMS values dominate the statistic. At high values of 𝛼, the
allocation scores of each solution give the same relative information as the mean Overtime of
each solution. Figure 1 shows that from 𝛼 = 2 𝑡𝑜 10, the ranking of highest to lowest allocation
score stays the same, suggesting that 𝛼 > 1 only gives extraneous information. We plotted the
Allocation Score values for our 14 candidate solutions and decided to set 105 as our cutoff score
for eliminating different candidate solutions.
Figure 1. Allocation Scores of all Candidate solutions at different values of alpha. This
figure was mainly used for determining which alpha value is used for analysis and our allocation score cutoffs.
We chose to attack Allocation Score first because our client ranked the importance of
Overtime and Excess Allocation highly. Candidate solutions 6, 8, 10, 11, and 12 were all
removed based on their Allocation Score values at ∝= 0. We then removed candidate solutions
7, 9, and 14 because of their Allocation Score values at ∝= 1.
17
We then considered patient flow and in particular looked at the weekly standard deviation
of patient flow for the three types. From that, we concluded that we should remove candidate
solution 13 because it had too much variance (𝑆𝐷 > 1) in all three patient types, especially in
Inpatient and Outpatient standard deviation. Thus, we evaluate five final candidate solutions in
more depth: 1, 2, 3, 4, and 5. See Appendix B.3 for the particular values of each statistic for each
candidate solution. Recall candidate solution 1 is the Consultant’s solution.
A Detailed Look at Our Candidate Solutions
We wanted to compare Overtime and Excess Allocation across our remaining candidate
solutions as compared to the Consultant’s solution. As a result, we found that our candidate
solutions had less Overtime and slightly more Excess Allocation than the Consultant’s solution.
Consider Figure 2 below, which demonstrates this phenomenon. This suggests that the
Consultant’s solution overall allocates less time to each block, while our candidate solutions
allocate more. Also, this trend suggests that the Consultant’s solution fails to allocate enough
time more frequently than our candidate solutions, as these measures are aggregate measures.
Overtime and Excess Allocation
Overtime
Excess Allocation
Consultant Solution Overtime
Consultant Solution Overtime and Excess Allocation
400
350
Time (Hours)
300
250
200
150
100
50
0
2
3
4
5
Candidate Solution
Figure 2. Overtime and Excess Allocation of Candidates 2-5. The Consultant’s
Overtime and Excess Allocation is plotted as the two lines.
18
We then wanted to look at the patient flow across the different candidate solutions. Recall
that patient flow is important due to hospital capacity constraints and that lower variation is
ideal. For the Consultant’s solution, consider Figure 3 which plots the mean and standard
deviation of patient flow across days of the week. The Consultant’s solution provides a very low
variation for Inpatient and Outpatient, but has a high variation in Admit patients.
Patient Allocation Across Days for Candidate Solution 1
18
16
Number of Patients
14
12
10
8
6
4
Admit
Inpatient
Outpatient
2
0
Monday
Tuesday
Wednesday
Thursday
Friday
Figure 3. Patient Allocation Across Days for Candidate Solution 1. (𝑆𝐷𝐴̅ , 𝑆𝐷𝐼̅ , 𝑆𝐷𝑂̅ ) = (1.8761, 0.5470, 0.5690)
A similar plot for Candidate Solutions 2, 3, 4, and 5 are included in Appendix B.2[4].
Candidate Solutions 2, 3, 4, and 5 all have worse Inpatient and Outpatient variation than the
Consultant’s solution. However, they have lower Admit patient variation. In addition, these
solutions have higher Outpatient mean values on Friday, which in effect lessens the need for
large staff over the weekend. The standard deviations across days and type for different
candidate solutions do not vary much.
Looking in more detail, it becomes evident that Friday sees fewer patients overall.
Oftentimes, physicians request operating rooms earlier in the week to avoid heavy staffing over
the weekends. While this is convenient for the physicians, this leads to some congestion earlier
19
in the week. We sought candidate solutions that minimized variation, especially for Admit, to
help alleviate this problem; however, again, it is up to the analyst to decide whether physician’s
preferences or overall hospital patient flow. Our candidate solutions thus increased the amount of
Admit patients on Friday so that the weekly Admit variation was not as staggering. Since this is
not very optimal for the physicians themselves, our candidate solutions made up for this by
increasing the amount of Outpatients on Friday. This means that patients are leaving the hospital,
and that fewer physicians and nurses will have to staff the Outpatient rooms.
Figure 4. Percent Utilization by Specialty for the Consultant’s Solution.
We lastly looked at the mean utilization of each candidate solution. Overall, the mean
utilization did not vary between the candidate solutions, as each mean utilization totaled to
around 68%. This differs from our allocation percentage because the amount of hours requested
by each specialty does not equal the amount of hours actually used. While these two values are
calculated similarly, the true patient data differs from the “theoretical” data, or the number of
requested hours.
20
Figure 4 shows the utilization of every specialty, along with the overall mean utilization,
for Candidate Solution 1. We see that Ortho, ENT, and Uro are the three specialties that get the
fewest hours. However Ortho is the only specialty that does not receive enough total hours for
that department to treat all of the patients (i.e., the utilization is greater than 1). This trend is the
same with the other solutions, with Ortho, ENT, and Uro being allocated fewer hours. This may
just the number of expected necessary hours (i.e. the requested hours) by the specialty is not
enough compared to how many hours the blocks will actually be used. Different candidate
solutions that increase the number of requested hours for these specialties may provide a
relatively more robust solution.
CONCLUSION
We have designed a system that can obtain new candidate solutions, creates statistics
based on historical patient data, and then evaluates the statistics. The most useful novel part of
this project is the Monte Carlo simulation and creation of statistics, as this has not been done in
other hospital OR scheduling reports. The simulation also does not necessarily require an
objective function or optimization – the user can just input a candidate solution and look at
meaningful physical measurements that can be useful for the hospital.
While we do have a list of candidate solutions and measures of evaluation, again, these
methods are not all encompassing. More candidate solutions can be determined and the ranking
of different solutions based on the importance of different metrics are at the hands of the analyst.
This report merely shows one way to evaluate the physical metrics obtained that is more
preferable to the Consultant.
21
Opportunities for Future Analysis
For future analysis, there are several assumptions that can be tweaked, including
completing a simulation across weeks as opposed to across days. The reason we chose to
simulate across days is because it seems more precise to allocate and simulate at a finer level, as
opposed to a more broad approach. Since we had the daily data rather than the weekly data, this
more precise approach was possible. We do not expect too much variation in the results if that
method were to be completed.
A more detailed approach for future analysis could be to use Data Envelopment Analysis
(DEA). The basic idea behind this analysis is to create a Pareto frontier in all of the dimensions
for each parameter of evaluation. From there, one could measure the distance between the
frontier and the given candidate solution, thus defining the quality of a candidate solution, as
tested against the parameters of evaluation.
Lastly, our list of candidate solutions is not the full population of candidate solutions, and
many other candidate solutions may be inputted into our simulation for analysis. For example,
examining a current hospital solution, an optimal hospital solution, and intermediate solutions
between the current and optimal solutions may provide many insights. These intermediate
solutions can initially start at the current solution and only have small changes until the optimal
solution is reached. This would then provide analysis of a series of OR schedules that lead from
current to optimal solution. While this would be desirable to examine in this report, our client
was unable to provide us with his affiliated hospital’s current solution.
22
BIBLIOGRAPHY
[1] S. F. Sufahani et al. “A Scheduling Problem for Hospital Operating Theatre.” Arxiv.org.
University Tun Hussein Onn Malaysia, 2012. Web. 1 Mar 2015.
<http://arxiv.org/pdf/1205.2108.pdf>.
[2] A. Romanyuk and A. Silva. “Optimization of an Operating Room Surgical Schedule.”
Ese.wustl.edu. Washingon University in St. Louis, 2012. Web. 1 Mar 2015.
<http://ese.wustl.edu/ContentFiles/Research/UndergraduateResearch/CompletedProjects/We
bPages/sp12/FinalReportSilvaRomanyuk.pdf>.
[3] F. Dexter et al. “An Operating Room Scheduling Strategy to Maximize the Use of Operating
Room Block Time: Computer Simulation of Patient Scheduling and Survey of Patients’
Preferences for Surgical Waiting Time.” citeseerx.ist.psu.edu. University of Iowa, 1999.
Web. 1 Mar 2015.
<http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.331.3313&rep=rep1&type=pdf>.
[4] Callaghan, Martina (2014). barwitherr(errors,varagin). MATLAB Central File Exchange.
Retrieved April 18, 2015. <http://www.mathworks.com/matlabcentral/fileexchange/30639barwitherr-errors-varargin-/content/barwitherr.m>
23
APPENDIX A: PROBLEM FORMULATION
Appendix A.1: Notation of Decision Variables
Mon
Tues
Wed
Thurs
Fri
Gen Ortho Tx
𝑥1,1,1 𝑥1,2,1 𝑥1,3,1
𝑥2,1,1 𝑥2,2,1 𝑥2,3,1
𝑥3,1,1 𝑥3,2,1 𝑥3,3,1
𝑥4,1,1 𝑥4,2,1 𝑥4,3,1
𝑥5,1,1 𝑥5,2,1 𝑥5,3,1
Mon
Tues
Wed
Thurs
Fri
Tx
𝑥1,9,1
𝑥2,9,1
𝑥3,9,1
𝑥4,9,1
𝑥5,9,1
Uro
𝑥1,10,1
𝑥2,10,1
𝑥3,10,1
𝑥4,10,1
𝑥5,10,1
ENT OMFS Uro
𝑥1,4,1 𝑥1,5,1 𝑥1,6,1
𝑥2,4,1 𝑥2,5,1 𝑥2,6,1
𝑥3,4,1 𝑥3,5,1 𝑥3,6,1
𝑥4,4,1 𝑥4,5,1 𝑥4,6,1
𝑥5,4,1 𝑥5,5,1 𝑥5,6,1
CT
𝑥1,11,1
𝑥2,11,1
𝑥3,11,1
𝑥4,11,1
𝑥5,11,1
PV
Neuro
𝑥1,12,1 𝑥1,13,1
𝑥2,12,1 𝑥2,13,1
𝑥3,12,1 𝑥3,13,1
𝑥4,12,1 𝑥4,13,1
𝑥5,12,1 𝑥5,13,1
Plas Neuro
𝑥1,7,1 𝑥1,8,1
𝑥2,7,1 𝑥2,8,1
𝑥3,7,1 𝑥3,8,1
𝑥4,7,1 𝑥4,8,1
𝑥5,7,1 𝑥5,8,1
Neuro
𝑥1,14,1
𝑥2,14,1
𝑥3,14,1
𝑥4,14,1
𝑥5,14,1
PV
𝑥1,15,1
𝑥2,15,1
𝑥3,15,1
𝑥4,15,1
𝑥5,15,1
CT
𝑥1,16,1
𝑥2,16,1
𝑥3,16,1
𝑥4,16,1
𝑥5,16,1
Shown above are decision variables 𝑥𝑑,𝑠,𝑡 for different combinations of day of week and
specialty/OR combination for block time of 8 hours. For 10 and 12 hours, 𝑡 is replaced
respectively with 2 and 3.
24
Appendix A.2: List of Constraints in Equation Form
16
3
3
∑ ∑ 𝑥 𝑑,𝑠,𝑡 = 13
∑ 𝑥𝑑,1,𝑡 ≤ 3
𝑠=1 𝑡=1
16
𝑡=1
3
∑ 𝑥𝑑,𝑠,1 ≤ 2
∑ 𝑥𝑑,2,𝑡 ≤ 4
𝑠=1
16
𝑡=1
∑ 𝑥𝑑,𝑠,2 ≤ 5
3
∑ ∑ 𝑥𝑑,𝑠,𝑡 ≤ 2
𝑠=1
16
𝑠=3,9 𝑡=1
3
∑ 𝑥𝑑,𝑠,3 ≤ 8
∑ 𝑥𝑑,4,𝑡 ≤ 2
𝑠=1
8
3
𝑡=1
3
∑ ∑ 𝑥 𝑑,𝑠,𝑡 ≤ 11
∑ 𝑥𝑑,5,𝑡 ≤ 2
𝑠=1 𝑡=1
13 3
𝑡=1
∑ ∑ 𝑥 𝑑,𝑠,𝑡 ≤ 1
3
∑ ∑ 𝑥𝑑,𝑠,𝑡 ≤ 2
𝑠=9 𝑡=1
3
𝑠=6,10 𝑡=1
3
∑ 𝑥𝑑,14,𝑡 ≤ 1
∑ 𝑥𝑑,7,𝑡 ≤ 2
𝑡=1
3
𝑡=1
∑ 𝑥𝑑,15,𝑡 ≤ 1
𝑡=1
3
∑ 𝑥𝑑,16,𝑡 ≤ 1
𝑡=1
𝑥𝑑,𝑠,𝑡 ∈ ℕ
∑
3
∑ 𝑥𝑑,𝑠,𝑡 ≤ 2
𝑠=8,13,14 𝑡=1
3
∑ ∑ 𝑥𝑑,𝑠,𝑡 ≤ 2
𝑠=11,16 𝑡=1
3
∑ ∑ 𝑥𝑑,𝑠,𝑡 ≤ 2
𝑠=12,15 𝑡=1
25
APPENDIX B: RESULTS
Appendix B.1: Result of Allocation Score Algorithm Calculations
Candidate Solution
𝛼
0
1
2
3
4
5
6
7
8
9
10
1
100.0
110.5
116.0
119.0
120.9
122.2
123.2
123.9
124.5
124.9
125.3
2
103.1
100.3
100.3
100.3
100.3
100.3
100.3
100.4
100.4
100.4
100.4
3
103.4
100.8
101.0
101.1
101.2
101.2
101.2
101.3
101.3
101.3
101.3
4
101.2
100.5
101.4
101.9
102.2
102.4
102.6
102.7
102.8
102.9
102.9
5
102.9
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
100.0
6
102.9
114.0
119.8
122.9
124.9
126.3
127.3
128.1
128.7
129.2
129.6
7
101.0
111.6
117.1
120.2
122.1
123.4
124.3
125.1
125.6
126.1
126.5
8
110.1
118.5
123.3
125.9
127.5
128.6
129.5
130.1
130.6
131.0
131.3
9
100.3
110.9
116.4
119.5
121.4
122.7
123.7
124.4
125.0
125.4
125.8
10
107.1
116.4
121.6
124.3
126.1
127.3
128.2
128.9
129.4
129.8
130.2
11
106.6
116.0
121.1
123.9
125.6
126.8
127.7
128.4
128.9
129.4
129.7
12
113.4
121.2
125.8
128.3
129.9
131.0
131.8
132.4
132.9
133.2
133.6
13
100.9
103.9
106.3
107.7
108.5
109.1
109.5
109.8
110.1
110.3
110.4
14
103.3
110.0
113.9
116.1
117.5
118.4
119.1
119.6
120.0
120.4
120.7
26
Appendix B.2: Patient Flow Variation for Selected Candidate Solutions
Patient Allocation Across Days for Candidate Solution 3
Patient Allocation Across Days for Candidate Solution 2
16
18
14
16
14
Number of Patients
Number of Patients
12
10
8
6
4
10
8
6
4
Admit
Inpatient
Outpatient
2
0
12
Monday
Tuesday
Wednesday
(𝑆𝐷𝐴
̅ , 𝑆𝐷𝐼̅ , 𝑆𝐷̅̅̅
𝑂 ) = (1.3743,
Thursday
0
Friday
1.4303, 0.9300)
14
14
12
12
Number of Patients
Number of Patients
16
6
Wednesday
Thursday
Friday
1.6848, 0.8947)
10
8
6
4
4
Admit
Inpatient
Outpatient
2
0
Tuesday
Patient Allocation Across Days for Candidate Solution 5
Patient Allocation Across Days for Candidate Solution 4
8
Monday
(𝑆𝐷𝐴
̅ , 𝑆𝐷𝐼̅ , 𝑆𝐷̅̅̅
𝑂 ) = (1.4790,
16
10
Admit
Inpatient
Outpatient
2
Monday
Tuesday
Wednesday
(𝑆𝐷𝐴
̅ , 𝑆𝐷𝐼̅ , 𝑆𝐷̅̅̅
𝑂 ) = (1.3790,
Thursday
Friday
1.3999, 0.9049)
Admit
Inpatient
Outpatient
2
0
Monday
Tuesday
Wednesday
(𝑆𝐷𝐴
̅ , 𝑆𝐷𝐼̅ , 𝑆𝐷̅̅̅
𝑂 ) = (1.3825,
Thursday
Friday
1.4216, 0.9504)
27
Appendix B.3: Statistics for All Candidate Solutions
̅
𝑈
𝑉̅
𝑉𝑟𝑚𝑠
𝐸̅
𝐸𝑟𝑚𝑠
𝐴1̅
𝐴̅2
𝐴̅3
𝐴̅4
𝐴̅5
𝐼1̅
𝐼2̅
𝐼3̅
𝐼4̅
𝐼5̅
𝑂̅1
𝑂̅2
𝑂̅3
𝑂̅4
𝑂̅5
𝑆𝐷𝐴̅
𝑆𝐷𝐼 ̅
𝑆𝐷𝑂̅
𝑆𝐷𝐴1
𝑆𝐷𝐴2
𝑆𝐷𝐴3
𝑆𝐷𝐴4
𝑆𝐷𝐴5
𝑆𝐷𝐼1
𝑆𝐷𝐼2
𝑆𝐷𝐼3
𝑆𝐷𝐼4
𝑆𝐷𝐼5
𝑆𝐷𝑂1
𝑆𝐷𝑂2
𝑆𝐷𝑂3
𝑆𝐷𝑂4
𝑆𝐷𝑂5
1
0.702
56.04
59.77
248.41
250.03
10.45
7.12
7.43
10.37
9.32
12.05
9.89
13.47
10.48
10.21
7.84
10.62
8.64
7.89
7.31
1.88
0.55
0.57
3.35
2.45
3.23
2.55
2.35
3.95
3.96
3.79
3.96
3.43
3.33
3.26
2.99
3.42
3.03
2
0.667
37.19
40.18
269.17
270.63
9.72
9.32
6.91
7.42
10.24
11.19
11.90
11.62
13.37
10.25
8.17
8.39
9.21
8.62
8.96
1.37
1.43
0.93
3.05
2.45
3.23
2.24
2.43
3.74
3.21
3.68
3.04
3.12
3.43
3.47
3.28
3.32
3.54
Candidate Solution
3
4
5
0.679
0.654
0.668
40.18
37.19
37.24
43.12
40.29
40.24
267.14
273.43
267.97
268.65
274.83
269.45
11.12
8.13
6.48
10.00
9.85
7.45
10.57
9.74
9.62
6.81
9.62
9.67
7.62
7.36
10.68
11.64
12.55
11.36
8.53
11.21
9.17
11.98
10.75
12.03
10.49
11.20
11.15
13.76
11.58
12.09
7.33
8.90
8.10
8.76
8.73
8.46
6.97
7.84
8.05
9.67
7.46
8.19
7.09
9.75
10.12
1.48
1.38
1.38
1.68
1.40
1.42
0.89
0.90
0.95
3.33
3.06
3.11
2.49
2.44
2.50
3.23
3.20
3.20
2.26
2.23
2.24
2.46
2.45
2.44
4.04
3.71
3.76
3.20
3.22
3.22
3.67
3.68
3.72
3.04
3.01
3.06
3.11
3.16
3.14
3.54
3.44
3.44
3.38
3.46
3.34
3.29
3.27
3.26
3.35
3.30
3.36
3.52
3.52
3.54
6
0.697
56.06
60.06
250.86
252.49
9.28
7.07
8.60
9.81
8.79
11.91
9.91
12.12
11.48
10.98
8.36
10.57
8.82
7.11
6.95
1.75
1.19
1.00
3.33
2.43
3.01
2.52
2.39
3.96
3.96
3.47
4.07
3.64
3.13
3.23
3.11
3.43
3.18
7
0.670
49.67
53.57
257.44
259.00
10.02
9.21
6.18
8.61
10.14
8.51
11.92
11.94
12.10
11.13
8.77
8.39
7.94
8.81
8.34
1.69
0.57
0.66
3.33
2.37
3.10
2.54
2.31
3.84
3.86
3.76
3.79
3.57
2.99
3.12
3.36
3.60
2.83
28
̅
𝑈
𝑉̅
𝑉𝑟𝑚𝑠
𝐸̅
𝐸𝑟𝑚𝑠
𝐴1̅
𝐴̅2
𝐴̅3
𝐴̅4
𝐴̅5
𝐼1̅
𝐼2̅
𝐼3̅
𝐼4̅
𝐼5̅
𝑂̅1
𝑂̅2
𝑂̅3
𝑂̅4
𝑂̅5
𝑆𝐷𝐴̅
𝑆𝐷𝐼 ̅
𝑆𝐷𝑂̅
𝑆𝐷𝐴1
𝑆𝐷𝐴2
𝑆𝐷𝐴3
𝑆𝐷𝐴4
𝑆𝐷𝐴5
𝑆𝐷𝐼1
𝑆𝐷𝐼2
𝑆𝐷𝐼3
𝑆𝐷𝐼4
𝑆𝐷𝐼5
𝑆𝐷𝑂1
𝑆𝐷𝑂2
𝑆𝐷𝑂3
𝑆𝐷𝑂4
𝑆𝐷𝑂5
8
0.693
54.05
58.16
259.28
261.03
9.90
10.03
9.65
6.56
6.55
11.25
8.48
11.17
12.83
10.26
8.73
8.70
7.49
8.35
10.93
1.31
1.36
0.33
3.13
2.33
3.11
2.57
2.41
3.65
3.89
3.59
4.12
3.91
3.26
3.01
3.17
3.46
2.86
9
0.692
54.53
58.44
254.06
255.59
7.47
9.90
9.40
9.62
7.50
9.10
11.22
10.83
11.19
14.14
8.44
8.73
7.38
7.49
7.83
1.79
1.29
1.13
3.36
2.43
3.06
2.52
2.40
3.97
3.93
3.46
4.09
3.38
3.15
3.28
3.15
3.45
3.29
Candidate Solution
10
11
12
0.673
0.678
0.618
48.95
49.68
36.46
53.18
53.76
41.33
260.02
256.28
277.65
261.77
257.99
279.51
7.11
10.07
10.01
7.43
7.10
10.72
9.70
8.29
6.63
9.74
8.83
8.79
9.90
9.98
10.00
9.88
12.75
8.53
9.17
9.89
12.60
11.19
13.58
13.49
10.81
10.74
13.12
11.66
9.66
10.52
10.62
8.74
8.74
8.44
10.67
6.54
7.76
8.24
7.71
7.88
7.90
9.07
6.43
8.08
9.97
1.37
1.42
1.24
0.71
1.33
1.45
0.69
0.63
1.13
3.16
3.11
3.07
2.41
2.37
2.51
3.04
2.84
3.06
2.56
2.59
2.50
2.40
2.39
2.47
3.67
3.65
3.56
3.95
3.91
3.60
3.60
3.37
3.52
3.82
4.08
4.09
3.93
3.93
3.49
3.26
3.26
3.24
3.28
3.13
3.16
3.01
3.26
3.33
3.60
3.63
3.23
2.97
2.78
3.35
13
0.696
47.25
51.01
244.69
246.42
9.87
9.73
10.78
6.22
8.35
11.19
11.15
12.56
13.49
13.19
8.70
8.13
6.56
7.62
9.13
1.74
1.50
1.65
3.33
2.45
3.14
2.33
2.50
3.94
3.91
3.61
3.28
4.04
3.52
3.03
3.38
3.71
2.87
14
0.696
45.19
49.64
225.91
228.08
7.47
10.41
9.70
9.39
6.45
9.17
10.47
11.20
11.09
12.86
8.41
7.86
8.21
8.69
7.84
1.56
1.50
1.35
3.13
2.52
3.05
2.52
2.40
3.71
3.63
3.57
4.03
3.73
3.12
3.25
3.56
3.38
3.02
29