How to report the DoE part of a Design Space document
-using MODDE for Design of Experiments
Introduction
Quality by Design and Design Space are concepts that have been around for quite some time
in the pharmaceutical industry. Slowly the industry has warmed up to these concepts. Since
the authorities provided the industry with guidelines, the interest has increased even more.
The vision of filing a Design Space is ambitious and may feel overwhelming when first proposed. However, over the years more and more knowledge has accumulated and the industry
seems ready to take the next step. When taking this step a frequently asked question is: “How
do we report the DoE part of a Design Space document internally as well as to the authorities?”
In Design Space projects different departments are commonly involved. Therefore a Standard
Operating Procedure (SOP) to avoid errors due to misunderstandings is necessary.
This white paper will focus on the DoE part of the Design Space, and will also briefly touch
on the workflow to establish before creating a successful report.
Establishing a Design Space
Establishing a Design Space is a process starting by defining the Critical Process Parameters
(CPP’s) and the Critical Quality Attributes (CQA’s). Design of Experiments (DoE) is used to
identify the important CPP’s and their impact on the CQA’s.
Case Types
The outcome of the DoE investigations should be judged on two criteria resulting in a case
categorization: (1) is the CQA inside or outside the specification limits and (2) is the model
statistically significant or not.
Case 1
Criteria
Next step and gained knowledge
The CQA’s are inside specification. A statistically significant model is obtained.
The acceptable range can be set as the investigated range
using the low and high values of the input parameters.
The model provides process knowledge through coefficient or
contour plots and ensures process robustness.
Case 2
Some CQA’s are outside specification. A
statistically significant model is obtained.
The model can be used to predict set points and acceptable
ranges for the CPP’s ensuring that all CQA’s are inside specified limits.
The model provides process knowledge and supports process
robustness by adjustment of CPP ranges. The Design Space
should be confirmed by additional experiments.
Case 3
Case 4
The CQA’s are inside specification. No or a
weak statistically significant model is obtained.
With all values inside the limits the design space can be set
using the low and high values of the CPP’s in the design.
Some CQA’s are outside specification. No or
a weak statistically significant model is
obtained.
A Design Space cannot be based on this data and a new design
is needed.
Without a statistical model little process knowledge is gained.
Preparation for a Design Space filing
When starting the project of establishing and filing a Design Space, it is extremely important
that the teams from the different departments “start on the same page”, communicate and use
the same terminology. Structure from the beginning to the end is Alpha and Omega for the
filing process to run smoothly. Trivial things, such as an inconsistent naming of the input parameters (CPP’s) and output parameters (CQA’s) and their abbreviations can lead to numerous rewrites, if not addressed at the beginning.
Communication is Key
Start the project by gathering a representative from each involved department and discuss the
topics in this section. Communication is the key and small misunderstandings can take a long
time to realize and to correct.
Naming and Abbreviations
Create a file where abbreviations and names are given in alphabetical order. Make this file
accessible to all involved parts in the project. Assign a person from each department to communicate with the other departments so that the used names are unique and not used for another CPP’s or CQA’s.
The length of the abbreviations should be considered as this can be a limiting issue for plots
and graph in the Design of Experiment software.
Traceability
Choose software that allows the name of the experiment to be traceable and visible in all
plots.
Reporting a Design Space
Creating the report of the Design Space runs rather smoothly if all critical aspect mentioned
earlier in this white paper are addressed correctly. The report can be build around a predefined
template, based on the 4 cases mentioned above. A template created by Umetrics and can be
accessed by contacting Umetrics. In the sections below this template will be explained and to
some extent demonstrated.
Report template general content
The template includes the following general content:
• A list of the CPP’s also referred to as the input parameters.
• A list of the CQA’s, also referred to as the quality product parameters, and the process
performance parameters.
• The used design and the model terms possible to estimate with this type of design.
• The design illustrated as a graph. This makes it easier to understand the design for the
(untrained) reader. This can be done with up to 5 factors (CPP’s) in Umetrics DoE
software MODDE.
• A matrix displaying the design. In many complex processes it can be difficult to perform exactly the experiments suggested by the DoE. In these cases the report should
include both the theoretical design and the actual design including an evaluation on
whether or not the design has been distorted by the changes. An estimation of the distortion is the condition number.
The four cases
The analysis part of the report template begins with a table summarizing the quality of the
models. This table contains the specification limits of the CQA’s as well as any experiments
that did not fulfill specifications. The quality of the model is described with the R2, Q2 and
validity of the models and the standard deviation of the replicates.
The illustration and decision part of the analysis section is built around the 4 cases mentioned
in the beginning of this white paper. For each case, specific plots are suggested.
Below is a list of plots to use for the four cases. Optional plots are also listed with the rational
on when to use them.
Diagnostic tool
Description and Decisions
Replicate plot
The replicate plot displays the variation of the replicated runs. The plot allows quick
identification of runs outside specification and displays the repeatability of the replicated runs and the position of the center points in comparison with the response
range
Coefficient plot
The coefficient plot informs on which CPP is most relevant for the system.
Validated Design Space
If one or more of the experiments are very close to the limit(s) we recommend to use
Monte Carlo simulations to validate the Design Space and estimate the risk of failure. Design Space validation, based on Monte Carlo simulations, is a straightforward
approach to identify the safe region for operations. As no model is obtained, it is
reasonable to anticipate that all variation in the experiments can be interpreted as a
variation around the mean. This variation can then be estimated as the mean value
+/- the t-value*standard deviation.
Replicate plot
See above.
Coefficient plot
See above.
Contour plot
The contour plot can be used to illustration and to some degree understand the
relationship and the position of the Design Space up to 4 CPP’s. However, the contour plot lacks a risk of failure estimate and the illustration of higher dimensions
spaces.
Estimated Design Space
By identifying the optimal setting (set point) and applying Monte Carlo simulations
around this point the largest possible Design Space for the selected set point can be
estimated with the acceptable risk of failure. The calculation of the Design Space is a
search function that expands the possible factor ranges from a set point to the largest possible range where all CQA’s predictions are still within the specifications. The
resulting distribution of predictions simulates a real situation with a random combination of factor setting disturbances within a given range.
Interaction and
prediction plots
In cases where additional practical constraints (cost, time etc.) are present the interaction and prediction plots can be used to illustrate a suitable Design Space that
fulfils the practical constraints.
Case 3
Replicate plot
A Design Space is established.
Case 4
Replicate plot
No Design Space is established. Identify problem.
Case 1
Case 2
Note that case 2 has many opportunities for different illustrations. This is of course related to
the fact that this case does not represent a robust system as the CQA’s are outside specification and there exists a significant relationship between changing the CPP’s and the CQA’s.
Below is an example of a report for a case 2 investigation. The example is based on GE
Healthcares Capto S column. A full description of the analytical work is given in the article
”Capto S Cation Exchanger for post-Protein A purification of monoclonal antibodies” at
www.gelifesciences.com.
Report example, illustrating case 2
Input parameters (CPP’s)
The three input parameters and the upper and lower limits of the design are listed in the table
below.
Input parameter
Abbreviation
Unit
Lower limit
Upper limit
pH
pH
-
4,5
5,5
Residence time
Res
min
2
6
Conductivity
Con
mS/cm
5
15
Output parameters
Output parameter
Abbreviation
Unit
Lower limit
Upper limit
Target
Dynamic binding capacity at 10% breakthrough
QB10%
-
110
-
Maximize
Study design
The design used is a Central Composite Face centered design (CCF) with three input parameters (CPP’s). The CCF design allows for all main, interaction and quadratic terms to be resolved in the system. Three center points were added for model diagnostics. Below the design
is depicted.
The output parameter (CQA) and the specifications for this parameter are found in the table
below.
The theoretical and the actual design plans were identical with a condition number of 4,44,
see below.
Run
pH
Residence time
Conductivity
QB10%
-
min
mS/cm
-
pH
Residence time
Conductivity
QB10%
Run
-
min
mS/cm
-
N1
4,5
2
5
76
N2
5,5
2
5
126
N3
4,5
6
5
116
N4
5,5
6
5
149
N5
4,5
2
15
99
N6
5,5
2
15
0
N7
4,5
6
15
105
N8
5,5
6
15
8
N9*
5
4
10
121
N10*
5
4
10
137
N11*
5
4
10
143
N12
4,5
4
10
119
N13
5,5
4
10
84
N14
5
2
10
125
N15
5
6
10
121
N16
5
4
5
139
N17
5
4
15
54
*Center point
Model diagnostics
Below, the model diagnostics are displayed. We conclude that the investigation is a case 2
investigation as the model is significant (R2, Q2 and Model validity are close to 1) and 7 runs
are outside the specification for the dynamic binding capacity of 10%.
Output
Dynamic binding
capacity at 10%
breakthrough
Upper
limit
-
Lower
limit
Runs outside
specifications
R
110
N1
N5
N6
N7
N8
N13
N17
0,96
2
Q
2
0,87
Model
validity
SD of
replicates
0,86
11,4
Raw data illustration
The replicate plot is a simple illustration of the raw data. Note that the variation in the center
points (N9, N10 and N11) is small compared to the overall variation in the data.
Model illustration
The coefficient plot illustrates which model terms that have the highest influence. In this case
it is clear that conductivity and pH have a high influence on the output and residence time has
a low influence.
Design Space illustration
A common way of illustrating the Design Space (see ICH Q8) is by using a contour plot. The
quadratic term for pH and conductivity and their interaction term are illustrated by selecting
them on the inner axes, as these two CPPs are the most important and their interaction is highly significant. The contour plots shows the accepted QB10% larger than 110.
However, the contour plot point estimate does not show any probability estimate. For a probability estimate Monte Carlo simulations can be used around a process set point. The optimizer tool in MODDE was used to find the set point, followed by the Design Space estimating
tool to find the region with low risk of failure.
The list and plots below shows the resulting Design Space.
Conclusion and what to next
Using the CPP settings in the Design Space table the predicted response profile clearly states
that the risk of failure is less than 0.1% .
The next step, validation, is to conduct a design using the high and low limits of the estimated
Design Space and the optimum as the new center point. This will result in a case 1 or 3 where
a design space is possible to establish. A design space validation can then be performed on the
design space in order to estimate the risk of failure.