W E I B AY E S T E S T I N G : W H AT I S T H E I M PA C T I F A S S U M E D
B E TA I S I N CO R R E C T ? 1
David Nicholls, RIAC (Quanterion Solutions Incorporated)
Paul Lein, RIAC (Quanterion Solutions Incorporated)
Weibayes is a one-parameter Weibull analysis technique developed by Dr. Robert Abernethy and other engineers at Pratt &
Whitney Aircraft in the 1970’s to “solve problems when traditional Weibull analysis has large uncertainties or cannot be used
because there are no failures.”[1] A basic assumption governing the accuracy of Weibayes analysis and its associated test
regimens, as stated by Abernethy, is that the value of the Weibull
shape parameter, b, is known or can be reasonably estimated.
Knowledge of b can be derived from historical failure data, prior
experience, or from engineering knowledge of the physics of the
failure. Engineering knowledge of failure physics and consistent use of accurate, representative b values is best supported by
an historical library of Weibull beta plots based on actual corporate and product experience.
Defining the Problem
In the absence of “good” engineering knowledge of b, inaccuracies in the assumption of its value can have a significant impact
on (1) the Weibayes test regimen used to define a reliability
requirement, and (2) the interpretation of Weibayes test results
used to demonstrate whether that reliability requirement has
been met. Marketplace pressure towards decreased test times,
fewer test samples, and zero- or sudden death-based test regimens are intuitively beneficial because they lower the total test
cost and allow decisions to be made earlier. As one hypothetical
example, assume that we are given a stated reliability requirement of 0.90 at 1000 hours (i.e., the B10 life). A Weibayes zero
failure test plan is formulated using a sample size of 5 and an
“assumed” value of 1.5 for b. The test time required to demonstrate that this requirement has been met is calculated to be 1533
hours per test sample. What are the corresponding implications
and risk in the interpretation of the test results if the “true”
value of b is something different? The purpose of this paper
is to examine and quantify the risk of assuming an “incorrect”
value of b that is higher than the “true” value when performing
Weibayes zero-failure or sudden death testing, and the subsequent impact on the analysis and interpretation of the results.
Reliability at a specified design life can be calculated based on
Equations (1) and (2). From the standard Weibull equation:
R( t d ) = e
⎛ t ⎞β
−⎜ d ⎟
⎝η ⎠
where,
R(td) = Reliability at the design life, td
td = Design life to be demonstrated
h = Characteristic life at CDF = 63.2%
b = Weibull shape parameter
For Weibayes analysis, the characteristic life is expressed by
Abernethy [2] as:
1
⎡ N T β ⎤β
η = ⎢∑ i ⎥
⎣ i=1 r ⎦
where,
Ti = Test time for each sample
r = Number of failures
N = Sample size
h, b As defined above
In order to determine an appropriate test time per sample, Equations (1) and (2) can be combined and rearranged to give:
1
β
⎡
⎤β
−t d
⎥
Ti = ⎢
⎢⎣ N * ln( R( t d )) ⎥⎦
(3)
Substituting the values from the Introduction yields a 1533-hour
test requirement per sample for our Weibayes zero-failure test
example to demonstrate that the design life of 1000 hours at 0.90
reliability has been achieved.
There are other test options available to demonstrate this requirement, however. Sample sizes can be changed. As an alternative,
since the “true” value of b is unknown, a different “assumed”
value of b can be used. The question then becomes, what is the
“best” test plan to use, given that the 1000-hour design life at
0.90 reliability is a firm requirement and there are cost, resource
and schedule constraints to be considered. Using Equation (3),
a range of possible test scenarios can be generated to provide
visibility into options for sample sizes that include potentially
“better” assumptions for b. Table 1 illustrates one example of
potential zero-failure test scenarios, based on “assumed” values
of 1.5 and 3.0 for the Weibull shape parameter.
(1)
1 . This article is adapted, with permission, from the 2009 Proceedings of the Annual Reliability and Maintainability Symposium. © 2009 IEEE.
THE JOURNAL OF THE RELIABILITY INFORMATION ANALYSIS CENTER
(2)
FIRST QUARTER - 2009
Design Life (td) = 1000 Hours
Maximum Likelihood Estimate (MLE)
Sample
Size (N)
Per-Item Test Time (Ti) to Demonstrate B10 Life
(R = 0.90)
b = 1.5
b = 3.0
1
4483
2117
2
2824
1680
3
2155
1468
4
1779
1334
5
1533
1238
10
966
983
15
737
859
Table 1: Weibayes Zero-Failure Test Scenarios Example
For this hypothetical example, let’s say that organizational
constraints dictate that we cannot afford to use more than 5 test
samples, and there are significant schedule pressures to get this
product to market. From the table, under the “b = 1.5” column,
using fewer test samples requires additional test time per
sample (unacceptable within our schedule constraints). Using
more samples reduces test time per item, but this has already
been excluded from consideration. One remaining option,
since the “true” value of b is unknown anyway, is to assume
a different value for b. This allows us to use the same number
of samples, yet we gain some relief in the test time per sample
(approximately 300 hours if we test concurrently, or 1500 hours
if we test serially). In the competitive marketplace, every hour
counts. How much impact can using this different value of b
really have, anyway? The remainder of this paper will provide
some insight into the answer.
The Relationship Between
Test Sample Size and the Beta
Intersection Point
If you review Table 1 for a given sample size and compare the
required test times at b = 1.5 and b = 3.0, you will observe that,
for sample sizes less than or equal to 5, the per-sample test
times for the larger value of b are noticeably shorter than those
for the smaller b. For sample sizes of “N” equal to 10 and 15,
however, this relationship is reversed. What is the reason for
this? Simply put, the intersection of the “assumed” and “true”
beta plots represents a pivot point that can influence decisions
about Weibayes test plans that may subsequently result in bad
decisions and unacceptable, but unidentified, risk based on conclusions about the demonstrated reliability.
The two parameter Weibull distribution [3] defines the Cumulative Density Function (CDF) as:
F ( t ) = 1− e
⎛ t ⎞β
−⎜ ⎟
⎝η ⎠
(4)
where,
t = Time (in hours)
h = Characteristic life at CDF = 63.2%
b = Weibull shape parameter
Mathematical manipulation of Equation (4) leads to Equation
(5), which represents the straight-line solution to be plotted on
Weibull graph paper [4].
⎛ 1 ⎞
lnln⎜
⎟ = β ln( t ) − β ln(η)
⎝ 1− F ( t ) ⎠
(5)
⎛ 1 ⎞
lnln⎜
⎟
⎝ 1− F ( t ) ⎠
ln( t ) =
+ ln(η)
β
The location of the intersection point of two Weibull solution
lines with different betas (based on the same number of hours, t)
requires that ln(t1) be set equal to ln(t2) and:
⎛ 1 ⎞
⎛
⎞
1
lnln⎜
⎟ = lnln⎜
⎟
⎝ 1− F ( t1 ) ⎠
⎝1− F ( t 2 ) ⎠
Substituting
⎛ 1 ⎞
y = lnln⎜
⎟
⎝ 1− F ( t ) ⎠
(6)
and equating the two beta
equations (derived from Equation 5) yields:
continued on next page ›››
http://theRIAC.org
3
WEIBAYES TESTING: WHAT IS THE IMPACT IF ASSUMED BETA IS INCORRECT?
continued from page 3
y=
β1β 2
(ln(η2 ) − ln(η1))
β 2 − β1
(7)
Table 2 provides a reference for the beta intersection point for
various test sample sizes.
For Weibayes zero-failure testing with equivalent test times, the
number of failures, r, from Equation (2) is equal to 1 and the
summation term becomes the sample size, N. Substituting the
expression for h from Equation (2) into Equation (7) yields the
two-beta equation:
y=
1 ⎞
1 ⎞⎤
⎛
β1β 2 ⎡ ⎛
β2 β
β1 β
2
−
ln
NT
ln
NT
⎢ ⎜(
) ⎟⎠ ⎜⎝( ) 1 ⎟⎠⎥
β 2 − β1 ⎣ ⎝
⎦
(8)
The intersection of the two beta lines can then be algebraically
shown to occur at the By life percent value, where:
By life % = e y *100% =
1
*100%
N
(9)
This result is shown graphically in Figures 1 and 2 for a sample
size of 5 and 10, respectively.
Figure 2: Beta Intersection Point for Sample Size = 10
Sample Size (N)
Beta Intersection CDF
(By Life)
2
50.00
3
33.33
4
25.00
5
20.00
10
10.00
25
4.00
50
2.00
100
1.00
1000
0.10
Table 2: Beta Intersection Point as a Function of Sample Size
Figure 1: Beta Intersection Point for Sample Size = 5
Each Weibull graph includes a line plotted at b = 1.5 and b =
3.0. For N = 5, the beta intersection point occurs at approximately CDF = 20% (or the B20 life), disregarding any inherent
small sample bias that is associated with the Weibull Maximum
Likelihood Estimate (MLE). For N = 10, the beta intersection
point occurs at CDF = 10% (or the B10 life). Note that the beta
intersection point is independent of both the design life, td and
the test time, Ti (assuming that the test time on each of the “N”
samples is equal). In other words, different values for the design
life will shift the beta plot pair left or right along the x-axis, but
the beta intersection point will not deviate from the By life value.
THE JOURNAL OF THE RELIABILITY INFORMATION ANALYSIS CENTER
The Relationship Between the
Confidence Level and the Beta
Intersection Point
What happens to the beta intersection points when lower-sided
confidence bounds are introduced into the analysis? Simply
stated, with increasing confidence level the beta intersection
point will shift vertically at design life, td. While it will be the
focus of a future paper to determine the mathematical relationship that governs this shift, Figures 3 and 4, respectively, illustrate the concept. In this example, the design life is set at 1000
hours. The resulting CDF is 37% at the 80% lower confidence
bound (LCB) and 49% at the 90% LCB (compared to the CDF of
approximately 20% at the MLE based on the sample size of 5).
FIRST QUARTER - 2009
››In
considering the use of confidence bounds, as the
desired lower confidence bound increases, the CDF
value of the beta intersection point also increases.
Based upon these general relationships, we can now make some
specific observations regarding the presence and quantification
of risk associated with Weibayes testing if there is a difference
between the “assumed” and “true” values of beta that govern
the test length and the subsequent interpretation of the test
results.
Figure 3: Beta Intersection Point at 80% Lower CB
Risk as it Relates to the Difference
Between the Assumed and “True”
Values of Beta
As alluded to in the introduction and problem definition sections
of this paper, there is risk associated with using an “assumed”
value of b in establishing Weibayes test plans and interpreting
Weibayes plots when the “true” value of b is something different.
The risk discussed in this paper is limited to underestimating the
required Weibayes test time to demonstrate a specified design
life by assuming a value of b that is higher than the “true” value.
First, a general statement:
››The higher the CDF level of the beta intersection point,
the greater the risk of reaching an overly optimistic
conclusion for a fixed By life below that intersection point.
Figure 4: Beta Intersection Point at 90% Lower CB
General Relationships of the Beta
Intersection Point
Upon examination, the following general comments can be
made for a given design life:
››There is an intersection point between Weibayes plots of
two values of b, below which the required test time to
demonstrate that the design life has been met is shorter
for the higher value of beta than the lower value (see
Table 1 for n < 5). The opposite is true above the beta
intersection point (see Table 1 for n = 10 and 15).
››The smaller the Weibayes test sample size, the higher the
CDF value at which the beta intersection point occurs
(see Figures 1 and 2 and Table 2).
For the purposes of quantifying the risk, we will work with
Figures 5 and 6. This hypothetical example is based on a B10 life
requirement of 1000 hours.
Due to organizational budget and schedule constraints, it was
decided to proceed with a Weibayes zero-failure test based on a
sample size of three. The organization does not have a Weibull
library of beta values for its products, or adequate knowledge
of the predominant physical failure modes of the design, so it
needs to rely on engineering judgment to establish an “appropriate” value of beta for the test (the “true” value of beta is
unknown). Best engineering judgment concluded that, at the
1000-hour B10 design life, the product would exhibit signs of
wearout, so it was decided that b = 3.0 would be appropriate
(especially since it resulted in a shorter test time that supported
management budget, resource and schedule constraints). Based
on these criteria, the three samples were tested for 1468 hours
each (based on Table 1 for b = 3.0).
continued on next page ›››
http://theRIAC.org
5
WEIBAYES TESTING: WHAT IS THE IMPACT IF ASSUMED BETA IS INCORRECT?
continued from page 5
failures for this example is to simply read the results directly
from Figure 6. At T = 1000 hours, the “assumed” b = 3.0 line
indicates a CDF of 10%, indicating that 10% of the population
are expected to have failed by that time. The “true” b = 1.5
line at T = 1000 hours indicates that approximately 17% of the
population will have failed.
Figure 5: Quantifying Risk When “Assumed” Beta is Higher
Than “True” Beta
The test was run and no failures were experienced, so the organization was confident that the design life requirements had been
met. Unfortunately, returns from the field during warranty did
not support this conclusion. What might have gone wrong?
If the organization had been in a position to apply the resources
needed to collect and analyze their data and characterize the
product’s dominant physical failure modes, they might have
discovered that the “true” beta for their product was actually
1.5, meaning that the product wasn’t wearing out as fast as engineering judgment had assumed. Superimposing a Weibayes
plot with a beta of 1.5 on Figure 5 provides some clarification of
the impact of this assumption. From Table 1, the organization
tested their product for the required time based on a sample of
three and an “assumed” beta value of 3.0. If they had known
that the “true” beta was actually 1.5, then they would (or should)
have tested each of the three samples for 2155 hours (with zero
failures) to support a conclusion that the design life requirement
had been met. As a consequence, each sample was undertested
by 687 hours (2155 hours – 1468 hours).
The impact of insufficient testing on the conclusions drawn
from the analysis can be read directly from the Weibayes plot in
Figure 5. Instead of demonstrating a B10 life of 1000 hours, the
testing actually demonstrated a B10 life of only 685 hours, which
is 68.5% of the requirement.
The analysis can easily be extended to determine the increase
in the expected number of failures over any time period and
population size of interest. Additionally, the increase in associated repair and support costs resulting from the lack of knowledge of the “true” beta value can also be determined. One
approach to estimating the increase in the expected number of
THE JOURNAL OF THE RELIABILITY INFORMATION ANALYSIS CENTER
Figure 6: Increase in Expected Number of Failures
Mathematically, the exact values can be found from Equations
(1) and (2). Using the results from the analysis above and substituting them into Equation (2) yields:
1
⎡ 3 1468(1.5) ⎤1.5
η = ⎢∑
⎥ = 3054 hours
1 ⎦
⎣ i=1
(10)
where,
3 = Sample size, N
1468 = Test time per sample, Ti
1.5 = “True” Weibull shape parameter, b
Substituting the value of h into Equation (1) results in:
R(1000) = e
⎛ 1000 ⎞1.5
−⎜
⎟
⎝ 3054 ⎠
= 0.829
CDF = 1− R(1000) = 0.171 or 17.1%
where,
(11)
1000 = Design life, td
3054 = Characteristic life, h
1.5 = “True” Weibull shape parameter, b
0.829 = Reliability at the design life, td
0.171 = CDF at the design life, td = (1-R(td))
Suppose that there are 10,000 products (defined as “P”) in
the field and it costs $5,000 to repair each returned item. The
FIRST QUARTER - 2009
increase in the expected number of returns at 1000 hours, based
on a “true” beta of 1.5, will be:
([CDF(btrue)*P] – [CDF(bassumed)*P]) =
(1710) – (1000) = 710 additional returns
(12)
At $5,000 per repair, these additional returns will result in approximately $3.55 million in unanticipated cost to the organization.
MS Excel® Spreadsheet
Quanterion Solutions Incorporated (QSI), in its role in the operation of the Reliability Information Analysis Center (RIAC), has
developed a MS Excel® spreadsheet based on the concepts presented in this paper. The spreadsheet supports:
››Calculation of a Weibayes zero or sudden death failure test
plan as a function of (1) sample size, (2) required design
life, (3) the value of “assumed” beta, and (4) the potential
value of “true” beta. The output is the required test time
to demonstrate that a design life requirement has been met
under the stated conditions.
››Calculation of the error in the estimated design life based
on the value of an “assumed” beta and the potential “true”
beta. This error is translated into the “actual” design life
based on the “true” beta in comparison to the “expected”
design life based on the “assumed” beta.
››Graphical representation of the error and resulting design
life for a range of “true” beta values in comparison to the
single “assumed” value of beta
››Calculation of the increase (or decrease) in the expected
number of failures over a user-defined time period based
on the difference between the “assumed” and “true” value
of beta.
››Simple calculation of the cost impact associated with the
increase (or decrease) in the expected number of failures
over the user-defined time period
The spreadsheet is available for download from http://www.
theriac.org/informationresources/demosanddownloads.swn
the location of the design life CDF requirement in relation to the
beta intersection point, and (3) the relative difference between
the values of the “assumed” and “true” beta.
This paper addresses only the risk associated with using an
“assumed” beta value that is higher than the “true” beta value,
as this represents the most damaging technical and financial
risk scenario for the organization. As such, the statement can be
made that the greater the difference between the values of the
“assumed” and “true” values of beta (with the “assumed” beta
being higher), the greater the organization’s risk through undertesting a product to demonstrate that a required design life has
been met, and being overly optimistic in the interpretation of the
demonstrated reliability.
General recommendations that can be made through this investigation are that:
››Each
organization should strive to thoroughly understand and differentiate the physical modes and mechanisms associated with the root failure causes of its products in order to identify their “true” beta values
››Each organization should establish its own library of
Weibull plots and beta values that are representative of
its specific products and applications, and are based on
the physical modes and mechanisms associated with the
root failure causes of its products
››Where cost and schedule permit, an organization should
use the exact number of test samples that correspond
with the By design life that is to be demonstrated, as
that value determines the intersection point between
the “assumed” and “true” value of beta. For example,
to demonstrate that a B10 life requirement is met, use a
sample size of 10, as the intersection of the “assumed”
and “true” beta plots will be at the required design life
(i.e., “no” risk if the “assumed” and “true” values of beta
are different).
References
1.
Abernethy, R.B., “The New Weibull Handbook – Fifth
Edition”, Dr. Robert B. Abernethy, June 2007, pg. 6-1
We have shown that there is a mathematically supported relationship between the intersection point for an “assumed” and
“true” value of beta that can be used to assess and quantify risk
when performing Weibayes zero failure or sudden death testing
and analysis.
2.
Abernethy, R.B., “The New Weibull Handbook – Fifth
Edition”, Dr. Robert B. Abernethy, June 2007, pg. 6-2
3.
Weibull, W., “A statistical distribution function of wide
applicability”, J. Appl. Mech.-Trans. ASME, September
1951, 18(3), pg. 293-297
The level of risk is a direct function of (1) the vertical location of
the beta intersection point on the Weibayes plot (mathematically
proven to be a function of the number of samples tested), (2)
4.
Nicholls, D. (Editor/Co-Author), “System Reliability
Toolkit”, Reliability Information Analysis Center/Data
and Analysis Center for Software, December 2005, pg. 526
Conclusions and Recommendations
http://theRIAC.org
7