Roberta L. Grant, Susan L. Santos, Mike L. Dourson, Stephanie
Shirley, Neeraja K. Erraguntla, R. Jeffrey Lewis, and Nancy B. Beck
Society of Toxicology, March 22-26, 2015
San Diego, CA
Texas Commission on Environmental Quality
•
•
•
Focused on US Environmental Protection Agency’s
Integrated Risk Information System (IRIS) as an example
• Considered experiences in other programs with
varied mandates
Used specific case studies to explore diverse approaches
(including some very basic ideas)
Work is continuing to develop implementable approaches
showing how information can be presented differently to
improve the consideration and use of uncertainty
information by risk managers
*American Chemistry Council’s Center for the Advancement of Risk Assessment Science and Policy
2
•
The Workshop examined four approaches:
1. Comparing Values to Other Relevant Peer-Reviewed
Numbers
2. Unpacking Toxicity Assessments to Understand and
Improve Confidence – Focus of this talk
3. Presenting Toxicological Information Visually in the
Context of Alternative Values, Exposure Levels, and
Biomonitoring Equivalents
4. Improving Transparency in Dose-Response Decision
Making
*American Chemistry Council’s Center for the Advancement of Risk Assessment Science and Policy
3

Completeness of Database

Study Quality

Reference Concentration (RfC)/ Reference
Dose (RfD)
* USEPA (1994) Methods for Derivation of Inhalation Reference
Concentration and Application of Inhalation Dosimetry
4
•
•
Many steps in toxicity assessment – the major steps
are referred to as ‘elements’
8 major ‘elements’ of a toxicity assessment and 3
‘confirmatory elements’ assessed
• for the purposes of our example, each element
is treated as being equally important
• how confident are we that the value is likely to
be accurate/precise/predictive



Accuracy: degree of closeness of the
measurements to that quantities actual (true)
value
Precision: the degree to which repeated
measurements under unchanged conditions
show the same value
Predictive: is there a balance between being
protective and reasonably accurate.
6
•
•
Used clear criteria to judge confidence and implement
scaling (numerical or descriptive values)
• 1 = low confidence; 5= high confidence
Audience = risk assessor
• help explain approach and confidence to a risk
manager/decision maker
7
Non-Cancer Toxicity Assessment Elements
Element
Description for High Confidence
Database
Completeness
Database included investigations of a comprehensive array of non-cancer
toxicity endpoints, established from studies of chronic duration in various
mammalian species (refer to EPA 1994)
Systematic Review
A systematic approach was used to identify studies, evaluate their quality
and integrate them.
Key Study Quality
The key study(ies) are well-conducted and can be used without restrictions.
Critical Effect
The database is sufficient to identify the effect occurring at early time points
(i.e. critical effect). This should protect against all other adverse effects. MOA
information, if available, helps to determine if the earliest critical effect has
been identified.
Relevance of Critical
Effect
The critical effect is known to be related to human findings. If only animal
studies are available, MOA information, if available, helps to determine if the
critical effect is relevant to humans.
Point of Departure
(POD)
Dose response is well understood and NOAEL and LOAEL are identified.
Ideally, BMC/BMD modeling was performed with small differences between
BMD and BMDL.
Human Equivalent
POD
Human data are available or human equivalent dose/concentration is known
from PBPK or similar model.
Sensitive Populations Human data on sensitive subpopulations are available or PBPK or similar
model is available to account for TK/TD differences between general and
sensitive populations.
8
Cancer Toxicity Assessment Elements
Element
Description for High Confidence
Carcinogenic Potential
Using a weight of evidence approach, adequate data exists to classify the
chemical into EPA/IARC categories (e.g., not carcinogenic, possibly carcinogenic,
known carcinogen, etc.).
Systematic Review
A systematic approach was used to identify studies, evaluate their quality and
integrate them.
Key Study Quality
The key study(s) are well-conducted and can be used without restrictions.
Relevance of Critical
Effect
The tumor type/site is known to be related (or may be related) to human
findings. If only animal studies are available, MOA information, if available,
helps to determine if the critical effect is relevant to humans.
Point of Departure
Dose response is well understood. Ideally, BMC/BMD modeling was performed
with small differences between BMD and BMDL.
Human Equivalent POD
Human data are available or human equivalent dose/concentration is known
from PBPK or similar model.
Low Dose Extrapolation
A biologically based model or PBPK model is available and MOA understanding
leads to extrapolation to lower doses with confidence.
Sensitive Populations
Human data on sensitive subpopulations are available or PBPK or similar model
available to account for TK/TD differences between general and sensitive
populations. If the MOA is mutagenic, then age-dependent adjustment factors
were applied.
9
Elements for a Confirmation of Toxicity
Assessment
Element
Description for High Confidence
Peer Review
An external independent peer review was conducted
including opportunities for public comment, written peer
review report, and the Agency has responded
appropriately to peer review and public comments.
Validation
The Agency has evaluated whether the final toxicity
values are realistic and plausible based on available
information.
Risk Value
Comparison
Relevant values from high quality, peer reviewed sources,
are within three-fold of each other.
10
Validation or Reality Check
• In some cases, the aggregate impact of all risk assessment
decisions involved in a toxicity assessment results in a toxicity
value that is overly conservative and unrealistic.
• Example: applying UFs in multiple areas, and then
multiplying them together, may compound the
conservatism as the upper bounds on each of the factors
is used in the calculation
11
Toxicity Assessment Elements that
Were Not Considered
• Mode-of Action
• RfC/RfD Confidence
• Uncertainty Factors
12
Confidence Scoring: Example –
Hazard Identification: Quality of Key Study(ies)
Confidence
scale and
basis for
scoring
Implication
References
1 = Low: chosen study may have deficiencies, but is still considered useful.
2-3 = Medium: chosen study is reasonably well done and can be used with
some restrictions; extrapolation seems reasonable based on findings with other
chemicals
4-5 = High: chosen study is well done and can be used without restriction
Klimisch scores for evaluating quality of toxicology studies have international
support; other scales have also been widely used. A scoring method consistent
with Klimisch scores for assessing quality of human studies also exists, along
with other scales. Studies with Klimish scores associated with high confidence
can be used with little or no restriction.
Potential tools:






H.J. Klimisch, M. Andreae and U. Tillmann. 1997. A Systematic Approach for
Evaluating the Quality of Experimental Toxicological and Ecotoxicological
Data. Regulatory Toxicology and Pharmacology Vol 25 pp 1-5
EPA’s OPP Core grades (guideline, minimum, supplementary, invalid)
EPA IRIS principal study confidence scoring (high, medium or low)
Bevan and Strother, 2012
Money et al., 2013
Systematic Review tool quality assessment components (e.g. NTP OHAT
approach, Rooney, 2014)
Similar tables describing basis for confidence scoring available for other Major Elements described earlier
e.g., database completeness, systematic review, critical effect, etc.
13
Confidence Scoring: Example - Dose-Response
Assessment: Point of Departure (POD)a
Confidence
scale and
basis for
scoring
1 = Low: many uncertainties exist in the POD; only a free-standing NOAEL or
LOAEL identified; few dose groups; BMD modeling not possible
2-3 = Medium: some uncertainty exists in identified POD, NOAEL or LOAEL
identified, but few dose groups; BMD modeling was conducted; difference
between BMD and BMDL is large
4-5 = High: dose response and basis for POD are well understood: NOAEL and
LOAEL identified; multiple dose groups, BMD modeling conducted; difference
between BMD and BMDL is small (approximately 2-fold or less)
Implication
When BMC/BMD modeling can be performed, the entire dose-response curve is
used to determine the POD. Such determination improves the basis of risk
management decisions.
References
Many references here (e.g., IPCS, 2005; EPA, 2002; EPA, 2012)
a
slope of the dose-response curve is critical information if the RfC or RfD is exceeded. A steep slope
increases the importance of the exposure assessment.
Similar tables describing basis for confidence scoring available for other Major Elements described
earlier e.g., database completeness, systematic review, critical effect, etc.
14
Confidence Scoring Results for Inhalation RfC
for Carbon Tetrachloride (CCl4)
Major
Elements
Peer Review
+++
Validation
-
Toxicity Value Comparison
-
Confirmation
Elements
15
Confidence Scoring Results for Inhalation
Reference Value (ReV) for 4-vinylcyclohexene
(4-VCH)
0
1
2
3
4
5
Database Completeness
Major
Elements
Systematic Review
Key Study Quality
Critical Effect
Relevance of Critical Effect
Point of Departure (POD)
Human Equivalent POD
Sensitive Populations
Peer Review
++
Validation
-
Toxicity Value Comparison
-
Confirmation
Elements
16
Confidence Scoring for RfC for CCl4
Confidence Scoring for ReV for 4-VCH
17
Confidence Scoring for RfC for CCl4
Element
Score
Basis
Database Completeness
3
Medium
Developmental study in different species and multigeneration study lacking
(confidence from EPA (2010) was medium)
Systematic Review
1
Low
At the time of this assessment, IRIS did not employ a systematic procedure
for data gathering, analysis and internal review
Key Study Quality
5
High
The chosen study is well done and can be used without restriction
(confidence from EPA (2010) was high)
Critical Effect
4
High
Studies are sufficient to determine the critical effect with confidence; fatty
change in liver is moderate severity
Relevance of Critical Effect
5
High
The critical effect of liver toxicity is appropriate to humans. Extrapolation
seems reasonable based on findings with humans and other experimental
animal species. Critical effect matches human experience
Point of Departure (POD)
5
High
A lower limit on the BMD was used as the POD. Multiple dose groups
Human Equivalent POD
4
High
HEC and duration adjustments were derived using a PBPK model
Sensitive Populations
3
Medium
+++
High
Peer Review
Available life stage information does not suggest increased childhood
susceptibility
The external peer review seemed adequate and EPA appeared to take
comments into consideration
Validation
-
Not conducted
Risk Value Comparison
-
Not conducted
18
Confidence Scoring for ReV for 4-VCH
Element
Score
Database Completeness
3
Medium
Systematic Review
Key Study Quality
Critical Effect
Relevance of Critical Effect
1
Low
3
Medium
The chosen study was conducted using GLP in rats and mice, although only10
animals/sex were evaluated. (confidence from TCEQ (2011) was medium).
2
Medium
Studies are sufficient to determine the critical effect with confidence. Three
concentrations were tested and multiple endpoints evaluated. The following critical
effects occurred at the highest concentration: ovarian atrophy and mortality (severe
effects) and lethargy/ tremor (moderate effects).
Mice are sensitive for ovarian atrophy because they produce more reactive
metabolite than humans. However, since it is possible that humans produce the
reactive metabolite, a default assumption was made that ovarian atrophy may occur
in humans. The MOA for tremor/lethargy is not known, so it was assumed these
effects were relevant to humans.
BMC modeling was not conducted because adverse effects only occurred at the
highest concentration. A NOAEL and a LOAEL were identified.
1
Low
Point of Departure (POD)
2
Medium
Human Equivalent (POD)
3
Medium
1
Low
++
Medium
Sensitive Populations
Peer Review
Basis
A subchronic inhalation study was available in two species. Inhalation
developmental study and multigeneration inhalation study lacking. An oral two
generation reproductive/developmental study in mice showed no effects on
reproductive function (confidence from TCEQ (2011) was medium).
At the time of this assessment, TCEQ did not employ a systematic procedure for
data gathering, analysis and internal review.
Default duration adjustments and animal-to-human adjustments were conducted.
Available life stage information was not available to indicate sensitive populations
exist.
Peer input, a 90-day public comment period, and comments were addressed.
Validation
-
Data are not available to conduct a reality check.
Risk Value Comparison
-
Chronic inhalation values from other sources were not available.
19
•
•
Presentation of risk assessment results in clear and
concise manner is challenging
• Risk Managers must apply this information to make
decision
No ‘right’ way to communicate this type of information
• Different approaches may appeal to different people
• Consideration should be given to testing (e.g. focus
groups) before adopting or adapting approaches
20
•
•
•
Method presented treats each ‘element’ as equally
important
• But ‘element’ weighting could be done
Many ‘elements’ interrelated (e.g., mode of action
information important to identification of critical effect
and human relevance)
The working groups welcome input and feedback on
the approaches
21





Roberta L. Grant, Neeraja K. Erranguntla, and
Stephanie Shirley (Texas Commission on
Environmental Quality (TCEQ))
Susan L. Santos (Focus Group Consulting &
Rutgers University)
Mike L. Dourson (Toxicology Excellence for
Risk Assessment (TERA))
R. Jeffery Lewis (Exxon Mobile Biomedical
Services)
Nancy B. Beck (American Chemistry Council)
22
Thank you!
[email protected]