“Why do Environmental Health and Regulatory Programs Ignore Laboratory QC data?” Henry Leibovitz, Ph.D. Environmental QA/Lab Cert Officer RI State Health Laboratories Laboratory Data Users Federal and State laws have created the reliance on laboratory testing to support environmental health, regulatory and monitoring programs. CWA, SDWA, RCRA, CERCLA, Brownfields, CAA, etc. Relying on Laboratory Results Federal, state and municipal Environmental Health and Regulatory Programs utilize laboratory test results to inform decisions affecting public health and the environment. Environmental programs and utilities may use their own laboratories or contract the testing to laboratories that meet requirements specified in methods. Laboratory Certification Most federal, state and municipal environmental programs require support laboratories to be certified by a recognized Accreditation Authority. Quality Assurance and Control Environmental programs and laboratories each maintain and follow their own Quality Assurance Plans to ensure mission and data quality objectives are met. Programs Expect Laboratories • Maintain certification and reliability • Follow approved methods • Meet method requirements including quality controls • Produce accurate and precise data reports Laboratory Quality Assurance Systematic process laboratories follow to produce reliable and reproducible data. Laboratory Quality Control Quality control samples are tested with environmental samples in the laboratory to demonstrate testing is in control (accurate and precise.) Laboratory Reports Include: • Sample test results (program required) • QC test results (program discretion) Lab Reports will include QC results at no extra cost to programs. Reasons Laboratory QC Data Are Ignored • Assumption - if laboratory is certified then no need to verify data. • Most federal and state programs don’t require QC or sample data submittal. • Data review not defined in program Quality Plans • Programs choose not to use QC data Laboratory Certification Assures Capability • • • • • • • Quality Assurance Plan Method Detection Limit (MDL) studies Initial Demonstration of Capability Standard Operating Procedures Annual Proficiency Testing Training Documentation On-site assessments every 2-3 years Laboratory Certification Does Not Assure Accountability • Are sample testing Quality Controls met? • Are test results and reports reviewed? • Is compliance and monitoring data acceptable? • Did the laboratory follow approved methods? Are Laboratory Results Acceptable Who Decides ? • Laboratories decide data is acceptable for public health decision making. • Programs forfeit this responsibility EPA Methods Update Rule March 12, 2007 – 40 CFR Parts 122, 136, 141, 143, 430, 455 and 465 “This rule modifies the testing procedures approved for analysis and sampling under the Clean Water Act and Safe Drinking Water Act.” EPA March 2007 Methods Update Rule Most compliance methods require rejection of data that fail QC criteria. EPA March 2007 Methods Update Rule However, to do so when there is not opportunity for reanalysis, resampling or corrective action may cause a violation of (compliance) reporting requirements. EPA March 2007 Methods Update Rule Regulatory authority should decide whether to reject or accept data that have failed QC specifications in the method, or when problems have occurred during sample collection. Data Review, Verify or Validate? Environmental Programs may accept or reject sample results based on review, verification or validation of QC data results. Data Quality Objectives (DQO) Program Quality Plans • Define how laboratory QC data will be used to meet DQO, • Describe level of review necessary to determine if DQO are met. EPA OIG Report on Laboratory Fraud • Real concerns about the reliability and quality of laboratory data • Findings based on improper laboratory practices • Failure of laboratory to follow method requirements Unscrupulous Laboratories • • • • Fail to use approved methods Fail to perform QA/QC testing Fail to calibrate instruments properly QA/QC testing “after the fact” to satisfy an audit or data submission request What to do about Improper Laboratory Practices? • Addressed by data review and verification or validation • Identify options and resources to assure the laboratories are certified and sample results are acceptable Program Resources are Limited Many programs do not have the resources or the training necessary to review, verify or validate laboratory data. Support using 3-Way Ties Programs and laboratories rely on each other and on the Laboratory Certification Officer – Build ties between all three “Solutions to Analytical Chemistry Problems with Clean Water Act Methods” (“Pumpkin Book” EPA 821-R-07-002) March 2007 Strongly recommends that the supporting QA/QC data be submitted along with analytical results EPA “Pumpkin Book” – Principles of Data Review Applicable to laboratory methods: • Drinking water methods (500 – series) • SW-846 methods (RCRA methods) • ASTM International Committee D19 (water methods) • Standard Methods Laboratory Accountability – Require QC Results in Reports Problem with Retrieving QC Only When Questions Arise • Programs rely upon laboratories to keep QA/QC records unless requested • QA/QC data maybe components of analytical results for a variety of clients • Resolving questions of compliance is time consuming and confusing for the program Retrieving Data on Request Data provided by laboratories may have been compliant . . . . . . if QC data becomes misplaced there is no way to prove the data was compliant. Including QC Data in Report • Quality of the data evaluated directly • Program decides to accept or reject data • Supporting data becomes property of program • Programs can defend their decisions by verifying supporting data Laboratory - Program Accountability Finding the Resources • Develop data review capability through training workshops • Use laboratory and Certification Officer expertise as learning resource • Utilize new electronic data review software to verify or validate electronic reporting • Third Party Validation Data Review - What Level? Programs must decide and define the level of data review they will require in in their Quality Plan prior to the onset of the project. Programs should seek a predefined level they can achieve with their resources 1. Review QC results only 2. Review and verify QC and sample data 3. Validate the entire calibration, QC and sample data. Random Data Audits Programs may randomly request entire data set including original QC, calibration and sample data for assessment. Advantage – Laboratories must always be prepared for data validation by the program. Standardized Laboratory QA/QC (reviewed by certification officer) • Method detection limit (MDL) and minimum level of quantitation • Initial precision and recovery • Ongoing precision and recovery • Statements of data quality from the laboratory Standardized Laboratory QC (determined by validation) • Purity and traceability of reference standards • Number of calibration points • Linearity of calibration • Calibration verification Standardized Laboratory QC (reviewed by program) • Analysis of blanks (Method Blank) • Recovery of analytes spiked into a laboratory reagent blank (LFB, LCS) • Recovery of analytes spiked into the sample matrix (MS, LFM) Method Flexibility vs. Deviations Data review must distinguish between program allowed flexibility and improper deviations in method implementation. Method Flexibility vs. Deviations For unanticipated situations or where there is ambiguity in the regulations or analytical methods, try working mutual solutions with accreditation or regulatory authority. Three Way Support Laboratories Environmental Programs Laboratory Certification Improving Communication Environmental Programs Laboratory Certification Rarely communicate with laboratories Routinely communicate with all laboratories Establish analytical and reporting requirements Communicate program analytical and reporting requirements Lack access to laboratories Access to all laboratories Program Support Through Communication Programs Establish Requirements, Verify Data Quality – Approve or Reject Data Lab Certification Reinforces/Verifies Lab Capability of Meeting Program Requirements Laboratories Provide Verifiable Data Overlapping Responsibility Programs DATA QUALITY Laboratory Certification Laboratories It Takes a Laboratory! Have you ever received QC data with your routine clinical laboratory test result? It Takes a Laboratory! Clinical Requires 1000 tests to determine the public health impact on 1000 people It Takes a Laboratory! Environmental ONE test to determine the public health impact to an entire population! Programs and laboratories together are engaged in the responsibility to protect public health. Programs ultimately are responsible to make the decisions that impact it. Laboratory accountability is necessary to protect public health and the environment against damages caused by the questions surrounding laboratory reliability and integrity that will always be a factor in making the correct decisions. The final decision on accepting or rejecting laboratory results belongs to Environmental Programs. Questions or Comments to: Henry Leibovitz, Ph.D. RI State Health Laboratories [email protected] Thank you!