How to Create a Business Focused Data Quality Assessment Dylan Jones,

How to Create a Business
Focused Data Quality Assessment
Dylan Jones, Editor/Community Manager
[email protected]
Why Do We Need a Data Quality Assessment?
We need to perform a data quality assessment
[DQA] for many reasons...
Data Warehouse
Implementation
Data Migration
Project
CRM / ERP
Monitoring
Integrating Billing
Systems
Potential Merger or
Acquisition
Revenue Assurance
or Recovery
Risk Analysis or
Compliance
Fraud Investigation
Integration
Integrating Billing
Systems
On a billing integration we want
to know if our processes are
working correctly so that we’re
compliant with regulations and
also protecting our revenue
Migration
Data Migration
Project
On a migration we want to know
if the legacy data will support
integration, transformation, load
and future target functions
Merger
Potential Merger or
Acquisition
On a merger, we want to verify
that the company is being
accurate with its statement of
accounts
e.g. “We have 20,000 customers”
Compliance
Risk Analysis or
Compliance
For compliance we may want to
check that a company has
handed over all information of
customers that meet certain
criteria e.g. Terrorists/Criminals
DQA helps us understand...
“Does our data perform
against expectations, goals or
requirements?”
Regulatory
Performance
Financial
Legal
What Happens During a DQA?
We use technology to assess data against a
series of metrics or criteria
●
●
●
●
17,121 Customer Records have a missing postal code
35% of Equipment.Install_Date values are invalid
99.2% of Financial Transactions have a valid timestamp
300 out of 192,213 customer accounts are duplicated
What are some common metrics (or
data quality dimensions)?
Completeness
Validity
Conformance
Accuracy
Consistency
Integrity
Uniqueness
Timeliness
Dependency
Coverage
Business Rule
Violations
etc…..
DQA’s can be complex...
Result #1
Assessment #1
Result #2
Assessment #2
Result #3
Assessment #3
Findings #1
Findings #2
Findings #3
(Local Data
Stewards)
(Legal &
Compliance)
(Head of
Operations)
So, what’s the problem?
● DQA’s are often carried out by technical people who
assess data in isolation (e.g. by column, table, system)
● The findings lack business impact because they present
primarily on data quality metrics
● The business can’t engage because there is no
business relevance to create a story around
But we’ve bought a data quality tool!
Yes, that’s a wise move because …
● You can leverage your data quality functionality to tell a
bigger (+ better) story by integrating business metrics
● You can automate the assessment process and put it
into business-as-usual
How can you improve the
conventional approach?
Understand Your
Profitability Drivers
Build a Financial
Performance Model
1
2
Create an Holistic
Data Quality System
3
Create an
Interrogative
Reporting Layer
4
Step 1: Understand Your
Profitability Drivers
Understand Your
Profitability Drivers
1
Case Study: Revenue Assurance
Telecoms companies need to ensure that
when they implement complex B2B
customer telecoms solutions they are
optimising their internal cost centres,
recovering all client revenues and ensuring
service level continuity.
American Bank requests an
international telecoms solution
Understand Your
Profitability Drivers
1
Telco initiates multiple processes
Understand Your
Profitability Drivers
1
An example of a Telecoms process
System
System
System
System
You need to understand the
profitability of your core process
●
●
●
●
●
Cost of installing 3rd party equipment
Cost of leasing international lines
Cost of hiring contractors or local workers
Cost of lead generation and sales
Revenues generated from service
Understand Your
Profitability Drivers
1
Performance is also useful...
●
●
●
●
How long does it take to service a line?
Which team receives the most orders?
Which engineers perform the most revisits?
What is the Process Cycle Efficiency?
Understand Your
Profitability Drivers
1
“Huh? But we don’t have this
information to hand because…”
●
●
●
●
●
Some of it belongs to a different owner
Some of it is in different systems
Some of it is in paper records
Some of it doesn’t connect
Some of it...
Understand Your
Profitability Drivers
1
Good. Poor information gives you a
perfect opportunity to add value.
Understand Your
Profitability Drivers
1
Flex your data quality muscles to help create a better
view of costs, revenues and performance
Engineering
Planning
Data Quality
System
Procurement
Fulfillment
Billing
Understand Your
Profitability Drivers
1
Once you start linking and enriching
your data you can gain a clearer
view of profitability and performance
Understand Your
Profitability Drivers
1
Example: Telecoms Order System
Links to other systems, you can
enrich with financial and
performance data
Performance Metrics and
data quality defects
Step 2: Build a Financial
Performance Model
Build a Financial
Performance Model
2
We need to connect clusters of financial, performance
and data quality metrics
Data Quality Management Reporting System
Build a Financial
Performance Model
2
How do you do it?
Demonstration...
Build a Financial
Performance Model
2
But isn’t this just business
intelligence?
Precisely. It’s where modern data quality technology is heading.
Data Quality metrics and business performance metrics are measured
the same way but for years they’ve been separated.
You need to bring them into the same environment if you want to
engage and focus the business.
Build a Financial
Performance Model
2
Step 3: Create an Holistic Data
Quality System
Create an Holistic
Data Quality System
3
Why bother with an holistic data
quality system ?
Each system can have great quality data
when measured against isolated data
quality metrics but...
… if you measured data quality and financial impact
across a broader process it could still result in
significant lost revenue and profits
Key Tip: Assessing the quality of each system in
isolation it won’t give you the full story
Create an holistic
data quality system
3
For example: Completeness
Completeness in the
[Planning.Inventory]
column tells us if there
are any missing values
but...
Value
Complete?
Juniper M320
YES
Juniper M320
YES
Juniper M312
YES
Juniper m320
YES
JUNIPER (M320)
YES
Create an holistic
data quality system
2
3
For example: Completeness
... it doesn’t tell us if there are other values that should be
there from [Stocks.Inventory]
Master Stock Item
Qty
Juniper M320
10
Juniper M312
7
Juniper M315
3
Juniper M325
2
Juniper M327
15
Lack of
coverage
Value
Complete?
Juniper M320
YES
Juniper M320
YES
Juniper M312
YES
Juniper m320
YES
JUNIPER (M320)
YES
Create an holistic
data quality system
2
3
For example: Completeness
... it doesn’t tell us if there are other values that should be
there from [Stocks.Inventory]
Procurement
Catalogue
Cost
Juniper M320
40000
Juniper M312
90000
Juniper M315
890
Juniper M325
500
Juniper M327
100
Lack of price
context
Value
Complete?
Juniper M320
YES
Juniper M320
YES
Juniper M312
YES
Juniper m320
YES
JUNIPER (M320)
YES
Create an holistic
data quality system
2
3
Step 3: Create an Interrogative
Reporting Layer
Create an Interrogative
Reporting Layer
4
Reporting Layer
By this point you will have a consolidated view
of your:
● Financial Metrics
● Data Quality Metrics
● Performance Metrics
Create an interrogative
reporting layer
4
What does the overall
architecture look like?
Dashboards and Visualisation
DQ Assessment
Rules
Data Profiling/
Discovery
Data Repository
and Storage
Information Chain
Management
DQ Monitoring and
Alerts
Data Integration/
Movement
Create an interrogative
reporting layer
4
This gives you much greater clarity
of the impact of poor quality data
● How long does it take to complete an order?
● What staff costs are involved in fulfillment?
● What costs are incurred when data quality
errors force a re-design?
● What is the frequency of delayed orders over
the last 12 months?
Create an interrogative
reporting layer
4
Adding performance and financial
data creates focus
One telecoms company
found their fulfillment
processes were taking
longer and were
increasing their stock of
unplanned equipment
Create an interrogative
reporting layer
4
The issue lay with a rogue software release that
introduced defects into the information chain
Software
Change
Added
Defects
enter the
information
chain
Holistic Reporting
System monitors
data quality,
financial and
performance metrics
Alarm Raised
Create an interrogative
reporting layer
4
Summary
Takeaway #1: Stop thinking of data quality
assessments in terms of isolated analysis
Summary
Takeaway #2: Understand how your business
model operates and create an end-to-end view
of it, identifying data sources along the way
Summary
Takeaway #3: Focus on the goal of your data
quality assessment and create an architecture
to support that objective
e.g. Data Quality impact on Service Levels
Summary
Takeaway #4: To create engagement and
focus from the business make sure that you’re
integrating business metrics into your reporting
and underlying assessment architecture.
Summary
Takeaway #5: Don’t spoon-feed the business
with canned reports, give them an environment
that they can interrogate, let them discover the
issues that matter to them most (hint: it will
change over time too)
Questions?
Contact me: Dylan Jones ([email protected]
Search on Data Quality Pro: http://dataqualitypro.com
Data Quality Assessment Guide: http://bit.ly/dq-assess