Risk & Innovation in Cybersecurity
Investments
Sponsored by Lockheed Martin
Independently conducted by Ponemon Institute LLC
Publication Date: April 2015
Ponemon Institute© Research Report
Risk & Innovation in Cybersecurity Investments
Ponemon Institute, April 2015
Part 1. Introduction
Ponemon Institute is pleased to present the results of Risk & Innovation in Cybersecurity
Investments, an industry survey sponsored by Lockheed Martin. The purpose of this research is
to understand how people, processes
and the desire to be innovative affect
cybersecurity technology investment
decisions.
We surveyed 618 U.S.-based
information technology (IT) and
security practitioners involved in
determining investments in
cybersecurity technologies. As shown
in Figure 1, business objectives are
most influential in deciding on a
specific technology (73 percent of
respondents) with security risk a close
second (68 percent of respondents).
Compliance with regulations is least
influential. In the context of this
research, we define security
innovation as “the use of enabling
technologies and personnel in new
ways to create a more secure and
efficient organization and improve alignment between security initiatives and business goals”. We
asked respondents to rate their organizations’ level of security innovation. Only 32 percent of
respondents felt their company is achieving a high level of innovation.
Key findings
Relying solely on Return on Investment (ROI) and Total Cost of Ownership (TCO) metrics
can lead to poor investment decisions. Seventy percent of respondents believe ROI and TCO
are important metrics for investment and measuring a technology’s economic benefits. However,
the same percentage say it is difficult to calculate an accurate ROI for a given security solution or
technology. TCO is also difficult to determine, according to 61 percent of respondents.
Incorporating other metrics into the decision process could result in smarter investments.
Fifteen percent of respondents say their organizations do not use ROI or TCO at all. These
organizations are most likely to look instead at improvements in the efficiency of security
operations (56 percent of respondents) or reduction in downtime (50 percent of respondents) as
ways to determine a technology’s viability.
Security investments are driven by cost. Sixty-four percent of respondents say cost and 56
percent of respondents say performance and vendor support are the most important factors when
investing in security technologies. Features such as interoperability, proven risk reduction and
lack of complexity are not considered as important (39 percent, 11 percent and 8 percent,
respectively).
Shelfware is every organization’s problem. Ninety percent of respondents say their
organization has invested in a security technology that was ultimately discontinued or scrapped
before or soon after deployment. On average, 31 percent of security technologies purchased by
organizations represented in this research over the past 24 months were never fully deployed
Ponemon Institute© Research Report
Page 1
Part 2. Key findings
In this section, we provide an analysis of the key findings. The complete audited findings are
presented in the appendix of this report. We have organized the report according to the following
topics?
§
§
§
The use of ROI and TCO to determine cybersecurity investments
When investments go wrong: the problem of shelfware
How risk and innovation affects security investments
The Use of ROI and TCO to Determine Cybersecurity Investments
Organizations are making investment decisions based on inaccurate and unreliable
numbers. As shown in Figure 2, 70 percent of respondents believe ROI and TCO are important
metrics for investment and measuring a technology’s economic benefits.
Figure 2. The importance of ROI and TCO in making a technology investment
40%
35%
35%
30%
25%
20%
26%
25%
19%
19%
16%
15% 15%
15% 15%
Not important
We do not use
ROI/TCO
15%
10%
5%
0%
Essential
Very important
Important
Importance of ROI
Ponemon Institute© Research Report
Importance of TCO
Page 2
However, according to Figure 3, 70 percent of respondents say it is very difficult (26 percent) or
difficult (44 percent) to calculate an accurate ROI for a given security solution or technology.
Similarly, it is difficult to calculate an accurate TCO, according to 61 percent of respondents.
More than half of respondents say it is difficult to be precise when calculating ROI for each
investment decision made (59 percent of respondents) and calculating a precise TCO (55 percent
of respondents) for each investment decision made.
Figure 3. How difficult is it to calculate a precise ROI and TCO for a security technology?
50%
44%
45%
38%
40%
35%
30%
25%
26%
25%
23%
18%
20%
15%
9%
10%
11%
3%
5%
3%
0%
Very difficult
Difficult
Not difficult
Calculate a precise ROI
Easy
Unsure
Calculate a precise TCO
What ROI reveals about technologies frequently purchased. According to respondents, the
average threshold ROI necessary to achieve a favorable investment decision in a given security
solution or technology is 13.7 percent. As shown in Figure 4, the security technologies with the
highest ROI are identity & access management (31 percent ROI), SIEM and security technology
(29 percent ROI) and encryption for data at rest and in motion (both 25 percent ROI).
Figure 4. Security technologies with the highest ROI
Identity & access management
31%
SIEM and security intelligence
29%
Encryption for data at rest
25%
Encryption for data in motion
25%
Anti-virus & anti-malware
25%
0%
Ponemon Institute© Research Report
5%
10%
15%
20%
25%
30%
35%
Page 3
According to Figure 5, access governance systems (9 percent), IT & credentialing system (8
percent), automated policy generation (8 percent), traditional firewalls (7 percent), and perimeter
or location surveillance (6 percent) are technologies with the lowest ROI.
Figure 5. Security technologies with the lowest ROI
Access governance systems
9%
ID & credentialing system
8%
Automated policy generation
8%
Firewalls (traditional)
7%
Perimeter or location surveillance
6%
0%
1%
2%
3%
4%
5%
6%
7%
8%
9%
10%
As a means of evaluating the success of specific investments, 54 percent of respondents say
their organization reconciles the historic (original) ROI with actual ROI for most or some
investments, according to Figure 6. Actual ROI tends to be lower than the historic ROI (46
percent of respondents) or actual and historic ROI tend to be very close (accurate), according to
34 percent of respondents.
Forty-eight percent of respondents say their organization reconciles the historic (original) TCO
with actual TCO for most or some investments. In contrast to ROI, actual TCO tends be higher
than historic TCO, according to 44 percent of respondents. Thirty-six percent of respondents say
actual and historic TCO tend to be close or accurate
Figure 6. Do you reconcile actual ROI and TCO with historic ROI and TCO?
60%
48%
50%
42%
36%
40%
33%
30%
20%
18%
15%
10%
4%
4%
0%
Yes, for most
investments
Yes, for some
investments
Actual ROI is reconciled with historic ROI
Ponemon Institute© Research Report
No
Unsure
Actual TCO is reconciled with historic TCO
Page 4
Inaccurate TCO and ROI metrics are used to reject technologies that could potentially be
successful. Despite the difficulty in determining an accurate TCO, 56 percent of respondents say
over the past 24 months one or more proposed investments in a particular solution or technology
was rejected because of unfavorable TCOs, as shown in Figure 7. Over the past 24 months, one
or more proposed investments was rejected because of unfavorable ROIs, according to 50
percent of respondents.
Figure 7. Have proposed investments in a particular security or technology been rejected
because of unfavorable ROIs or TCOs?
56%
60%
50%
50%
45%
41%
40%
30%
20%
10%
5%
3%
0%
Yes
No
Unfavorable ROIs
Unsure
Unfavorable TCOs
Other metrics could be useful in evaluating technology’s economic viability. Fifteen percent
of respondents say their organizations do not use ROI or TCO in their decision making. Figure 8
shows these organizations are most likely to use improvements in the efficiency of security
operations (56 percent of respondents) or reduction in downtime (50 percent of respondents as
ways to determine a technology’s viability.
Figure 8. Other metrics used to make investment decisions
More than one response permitted
Improvement in the efficiency of security
operations
56%
Reduction in system downtime
50%
Reduction in data breach costs
47%
38%
Reduction in time to contain security incidents
Reduction is non-compliance costs including
fines, penalties and lawsuits
23%
Reduction in time to detect security incidents
12%
6%
Return on prevention
2%
Other
16%
None of the above
0%
Ponemon Institute© Research Report
10%
20%
30%
40%
50%
60%
Page 5
The Information Technology (IT) professional and IT security functions have similar
influence over cybersecurity investments. As shown in Figure 9, those with the most influence
in the purchase of security technologies are business unit leaders (59 percent), CIOs (53 percent)
and the CISO (45 percent). However, the chief information officer (CIO) and chief information
security officer (30 percent and 20 percent, respectively) hold the purse strings and are the final
authority on how cybersecurity dollars should be spent.
Figure 9. The main influencers and final authority on cybersecurity investment
LOB or business unit leader
59%
19%
Chief information officer
53%
30%
Chief information security officer
45%
20%
19%
19%
Chief technology officer
Chief security officer
2%
Chief risk officer
2%
Chief compliance officer
1%
6%
6%
5%
4%
4%
Chief financial officer
3%
3%
CEO/COO
0%
10%
20%
30%
40%
50%
60%
70%
Who influences what security technologies to purchase?
Who is the final authority what security technologies to purchase?
Ponemon Institute© Research Report
Page 6
When Investments Go Wrong: The Problem of Shelfware
Security investments are driven by cost. As shown in Figure 10, 64 percent of respondents
say cost and 56 percent of respondents say performance and vendor support are the most
important factors when investing in security technologies. Surprisingly, important features such as
interoperability, proven risk reduction and lack of complexity are not considered to be as
influential (39 percent, 11 percent and 8 percent, respectively).
Figure 10. Most important factors when investing in security technologies
Four responses permitted
Cost
64%
Vendor support
56%
Performance
56%
45%
Vendor reputation
Efficiency
44%
40%
Time to deploy
39%
Interoperability
Scalability
28%
Proven risk reduction
11%
Redundancy
8%
Lack of complexity
8%
Other
1%
0%
10%
Ponemon Institute© Research Report
20%
30%
40%
50%
60%
70%
Page 7
Shelfware is every organization’s problem. Ninety percent of respondents say their
organization has invested in a security technology that was ultimately discontinued or scrapped
before or soon after deployment. As shown in Figure 11, on average, 31 percent of security
technologies purchased by organizations represented in this research over the past 24 months
were never fully deployed.
Figure 11. The percentage of security technologies purchased over the past 24 months but
never fully deployed Extrapolated value = 31 percent
40%
36%
35%
28%
30%
25%
21%
20%
15%
12%
10%
3%
5%
0%
Less than 10%
10% to 25%
26% to 50%
51% to 75%
76% to 100%
According to Figure 12, the technologies most often shelved are data loss prevention (55
percent), identity & access management (51 percent), SIEM and security intelligence (49
percent), Web application firewalls (46 percent) and intrusion & detection management (44
percent).
Figure 12. Security technologies most often “shelved” before or soon after deployment
More than one response permitted
Data loss prevention (DLP)
55%
Identity & access management
51%
SIEM and security intelligence
49%
Web application firewalls (WAF)
46%
Intrusion & detection management
44%
Configuration & log management
40%
Endpoint security solutions
39%
Access governance systems
36%
Mobile device management
35%
Automated policy generation
35%
0%
Ponemon Institute© Research Report
10%
20%
30%
40%
50%
60%
Page 8
According to Figure 13, technologies least often shelved are tokenization tools (10 percent),
perimeter or location surveillance (9 percent), encryption for data at rest (8 percent) and
traditional firewalls (5 percent), according to Figure 13.
Figure 13. Security technologies least often “shelved” before or soon after deployment
More than one response permitted
ID & credentialing system
14%
Database scanning and monitoring
14%
URL or content filtering
13%
Encryption for data in motion
12%
Anti-virus & anti-malware
12%
Virtual private network (VPN)
11%
Tokenization tools
10%
Perimeter or location surveillance
9%
Encryption for data at rest
8%
Firewalls (traditional)
5%
0%
Ponemon Institute© Research Report
2%
4%
6%
8%
10%
12%
14%
16%
Page 9
Complexity and difficulty in operating was the main reason the security technologies were
scrapped before or soon after deployment (77 percent of respondents). This is followed by a lack
of in-house expertise to deploy and operate the technology (55 percent of respondents). It is
interesting that the primary reasons for purchasing a particular technology are cost and
performance, when complexity of a system is most to blame for creating shelfware. Thus, level of
complexity should become a more important factor in the purchasing decision.
Figure 14. Why security technologies are scrapped before or soon after deployment
More than one response permitted
The technology was overly complex and too
difficult to operate
77%
Lack of in-house expertise to deploy and operate
the technology
55%
The technology was too expensive to maintain
41%
27%
Lack of vendor support and service
The technology did not meet the needs of our
cybersecurity strategy
16%
The technology diminished the performance of
our systems
12%
The technology diminished the productivity of our
employees
Other
10%
2%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
Ponemon Institute© Research Report
Page 10
How did the shelfware experience change cybersecurity investments? Sixty-five percent of
respondents say their organizations expanded the effort to evaluate vendors’ customer service
and support. Organizations are also insisting on contracts (SLAs) that hold vendors’ accountable
to their commitments (59 percent of respondents) and performance of additional risk
assessments (47 percent of respondents).
Figure 15. How did the shelfware experience change the organization’s approach to
purchasing cybersecurity technologies?
Expanded the effort to evaluate vendors’
customer service and support
65%
Insisted on contracts (SLAs) that hold vendors’
accountable to their commitments
59%
47%
Performed additional risk assessments
Conducted extensive testing of the technology
prior to purchase
33%
Implemented pilot or "beta" tests before investing
in the solution
25%
No change
8%
Other
2%
0%
10%
20%
30%
40%
50%
60%
70%
How Risk and Innovation Affects Security Investments
Are organizations innovative in their use of security technologies? In the context of this
research we define security innovation as “the use of enabling technologies and personnel in new
ways to create a more secure and efficient organization and improve alignment between security
initiatives and business goals”. Figure 16 reveals that almost half (49 percent) believe innovation
is essential or very important to achieving a strong security posture.
Figure 16. How important is innovation to achieving a strong security posture
40%
34%
35%
30%
26%
25%
20%
15%
15%
13%
12%
Not important
Irrelevant
10%
5%
0%
Essential
Very important
Ponemon Institute© Research Report
Important
Page 11
Only 32 percent of respondents say their organizations achieve a high level of innovation. Figure
17 shows how differently respondents define innovation. Seventy-five percent of respondents say
the reason they believe their organizations are innovative is because they use existing
technologies in ways that are more efficient and cost effective. Sixty-seven percent of
respondents say they use these technologies to create a more secure and efficient organizations.
Figure 17. What makes an organization innovative?
More than one response permitted
Uses existing technologies in ways that are more
efficient and cost effective
75%
Uses enabling technologies to create a more
secure and efficient organization
67%
Is not dependent upon any one vendor or
solution to achieve a strong security posture
60%
Uses technologies to improve alignment between
security initiatives and business goals
55%
Creative in using existing technologies to
address new threats
46%
Other
4%
0%
Ponemon Institute© Research Report
10% 20% 30% 40% 50% 60% 70% 80%
Page 12
Those who believe their organizations are not innovative, cite several reasons. More than half (56
percent) feel their organization is overly dependent upon vendors to make technology decisions,
53 percent of respondents believe their organizational culture inhibits innovation and 40 percent
say they do not have the right in-house expertise, as shown in Figure 18. Only 34 percent say
their level of investment in security innovation has changed over the past 24 months.
Figure 18. What keeps an organization from being innovative?
More than one response permitted
Overly dependent upon vendors to make
technology decisions
56%
53%
Organizational culture inhibits innovation
40%
Lack of in-house expertise
37%
Lack of resources
Belief that innovation increases security risk
28%
Overly dependent upon what industry peers are
doing
16%
2%
Other
0%
Ponemon Institute© Research Report
10%
20%
30%
40%
50%
60%
Page 13
Part 3. Conclusion
In this study, we explore how organizations are making decisions that will have a significant
impact on their ability to prevent and detect cyber threats. Our findings reveal how organizations
can avoid the problem of shelfware and invest instead in technologies that will pay dividends by
reducing cybersecurity risks.
§
Reduce dependency on ROI and TCO to make investment decisions. Instead, consider
such metrics as improvements in the efficiency of security operations, reduction in time to
detect security incidents and return on prevention.
§
Cost should not be the most important factor when investing in a security technology.
Companies find themselves investing heavily in technologies that are never deployed
because they are overly complex. Yet, only eight percent of respondents say their
organizations consider a lack of complexity in the investment decision. Shelfware would
become less pervasive a problem if companies prioritized level of complexity, interoperability
and proven risk reduction in their decision making
§
Innovation is important to a strong cybersecurity posture. The most innovative
organizations have found ways to use existing technologies that are more efficient and cost
effective and to create a more secure and efficient organization.
We believe these recommendations can be easily incorporated into an organization’s strategy to
improve the allocation of resources and achieve a strong cybersecurity posture.
Ponemon Institute© Research Report
Page 14
Part 4. Methods
The sampling frame is composed of 17,228 IT management and IT security practitioners who are
located in the United States. All respondents are familiar with and responsible for determining
their organizations’ investments in cybersecurity technologies. As shown in Table 1, 17,228
respondents completed the survey. Screening removed 73 surveys. The final sample was 618
surveys (or a 3.6 percent response rate). All survey responses were captured February 18 to
March 13, 2015.
Table 1. Sample response
Total sampling frame
Total returns
Rejected or screened surveys
Final sample
Freq
Pct%
17,228
691
73
618
100.0%
4.0%
0.4%
3.6%
We calculated a margin of error for all statistical survey questions that yielded a proportional or
percentage result. Most questions utilized the full sample size of n = 618 qualified respondents.
Assuming a confidence level at the 95 percent level, the margin of error for survey questions
ranted from ± 1.0 percent to ± 5.6 percent, with an overall average of ± 3.6.
Pie Chart 1 reports the current position or organizational level of the respondents. More than half
of respondents (57 percent) reported their current position as supervisory or above.
Pie Chart 1. Current position or organizational level
3%
4% 2% 1% 2%
16%
Senior Executive
Vice President
Director
Manager
Supervisor
34%
Technician
23%
Staff
Consultant
Contractor
15%
Ponemon Institute© Research Report
Page 15
Pie Chart 2 identifies the primary role the respondent has within the organization. Fifty-six percent
of respondents identified the chief information officer as the primary person they report to.
Another 21 percent indicated they report to the chief information security officer.
Pie Chart 2. Primary person you or your IT leader reports to within the organization
3%
3% 2% 2%
Chief Information Officer
6%
Chief Information Security Officer
Chief Risk Officer
7%
Data Center Management
56%
Other
Compliance Officer
21%
Chief Financial Officer
Chief Security Officer
Pie Chart 3 reports the primary industry classification of respondents’ organizations. This chart
identifies financial services (18 percent) as the largest segment, followed by federal government
(17 percent), healthcare (15 percent) and hi tech (11 percent).
Pie Chart 3. Primary industry focus
8%
4%
18%
8%
9%
17%
10%
11%
Financial services
Federal government
Healthcare
Hi Tech
Utilities
Energy, oil & gas
Pharmaceuticals
Telecom
All others
15%
Ponemon Institute© Research Report
Page 16
According to Pie Chart 4, more than half of the respondents (64 percent) are from organizations
with a global headcount of more than 1,000 employees.
Pie Chart 4. Worldwide headcount of the organization
8%
8%
9%
11%
< 100
100 to 500
501 to 1,000
1,001 to 5,000
19%
17%
5,001 to 25,000
25,001 to 75,000
> 75,000
28%
Table 2 reveals that in addition to the United States, 70 percent of respondents indicated their
organization has employees in Canada and sixty-nice percent responded Europe.
Table 2. Where are your employees located?
United States
Canada
Europe
Asia-Pacific
Latin America (including Mexico)
Middle East & Africa
Ponemon Institute© Research Report
Pct%
100%
70%
69%
49%
43%
39%
Page 17
Part 4. Caveats
There are inherent limitations to survey research that need to be carefully considered before
drawing inferences from findings. The following items are specific limitations that are germane to
most web-based surveys.
Non-response bias: The current findings are based on a sample of survey returns. We sent
surveys to a representative sample of individuals, resulting in a large number of usable returned
responses. Despite non-response tests, it is always possible that individuals who did not
participate are substantially different in terms of underlying beliefs from those who completed the
instrument.
Sampling frame bias: The accuracy is based on contact information and the degree to which the
list is representative of individuals who are IT or IT security practitioners in various organizations
in the United States. We also acknowledge that the results may be biased by external events
such as media coverage. We also acknowledge bias caused by compensating subjects to
complete this research within a specified time period.
Self-reported results: The quality of survey research is based on the integrity of confidential
responses received from subjects. While certain checks and balances can be incorporated into
the survey process, there is always the possibility that a subject did not provide accurate
responses.
Ponemon Institute© Research Report
Page 18
Appendix: Detailed Survey Results
The following tables provide the frequency or percentage frequency of responses to all survey
questions contained in this study. All survey responses were captured February 18 to March 13,
2015.
Survey response
Total sampling frame
Total returns
Rejected or screened surveys
Final sample
Freq
17228
691
73
618
Part 1. Screening questions
S1. How familiar are you with your organization’s investments in cybersecurity
technologies?
Very familiar
Familiar
Somewhat familiar
No knowledge (Stop)
Total
Pct%
33%
36%
31%
0%
100%
S2. Do you have any responsibility for determining investments in cyber security
technologies?
Yes, full responsibility
Yes, some responsibility
Yes, minimum responsibility
No responsibility (Stop)
Total
Pct%
25%
59%
16%
0%
100%
Part 2. Attributions
Q1a. My organization believes a strong cybersecurity posture is a competitive
advantage.
Strongly agree
Agree
Unsure
Disagree
Strongly disagree
Total
Pct%
12%
18%
23%
25%
22%
100%
Q1b. My organization’s senior leadership understands the cybersecurity risks it
faces.
Strongly agree
Agree
Unsure
Disagree
Strongly disagree
Total
Pct%
15%
19%
32%
22%
12%
100%
Q1c. My organization is effective in using cybersecurity technologies to reduce
the risk.
Strongly agree
Agree
Unsure
Disagree
Strongly disagree
Total
Pct%
25%
24%
25%
16%
10%
100%
Ponemon Institute© Research Report
Pct%
100.0%
4.0%
0.4%
3.6%
Page 19
Q1d. My organization is innovative in how it invests and deploys cybersecurity
technologies
Strongly agree
Agree
Unsure
Disagree
Strongly disagree
Total
Pct%
16%
31%
25%
18%
10%
100%
Q1e. My organization’s senior leadership does not view investment in
cybersecurity technologies as a strategic priority.
Strongly agree
Agree
Unsure
Disagree
Strongly disagree
Total
Pct%
26%
38%
16%
12%
8%
100%
Q1f. My organization had made investments in security technologies that did not
improve its cybersecurity posture.
Strongly agree
Agree
Unsure
Disagree
Strongly disagree
Total
Pct%
29%
29%
16%
16%
10%
100%
Part 3. The measurement of security technology investment
Q2a. How important is Return on Investment (ROI) in making a decision to
invest in a given security solution or technology?
Essential
Very important
Important
Not important
We do not use ROI to evaluate investments in security solutions or technologies
[skip to Q3a]
Total
Pct%
19%
25%
26%
15%
15%
100%
Q2b. How difficult is it to calculate a precise (accurate) ROI for a given security
solution or technology?
Very difficult
Difficult
Not difficult
Easy
Unsure
Total
Pct%
26%
44%
18%
9%
3%
100%
Q2c Were you able to determine a precise ROI for each investment decision
made?
Yes
No
Total
Pct%
41%
59%
100%
Ponemon Institute© Research Report
Page 20
Q2d. Over the past 24 months, were one or more proposed investments in a
particular security solution or technology rejected because of unfavorable ROIs?
Yes
No
Unsure
Total
Pct%
50%
45%
5%
100%
Q2e-1. Does your organization reconcile the historic (original) ROI with actual
ROI for investments in security solutions or technologies?
Yes, for most investments
Yes, for some investments
No
Unsure
Total
Pct%
18%
36%
42%
4%
100%
Q2e-2. If yes, what best describes your organization’s experience?
Actual ROI tends to be lower than the historic ROI
Actual ROI tends to be higher than the historic ROI
Actual and historic ROI tend to be very close (accurate)
Unsure
Total
Pct%
46%
15%
34%
5%
100%
Q2f. Approximately, what is the threshold ROI necessary to achieve a favorable
investment decision in a given security solution or technology within your
organization?
Less than 5%
5% to 10%
11% to 15%
16% to 20%
21% to 25%
26% to 30%
More than 30%
Total
Pct%
11%
21%
34%
18%
11%
4%
1%
100%
Q3a. How important is Total Cost of Ownership (TCO) in making a decision to
invest in a given security solution or technology?
Essential
Very important
Important
Not important
We do not use TCO to evaluate investments in security solutions or technologies
[skip to Q4]
Total
Q3b. How difficult is it to calculate a precise (accurate) TCO for a given security
solution or technology?
Very difficult
Difficult
Not difficult
Easy
Unsure
Total
Ponemon Institute© Research Report
Pct%
16%
19%
35%
15%
15%
100%
Pct%
23%
38%
25%
11%
3%
100%
Page 21
Q3c. Were you able to determine a precise TCO for each investment decision
made?
Yes
No
Total
Pct%
45%
55%
100%
Q3d. Over the past 24 months, were one or more proposed investments in a
particular security solution or technology rejected because of unfavorable
TCOs?
Yes
No
Unsure
Total
Pct%
56%
41%
3%
100%
Q3e-1. Does your organization reconcile the historic (original) TCO with actual
TCO for investments in security solutions or technologies?
Yes, for most investments
Yes, for some investments
No
Unsure
Total
Pct%
15%
33%
48%
4%
100%
Q3e-2. If yes, what best describes your organization’s experience?
Actual TCO tends to be lower than the historic TCO
Actual TCO tends to be higher than the historic TCO
Actual and historic TCO tend to be very close (accurate)
Unsure
Total
Pct%
15%
44%
36%
5%
100%
Q4. If ROI or TCO is not used by your organization, what else is used to
evaluate the economic viability of a particular security solution or technology?
Please select all that apply.
Improvement in the efficiency of security operations
Reduction in system downtime
Reduction in data breach costs
Reduction in time to contain security incidents
Reduction is non-compliance costs including fines, penalties and lawsuits
Reduction in time to detect security incidents
Return on prevention
Other (please specify)
None of the above
Pct%
56%
50%
47%
38%
23%
12%
6%
2%
16%
Ponemon Institute© Research Report
Page 22
Part 4. The security technology investment process
Q5. What are the most important factors when investing in security
technologies? Please select the top four.
Cost
Performance
Vendor support
Vendor reputation
Efficiency
Time to deploy
Interoperability
Scalability
Proved risk reduction
Lack of complexity
Redundancy
Other (please specify)
Total
Pct%
64%
56%
56%
45%
44%
40%
39%
28%
11%
8%
8%
1%
400%
Q6. In your organization, who is the final authority on what security technologies
to purchase?
Chief information officer
Chief information security officer
Chief technology officer
LOB or business unit leader
Chief financial officer
CEO/COO
Chief security officer
Chief risk officer
Chief compliance officer
Other (please specify)
Total
Pct%
30%
20%
19%
19%
4%
3%
2%
2%
1%
0%
100%
Q7. In your organization, who influences what security technologies to
purchase? Please select the top two choices.
LOB or business unit leader
Chief information officer
Chief information security officer
Chief technology officer
Chief security officer
Chief risk officer
Chief compliance officer
Chief financial officer
CEO/COO
Other (please specify)
Total
Pct%
59%
53%
45%
19%
6%
6%
5%
4%
3%
0%
200%
Q8. What best describes the investment process for security technologies?
A structured approach that is applied consistently throughout the organization
A structured approach that is not applied consistently throughout the
organization
An unstructured approach that applied consistently throughout the organization
An unstructured approach that is not applied consistently throughout the
organization
An “ad hoc” approach
Total
Ponemon Institute© Research Report
Pct%
31%
18%
4%
28%
19%
100%
Page 23
Q9. Which of the following security technologies has the highest ROI? Please
estimate the ROI for each technology using the percentage range in the
adjacent scale.
Identity & access management
SIEM and security intelligence
Encryption for data at rest
Encryption for data in motion
Anti-virus & anti-malware
Web application firewalls (WAF)
Tokenization tools
Endpoint security solutions
Intrusion & detection management
Mobile device management
URL or content filtering
Next generation firewalls (NGFW)
Data loss prevention (DLP)
Sandboxing or isolation tools
Virtual private network (VPN)
Configuration & log management
Big data analytics
Database scanning and monitoring
Device anti-theft solutions
Access governance systems
ID & credentialing system
Automated policy generation
Firewalls (traditional)
Perimeter or location surveillance
Average
ROI
31%
29%
25%
25%
25%
24%
23%
23%
21%
20%
18%
18%
17%
15%
14%
12%
12%
12%
11%
9%
8%
8%
7%
6%
17%
Q10. Has your organization ever invested in a security technology that was
ultimately discontinued or scrapped before or soon after deployment (i.e.
shelfware)?
Yes
No or unsure (skip to Q15)
Total
Pct%
90%
10%
100%
Q11. Approximately, what is the percentage of security technologies purchased
over the past 24 months but never fully deployed?
Less than 10%
10% to 25%
26% to 50%
51% to 75%
76% to 100%
Total
Pct%
21%
28%
36%
12%
3%
100%
Ponemon Institute© Research Report
Average
31%
Page 24
Q12. Which of the following security technologies were “shelved” before or soon
after deployment?
Data loss prevention (DLP)
Identity & access management
SIEM and security intelligence
Web application firewalls (WAF)
Intrusion & detection management
Configuration & log management
Endpoint security solutions
Access governance systems
Automated policy generation
Mobile device management
Big data analytics
Device anti-theft solutions
Sandboxing or isolation tools
Next generation firewalls (NGFW)
Database scanning and monitoring
ID & credentialing system
URL or content filtering
Anti-virus & anti-malware
Encryption for data in motion
Virtual private network (VPN)
Tokenization tools
Perimeter or location surveillance
Encryption for data at rest
Firewalls (traditional)
Average
Pct%
55%
51%
49%
46%
44%
40%
39%
36%
35%
35%
27%
27%
21%
16%
14%
14%
13%
12%
12%
11%
10%
9%
8%
5%
26%
Q13. What are the main reasons these security technologies were scrapped
before or soon after deployment?
The technology was overly complex and too difficult to operate
Lack of in-house expertise to deploy and operate the technology
The technology was too expensive to maintain
Lack of vendor support and service
The technology did not meet the needs of our cybersecurity strategy
The technology diminished the performance of our systems
The technology diminished the productivity of our employees
Other (please specify)
Total
Pct%
77%
55%
41%
27%
16%
12%
10%
2%
240%
Q14. How did your organization’s experience with shelfware change its
approach to purchasing security technologies?
Expanded the effort to evaluate vendors’ customer service and support
Performed additional risk assessments
Conducted extensive testing of the technology prior to purchase
Implemented pilot or "beta" tests before investing in the solution
Insisted on contracts (SLAs) that hold vendors’ accountable to their
commitments
No change
Other (please specify)
Total
Ponemon Institute© Research Report
Pct%
65%
47%
33%
25%
59%
8%
2%
239%
Page 25
Part 5. Risk and innovation in investing in security technologies
Q15. How much does security risk influence the purchase of specific security
technologies? Please use the following 10-point scale from 1 = no impact to 10 =
significant impact.
1 or 2
3 or 4
5 or 6
7 or 8
9 or 10
Total
Pct%
8%
9%
15%
21%
47%
100%
Q16. How much do compliance requirements influence the purchase of specific
security technologies? Please use the following 10-point scale from 1 = no
impact to 10 = significant impact.
1 or 2
3 or 4
5 or 6
7 or 8
9 or 10
Total
Pct%
11%
18%
26%
24%
21%
100%
Q17. How much do business objectives influence the purchase of specific
security technologies? Please use the following 10-point scale from 1 = no
impact to 10 = significant impact.
1 or 2
3 or 4
5 or 6
7 or 8
9 or 10
Total
Pct%
5%
6%
16%
23%
50%
100%
Q18. What is more important a technology that reduces risk or one that supports
the businesses processes?
Reducing risk is more important
Supporting businesses processes are more important
Both are equally important
Total
Pct%
33%
32%
35%
100%
Q19. Please rate your organization’s level of security innovation today using the
following 10-point scale.
1 or 2
3 or 4
5 or 6
7 or 8
9 or 10
Total
Pct%
12%
28%
28%
23%
9%
100%
Q20. [For risk score at or above 6] If your organization is innovative, why?
Uses existing technologies in ways that are more efficient and cost effective
Uses enabling technologies to create a more secure and efficient organization
Is not dependent upon any one vendor or solution to achieve a strong security
posture
Uses technologies to improve alignment between security initiatives and
business goals
Creative in using existing technologies to address new threats
Other (please specify)
Total
Ponemon Institute© Research Report
Pct%
75%
67%
60%
55%
46%
4%
307%
Page 26
Q21. [For risk score at or below 5] If your organization is not innovative, why?
Overly dependent upon vendors to make technology decisions
Organizational culture inhibits innovation
Lack of in-house expertise
Lack of resources
Belief that innovation increases security risk
Overly dependent upon what industry peers are doing
Other (please specify)
Total
Pct%
56%
53%
40%
37%
28%
16%
2%
232%
Q22. How has your organization’s level of investment in security innovation
changed over the past 24 months?
Significant increase
Increase
No change
Decrease
Significant decrease
Total
Pct%
8%
26%
44%
15%
7%
100%
Q23. In your opinion, what is the relative importance of security innovation to
achieving a strong security posture?
Essential
Very important
Important
Not important
Irrelevant
Total
Pct%
15%
34%
26%
13%
12%
100%
Part 6. Your role and organization
D1. What organizational level best describes your current position?
Senior Executive
Vice President
Director
Manager
Supervisor
Technician
Staff
Consultant
Contractor
Other (please specify)
Total
Pct%
1%
2%
16%
23%
15%
34%
3%
4%
2%
0%
100%
D2. Check the Primary Person you or your IT security leader reports to within
the organization.
Chief Information Officer
Chief Information Security Officer
Chief Risk Officer
Data Center Management
Compliance Officer
Chief Financial Officer
Chief Security Officer
CEO/Executive Committee
General Counsel
Other (please specify)
Human Resources VP
Total
Pct%
56%
21%
7%
6%
3%
2%
2%
1%
1%
1%
0%
100%
Ponemon Institute© Research Report
Page 27
D3. What industry best describes your organization’s industry focus (stratified
sample)?
Financial services
Federal government (various departments)
Healthcare
Hi Tech
Utilities
Pharmaceuticals
Energy, oil & gas
Telecom
All others
Total
Pct%
18%
17%
15%
11%
10%
8%
9%
8%
4%
100%
D4. Where are your employees located? Please check all that apply:
United States
Canada
Europe
Asia-Pacific
Latin America (including Mexico)
Middle East & Africa
Total
Pct%
100%
70%
69%
49%
43%
39%
370%
D5. What is the worldwide headcount of your organization?
< 100
100 to 500
501 to 1,000
1,001 to 5,000
5,001 to 25,000
25,001 to 75,000
> 75,000
Total
Pct%
8%
9%
19%
28%
17%
11%
8%
100%
Ponemon Institute
Advancing Responsible Information Management
Ponemon Institute is dedicated to independent research and education that advances responsible
information and privacy management practices within business and government. Our mission is to conduct
high quality, empirical studies on critical issues affecting the management and security of sensitive
information about people and organizations.
As a member of the Council of American Survey Research Organizations (CASRO), we uphold strict
data confidentiality, privacy and ethical research standards. We do not collect any personally identifiable
information from individuals (or company identifiable information in our business research). Furthermore, we
have strict quality standards to ensure that subjects are not asked extraneous, irrelevant or improper
questions.
Ponemon Institute© Research Report
Page 28