Who Says a SOC Can’t Have Windows?
Rethinking the Modern SOC
Introduction
VP Product & Strategy MSS
2004
Built a 8x5 SOC for Nokia
2006
Outsourced it
2008
Built 24x7 Consumer Operations (ecommerce, accounts)
2010
Ran 24x7 Cloud Operations and
Automation Development for those
systems
2012
Developed Repeatable Security
Programs for Blackstone’s portfolio
(and ran network)
2014
Helped plan Accuvant acquisition
and now run Managed Services
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
2
How I describe my job to my friends:
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
3
What I leave unsaid:
Most of my analysts
want to see the sun!
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
4
But what’s the problem?
… Everyone loves it on the SOC tour!
0%
Unemployment
in security
© Copyright 2014 EMC Corporation. All rights reserved.
Finding people is hard,
making them work
without windows (24x7)
is even harder
#RSAsummit
5
This is what
they really want
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
6
Yes, a little tongue in cheek
… but this presentation is about rethinking assumptions
There are many drivers of change today:
• Technology and especially infrastructure management is transforming
• The potential impact of todays threats to the enterprise (and its visibility) are skyrocketing
• Various threat actors (not just state actors) willingness to use cyber weapons is increasing
• Technical Defenses are also rapidly evolving
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
7
Rethinking assumptions
In SOC Operations
• The SOC is not a NOC
• Teach Analytics not (just) SLA
• Ecosystem of tool chains vs. single toolset
• Software Driven Everything
• Constant QA not Red Team Exercises
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
8
SOC is not a NOC
Rethinking Tier 1
NOC Process:
“Quote goes here. Increase or decrease text
to fit.”









Something goes Red
Ping down. System not responding.
Assigned to Tier 1
Try something
Is it green?
Reboot
Is it green?
Escalate
Keep working until things become green
Generally it is clear something is wrong
The problem is how to fix it?
It is usually clear once it is fixed
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
9
SOC is not a NOC
Rethinking Tier 1
SOC Process:
“Quote goes here. Increase or decrease text
to fit.”
There is a lot of Yellow “stuff” and some Red
Assigned to Tier 1
Is something actually wrong? Are we vulnerable to this?
Oh, that machine is johnny-desktop01 let’s just re-image it
Great, no more alerts. I guess its all good. Close case
….
Few days later… hmm, how come Johnny keeps logging into the
domain controller? Didn’t we re-image his machine?
 OR: We re-imaged Johnny’s computer because of commodity
attack that didn’t even get actually compromised? Now he is
screaming at me for taking his computer away for 2 days!







It is completely unclear if something is wrong
“Fixing” it is easy
But it is never clear if the problem is really fixed
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
10
SOC Analyst as Frontline Medic
Not the Call Center Tier 1
 There is no green.
 Get to classification “Recon, Exploit, Exfiltrate”
 Depending on severity and confidence level
pull in more experienced people on a call
 Pool teams together for mix of skills and
experience as a team
 “Tier 1” has to be able to make informed
decisions and investigate, not just run through
a script
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
11
Analytics not (just) SLA
• Like the NOC case, we have looked at SLA as
a way to measure the quality of a SOC
• Time to Respond/Time to Resolve
• However not all cases are the same
• The countdown timer is efficient for maximizing
profit but doesn’t necessarily lead to best quality
• Need to drive the mentality of handling the
problem effectively not just quickly
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
12
Analytics not (just) SLA
 Measure using trends
–
–
–
–
–
What were the characteristics of the
longest cases to investigate?
How could we reduce the time?
On average how many tools used to reach
an FP or TP conclusions?
Can we consolidate?
What is normal?
 These are the same types of questions
analysts need to ask when investigating
events
–
–
–
Understanding what “normal” is
Looking for patterns in the data that
indicate potential issues or a false positive
Where have we seen similar cases?
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
13
Toolset: A Christmas gift favorite
Valhalla:
One tightly coupled solution suite to solve
Your CM, Monitoring, security, issues
One stop shop for security operations
© Copyright 2014 EMC Corporation. All rights reserved.
#RSAsummit
14
Reality is more like this
• Multiple tools to fit job
• Some tools you like
better than others
• Miscellaneous tools used
only once but needed
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
15
Ecosystems are the new SOA
Use them to your advantage
•
•
•
•
SOP Documentation
Automated
Versioning & Search
On call management
Automated
notification
•
•
Security Events
Situational
Awareness
Monitoring Health
QA Checklist
•
•
•
•
•
•
•
Event Analytics
Pattern detection
© Copyright 2014 EMC Corporation. All rights reserved.
Time tracking
Reporting
Email Automation
#RSAsummit
16
Engineers are grinning
But managers are shaking their heads:
WARNING: IVORY TOWER THINKING
• Of course you still need to balance best of breed with manageability
• But don’t get locked in
• Don’t be afraid to build the skills
Abort
Fail
Retry
AND …
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
17
“Magic” Piece of Advice
One of the best tools you have to retain your people
Is to support contributing back to the community
Intrinsic Motivators outweigh extrinsic factors once
people have their basic needs met
•
•
•
•
Wikipedia
Firefox
Linux
Toms Shoes
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
18
Software Driven Everything
Everything has an API now
 Which means everyone is a developer
(or has to become one)
– Everything is designed to be integrated
– A little python can build a toolkit that used to take months to put together
 More importantly, the way IT is built is changing
– Cloud/API driven compute
– Software Define Networking
– System Configuration Automation
Understand how these capabilities work
will be just as important
as understanding how to read a pcap file is today
© Copyright 2014 EMC Corporation. All rights reserved.
#RSAsummit
19
Software Driven Everything
Ways of Working are Just as Important as “Coding”
 Shared ops experience: sleeping in the data center
– It’s a shared experience that bonds a team’s experience
– And drives critical lessons like why Change
management is important
•
Developer Version: Don’t break the build!
•
Understanding how an SDLC is run will be critical
– Source Control
– REST and Callbacks
– QA Automation and build chains
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
20
Constant QA not Red Team Exercises
As infrastructure gets defined as software…
Security becomes part of QA
 Begin to think of your tooling like an SDLC
 When deploying new capabilities “Continuously Test”
– Unit Tests (is my firewall blocking?)
– End to End (is this attack detected and stopped?)
– Regression test (am I still checking for old exploits?)
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
21
What doesn’t change
• Core Security Operations Processes are still critical
– Hire the right people with the right attitude
– Define and follow critical process such as incident response, change
management, post mortems
– Practice your ability to execute these processes
– Continuously train
This presentation is to help you
get more out of what you do
foster an environment to retain the best people
adapt to the ever changing landscape
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
22
That’s still not all easy … can I outsource it?
• 24x7 is very hard to do (8x5 is bad enough)
–
–
–
–
It seems inevitable to need outsourced help
Just recruiting, training and retaining people for 8x5 is not easy
You still need some strong internal incident responders
There is no way to get all the tribal knowledge into an
outsourcer
• The Outsourced SOC should be a force multiplier
– Yes it can save save 3-4 FTE Cost,
– they should make your 1-2 FTE 10x more capable
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
23
What about SOC in the Cloud?
• Its inevitable
• What’s the difference though between Provider, Cloud, SaaS?
–
–
–
–
Not much on the surface
SIEM in Public Cloud as SaaS is more important (scaling is what matters)
Its happening already (ex: SumoLogic)
The value will be in reducing cost, improving quality , crowdsourcing & ecosystem
• Expect new on demand models
– QA: uTest
– Proxy: zScaler
– Tools: Cloudshark
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
24
How does Ecosystem work with Outsourcing?
• Not very well
• I’m a believer in eco-system  I’m a believer in co-managed
–
–
–
–
Customer has full access to the product they bought
Provider utilizes that product to act on their behalf
Keeps it healthy, tuned to be a force multiple for the Incident Handler
Its not easy to do
• Cloud based SIEM which you have full access to it that a 3rd Party is also
monitoring and providing crowdsourced data is the future
#RSAsummit
© Copyright 2014 EMC Corporation. All rights reserved.
25
Summary and Key Takeaways
• The SOC is not a NOC
– Tier 1 as your field medic
– There is no green
• Teach Analytics not (just) SLA
– Trends and patterns will be critical for security analysts
– Put emphasis on it even in daily work
• Ecosystem of tool chains vs. single toolset
– Take advantage of ecosystem of tools
– Focus your effort where you think it helps the most
• Software Driven Everything
– Look for software skills sets
– Training on software development methodologies
• Constant QA not scheduled Red Team exercises
– Regularly test your defenses
– Consider how software testing processes can be integrated
© Copyright 2014 EMC Corporation. All rights reserved.
#RSAsummit
26
THANK YOU