TAP User Guide
FireEye, Inc.
Next Generation Threat Protection
1440 McCarthy Blvd., Milpitas, CA 95035
www.FireEye.com
© 2014 FireEye, Inc. All rights reserved.
FireEye is a registered trademark of FireEye, Inc. All other brands, products, or service
names are or may be trademarks or service marks of their respective owners.
Document version: v1.0A
Contents
Contents
i
TAP Overview
iii
Events
iii
Intelligence
iv
TAP Architecture
v
Dashboard
vi
Search
vii
Run a Search
vii
Quick Mode
vii
Date Range
vii
History
viii
Favorites
viii
Lists
viii
Create, Update, Export Lists
viii
Import Lists
ix
Search Results
ix
Pivoting
ix
MQL Syntax
1
Search Clauses
1
Quotes
1
Numbers
2
Date and Time
2
Set Notation
3
Boolean Operators
3
Comparison Operators
4
Parenthetical Expressions
4
Prefix Search
4
Subsearch and Variable Expansion
5
Subsearch
5
Subearch with Variable Expansion
5
Variable Expansion
6
Regular Expressions
6
FireEye, Inc.
i
TAP User Guide
Filters
7
Directives
8
Transforms
8
Groupby
8
Histogram
9
Taxonomy
9
Meta-Classes and Classes
10
Fields
10
Aliases
11
IP, SRCIP, DSTIP
11
Host
12
ID
13
MAC
14
Port
14
Interface
15
Hash
15
Intel Hit Searches
15
Rules
17
Rule Packs
17
View Rules
18
Enable and Disable Rules
18
Create User-Defined Rules
18
Update User-Defined Rules
19
Delete Rules
19
Import and Export Rules
19
Alerts
20
Suppress Alerts
20
Add Alerts to Incidents
20
Incidents
21
Create New Incident
21
Add Events to Existing Incident
22
Assign Incident and Investigate
22
View Alert Details
22
FireEye, Inc.
ii
TAP User Guide
TAP Overview
The FireEyeThreat Analytics Platform (TAP) is a security incident detection and resolution tracking platform that identifies cyber threats and improves response by layering
enterprise-generated event data with real-time threat intelligence from FireEye.
TAP Overview
TAP is a cloud-based application that:
l
l
l
l
l
l
Collects and indexes database, security, network, and endpoint events from your
environment
Compares indicators in your events in real-time against FireEye intelligence and
generates alerts on hits
Applies both FireEye-defined rules and rules that you define to event data to generate alerts
Provides an incident workflow for tracking both events associated with alerts and
any events that you deem suspicious from investigation to remediation
Makes events available for efficient searching and pivoting
Provides visualizations of trending activity
Events
An event is any observable occurrence. Logging is the process of recording events to
provide an audit trail that can be used to understand the activity of a system. In the context of TAP, event refers to a specific log entry.
FireEye, Inc.
iii
TAP User Guide
Each event found within log data in TAP is assigned a uniqueID as it enters the TAP
application in the Comm Broker Sender within your environment.
Log data from each device is run through a parser if one is available. Parsers separate
the data in events into fields and label though fields according to the taxonomy. The taxonomy names and defines types of data that appear in events.
After being parsed, each event then has two parts: parsed fields and the raw message.
Parsed fields have the following advantages:
l
l
l
Can be searched using MQL syntax
Have pivoting capability when the event appears in search results
Provide more accurate data for matching against intelligence indicators
If an event is not parsed, the raw message is still indexed and can be searched as a
string. TAP still attempts to match the raw message against intelligence indicators but
more false positive hits may result.
When an event matches intelligence, TAP generates a synthetic event for that intel hit.
Having a synthetic event allows you to search more effectively for intel hits.
Intelligence
TAP applies FireEye Intelligence to events once in real-time as they are received. TAP
checks parsed fields in events and unparsed, raw messages for fully qualified domains
(FQDNs) and IPs that we believe are indicators of a compromise. When a match is
found, TAP generates an intel hit (along with a synthetic event) and an alert. Intel hits
based on matches of just raw data in an event are more likely to be false positive hits
than matches on parsed fields. Intelligence indicators are updated hourly.
FireEye intelligence is gained from our extensive incident response work as well as
research by our experts and includes these two types:
l
l
Commodity. Commodity intel is generated from analysis of over 1 million malware
samples per day. By denoting these samples and analyzing them, we generate
10,000 indicators per day. Our expert intel team reviews matches from customer
environments to these indicators to help ensure that only valid intel hits are sent to
your TAP instance.
Curated. Curated intel is generated from FireEye research; we closely track threat
families and Advanced Persistent Threats (APT) groups. From this research, we
compile malicious indicators and TAP looks for those indicators in your events.
This type of intel generally generates fewer hits but the hits that occur indicate high
risk compromises. When TAP is configured for your environment, the TAP operations team notes your external IPs. If FireEye finds any of these IPs sending data
to a known sinkhole (i.e., a known evil command and control server), TAP alerts
you.
FireEye, Inc.
iv
TAP User Guide
TAP Architecture
Your TAP instance resides in two environments: your environment and a Virtual Private
Cloud (VPC) within Amazon World Servies (AWS). Within your environment is one or
more Communication Broker Senders that send log data to a Communications Broker
Receiver within TAP in the VPC. The Comm Broker Receiver and all other
TAP components within the VPC are managed by the TAP Operations Team.
TAP High-Level Architecture
The data flow is as follows:
l
l
l
l
The Comm Broker sender listens receives log data in your environment and sends
it to the Comm Broker Receiver in the VPC. For security purposes, all data in
transit, including all metadata, is encrypted with Twofish with a 256-bit key. When
data is transmitted over the WAN to the Communication Broker Receiver, it is
double-encrypted with two layers of Twofish and 512 bits of key total. The Communication Broker Sender/Receiver combination never stores any customer data
in clear text.
Log data is parsed according to the TAP taxonomy and then indexed to make it
available for fast searches and pivoting. Log data that cannot be parsed is still
indexed as raw messages.
Both FireEye-defined and customer defined rules are applied to the events and
alerts generated if applicable.
FireEye intelligence is also applied to all events in real-time and alerts generated
for any hits.
FireEye, Inc.
v
TAP User Guide
Dashboard
The TAP application is designed to be viewed in Chrome.
The default page for the TAP application is the Dashboard page.
The Dashboard page is designed to provide a broad overview of the current status of
events, alerts, and incidents in TAP.
If there is a current alert, a banner appears at the top of the Dashboard page to provide
immediate pertinent information related to that alert and options for taking action with the
alert.
Depending on the data available in TAP, the Dashboard page shows:
l
l
l
Active alerts and a graphic showing the number of alerts per source (FireEyedefined rules, user-defined rules, or intel hits)
Open incidents, including the total number as well as the average time to close
Metrics about events such as the total number of events in the TAP index and the
daily high and low number of events.
FireEye, Inc.
vi
TAP User Guide
Search
TAP Search capability allows you to search events as both a starting point to find a
potential compromise and to locate specific events associated with alerts. TAP can
search billions of events in seconds.
Events are available for searching for 4, 8, or 12 months depending on how your TAP
instance was configured when it was deployed.
The TAP search depends on the event as follows:
l
l
For raw events, TAP is able to search for matching strings.
For parsed events that are normalized according to the TAP taxonomy, TAP is
able to search for common data all across all events from a variety of log sources.
Run a Search
Search capability is provided in TAP by entering a query written using Mandiant Query
Language (MQL) and selecting a date range option and selecting Quick Mode if desired.
TAP uses MQL as the syntax for constructing queries to search events. The complete
MQL syntax is supported in the Search box the top of every TAP page.
Once you have run one query, you use the pivot functionality to refine that query or create a new query.
Quick Mode
Quick Mode provides fast but partial search results when you run a query. To use Quick
Mode, check the Quick Mode option on the Search bar.
Quick Mode limits the number of results that TAP returns. The results returned are based
on events found but do not represent all the events that match the query (i.e., the count is
not accurate). Using Quick Mode, you can test whether a query will return results and if
those results are what you expected. This is particularly useful when building complex
queries.
Date Range
You can limit the query to a specific time period, by selecting one of the following
options:
l
l
l
l
l
Past hour
Past 12 hours
Past 24 hours
Past 7 days
Last 30 days
FireEye, Inc.
vii
TAP User Guide
l
l
Past 90 days
Custom range with a start and end date
To select a date, click the calendar icon in the Search bar.
The date that TAP uses to evaluate the query (and as the default order in search results)
is the date and time that the event arrived into TAP. When the Comm Broker Sender
receives log data from your environment, it adds the current time as a timestamp to the
event. This timestamp, includes the date and time down to the millisecond.
History
TAP saves all the search queries run. This History is available on the Search page.
To run a previous query, select it from the History menu or the Latest History box on the
Search page.
To clear the history, click Clear in the Latest History box on the Search page or select
Clear History from the History menu on the Search page.
Favorites
A Favorite is a search query that you would like to save to run again. To add a query to
Favorites, run the query in the Search bar then click Favorite.
To run a Favorite query, select it from the Favorites menu or the Top Favorites box on
the Search page.
To delete a Favorite from the list, click the X icon next to the query on the Favorites
menu or select Edit in the Top Favorites box.
Lists
Lists allow you to conduct searches for multiple items at one time (e.g., shared indicators, executive machine IPs).
If you have many fields that you would like to include in one or more queries, you can create a list and use the list name in the query. The list name serves as a variable in the
MQL syntax.
Create, Update, Export Lists
To create a new List, click Create New List on the Lists menu on the Search page. To
edit an existing List, select the list name on the Lists menu on the Search page.
On the Lists page, enter or update the following:
l
l
l
List name
List description
IPs and domains on separate lines in the Search for box.
The limit is 100K per list.
FireEye, Inc.
viii
TAP User Guide
To export the list, click Export.
Import Lists
If you have existing lists of domain and IPs stored in .json format, you can upload those
lists for use in TAP. To import list, select Import List from the Lists menu on the Search
page. On the Import List window, select the file to import. Only .json is supported.
Search Results
The results for the query appear on the Search page as a list of events. Each event
includes both the raw event data and the parsed fields, if applicable. Parsed fields have
pivoting options.
You can also visualize data results on a timeline. Click Timeline at the top of the search
results.
If you think results are missing, be sure that the search query was not run with Quick
Mode selected. After selecting events, you can add them to an incident.
After finding results, TAP has the following options to help you use the results more efficiently:
l
l
l
l
l
l
Highlight. To see the fields that matched the search query, click Highlight.
Geo. TAP uses data from the srcipv4 and dstipv4 fields in events to determine geographical IP. To display the geographical information for an event, such as the destination country, destination IP, or destination domain, click Geo.
Meta. To see the meta-classes in the events, click Meta.
Sort. By default, the events in the results are sorted by the newest first. You can
reverse the order by selecting Oldest for the Sort option. The date that TAP uses
is the date and time that the event arrived into TAP (which is also used in the
query).
Show and Select. By default, the search results list shows all the results. To view
only specific events, select those events and choose Selected for the Show
option. The Visible and None options for Select work also change the events displayed.
View. You can opt to see all the results which may include both raw events and
parsed events or you can opt to show just one or the other. Select an option for
View.
Pivoting
Pivoting through data is how you add context to an event to determine what to do next.
After completing a search, you may find data of interest in a parsed field in an event in
the results and want to refine your search or run a new search for the same data in other
events.
FireEye, Inc.
ix
TAP User Guide
When events appear in the search results, the parsed fields provide the following pivoting and drill-down options by clicking the down arrow next to the field in the event in the
results on the Search page:
l
l
l
l
l
New search. To search for the same field and data in other events, select New
search. A new query appears in the Search bar, which you can either run as is or
modify.
Add to current search. To add another field and data to the current query in the
Search bar, click Add to current search.
Exclude from search. To add a field and data to the current query as a “not” statement, click Exclude from search.
Groupby field. To use a field and its data in a groupby clause in the current query,
click Groupby field.
Copy to clipboard. To copy the field and its data to the clipboard for use in
another application or for your notes, click Copy to clipboard.
FireEye, Inc.
x
TAP User Guide
MQL Syntax
TAP employs Mandiant Query Language (MQL) for constructing queries. MQL is a data
analysis language that is used to retrieve events for further analysis in TAP.
MQL supports three types of clauses in the search query:
l
l
l
Search, which includes:
l Quotes
l Numbers
l Date and time
l Set notation
l Boolean operators
l Comparison operators
l Parenthetical expressions
l Prefix search
l Regular expressions
l Variable expansion
l Subsearch
l Filters
Directives
Transforms, which include:
l Groupby
l Histograms
MQL supports an unlimited nesting of queries; however performance may slow depending on result set.
The fields available to include queries are based on the taxonomy. Those fields include
classes and also alias options.
All fields and values are lowercase.
Search Clauses
A search clause specifies the data to be located based on exact matches, comparisons,
ranges, and expressions.
Quotes
Use “ “(double quotes) or ‘ ‘ (single quotes) to search for the following:
l
l
A space or an exact string that includes a space, use
Keywords such as “and” or “or”, use
FireEye, Inc.
1
TAP User Guide
To search for quotes, use the escape. For double quotes escape using the following:
l
\“
For single quote escape using the following:
l
\’
Numbers
MQL syntax supports the following for numbers:
l
l
l
l
Negative “-“ and positive “+” signs
Exponent as ‘e’ or ‘E’
Fraction as ‘.’ digit
Fraction exponent as ‘fracexp’
Date and Time
To limit events return by the search to a specific range, MQL supports the keywords
“start” and “end”. Time specified in the MQL syntax takes precedence over time specified
in the Search bar in TAP.
MQL supports dates as follows:
l
l
l
l
l
Calendar date with the date: “yyyymmdd”
Calendar date with just the year and monthy: “yyyymm”
Calendar date with the week: “yyyyww” or “yyyy-ww”
Calendar date with the week and day: “yyyywwd”
Ordinal date: “yyyy ordinalday”
Where:
l
l
l
l
l
l
yyyy is the year in 4-digit number format
mm is the month in 2-digit number format
dd is the date in 2-digit number format
ww is the week in 2-digit number format ranging between 1 and 52 starting the first
week of January
d is the weekday in 1-digit number format ranging between 1 and 7 starting with
Monday
ordinalday is the day in 3-digit number format ranging between 1 and 366 starting
on January 1
MQL support time as follows:
l
l
Hours, minutes, and seconds: “hhmmss”
Hours and minutes: “hhmm” or “hh:mm”
Where:
l
l
hh is the hour in 2-digit format
mm is the minute in 2-digit format
FireEye, Inc.
2
TAP User Guide
l
ss is the second in 2-digit format
MQL syntax also supports relative time such as:
l
l
l
l
yesterday
today
last with hour, day, week, month, or year
X hours, days, weeks, months, or years ago where X is a number
The time that MQL uses when evaluating queries with “start” and “end” is the time that
the event arrived into TAP.
When the Comm Broker Sender receives log data from your environment, it adds the current time as a timestamp to the event. This timestamp, includes the date and time down
to the millisecond, is the time field used to evaluate the start and end keywords and is
also the default order that search results are displayed on the Search page.
To evaluate the query using time values located in another field, you must specify that
field.
Set Notation
MQL syntax supports lists of comma separated values by placing the list between
square brackets “[]”.
For example:
scrip:[192.68.1.1,192.68.1.2]
Boolean Operators
Boolean operators include “and” and “or” and “not”.
By default, a space between query terms is the equivalent is considered an implicit “and”
and the search will be inclusive of all terms specified.
The order of precedence for “and” and “or” is:
l
l
Explicit “and” (i.e., and is entered) is higher than “or”
Implicit “and” (i.e., no “and” entered by just a space between terms) is lower than
“or”
The valid symbols for “and” are:
l
l
and
&&
The valid symbols for “or” are:
l
l
or
|| (double bars)
For example:
http and tcp or ftp : (http and tcp) or ftp
http tcp or ftp : http and (tcp or ftp)
FireEye, Inc.
3
TAP User Guide
The operator “not” binds to what is immediately after it. For example:
not ftp
The valid symbols for “not” are
l
l
l
NOT
Not
! (exclamation point) (with no space afterwards)
Comparison Operators
To find data in a specific field, use the equal operator which is one of the following:
l
l
= (equal sign)
: (colon)
Neither have spaces around them. The value specified after the operator must be valid
for the field type.
For example:
scrip:192.168.1.1
scrip=192.168.1.1
In addition to equals, the following other operator values supported are:
l
l
l
l
>
<
=<
>=
There are no spaces around the operators.
Operator values are the tightest binding.
For example:
recvdpackets>20
! recvdpackets<=20
Parenthetical Expressions
Parenthesis can be used to group terms for precedence. Parenthesis are also used to
designate subsearches.
For example:
tcp and (http or ftp)
not (http and tcp or not ftp)
Prefix Search
Parenthesis can be used to group terms for precedence. Parenthesis are also used to
designate subsearches.
FireEye, Inc.
4
TAP User Guide
For example:
tcp and (http or ftp)
not (http and tcp or not ftp)
Subsearch and Variable Expansion
Subsearch allows you to specify a query to run then use the results from that query in
another query. Subsearch can be run by itself or with variable expansion. Variable
expansion can also be run independently of subsearch.
Subsearch
Subsearch is supported using a parenthetical expression.
For example:
srcip:(rawmsg:Trojan) and hostname:important_machine
In this example query, the search engine will first locate “Trojan” in all raw messages
and return all the source IPs of those events. It will then search for events with one of
those source IPs and the hostname “important_machine” and return those events.
In this example if the subsearch (rawmsg:Trojan) would find 2 unique srcip addresses
(172.15.1.10 & 175.15.1.20) then the full searches performed will be:
srcip:172.15.1.10 and hostname:important_machine OR
srcip:175.15.1.20 and hostname:important_machine
You can also compare different variables to each other with the “in” operator to specify
the field in the subsearch to be used in the comparison.
For example:
IOC dsthost:(srchost in fireeye eventlog:/.*back.*/)
For this query, search first resolves the subsearch expression and returns the unique
srchosts which it then uses to search the dsthost as search terms in the main query. This
can be useful when the direction of the event is different (as with some of the events from
other FireEye products) or when trying to look for callback activity from the original infection source.
Subearch with Variable Expansion
You can also use subsearch in combination with variable expansion. Variable expansion allows you to specify a list of pre-populated values and then use the results from
that query in another query.
For example:
IOC (dsthost:$exec_pc)
FireEye, Inc.
5
TAP User Guide
For this query, search first resolves the subsearch expression by doing a search for all
the dsthosts that match the list $exec_pc and returns the result to add them as search
terms in the main query.
In this example if the exec_pc list contains the hostnames execcto and execcfo, the following search would be executed:
IOC dsthost:execcto OR IOC dsthost:execcfo
Variable Expansion
You can also use variable expansion outside of subsearches. The syntax is:
field_name:$keyword
For example:
dsthost:$exec_pc
In this example you would be executing the following search:
dsthost:execcto OR dsthost:execcfo
Which is the same as:
dsthost:[execcto,execcfo]
Regular Expressions
The MQL syntax supports regular expressions.
MQL regular expression patterns are always anchored (i.e., you do not need to specifically indicate whether the regex pattern starts at the beginning or end of the string).
The pattern provided must match the entire string. The exception is the use of an asterisk
(*) in a prefix search.
For example, to match a user-agent field of “Mozilla/5.0”, the regular expression must be
“User-agent=.*5\.0.*”; it will not match on “User-agent=5.0” or “User-agent=5\.0”.
Any characters may be used, but certain characters are reserved and must be escaped.
The reserved characters are:
. ? + * | { } [ ] ( ) " \
The character * is also used for prefix searches.
Any reserved character (including a backslash) can be escaped with a backslash (for
example, "\*").
Any characters (except double quotes) are interpreted literally when surrounded by
double quotes.
A period “.” can be used to represent any character.
The plus sign "+" can be used to repeat the preceding shortest pattern once or more
times.
The asterisk "*" can be used to match the preceding shortest pattern zero-or-more times.
FireEye, Inc.
6
TAP User Guide
The question mark "?" makes the preceding shortest pattern optional. It matches zero or
one times.
Curly brackets "{}" can be used to specify a minimum and (optionally) a maximum number of times the preceding shortest pattern can repeat. The allowed forms are:
l
l
l
{5} # repeat exactly 5 times
{2,5} # repeat at least twice and at most 5 times
{2,} # repeat at least twice
Parentheses "()" can be used to form sub-patterns.
The pipe symbol "|" acts as an OR operator. The match will succeed if the pattern on
either the left-hand side OR the right-hand side matches. The alternation applies to the
longest pattern, not the shortest.
Ranges of potential characters may be represented as character classes by enclosing
them in square brackets "[]". A leading ^ negates the character class. The allowed forms
are:
l
l
l
l
l
l
l
l
[abc] # 'a' or 'b' or 'c'
[a-c] # 'a' or 'b' or 'c'
[-abc] # '-' or 'a' or 'b' or 'c'
[abc\-] # '-' or 'a' or 'b' or 'c'
[^a-c] # any character except 'a' or 'b' or 'c'
[^a-c] # any character except 'a' or 'b' or 'c'
[-abc] # '-' or 'a' or 'b' or 'c'
[abc\-] # '-' or 'a' or 'b' or 'c'
Note that the dash "-" indicates a range of characters, unless it is the first character or if it
is escaped with a backslash.
Filters
MQL supports filtering on data that exists in a given field and data that is missing from a
given field.
Including the keyword “has” in a search query followed by a field name filters out events
which do not have that field.
For example:
has:dstcity
Including the keyword “missing” in a search query followed by a field name filters out
events which have that field.
For example:
missing:dstcity
FireEye, Inc.
7
TAP User Guide
Directives
Directives are modifiers that instruct the search engine how to query. Directives include:
l
l
l
l
l
l
Limit. Limit indicates the number of results to return. A limit of zero (which is the
default) means that all results are returned. Note: The default value for limit is ten
when in quick mode.
Offset. Offset indicates how far into a result set to go before returning results
Cutoff. Cutoff tells the search engine to stop searching after finding the indicated
number of records. The results will not be as accurate but will be returned faster.
Start. Start represents the earliest timestamp to return.
End. End represents the latest timestamp to return.
Sort. Sort indicates the field on which to order the events returned. Multiple fields
can be indicated with a comma (,). Sort has two options to indicate direction of the
sort: ascending (designated by “[asc]”) or descending (designated by “[desc]”)
For example:
start:yesterday end:today limit:10 offset:10 sort:timestamp
[desc],bytes_in[asc] cutoff:10
Transforms
A transform allows you to pass the results of a query through a function, which will add,
remove, or modify your search results.
Transforms are separated in a query by “|”. Queries can include multiple transforms.
Transforms include:
l
l
Groupby
Histogram
Groupby
Groupby returns the unique values for a given field and the count of the distinct values.
The events returned by the search when you use a groupby are representative events
which are grouped. The syntax for groupby is:
<search> | groupby field_name [integer]
Field_name is any string comprised of letters and numbers and special characters.
A positive integer searches for the most frequent occurrences. A negative integer
searches for the least frequent occurrences. You can specify a second integer to designate a minimum count threshold. The integer argument is optional.
For example:
rawmsg:dyn* | groupby srcipv4 10 1000
Returns the top ten srcipv4s which have at least 1000 occurrences.
FireEye, Inc.
8
TAP User Guide
tcp | groupby dstport 10 1000
Returns the 10 most frequent ports which also have counts greater than 1000
rawmsg:dyn* | groupby srcipv4 -10
Returns the top 10 scripv4s with the least number number of occurrences
tcp | groupby dstport 110 1000
Returns the 10 least frequent ports which also have counts greater than 1000
Histogram
To return the search results as a histogram (i.e., a graphical representation of the distribution of data), use the histogram transform.
There are two types of histograms:
l
l
Date histogram
Field value histogram
Date histogram has the following syntax: <search> | histogram eventtime interval]
Where interval is one of the following:
l
l
l
l
l
l
l
l
l
l
l
‘1d’
‘1h’
‘1w’
‘day’
‘hour’
‘minute’
‘month’
‘quarter’
‘week’
‘year’
‘datetime’
Field histogram has the following syntax: <search> | histogram keyword integer
For example:
tcp | groupby class | histogram bytes_in 500
Taxonomy
Because log data within an environment varies widely, TAP imposes a high-level standardized view of events and normalizes the data. This allows you to craft queries and to
interact with the data in a more predictable, standardized way, such as searching and
creating rules.
FireEye, Inc.
9
TAP User Guide
Log data from each device is run through a parser if one is available. Parsers separate
the data in events into fields and label though fields according to the taxonomy. The taxonomy names and defines types of data that appear in events.
The TAP taxonomy serves as the “dictionary” for the TAP parsers that normalize the
event data. TAP parses event data so that it is more easily compared and available to
use as a pivot in searches.
Meta-Classes and Classes
Classes in the TAP taxonomy represent types of events or log sources. For example, a
synthetic event created by an intel hit has the class “Intel_hit”.
A meta-class is simply a generic class name that refers to events from one or more
actual classes in TAP. Having meta-classes, allows you to refer to specific types of
events without knowing in which class they actually exist.
For example, if you have both Bluecoat HTTP proxies and a Palo Alto firewall with
HTTP inspection enabled, your event data may have classes called “bluecoat” and
“palo_alto_http”. Both contain similar data (i.e., logs of their own users browsing websites on the Internet). You can search either or both of these classes directly, but then
you must remember to use a query like “class=bluecoat OR class=palo_alto_http”, which
is more complicated and therefore they are more likely to make mistakes.
Instead you can use the meta-class called http_proxy, which can reference records from
both classes. In other words, the search “class=http_proxy” is the equivalent of “classs=bluecoat OR class=palo_alto_http”, but is easier to remember and type.
Fields
Field names in MQL queries consist of a string of letters and numbers. Each distinct field
is the string up to a white space or a string within quotes “”. Field names are determined
by the taxonomy. They are not case sensitive and can be either lowercase or uppercase.
Fields containing strings that have not been normalized (i.e., parsed using a TAP parser
against the taxonomy) will match only on the entire contents of the field. All parsed fields
(i.e., fields designated by a TAP parser against the taxonomy) will match on partial values.
All events in TAP (whether part of a class or meta-class) have the following fields:
l
l
l
l
l
l
rawmsg
rawmsgid
rawmsghostip or rawmsghostipv6
rawmsgtimeutc
class
classid
If the field does not apply to that event, that field is still present, but has no value. Please
refer to the Master MAP Taxonomy v2.xlsx for a description of what these fields mean.
FireEye, Inc.
10
TAP User Guide
Aliases
MQL Syntax recognizes aliases for field names. The alias searches for all fields that are
designated as corresponding to that alias. Using an alias allows you to search for multiple fields at the same time without knowing the exact field name of each field. An alias
will match on any field which it represents.
Caution: We recommend using aliases sparingly. While this type of query may be useful,
it is a time intensive query to run because the search must look for every field that the
alias represents.
Note: If you want to search for specific multiple field name, use a list instead.
Aliases include:
l
l
l
l
l
l
l
IP, SRCIP, and DSTIP
Host
ID
MAC
Port
Interface
Hash
For example:
hash=[hash or MD5 or SHA1 or SHA256 or SHA512]
IP, SRCIP, DSTIP
The alias ‘ip’ references fields which contain IP addresses such as:
l
l
l
l
l
l
l
l
l
l
l
l
l
l
rawmsghostipv4 (IPv4 address of the host sending the raw message)
rawmsghostipv6 (IPv6 address of the host sending the raw message)
ip (general purpose field for representing an IPv4 address that is only used when
source or destination is either unclear or unknown)
srcipv4 (IPv4 address of the source)
srcipv6 (IPv6 address of the source)
dstipv4 (IPv4 address of the destination)
dstipv6 (IPv6 address of the destination)
cidr (Classless Inter-Domain Routing (CIDR) notation)
transsrcip (Translated IPv4 address of the source)
transsdstip (Translated IPv4 address of the destination)
defgw (Default gateway typically referenced in network events)
ipmask (IP network mask)
intnatip (Internal NAT IP address used in NAT logs)
extnatip (External NAT IP address used in NAT logs)
FireEye, Inc.
11
TAP User Guide
l
l
l
xfwdforip (X-forwarded-for header value, comman+ space delimited if more than
one)
callingsrcip (Source IP of a remote calling identity, typically observed in Windows
event logs as calling address)
targetip (Typically observed in host IDS/IPS, AV, and other logs when referencing
a targeted system or user
For example:
ip=215.18.25.33
ip:215.18.25.0/24
Like IP, the alias ‘srcip’ references keywords that have directional orientation as being a
source IP value of some sort such as:
l
l
l
l
l
rawmsghostipv4 (IPv4 address of the host sending the raw message)
rawmsghostipv6 (IPv6 address of the host sending the raw message)
srcipv4 (IPv4 address of the source)
srcipv6 (IPv6 address of the source)
transsrcip (Translated IPv4 address of the source)
For example:
srcip:25.18.25.33
Like IP, the alias ‘dstip’ references keywords that have directional orientation as being a
destination IP value of some sort such as:
l
l
l
dstipv4 (IPv4 address of the destination)
dstipv6 (IPv6 address of the destination)
transsdstip (Translated IPv4 address of the destination)
For example:
dstip=215.18.25.33
Host
The ‘host’ alias references all keywords for a hostname. You can search for the complete hostname or portion of a fully qualified hostname. HOST references such fields as:
l
l
l
l
rawmsghostname (Hostname, when available, of the host sending the raw message. In some cases, this may be a log relay or forwarder.)
targethost (Typically observed in IDS/IPS, AV, and other logs when referencing a
targeted system or user)
hostname (Hostname which is used whenever the source or destination is unclear
or unknown)
srchost (Hostname of the source machine when direction is known and/or relevant
which typically resolve to srcip or srcipv6)
FireEye, Inc.
12
TAP User Guide
l
l
l
l
l
l
desthost (Hostname of the destination machine when direction is known and/or relevant which typically resolve to dstip or desipv6)
server (Hostname for when srchost or dsthost do not apply such as a central management server, proxy, router serving a nondirectional role)
node (Hostname for specific references to a node that is typically a sensor node)
agent (Name of an agent)
sensor (Name of a sensor)
workstation (Name of workstation which is used only when hostname is already
used and workstation is explicitly declared)
For example:
host=corp-12345
host:subdomain
host=mycompany.com
host:copr-12345.subdomain.mycompany.com
ID
The ‘id’ alias is a generic term for referencing all keywords that contain some type of ID.
ID references such keywords as:
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
classid (Class ID for event collection)
eventid (Specific event identifier )
protoid (Numerical representation of a protocol (6=TCP, 17=UDP, 47=GRE, etc.) )
connectionid (Specific connection identifier)
transactionid (Specific transaction identifier)
sessionid (Specific session identifier)
deviceid (Specific device identifier)
agentid (Specific agent identifier )
accountid (Specific account identifier)
uid (Used when a given user (i.e. joesample) also has a unique user ID or GUID
(i.e., 9473))
gid (Specific group identifier)
policyid (Specific policy identifier)
portid (Specific port id or terminal port id )
pid (Specific process ID, typically used for application PIDs)
ppid (Specific parent process ID, typically used for application PPIDs)
ruleid (Rule or Signature ID containing a unique ID for a given rule or signature)
referenceid (Specific reference ID relating 2 or more things together)
requestid (Specific request ID, specifying the identifier of a given request)
callid (Specific call identifier)
handleid (Specific handle identifier, referring to process handle IDs)
FireEye, Inc.
13
TAP User Guide
l
l
l
l
l
l
l
l
l
l
l
l
operationid (Specific operation identifier)
callinguid (User ID of a remote calling identity, typically observed in windows event
logs as calling id
cveid (Common Vulnerabilities and Exposures identifier, can be referenced by
either reference number or year)
processid (Specific process identifier)
creatorprocessid (Specific creator process identifier)
threadid (Specific thread identifier)
stationid (Specific station identifier)
fileid (Specific file identifier)
parentfileid (Specific parent file identifier. In Bro logs, this is the identifier associated with a container file from which the child (fileid) was extracted as part of the
file analysis)
sentfileid (Sent File identifier, found in Bro http logs as the orig_fuids value, indicates the file identifier of a file pertaining to an originator)
rcvdfileid. (Received File identifier, found in Bro http logs as the resp_fuids value,
indicates the file identifier of a file pertaining to a receiver)
lastalertid (In Bro ssl logs, the last_alert field is the last alert that was seen during
the connection)
For example:
id:5649
MAC
The ‘mac’ alias allows you to search for MAC regardless of hex value delimiter format
and include such keywords as:
l
l
l
l
mac (General purpose field for MAC address, used whenever)
macoui (Organizationally Unique Identifier of a MAC address)
srcmac (MAC address of the source when direction is known)
dstmac (MAC address of the destination when direction is known)
You can search by the OUI portion of the MAC address also.
For example:
mac=01:23:45:67:89:ab
mac:01-23-45
Port
The ‘port’ alias references keywords that contain a network port value such as:
l
l
srcport (Source port number)
dstport (Destination port number)
FireEye, Inc.
14
TAP User Guide
l
l
transsrcport (Translated source port number)
transdstport (Translated destination port number)
For example:
port=80
Interface
The ‘interface’ alias references keywords that contain a network interface value such as:
l
l
l
interface (Logical or physical network interface used for communications (eth0,
eth1, etc) when orientation is unknown)
localinterface (Local interface identifier, typcially network or firewall events specifying interface orientation)
loreigninterface (Foreign interface identifier, typically network or firewall events specifying interface orientation)
For example:
(interface=eth0 or interface=eth1) and ip=22.33.44.55
Hash
The ‘hash’ alias references keywords that contain a cryptographic hash value and can
be used in cases where you have not memorized or do not know the specific cyrptographic hash algorithm used to generate the hash such as:
l
l
l
l
l
hash (General purpose field for storing any type of message digest hash value)
MD5 (computed MD5 hash of an object)
SHA1 (computed SHA1 hash of an object)
SHA256 (computed SHA256 has of an object)
SHA512 (computed SHA512 has of an object)
For example:
hash=24a938a1fcc5df0a7e78267aac0a41ca
Intel Hit Searches
Each intel hit creates a synthetic event with the class of “intel_hit”. To find all events that
are intel hits, run the query class=intel_hit and select specific dates in the Search
box.
To further refine the intel_hits that the search returns:
l
l
Type. To see the number of hits that are based on commodity and curated intelligence, use the query class=intel_hit | groupby type.
Intelscore. To see the number of intel hits with an intelligence score (i.e., low,
medium, high, or critical) use the query class=intel_hit | groupby
intelscore
FireEye, Inc.
15
TAP User Guide
l
Malware family. To show how many intel hits are from a specific intel malware family, use the query class=intel_hit | groupby intelmalwarefamily
(replacing intelmalwarefamily with the family name)
FireEye, Inc.
16
TAP User Guide
Rules
A TAP rule is a search query that is run automatically to locate matches (aka “hits”).
When matches are found, the rule generates an alert based on the rule’s frequency and
distinguishers.
Threshold and time window options work together to determine the frequency with which
the rule generates an alert. The threshold is the number of times that an rule must hit
within the time window specified for an alert to be generated. For example, if the rule generates five hits (i.e., it matches five different events) within one hour then an alert is generated.
A distinguisher is a field in an event that a rule uses to differentiate hits for the purpose of
creating alerts. The distinguisher is typically a hostname or IP address but can be any
field.
In the case where there are multiple events referring to the same type of activity, you
might want to have a single alert instead of multiple alerts. For example, you have a rule
that detects RAR files being transferred over the network. You do not want an alert for
each time a RAR file is transferred from the same host. By adding “scrip” (source IP) to
the rule, then only one alert is generated for each host sending RAR files.
You can also have a single rule generates multiple alerts based on its distinguishers.
For example, a rule has two distinguishers: source IP and destination IP. For every combination of srcIP and dstIP found to match the rule criteria, TAP will create an alert.
Each rule is assigned a unique ID for tracking.
To be effective, some rules require events from specific types of log sources and the
events must be parsed.
There are two types of rules:
l
l
FireEye-defined rules. FireEye experts create rules within rule packs to attempt to
detect a wide range of malicious activities.
Customer-defined rules. You can use custom-defined rules to detect events specific to your environment and organizational needs and generate alerts based on
those rules.
Rule Packs
Rules are grouped together into Rule Packs. Rule Packs serve as containers for groups
of rules. FireEye-defined rules are assigned to FireEye rules packs. Any rules that you
define can only be assigned to rule packs that you define.
To create a rule pack, select Rule Packs at the top of the Rules page then click Create
New Rule Pack. On the Create New Rule Pack window, enter a name for the rule pack.
FireEye, Inc.
17
TAP User Guide
Once you have created a rule pack, you can assign a rule to it when the rule is created
or by updating the rule.
View Rules
To view the details for a rule as well as its revision history, select View/Edit from the
action menu when the rule is selected on the Rules page.
Enable and Disable Rules
Individual rules as well as all the rules within a rule pack can be enabled and disabled. If
a rule is producing lots of false positive alerts, you may decide to disable. When displaying Rules, the Rules page indicates whether a rule is enabled or disabled. To
change its status, select the Rule, click the action icon, and select either Enable or Disable.
When displaying Rule Packs, the Rules page indicates the number of enabled and disabled rules within each rule pack. To change all the rules in a rule pack to either
enabled or disabled, select the rule pack, click the action icon, and select Disable All or
Enable All.
Create User-Defined Rules
There are two ways for you to create new rules in TAP:
l
l
Create a rule on the Rules page
Create a rule from a search on the Search bar
When creating a rule, specify the following on the Create New Rule window:
l
l
l
l
l
Name
Status of enabled or disabled
Description
Query. When creating a rule from the Search bar, the query will be the query used
to find the search results.
Distinguisher. A distinguisher is a field in an event that a rule uses to differentiate
hits for the purpose of creating alerts. The distinguisher is typically a hostname or
IP address but can be any field.In the case where there are multiple events referring to the same type of activity, you might want to have a single alert instead of
multiple alerts. For example, you have a rule that detects RAR files being transferred over the network. You do not want an alert for each time a RAR file is transferred from the same host. By adding “scrip” (source IP) to the rule, then only one
alert is generated for each host sending RAR files.You can also have a single rule
generates multiple alerts based on its distinguishers. For example, a rule has two
distinguishers: source IP and destination IP. For every combination of srcIP and
dstIP found to match the rule criteria, TAP will create an alert.
FireEye, Inc.
18
TAP User Guide
l
l
l
l
Threshold as a number and time window as a number then select hours, minutes,
or seconds as the amount of time. Threshold and time window options work
together to determine the frequency with which the rule generates an alert. The
threshold is the number of times that an rule must hit within the time window specified for an alert to be generated. For example, if the rule generates five hits (i.e., it
matches five different events) within one hour then an alert is generated.
Rule pack
Confidence, which can be low, medium, or high, indicates how likely it is that the
rule will detect events that correspond to the type of activity anticipated (i.e., likelihood that the rule will produce true positives).
Severity, which can be low, medium, or high, indicates how much of an impact a hit
with this rule could have on an organization if verified to be a true positive.
The confidence and severity combine to form the risk attribute of the alert.
Update User-Defined Rules
FireEye-defined rules can be enabled or disabled but not updated. Any rules that you
create can be enabled or disabled and updated.
To update a rule that you have created, click Edit when viewing a rule. The options on
the Update Rule window are the same as when a new rule is created.
Delete Rules
To delete a rule or a rule pack, select it on the Rules page and from the action menu,
select Delete.
Caution: This action cannot be undone.
Deleting a rule pack deletes all the rules that are in that rule pack.
Import and Export Rules
Rules must be in .json format to be imported. To import a rule, when viewing rules on the
Rules page, click Import to open the Import Rule window. You can either select a rule
pack to which the new rule will be added or select the option to have the rule pack identified in the rule being imported. Choose a .json file to import.
Rules are exported in .txt format. To export a rule, click Export when viewing the rule.
FireEye, Inc.
19
TAP User Guide
Alerts
An alert is a notification that at least one event of interest has occurred. The event or
events may have possible security impacts or may be of interest based on some other criteria that you have defined. Alerts can be considered possible candidates for incidents.
Alerts originated from one of the following:
l
l
l
FireEye-defined rules
User-defined rules
Intel hits
TAPS assigns each alert one of the following risk values:
l
l
l
l
Critical
High
Medium
Low
Risk describes the overall potential risk to the organization if the alert is a true positive. It
is typically used to prioritize alert verification and response activities.
Suppress Alerts
There may be situations in which you prefer not to see specific alerts; for example, alerts
may continue to appear while an incident responder is actively engaged in responding
to a potential compromise.
TAP give you the option to suppress any alert’s that match an existing alert’s origin, trigger, and distinguishers for 1 hour, 12 hours, 24 hours, 2 days, 3 days, 1 week, 2 weeks,
or 1 month. A suppressed alert does not appear anywhere in TAP.
To suppress an alert, select the alert on the Alerts page and select Suppress from the
action menu. On the Suppress Alert window, select a time frame.
Add Alerts to Incidents
After determining that an alert requires further action, you can convert it to an incident by
using it to create a new incident.
When an alert is added to an incident, the status of the alert changes to Closed.
FireEye, Inc.
20
TAP User Guide
Incidents
An incident is a grouping of one or more events or alerts that combine to describe a situation that needs further investigation.
An incident may contain multiple alerts. For example, a targeted attack by a single
attacker may generate multiple alerts because different hosts across the environment
were compromised but all those alerts could be added to a single incident that is then
assigned to one person who leads the response.
Incidents have the following characteristics:
l
l
l
l
Priority. Like risks associated with alerts, priority can be critical, high, medium, or
low to provide an indication of the order in which the incident should be examined
in regards to other incidents.
Classification. Classification provides a mechanism for labeling the type of incident and includes the following labels: testing/demonstration, unauthorized access,
denial of service, malicious code, policy violation or poor practice, reconnaissance,
phishing, and other.
Status. Status designates the stage of the investigation and includes declared,
scoped, contained, recovered, and improved.
Assignee. Any TAP user can be designated as the assignee. Assigning an incident to one person avoids the issue of multiple people responding to the same issue
and duplicating efforts unnecessarily.
Create New Incident
TAP has three ways to create a new incident:
1. Create an incident on the Incident page then manually add events
2. Select events based on search results and use those events to create a new incident
3. Convert an alert and its corresponding events into an incident
To create an incident on the Incidents page, click Create New Incident. On the Create
New Incident window, enter a name and description. Select the priority, classification,
and initial status for the incident. Then add events to it.
To create an incident using events from a search, on the Search page, select events
from the Search results and click Add to Incident. The Create New Incident window
opens with the associated events listed. Select create new incident then complete the
name and description and select priority, classification, and initial status for the incident.
To create an incident by using an alert and its associated events, select an alert and
under the action menu click Add to Incident. The details for alert map to the details for
the incident with risk for the alert mapping to priority for the incident.
FireEye, Inc.
21
TAP User Guide
Add Events to Existing Incident
After running a search, you can add the events found to any existing incident. Select the
events on the Search page and click Add to Incident. Select the incident to which you
want to add the events.
Assign Incident and Investigate
Any user in TAP can be assigned to an incident. To designate an assignee for an incident, select it on the Incidents page and under the action icon select View/Edit. Select a
new status from the drop-down menu.
During the course of the investigation, you can also update the incident’s severity and
classification if needed. To update an incident, select it on the Incidents page and under
the action icon select View/Edit. Click Edit to open the Edit Incident window and make
any needed changes.
View Alert Details
New alerts are displayed prominently in an Alert box on the Dashboard page.
The Alerts page shows metrics on alerts such as the summary of active alerts and the
highest daily number of alerts and average number of daily alerts as well as a list of
alerts that you can filter by status.
For each alert, additional details are available by clicking the action icon and selecting
View Details.
FireEye, Inc.
22
TAP User Guide