1. No category

WebSphere Runtime Tooling Portfolio
How to get the tools to do the heavy lifting
1
© 2011 IBM Corporation
Agenda
Developer’s Dilemma
IBM Support Assistant – The one stop shop for “almost” everything
Is my application healthy? Profiling with the IBM Health Center
Garbage Collection & Memory Visualizer (GCMV)
JavaTM Memory Leaks – Catch me if you can
– Memory Analyzer Overview
– Best Practices for Troubleshooting Native Memory Problems
*Java is the registered trademark of Oracle Corp.
2
© 2011 IBM Corporation
3
© 2011 IBM Corporation
Developer Dilemma
How do I get the best
performance from my
application?
My application crashed with
a Java Out of Memory
Error. What do I do now?
4
My customer experiences
frequent Garbage
Collection cycles. What
can I do to alleviate this ?
Is my application’s heap
appropriately sized? How
can I find out?
© 2011 IBM Corporation
Java Runtime PD : Symptoms, Data & Tools
Problem
Causes
Memory Leaks
Java Heap OOM
Data
Tools
verboseGC log
GCMV
Heapdump
MAT
Long Pause times
verboseGC log
GCMV
Contention
Javacore
Health Center
Deadlocks
Binary Core
Thread Analyzer
Binary Core
Dump Analyzer
Fragmentation
Heap Size
Frequent GC
GC Performance
Hangs
Memory Corruption
Crashes
5
Un-initialized
Pointers
© 2011 IBM Corporation
Tooling strategy
Tools need to be
– Easy to obtain
– From a single source
– Easy to update
The answer is the IBM Support Assistant (ISA)
– Available as a free download from:
www.ibm.com/software/support/isa/
A framework for tools delivery
6
© 2011 IBM Corporation
IBM Support Assistant
The IBM Support Assistant (ISA) is a free local workbench (application) that simplifies support and
helps customers resolve questions and problems with IBM software products. ISA includes rich
features and serviceability tools for quick resolution to problems
7
© 2011 IBM Corporation
Meeting customers needs and simplifying support
ISA v4
Features to meet needs
Customer needs:
A robust search feature to query
IBM and non-IBM repositories
concurrently
•Quick access to key IBM product
information and support documentation
Support links for quick access to
key product info
•Entrée into helpful, well organized
relevant educational resources, product
simulations and tutorials
An enhanced service component
to expedite problem submission
Symptom based collection
•Readily available serviceability tools
designed to fix problems and guided
assistance tool to analyze and isolate
software challenges
•Shorten the time to resolution – make
reporting a problem easier and collect
the required data quicker and more
accurate
Education links and roadmaps
Easy add-on tools framework
Update manager for dynamic
updates for product customization
Meeting customer needs with IBM Support Assistant - www.ibm.com/software/support/isa
8
© 2011 IBM Corporation
Getting started with ISA
Step 2
Step 1
ISA Download Site
New ISA Customer
ISA Update Manager Site
Getting up and going in ISA is a simple 2 step process:
1. Download the ISA runtime
2. Use the Updater to customize ISA for preferred product and tool customization
Use the ISA
Updater
to customize
the ISA
components like
search and tools!
9
© 2011 IBM Corporation
IBM Support Assistant V4 interface
Activity-based user
interface helps you
find the information
you need to analyze
your problems and get
support
Configuration tools
and task-based
tutorials to help you
get started
Support Assistant news
feed for latest feature
updates, tools and
announcements
10
© 2011 IBM Corporation
“The game is afoot…”
11
© 2011 IBM Corporation
12
© 2011 IBM Corporation
Understanding Your Application through the Health Center
Answers to..
– What is my Java application doing ?
– Why is it doing that ?
– Why is my application going so slowly ?
– Is my application scaling well ?
– Do we need to tune the JVM ?
– Am I using the right options?
13
© 2011 IBM Corporation
Health Center Overview
Live monitoring tool with very low overhead
Can be run on production systems with minimal impact
Understand how your application is behaving
Diagnose potential problems, with recommendations
Works at the JVM level – no domain-specific (e.g. J2EE)
information
Suitable for all Java applications
Available in ISA
14
© 2011 IBM Corporation
Classes Subsystem
– Shows all loaded
classes
– Shows classes
loaded time
– Visualizes
classloading
activity
– Identifies shared
classes
– Makes
recommendations
15
© 2011 IBM Corporation
Environment Subsystem
Shows
– Version information
for the JVM
– Operating system
and architecture
information for the
monitored system
– Process ID
– All system
properties
– All environment
variables
16
© 2011 IBM Corporation
GC Subsystem
- Shows used heap (after
collection) and GC pause
times
- Identify memory leaks
- Provides tuning
recommendations and
analysis of GC data
17
© 2011 IBM Corporation
Locking Subsystem
- Always-on lock monitoring
- All lock usage is profiled
such as lock request totals,
blocking requests and hold
times
- Helps to identify points of
contention that prevents the
application from scaling
18
© 2011 IBM Corporation
Profiling Subsystem
- Sampling based profiler
- Instantly identifies hottest
methods in an application
- See full call stacks to
identify where methods are
being called from and what
methods they call
19
© 2011 IBM Corporation
New features added
I/O
–
Provides File open events
–
Provides File close events
–
Provides Details of files that are currently open
Native Memory
20
–
Provides native memory usage of the process
and system monitored
–
Does not provide a native memory perspective
view for the z/OS® 31-bit or z/OS 64-bit
platforms.
© 2011 IBM Corporation
What is your verbose GC trying to tell you?
I'm here to help!
21
© 2011 IBM Corporation
Monitoring GC Activity
Use of Verbose GC logging
– only data that is required for GC performance tuning
– Graph Verbose GC output using GC and Memory Visualizer (GCMV) from ISA
Activated using command line options
– -verbose:gc
– -Xverbosegclog:[DIR_PATH][FILE_NAME],X,Y
where:
[DIR_PATH]
is the directory where the file should be written
[FILE_NAME]
is the name of the file to write the logging to
X
is the number of files to write to
Y
is the number of GC cycles a file should contain
Performance Cost:
– (very) basic testing shows a 2% overhead for GC duration of 200ms
• eg. if application GC overhead is 5%, it would become 5.1%
22
© 2011 IBM Corporation
GCMV usage scenarios
Investigate performance problems
– Long periods of pausing or unresponsiveness
Evaluate your heap size
– Check heap occupancy and adjust heap size if needed
Garbage collection policy tuning
– Examine GC characteristics, compare different policies
Look for memory growth
– Heap consumption slowly increasing over time
– Evaluate the general health of an application
23
© 2011 IBM Corporation
Introduction to GCMV
Garbage Collection and Memory Visualizer
–
Verbose GC data visualizer
–
Eclipse based tool available as plugin in ISA and as a standalone tool.
–
Parses and plots all verbose GC logs
–
Extensible to parse and plot other forms of input
–
Provides graphical display of wide range of verbose GC data values
–
Handles optthruput, optavgpause, and gencon GC modes
–
Has raw log, tabulated data and graph views and can save data to jpeg or
.csv files (for export to spreadsheets)
24
© 2011 IBM Corporation
Raw Data
Report with recommendations
Tabbed Data
Line Plot
Structured Data
25
© 2011 IBM Corporation
GC and Memory Visualizer Capabilities
Analyses heap usage, heap size, pause times, and
many other properties
Compare multiple logs in the same plots and
reports
Many views on data
–
Reports
–
Graphs
–
Tables
Can save data to
–
HTML reports
–
JPEG pictures
–
CSV files
© 2011 IBM Corporation
Plotting data with the GCMV
Use File > Open
to open a new
input file
The VGC Data
menu allows
you to choose
what data to
display
Use File > Add
to add multiple
input files to a
single data set
for comparison
and aggregated
display
Right-click on the
plot and use the
context menu to
export data
27
The Axes
panel supports
customized
units and panand-zoom
The Line plot tab contains
the data visualization
© 2011 IBM Corporation
Salient Features of GCMV
Axis Control
Zooming
Comparison of two verbosegc logs
Can Save the results in images
Performance and Memory usage indicators
Recommendations
28
© 2011 IBM Corporation
Memory Leak Example
29
© 2011 IBM Corporation
Java Memory Leaks: Catch me if you Can
Debugging memory leaks using the Memory Analyzer (MAT)
30
© 2011 IBM Corporation
How to interpret the memory leak analysis results?
Not an exact science, analysis will point out suspects
Likelihood of leak depends on the
– Size of the leaking data structure,
– Drop size
– Number of leaking units
– Number of instances of objects in Ownership Context Graph Nodes
Better comparative analysis results, if baseline heap dump
is taken early and primary heap dump is taken after a large
increase in the occupied heap size
Better single dump analysis, if primary heap dump is taken
after a large increase in the occupied heap size
31
© 2011 IBM Corporation
How to interpret the Memory Leak Analysis Results?
What is leaking?
– What is the object (e.g. a HashMap) holding all the
leaking objects i.e. leak container.
– What are the objects getting added to the leak
container i.e. leak unit.
– Who is holding the leak container in memory? What
are the object types and package names of objects on
the chain of references from a root object to the leak
container i.e. owner chain.
32
© 2011 IBM Corporation
View of the leaking data structure
Leak Root
Significant entities
– An owner chain
MyClass
Container
HashSet
Leaking
Unit
HashMap
Owner
Chain
– A leak root
– A container
– The unit of the leak
– Leak contents
Contents
33
HM$Entry
HM$Entry
HM$Entry
String
String
String
Char[]
Char[]
Char[]
© 2011 IBM Corporation
Understanding Shallow & Retained Size
A is the root object
100
A has outgoing references (Children) to B and C
A
B has an incoming reference (Parent) from A
and outgoing reference (Children) to B1 and B2
B
10
10
B1
5
34
C
Shallow size = Size of an object
Retained Size = Total size of the subtree
B2
C1
C2
5
5
5
© 2011 IBM Corporation
Understanding Shallow Size & Retained Size
100
Size of A = 100 (Shallow Size)
A
Total Size of A = 140 (Retained Size)
(A+B+B1+B2+C+C1+C2)
B
10
10
35
C
B1
B2
C1
C2
5
5
5
5
© 2011 IBM Corporation
Understanding Shallow Size & Retained Size
Consider the Size of B1 = 1000 (Shallow Size)
100
Total Size of A (root object) = 1135 (Retained Size)
A
B
10
10
C
B1
B2
C1
C2
1000
5
5
5
36
© 2011 IBM Corporation
Why MAT?
Allows developers to readily track the memory leaks in java
applications during runtime
Provides a instance-level view for the application under test
Graphical representation of memory utilization
Ability to filter and view graphical reports of specific criteria
Provision to view the trace details and reference class for a
particular class instance
Ability to add and modify the threshold count for a particular class
instance and view threshold reports
Provision to filter and view instance details of specific criteria
37
© 2011 IBM Corporation
Acquiring Heap Dumps
Oracle/Sun VMs
– -XX:+HeapDumpOnOutOfMemoryError
– Jconsole
– Jmap -dump
IBM VMs
– Automatic on OutOfMemoryErrors
– Configurable on many other events using -Xdump command line option
• SIGQUIT, Full GC, Exceptions, Class Load/Unload, Method entry/exit
Acquire Heap Dump option on MAT
– Acquires dumps from processes on same machine
– Oracle/Sun VMs – triggered via jps/jmap
– IBM VMs – target must be running Java 6 SR6 or later
http://wiki.eclipse.org/index.php/MemoryAnalyzer#Getting_a_Heap_Dump
© 2011 IBM Corporation
Dump Types
HPROF – can be generated on OutOfMemory
IBM System Dump – core dump processed with Jextract
IBM Portable Heap Dump – default on OutOfMemory, smaller & faster
Objects,
classes,
class
loaders
HPROF
IBM
System
IBM
PHD
Thread
details
Field
name
s
Field
&
array
refs
Primitive Primitiv
fields
e array
content
s
with
javacore
Accurat
e GC
roots
Native
memor
y&
thread
s
© 2011 IBM Corporation
Features - Overview
40
© 2011 IBM Corporation
Features – Leak Suspects
41
© 2011 IBM Corporation
Features – System Overview
42
© 2011 IBM Corporation
Features – Top Consumers
43
© 2011 IBM Corporation
Features – Thread details/attributes
44
© 2011 IBM Corporation
Component Report – memory waste
45
© 2011 IBM Corporation
Features – Query Language
Very similar to SQL:
– think of classes as tables, objects as rows and attributes as columns.
– SELECT * FROM java.lang.String
– SELECT * FROM java.lang.String s WHERE s.length < 1024
– SELECT * FROM java.lang.String s WHERE dominatorof(s) != NULL and
classof(dominatorof(s)).@name = "com.test.Test“
Produces table of results from which you can do interactive analysis
46
© 2011 IBM Corporation
MA Extension Plug-ins
MA provides an extension point for writing analysis plug-ins
MA provides a public API for accessing Java objects, classes and fields on
the Java heap
– Also provides APIs for looking at relationships between objects
– Also provides APIs for generating reports and charts
47
© 2011 IBM Corporation
MA : Running in “Batch” Mode
Batch processing of dumps is possible with MA
– Allows initial processing and report generation to be handled on higher end boxes
– Produces standard reports: Leak Suspects, System Overview, Top Components
– Produces “index” files
Command to run in Batch Mode
– [JAVA_HOME]\jre\bin\java –Xmx[nnnnM] –
Dosgi.bundles=org.eclipse.mat.dtfj@4:start,org.eclipse.equin
ox.common@2:start,org.eclipse.update.configurator@3:start,or
g.eclipse.core.runtime@start -jar
[MA_HOME]\plugins\org.eclipse.equinox.launcher*.jar consoleLog -application org.eclipse.mat.api.parse [heapdump]
[MA_Reports]
– Where MA_Reports can be
• Overview - "org.eclipse.mat.api:overview"
• Leak Suspects - "org.eclipse.mat.api:suspects"
• Top Components - "org.eclipse.mat.api:top_components“
48
© 2011 IBM Corporation
Thai
Hindi
Traditional Chinese
Russian
Gracias Thank
Spanish
Merci
You
French
Obrigado
English
Brazilian Portuguese
Arabic
Danke
German
Grazie
Simplified Chinese
Italian
Japanese
Korean
Tamil
Teşekkürler
turkish
49
© 2011 IBM Corporation
Resources
www.ibm.com/developerworks/java
www.alphaworks.ibm.com/java
http://www.ibm.com/developerworks/forums/dw_jforums.jsp
http://ww.ibm.com/developerworks/java/jdk/diagnosis
50
© 2011 IBM Corporation
51
© 2011 IBM Corporation