1. technology and computing
2. operating systems
3. unix

2011
IBM Power Systems Technical University
October 10-14 | Fontainebleau Miami Beach | Miami, FL
򔻐򗗠򙳰
PE62 Part II: Updated Real-world Case
Histories -- How to Monitor and Analyze
the VMM and Storage I/O Statistics of a
Power/AIX LPAR
Earl Jew ([email protected])
310-251-2907 cell
Senior IT Management Consultant - IBM Power Systems and IBM Systems Storage
IBM Lab Services and Training - US Power Systems (group/dept)
400 North Brand Blvd., c/o IBM 8th floor, Glendale, CA 91203
[Extended: April 4th, 2013]
© Copyright IBM Corporation 2012
Materials may not be reproduced in whole or in part without the prior written permission of IBM.
5.3
Part I: Updated Concepts and Tactics -- How to Monitor and Analyze
the VMM and Storage I/O Statistics of a Power/AIX LPAR
ABSTRACT
This presentation updates AIX/VMM (Virtual Memory Management) and LVM/JFS2
storage IO performance concepts and tactics for the day-to-day Power/AIX system
administrator. It explains the meaning of the numbers offered by AIX commands
(vmstat, iostat, mpstat, sar, etc.) to monitor and analyze the AIX VMM and storage
IO performance and capacity of a given Power7/AIX LPAR.
These tactics are further illustrated in Part II: Updated Real-world Case Histories -How to Monitor and Analyze the VMM and Storage I/O Statistics of a Power/AIX
LPAR.
© Copyright IBM Corporation 2012
2
Part II: Updated Real-world Case Histories -- How to Monitor and Analyze
the VMM and Storage I/O Statistics of a Power/AIX LPAR
ABSTRACT
These updated case-histories further illustrate the content presented in Part I:
Updated Concepts and Tactics -- How to Monitor and Analyze the VMM and Storage
I/O Statistics of a Power/AIX LPAR.
This presentation includes suggested ranges and ratios of AIX statistics to guide VMM
and storage IO performance and capacity analysis.
Each case is founded on a different real-world customer configuration and workload
that manifests characteristically in the AIX performance statistics -- as performing:
intensely in bursts, with hangs and releases, AIX:lrud constrained, AIX-buffer
constrained, freely unconstrained, inode-lock contended, consistently light,
atomic&synchronous, virtually nil IO workload, long avg-wait's, perfectly ideal, long
avg-serv's, mostly rawIO, etc.
© Copyright IBM Corporation 2012
3
Strategic Thoughts, Concepts, Considerations, and Tactics
• Monitoring AIX – Usage, Meaning and Interpretation
– Review component technology of the infrastructure, i.e. proper tuning-by-hardware
– Review implemented AIX constructs, i.e. “firm” near-static structures and settings
– Review historical/accumulated AIX events, i.e. usages, pendings, counts, blocks, etc.
– Monitor dynamic AIX command behaviors, i.e. ps, vmstat, mpstat, iostat, etc.
• Recognizing Common Performance-degrading Scenarios
– High Load Average relative to count-of-LCPUs, i.e. “over-threadedness”
– vmstat:memory:avm near-to or greater-than lruable-gbRAM, i.e. over-committed
– Continuous low vmstat:memory:fre with persistent lrud (fr:sr) activity
– Continuous high ratio of vmstat:kthr:b relative to vmstat:kthr:r
– Poor ratio of pages examined to pages freed (fr:sr ratio) in vmstat -s output
© Copyright IBM Corporation 2012
4
High Load Average relative to count-of-LCPUs, i.e. “over-threadedness”
System Configuration: lcpu=30 mem=123903MB
kthr
memory
page
faults
cpu
time
----------- --------------------- ------------------------------------ ------------------ ----------- -------r
b
p
avm
fre
fi
fo
pi
po
fr
sr
in
sy
cs us sy id wa hr mi se
175
5
0
15545407
6363904 4659
378
0
0 4519 18608 6995 1798098 17456 89 11 0 0 07:39:31
169
5
0
15564915
6344425 4130
372
0
0 4269 25154 7503 1818519 17848 88 12 0 0 07:39:32
165
5
0
15592443
6316497 4697
496
0
0 4401 24161 7645 1722401 18499 88 12 0 0 07:39:33
175
4
0
15616366
6292662 5204
522
0
0 5576 39136 8115 1862990 19747 88 12 0 0 07:39:34
177
5
0
15619267
6290033 4224
673
0
0 4903 21061 7792 1772567 19948 88 12 0 0 07:39:35
186
7
0
15639664
6269361 4519
586
0
0 4394 19688 8235 1804175 22383 87 13 0 0 07:39:36
191
5
0
15651883
6257403 4158
286
0
0 4521 19371 6383 1833902 16902 88 12 0 0 07:39:37
199
2
0
15670551
6238481 2623
362
0
0 2470 11057 5157 1837258 13450 88 12 0 0 07:39:38
204
1
0
15691425
6217629 1823
292
0
0 1941
9109 3527 1885205 9547 89 11 0 0 07:39:39
207
4
0
15698443
6210608 2539
442
0
0 2718 13941 7907 1843474 19631 88 12 0 0 07:39:40
224
3
0
15715376
6194061 2113
230
0
0 2592 13741 5283 1853315 12240 88 12 0 0 07:39:41
0
0
0
15728361
6180673 2142
236
0
0 1814
9105 5295 1859673 12272 88 12 0 0 07:39:42
224
3
0
15737615
6171916 2275
220
0
0 2839 17585 5058 1931829 12176 88 12 0 0 07:39:43
238
4
0
15737613
6171746 3182
290
0
0 3108 16083 6011 1883330 14504 88 12 0 0 07:39:44
243
4
0
15739367
6169632 3016
356
0
0 2839 13574 4945 1917855 14129 89 11 0 0 07:39:45
245
2
0
15742352
6166712 2270
376
0
0 2463 15306 3546 1941029 10252 89 11 0 0 07:39:46
243
2
0
15754661
6154318 2280
312
0
0 2332 15982 3393 1892718 9638 89 11 0 0 07:39:47
244
3
0
15737393
6172133 1958
353
0
0 2592 14843 4138 1918667 11481 89 11 0 0 07:39:48
246
3
0
15737126
6172074 1682
311
0
0 1426 14808 4146 1922001 11942 89 11 0 0 07:39:49
242
2
0
15758610
6150400 1668
244
0
0 1555
8393 4324 1924860 10869 88 12 0 0 07:39:50
kthr
memory
page
faults
cpu
time
----------- --------------------- ------------------------------------ ------------------ ----------- -------r
b
p
avm
fre
fi
fo
pi
po
fr
sr
in
sy
cs us sy id wa hr mi se
251
5
0
15790370
6118650 1985
452
0
0 2207 13685 5625 1903786 12530 88 12 0 0 07:39:51
259
2
0
15797418
6111729 1660
266
0
0 1937 12706 6979 1957108 16857 88 12 0 0 07:39:52
© Copyright IBM Corporation 2012
5
Strategic Thoughts, Concepts, Considerations, and Tactics
• Monitoring AIX – Usage, Meaning and Interpretation
– Review component technology of the infrastructure, i.e. proper tuning-by-hardware
– Review implemented AIX constructs, i.e. “firm” near-static structures and settings
– Review historical/accumulated AIX events, i.e. usages, pendings, counts, blocks, etc.
– Monitor dynamic AIX command behaviors, i.e. ps, vmstat, mpstat, iostat, etc.
• Recognizing Common Performance-degrading Scenarios
– High Load Average relative to count-of-LCPUs, i.e. “over-threadedness”
– vmstat:memory:avm near-to or greater-than lruable-gbRAM, i.e. over-committed
– Continuous low vmstat:memory:fre with persistent lrud (fr:sr) activity
– Continuous high ratio of vmstat:kthr:b relative to vmstat:kthr:r
– Poor ratio of pages examined to pages freed (fr:sr ratio) in vmstat -s output
© Copyright IBM Corporation 2012
6
vmstat:memory:avm near-to or greater-than lruable-gbRAM;
memory over-committed 3986734*4096=16329mb vs 15744mb
System configuration: lcpu=8 mem=15744MB
kthr
memory
page
faults
cpu
time
----------- --------------------- ------------------------------------ ------------------ ----------- -------r
b
p
avm
fre
fi
fo
pi
po
fr
sr
in
sy
cs us sy id wa hr mi se
1
1
0
3986577
2652 1944
797
0
0 1536 12803
880
2377 4459 10 4 55 31 14:17:58
2
2
0
3986576
2553 1863
757
0
0 2557 37067
852
4053 4446 11 4 55 30 14:18:00
2
1
0
3986574
2206 1959
799
0
0 2559 37499 1009
2523 4559 10 6 53 31 14:18:02
0
3
0
3986573
2597 2044
843
0
0 3069 42804
912
2377 4553 11 4 55 30 14:18:04
1
2
0
3986571
2511 1870
754
0
0 2559 167438
804
2203 4247 10 4 56 30 14:18:06
0
2
0
3986571
2197 1944
787
0
0 2560 102054
814
2310 4063 10 4 56 30 14:18:08
0
2
0
3986570
2872 1960
792
0
0 3070 42557
889
4148 4532 11 4 54 30 14:18:10
1
2
0
3986569
3752 1876
764
0
0 3070 65622
933
2363 4834 10 5 53 32 14:18:12
1
2
0
3986568
3864 1787
730
0
0 2559 49907
880
2135 4617 9 4 53 33 14:18:14
1
1
0
3986567
2634 1915
767
0
0 2047 30676
785
2774 3948 10 4 55 31 14:18:16
0
3
0
3986567
2523 1890
759
0
0 2552 27693
877
2646 4443 10 4 55 32 14:18:18
1
2
0
3986573
2040 2008
810
0
0 2557 23419
928
5155 4671 12 4 54 30 14:18:20
1
2
0
3986572
1962 1878
761
0
0 2554 52663
905
2525 4795 10 4 56 29 14:18:22
2
2
0
3986587
2652 1960
798
3
0 3071 14081 1030 11377 7789 13 9 51 27 14:18:24
2
2
0
3986570
2363 1938
781
0
0 2558 30570
836
3004 5732 10 5 56 29 14:18:26
2
1
0
3986734
2056 1884
762
1
0 2557 32017
888 31414 6058 15 11 47 26 14:18:28
2
0
0
3986617
1933 1920
779
2
0 2558 15377
933 22108 5545 15 9 48 28 14:18:30
1
0
0
3986612
2463 2008
826
0
0 3069 25129 1192
2823 5935 11 9 52 28 14:18:32
1
2
0
3986586
3073 1988
810
0
0 3064 15116
816
2732 4430 10 4 56 30 14:18:34
0
1
0
3986587
3402 1719
685
0
0 2555 24262
799
3395 4429 9 4 58 29 14:18:36
kthr
memory
page
faults
cpu
time
----------- --------------------- ------------------------------------ ------------------ ----------- -------r
b
p
avm
fre
fi
fo
pi
po
fr
sr
in
sy
cs us sy id wa hr mi se
0
3
0
3986582
2347 1841
748
0
0 2047 14678
865
2352 4683 10 4 56 31 14:18:38
0
1
0
3986580
3068 1945
784
0
0 3070 24649
784
4741 4233 11 4 55 29 14:18:40
0
2
0
3986583
2797 1929
780
0
0 2559 16436
806
2466 4205 10 4 57 29 14:18:42
© Copyright IBM Corporation 2012
7
Strategic Thoughts, Concepts, Considerations, and Tactics
• Monitoring AIX – Usage, Meaning and Interpretation
– Review component technology of the infrastructure, i.e. proper tuning-by-hardware
– Review implemented AIX constructs, i.e. “firm” near-static structures and settings
– Review historical/accumulated AIX events, i.e. usages, pendings, counts, blocks, etc.
– Monitor dynamic AIX command behaviors, i.e. ps, vmstat, mpstat, iostat, etc.
• Recognizing Common Performance-degrading Scenarios
– High Load Average relative to count-of-LCPUs, i.e. “over-threadedness”
– vmstat:memory:avm near-to or greater-than lruable-gbRAM, i.e. over-committed
– Continuous low vmstat:memory:fre with persistent lrud (fr:sr) activity
– Continuous high ratio of vmstat:kthr:b relative to vmstat:kthr:r
– Poor ratio of pages examined to pages freed (fr:sr ratio) in vmstat -s output
© Copyright IBM Corporation 2012
8
Continuous low vmstat:memory:fre with persistent lrud (fr:sr) activity
Continuous high ratio of vmstat:kthr:b relative to vmstat:kthr:r
System configuration: lcpu=8 mem=20480MB ent=3.90
kthr
memory
page
faults
cpu
----------- --------------------- ------------------------------------ ------------------ ----------------------r
b
p
avm
fre
fi
fo
pi
po
fr
sr
in
sy
cs us sy id wa
pc
ec
1 11
0
2381462
12283
574
41
4
0
0
0
282 14044 3759 36 3 36 24 1.59 40.8
5
6
0
2383434
9229
537
21
15
0
0
0
214
9309 1221 65 8 22 5 2.94 75.4
3
5
0
2381734
10665
571
5
4
0
451
1303
442
8902 3345 83 6 5 6 3.53 90.4
3 10
0
2383409
7700
666
1
3
0
0
0
232
2606 1828 46 7 20 28 2.08 53.3
6 10
0
2383410
7743
611
0
0
0
661
1956
270
5229 3206 93 2 2 4 3.72 95.3
5
8
0
2384836
7559
544
35
1
45 1150
3962
475
9629 4267 48 10 22 19 2.33 59.7
4
7
0
2385533
8572
569
82
1
30 1437
3311
259
8388 1512 83 2 6 10 3.33 85.3
2
8
0
2387636
7716
411
90
5
7
967
2971
467 40650 2367 58 15 12 16 2.89 74.1
5
7
0
2390359
7622
542
20
19
11 1908
5953
360 18005 2195 77 4 9 10 3.19 81.8
3 11
0
2390756
7956
511
0
91
10
977
3899
178
3897 1655 59 2 17 22 2.40 61.5
3
9
0
2390761
7750
487
16
87
0
504
1368
471
3018 2893 45 2 33 19 1.89 48.5
6
7
0
2392294
7597
483
4
80
40 1227
3822
233
4682 2070 69 5 17 9 2.92 75.0
4
8
0
2392294
7837
413
30
53
0
571
2025
416
4990 5019 85 2 5 8 3.45 88.4
8
8
0
2392294
7704
409
0
0
0
385
983
184
2894 3480 53 1 25 20 2.16 55.3
11
8
0
2387726
19279
325
114
118
177 3971 13773
224 39964 2564 84 15 0 1 3.88 99.6
4 13
0
2402933
7698
390
656
189
118 3732 14638
436 39993 4822 53 24 7 16 3.06 78.6
5 11
0
2403390
8018
396
57
95
6
946
3780
263 23753 1784 93 3 1 3 3.77 96.6
3 11
0
2402851
7753
424
17
73
0
64
98
393
6538 2994 52 6 9 33 2.31 59.3
5 11
0
2404724
7710
354
10
49
0 1356
5981
239
4234 1928 70 6 16 9 2.96 75.8
3 11
0
2404716
8279
324
23
36
0
586
2056
436
6414 3849 76 2 8 13 3.12 80.0
kthr
memory
page
faults
cpu
----------- --------------------- ------------------------------------ ------------------ ----------------------r
b
p
avm
fre
fi
fo
pi
po
fr
sr
in
sy
cs us sy id wa
pc
ec
5 13
0
2404718
8054
373
0
0
0
274
784
192
1603 2058 56 1 26 17 2.36 60.4
5 13
0
2404716
7965
311
0
0
0
273
583
237
2342 2394 79 1 7 13 3.16 81.0
2 12
0
2388769
26143
353
22
35
3
322
590
390
3550 3130 48 4 25 24 2.06 52.9
© Copyright IBM Corporation 2012
time
-------hr mi se
02:00:03
02:00:05
02:00:07
02:00:09
02:00:11
02:00:13
02:00:15
02:00:17
02:00:19
02:00:21
02:00:23
02:00:25
02:00:27
02:00:29
02:00:31
02:00:33
02:00:35
02:00:37
02:00:39
02:00:41
time
-------hr mi se
02:00:43
02:00:45
02:00:47
9
Strategic Thoughts, Concepts, Considerations, and Tactics
• Monitoring AIX – Usage, Meaning and Interpretation
– Review component technology of the infrastructure, i.e. proper tuning-by-hardware
– Review implemented AIX constructs, i.e. “firm” near-static structures and settings
– Review historical/accumulated AIX events, i.e. usages, pendings, counts, blocks, etc.
– Monitor dynamic AIX command behaviors, i.e. ps, vmstat, mpstat, iostat, etc.
• Recognizing Common Performance-degrading Scenarios
– High Load Average relative to count-of-LCPUs, i.e. “over-threadedness”
– vmstat:memory:avm near-to or greater-than lruable-gbRAM, i.e. over-committed
– Continuous low vmstat:memory:fre with persistent lrud (fr:sr) activity
– Continuous high ratio of vmstat:kthr:b relative to vmstat:kthr:r
– Poor ratio of pages freed to pages examined (fr:sr ratio) in vmstat -s output
© Copyright IBM Corporation 2012
10
Poor ratio of pages freed to pages examined (fr:sr ratio) in vmstat -s output
$ uptime ; vmstat –s
02:17PM
up 159 days, 21:47, 1 user, load average: 1.31, 1.52, 2.97
42359798551 total address trans. faults
25149263165 page ins
17902490831 page outs
52357061 paging space page ins
59626441 paging space page outs
0 total reclaims
13778823141 zero filled pages faults
804184 executable filled pages faults
580181169061 pages examined by clock
310896 revolutions of the clock hand
29494284299 pages freed by the clock
4191584238 backtracks
149218393 free frame waits
0 extend XPT waits
4506482991 pending I/O waits
29188011653 start I/Os
8946597697 iodones
204899338951 cpu context switches
26163416710 device interrupts
699186076 software interrupts
31029975857 decrementer interrupts
15560545 mpc-sent interrupts
15560524 mpc-receive interrupts
53335915 phantom interrupts
0 traps
432963088862 syscalls
© Copyright IBM Corporation 2012
11
Poor ratio of pages freed to pages examined (fr:sr ratio) in vmstat -s output
Given sustained fr:sr ratios: 1:1.1/blue 1:3/green 1:5/warning 1:10/red
$ uptime ; vmstat –Iwt 2
02:17PM
up 159 days, 21:47, 1 user,
System configuration: lcpu=8 mem=15744MB
load average: 1.31, 1.52, 2.97
kthr
memory
page
faults
cpu
time
----------- --------------------- ------------------------------------ ------------------ ----------- -------r
b
p
avm
fre
fi
fo
pi
po
fr
sr
in
sy
cs us sy id wa hr mi se
1
1
0
3986577
2652 1944
797
0
0 1536 12803
880
2377 4459 10 4 55 31 14:17:58
2
2
0
3986576
2553 1863
757
0
0 2557 37067
852
4053 4446 11 4 55 30 14:18:00
2
1
0
3986574
2206 1959
799
0
0 2559 37499 1009
2523 4559 10 6 53 31 14:18:02
0
3
0
3986573
2597 2044
843
0
0 3069 42804
912
2377 4553 11 4 55 30 14:18:04
1
2
0
3986571
2511 1870
754
0
0 2559 167438
804
2203 4247 10 4 56 30 14:18:06
0
2
0
3986571
2197 1944
787
0
0 2560 102054
814
2310 4063 10 4 56 30 14:18:08
0
2
0
3986570
2872 1960
792
0
0 3070 42557
889
4148 4532 11 4 54 30 14:18:10
1
2
0
3986569
3752 1876
764
0
0 3070 65622
933
2363 4834 10 5 53 32 14:18:12
1
2
0
3986568
3864 1787
730
0
0 2559 49907
880
2135 4617 9 4 53 33 14:18:14
1
1
0
3986567
2634 1915
767
0
0 2047 30676
785
2774 3948 10 4 55 31 14:18:16
0
3
0
3986567
2523 1890
759
0
0 2552 27693
877
2646 4443 10 4 55 32 14:18:18
1
2
0
3986573
2040 2008
810
0
0 2557 23419
928
5155 4671 12 4 54 30 14:18:20
1
2
0
3986572
1962 1878
761
0
0 2554 52663
905
2525 4795 10 4 56 29 14:18:22
2
2
0
3986587
2652 1960
798
3
0 3071 14081 1030 11377 7789 13 9 51 27 14:18:24
2
2
0
3986570
2363 1938
781
0
0 2558 30570
836
3004 5732 10 5 56 29 14:18:26
2
1
0
3986734
2056 1884
762
1
0 2557 32017
888 31414 6058 15 11 47 26 14:18:28
2
0
0
3986617
1933 1920
779
2
0 2558 15377
933 22108 5545 15 9 48 28 14:18:30
1
0
0
3986612
2463 2008
826
0
0 3069 25129 1192
2823 5935 11 9 52 28 14:18:32
1
2
0
3986586
3073 1988
810
0
0 3064 15116
816
2732 4430 10 4 56 30 14:18:34
0
1
0
3986587
3402 1719
685
0
0 2555 24262
799
3395 4429 9 4 58 29 14:18:36
kthr
memory
page
faults
cpu
time
----------- --------------------- ------------------------------------ ------------------ ----------- -------r
b
p
avm
fre
fi
fo
pi
po
fr
sr
in
sy
cs us sy id wa hr mi se
© Copyright IBM Corporation 2012
12
Next: Case-histories to illustrate indications of performance issues
We will next review a parade of customer case-history textfiles.
•
•
•
•
•
These case-histories are founded on a mundane AIX command script, (see appendix)
Each sanitized textfile illustrates a common indicated performance issue, if not several.
Remedies to resolve will be offered, but not illustrated; most remedies are surprisingly simple.
Except for inexplicably poor performance, there are typically no other apparent issues.
In other words, Recognition is notably more problematic than Resolution.
Some indicated performance issues are:
•
•
•
•
A simple lack of CPU and/or gbRAM for the given workload, i.e. poor Tuning-by-Hardware
Improperly implemented tactics, i.e. AIX VMM parameter values that are far out-of-whack
Simply continuing to use old technologies when better technologies are free and available
Implementing tactics without understanding their purpose, appropriateness or compromise
Heads-up on 2 Hot Tips:
• Understand JFS2 rbr,rbw,rbrw mount-options for predominantly sequential IO workloads
• Understand the JFS2 cio mount-option for concurrently read-write IO workloads
© Copyright IBM Corporation 2012
13
An AIXperftuning tactic: The Tractor for move-the-data Sequential Read
workloads – How to Qualify this as an Appropriate Tactic before Implementation
• Objective: Substantially reducing near-continuous burns of lrud’s “fr:sr” activity
• Mandatory Qualifications are:
– deploying a SAN storage system of sufficient performance, i.e. V7000,DS8000,XIV,etc.
– using AIX:LVM/JFS2 filesystems in-service of an RDBMS w/internal buffer cache
– observing near-continuous burns of lrud’s “fr:sr” activity in vmstat -Iwt 2
– observing high runtime for lrud; check ps –ek|egrep “lrud|syncd|TIME”
– observing a sustained 5-digits or more of AIX:vmstat –Iwt 2:page:fi readIO
– a confirming 10:1 or greater ratio of AIX:vmstat –s:start I/Os-to-iodones
• Bluntly, if your Power/AIX LPAR infrastructure/workload meets the above criteria,
then implementing this simple tactic is Gonna-Rock BIG TIME !!!
– If implemented on a workload less intense than the above,
…
then the rock will be more like a pebble
– Of course, this tactic assumes all other dependencies are properly/sufficiently tuned.
© Copyright IBM Corporation 2012
14
An AIXperftuning tactic: The Tractor for move-the-data Sequential Read
workloads – What to do to Implement this Tactic i.e. mount –o rbr <…>
From: http://pic.dhe.ibm.com/infocenter/aix/v6r1/index.jsp?topic=%2Fcom.ibm.aix.cmds%2Fdoc%2Faixcmds3%2Fmount.htm
AIX 6.1 information > Commands > m
mount Command
Purpose
Makes a file system available for use.
Syntax
mount [ -f ] [ -n node ] [ -o options ] [ -p ] [ -r ] [ -v vfsname ] [ -t type | [ device | node:directory ] directory | all | -a ] [-V [generic_options]
special_mount_points ]
Description
The mount command instructs the operating system to make a file system available for use at a specified location (the mount point). In addition, you can use
the mount command to build other file trees made up of directory and file mounts. The mount command mounts a file system expressed as a device
using the device or node:directory parameter on the directory specified by the directory parameter. After the mount command has finished, the directory
specified becomes the root directory of the newly mounted file system.
...
...
...
mount –o rbr <…>
Mount file system with the release-behind-when-reading capability. When sequential reading of a file in this file system is
detected, the real memory pages used by the file will be released once the pages are copied to internal buffers. If none of
the release-behind options are specified, norbrw is the default.
Note: When rbr is specified, the D_RB_READ flag is ultimately set in the _devflags field in the pdtentry structure.
mount –o rbw <…>
Mount file system with the release-behind-when-writing capability. When sequential writing of a file in this file system is
detected, the real memory pages used by the file will be released once the pages written to disk. If none of the releasebehind options are specified, norbrw is the default.
Note: When rbw is specified, the D_RB_WRITE flag is set.
mount –o rbrw <…>
Mount file system with both release-behind-when-reading and release-behind-when-writing capabilities. If none of the releasebehind options are specified, norbrw is the default.
Note: If rbrw is specified, both the D_RB_READ and the D_RB_WRITE flags are set.
© Copyright IBM Corporation 2012
15
An AIXperftuning tactic: The Tractor for move-the-data Sequential Read
workloads – What does mount –o rbr <…> do to work so effectively?
• AIX buffers JFS2 SAN IO:
– to implement its ReadAhead and WriteBehind algorithms (IO coalescence is Good)
– to optimize JFS2 read-rehits and write-rehits (note: “rehits” are usually random IO)
• Substantial streams of Sequential IO almost never read-rehit or write-rehit in the JFS2
buffer cache – thus buffering Sequential IO offers little, if any, rehit benefit
• AIX:lrud indiscriminately/non-selectively uses its Clockhand to scan for Least
Recently Used buffer-cache’d IOs to steal&free in-order to supply free memory
– the kernel processing overhead of lrud fr:sr is often critically overwhelming
– unfortunately howsoever overwhelming, it is also virtually always overlooked
Question:
Is there a way to scan|steal|free Sequential IO, and only buffer-cache Random IO (for JFS2
rehits) – without suffering the kernel processing overhead of lrud fr:sr ?
© Copyright IBM Corporation 2012
16
An AIXperftuning tactic: The Tractor for move-the-data Sequential Read
workloads – Question & Answer explanation regarding mount –o rbr <…>
Question:
Is there a way to scan|steal|free Sequential IO, and only buffer-cache Random IO (for JFS2
rehits) – without suffering the kernel processing overhead of lrud fr:sr ?
Answer:
Yes. Use mount –o rbr <…> to mount JFS2 RDBMS data filesystems.
– rbr replaces lrud fr:sr by immediately freeing only the memory used to convey
Sequential Read’s to the RDBMS (thus rbr for release-behind-read).
– Unlike lrud, rbr is selective: It does no scanning of the buffer cache !!!
– rbr only works when “sequential reading of a file in this file system is detected”.
Thereafter, only “the real memory pages used by the file will be released once the
pages are copied to internal buffers”. These internal buffers can be the RDBMS itself.
Result:
– Sequential Reads of a mount –o rbr <…> JFS2 filesystem are not buffer-cached.
– This also means the Random Reads are buffer-cached for read-rehits.
– The kernel processing overhead of lrud fr:sr is substantially reduced.
– Overall SAN IO performance/throughput is noticeably improved with The Tractor.
© Copyright IBM Corporation 2012
17
An AIXperftuning tactic: The Tractor for move-the-data Sequential Read
workloads – Other good tactics to go along with mount –o rbr <…>
Construct your LUN/hdiskVG/LV/JFS2 filesystems for the best IO performance
– Ask your storage admin for a RAID5/6/10 LUN-map to answer: Which LUNs share the same
RAIDset, (and which one’s don’t), etc.?
– Do not share and reshare the same /dev/loglv01…99 log devices with more than one JFS2
filesystem. That is, only ever assign dedicated /dev/loglv’s.
– As well, howsoever convenient, try not to use INLINE jfs2log devices; they are about 5% slower
than dedicated jfs2log devices.
– Create /dev/loglv01…99 on a different (not co-resident) set of RAID5 LUNs apart from its
associated data LUN/LV, (study all LUN/hdisk->LVM:vg->lv/JFS2 filesystem mappings)
– Universally adopt the use of mount –o noatime.
– Monitor&tune AIX:vmstat –v:pbuf|psbuf|fsbuf blocked IOs (see Part I, 39-47)
– Consider using the counterpart mount –o rbw <…> for Sequential Write JFS2 filesystem
workloads. Most RDBMS have a Write-Once-Only Sequential-Write logging mechanism; when
there is no chance of JFS2 write-rehits for Sequential-Write’s, rbw is likely appropriate.
© Copyright IBM Corporation 2012
18
Criteria for Creating a Write-Expedient pagingspace_vg
The first priority should be to preclude any pagingspace-pageouts. Thus, a write-expedient pagingspace is only needed
if you have any unavoidable pagingspace-pageout activity. Ultimately, if we must suffer any pagingspace-pageouts,
we want them to write-out to the pagingspace as quickly as possible (thus my term: write-expedient).
So, for the sake of prudence, we should always create a write-expedient pagingspace. The listed traits below are
optimal for write-expediency; include as many as you can (but always apply the key tuning tactic below):
•
Create a dedicated AIX:LVM:vg (VolumeGroup) called pagingspace_vg
•
Create the pagingspace_vg using FC-SAN storage LUNs (ideally RAID5 LUNs on SSD, FC or SAS technology disk drives,
and not on SATA disk drives (which are slower and employs RAID6), nor on any local/internal SAS disks)
•
The total size of the pagingspace in pagingspace_vg should match the size of installed LPAR gbRAM
•
Assign 3-to-8 LUN/hdisks to pagingspace_vg and size each LUN to be an even fraction of installed gbRAM. For instance, if
the LPAR has 18gbRAM, then assign three 6gb LUN/hdisks to pagingspace_vg
•
Configure one AIX:LVM:VG:lv (logical volume) for each LUN/hdisk in pagingspace_vg; do not deploy PP-striping
(because it messes-up discrete hdisk IO monitoring) –- just map one hdisk to one lv
•
The key tuning tactic: With root-user privileges, use AIX:lvmo to set pagingspace_vg:pv_pbuf_count=2048.
This will ensure pagingspace_vg:total_vg_pbufs will equal [<VGLUNcount> * pv_pbuf_count].
•
To set the pv_pbuf_count value to 2048, type the following:
lvmo -v pagingspace_vg -o pv_pbuf_count=2048
© Copyright IBM Corporation 2012
19
vmstat –I 2 # Best 6-in-1 monitor; no-load leave-it-up all-day VMM monitor
kthr
Number of kernel threads in various queues averaged per second over the sampling
interval. The kthr columns are as follows:
r
Average number of kernel threads that are runnable, which includes threads that
are running and threads that are waiting for the CPU. If this number is greater
than the number of CPUs, then there is at least one thread waiting for a CPU
and the more threads there are waiting for CPUs, the greater the likelihood of a
performance impact.
b
Average number of kernel threads in the VMM wait queue per second. This
includes threads that are waiting on filesystem I/O or threads that are blocking
on a shared resource, i.e. inode-lock.
p
For vmstat -I The number of threads waiting on I/Os to raw devices per second.
Threads waiting on I/Os to filesystems would not be included here.
© Copyright IBM Corporation 2012
20
vmstat –I 2 # Best 6-in-1 monitor; no-load leave-it-up all-day VMM monitor
memory
Provides information about the real and virtual memory.
avm
The Active Virtual Memory, avm, column represents the number of active virtual
memory pages present at the time the vmstat sample was collected. It is the
sum-total of all computational memory – including content paged-out to the
pagingspace. The avm statistics do not include file pages.
fre
The fre column shows the average number of free memory pages. A page is a 4
KB area of real memory. The system maintains a buffer of memory pages, called
the free list, that will be readily accessible when the VMM needs space. The
minimum number of pages that the VMM keeps on the free list is determined by
the minfree parameter of the vmo command.
© Copyright IBM Corporation 2012
21
vmstat –I 2 # Best 6-in-1 monitor; no-load leave-it-up all-day VMM monitor
page [ fi and fo are only included with vmstat –I ]
Information about page faults and paging activity. These are averaged over the
interval and given in units per second.
fi
The fi column details the number of pages paged-in from persistent storage, i.e.
pages read-in from JFS/JFS2 file systems on disk. This does not include
pagingspace-pagein’s from the pagingspace; rather, these are filesystem-reads.
fo
The fo column details the number of pages paged-out to persistent storage, i.e.
pages written-out to JFS/JFS2 file systems on disk. This does not include
pagingspace-pageout’s to the pagingspace; rather, these are filesystem-writes.
© Copyright IBM Corporation 2012
22
vmstat –I 2 # Best 6-in-1 monitor; no-load leave-it-up all-day VMM monitor
Page (continued)
Information about page faults and paging activity. These are averaged over the interval and given in units per
second.
pi
The pi column details the number of pages paged in from paging space. Paging space is the part of virtual
memory that resides on disk. It is used as an overflow when memory is over committed. Paging space
consists of logical volumes dedicated to the storage of working set pages that have been stolen from
real memory. When a stolen page is referenced by the process, a page fault occurs, and the page must
be read into memory from paging space.
Due to the variety of configurations of hardware, software and applications, there is no absolute number to
look out for. This field is important as a key indicator of paging-space activity. If a page-in occurs, there
must have been a previous page-out for that page. It is also likely in a memory-constrained environment
that each page-in will force a different page to be stolen and, therefore, paged out.
po
The po column shows the number (rate) of pages paged out to paging space. Whenever a page of working
storage is stolen, it is written to paging space, if it does not yet reside in paging space or if it was
modified. If not referenced again, it will remain on the paging device until the process terminates or
disclaims the space. Subsequent references to addresses contained within the faulted-out pages results
in page faults, and the pages are paged in individually by the system. When a process terminates
normally, any paging space allocated to that process is freed. If the system is reading in a significant
number of persistent pages, you might see an increase in po without corresponding increases in pi. This
does not necessarily indicate thrashing, but may warrant investigation into data-access patterns of the
applications.
© Copyright IBM Corporation 2012
23
vmstat –I 2 # Best 6-in-1 monitor; no-load leave-it-up all-day VMM monitor
page (continued)
Information about page faults and paging activity. These are averaged over the interval and
given in units per second.
fr
Number of pages that were freed per second by the page-replacement algorithm during the
interval. As the VMM page-replacement routine scans the Page Frame Table, or PFT, it
uses criteria to select which pages are to be stolen to replenish the free list of available
memory frames. The criteria include both kinds of pages, working (computational) and file
(persistent) pages. Just because a page has been freed, it does not mean that any I/O has
taken place. For example, if a persistent storage (file) page has not been modified, it will
not be written back to the disk. If I/O is not necessary, minimal system resources are
required to free a page.
sr
Number of pages that were examined per second by the page-replacement algorithm during
the interval. The page-replacement algorithm might have to scan many page frames
before it can steal enough to satisfy the page-replacement thresholds. The higher the sr
value compared to the fr value, the harder it is for the page-replacement algorithm to find
eligible pages to steal.
© Copyright IBM Corporation 2012
24
vmstat –I 2 # Best 6-in-1 monitor; no-load leave-it-up all-day VMM monitor
faults
Information about process control, such as trap and interrupt rate. The faults columns are as follows:
in
Number of device interrupts per second observed in the interval.
sy
The number of system calls per second observed in the interval. Resources are available to user
processes through well-defined system calls. These calls instruct the kernel to perform
operations for the calling process and exchange data between the kernel and the process.
Because workloads and applications vary widely, and different calls perform different functions, it
is impossible to define how many system calls per-second are too many. But typically, when the
sy column raises over 10000 calls per second on a uniprocessor, further investigations is called
for (on an SMP system the number is 10000 calls per second per processor). One reason could
be "polling" subroutines like the select() subroutine. For this column, it is advisable to have a
baseline measurement that gives a count for a normal sy value.
cs
Number of context switches per second observed in the interval. The physical CPU resource is
subdivided into logical time slices of 10 milliseconds each. Assuming a thread is scheduled for
execution, it will run until its time slice expires, until it is preempted, or until it voluntarily gives up
control of the CPU. When another thread is given control of the CPU, the context or working
environment of the previous thread must be saved and the context of the current thread must be
loaded. The operating system has a very efficient context switching procedure, so each switch is
inexpensive in terms of resources. Any significant increase in context switches, such as when cs
is a lot higher than the disk I/O and network packet rate, should be cause for further
investigation.
© Copyright IBM Corporation 2012
25
vmstat –I 2 # Best 6-in-1 monitor; no-load leave-it-up all-day VMM monitor
cpu
Percentage breakdown of CPU time usage during the interval. The cpu columns are as follows:
us
The us column shows the percent of CPU time spent in user mode. A UNIX® process can execute in either user
mode or system (kernel) mode. When in user mode, a process executes within its application code and does
not require kernel resources to perform computations, manage memory, or set variables.
sy
The sy column details the percentage of time the CPU was executing a process in system mode. This includes
CPU resource consumed by kernel processes (kprocs) and others that need access to kernel resources. If a
process needs kernel resources, it must execute a system call and is thereby switched to system mode to
make that resource available. For example, reading or writing of a file requires kernel resources to open the file,
seek a specific location, and read or write data, unless memory mapped files are used.
id
The id column shows the percentage of time which the CPU is idle, or waiting, without pending local disk I/O. If
there are no threads available for execution (the run queue is empty), the system dispatches a thread called
wait, which is also known as the idle kproc. On an SMP system, one wait thread per processor can be
dispatched. The report generated by the ps command (with the -k or -g 0 option) identifies this as kproc or wait.
If the ps report shows a high aggregate time for this thread, it means there were significant periods of time
when no other thread was ready to run or waiting to be executed on the CPU. The system was therefore mostly
idle and waiting for new tasks.
wa
The wa column details the percentage of time the CPU was idle with pending local disk I/O and NFS-mounted
disks. If there is at least one outstanding I/O to a disk when wait is running, the time is classified as waiting for
I/O. Unless asynchronous I/O is being used by the process, an I/O request to disk causes the calling process to
block (or sleep) until the request has been completed. Once an I/O request for a process completes, it is placed
on the run queue. If the I/Os were completing faster, more CPU time could be used.
A wa value over 25 percent could indicate that the disk subsystem might not be balanced properly, or it might be
the result of a disk-intensive workload.
© Copyright IBM Corporation 2012
26
Exercise&experiment with the JFS2 default mount and Raw I/O
By default, file pages can be cached in real memory for file systems. The caching can be disabled using direct
I/O or concurrent I/O mount options; also, the Release-Behind mount options can be used to quickly
discard file pages from memory after they have been copied to the application's I/O buffers if the readahead and write-behind benefits of cached file systems are needed.
JFS2 default mount -- AIX uses file caching as the default method of file access. However, file caching
consumes more CPU and significant system memory because of data duplication. The file buffer cache
can improve I/O performance for workloads with a high cache-hit ratio. And file system readahead can
help database applications that do a lot of table scans for tables that are much larger than the database
buffer cache.
Raw I/O -- Database applications traditionally use raw logical volumes instead of the file system for
performance reasons. Writes to a raw device bypass the caching, logging, and inode locks that are
associated with the file system; data gets transferred directly from the application buffer cache to the disk.
If an application is update-intensive with small I/O requests, then a raw device setup for database data and
logging can help performance and reduce the usage of memory resources.
© Copyright IBM Corporation 2012
27
Exercise&experiment with the JFS2 Direct I/O and Concurrent I/O mount options
By default, file pages can be cached in real memory for file systems. The caching can be disabled using direct
I/O or concurrent I/O mount options; also, the Release-Behind mount options can be used to quickly
discard file pages from memory after they have been copied to the application's I/O buffers if the readahead and write-behind benefits of cached file systems are needed.
• Direct I/O – DIO is similar to rawIO except it is supported under a file system. DIO bypasses the file
system buffer cache, which reduces CPU overhead and makes more memory available to others (that is, to
the database instance). DIO has similar performance benefit as rawIO but is easier to maintain for the
purposes of system administration. DIO is pro-vided for applications that need to bypass the buffering of
memory within the file system cache. For instance, some technical workloads never reuse data because of
the sequential nature of their data access. This lack of data reuse results in a poor buffer cache hit rate,
which means that these workloads are good candidates for DIO.
• Concurrent I/O -- CIO supports concurrent file access to files. In addition to bypassing the file cache, it
also bypasses the inode lock that allows multiple threads to perform reads and writes simultaneously on a
shared file. CIO is designed for relational database applications, most of which will operate under CIO
without any modification. Applications that do not enforce serialization for access to shared files should not
use CIO. Applications that issue a large amount of reads usually will not benefit from CIO either.
© Copyright IBM Corporation 2012
28
Exercise&experiment with the JFS2 Release-Behind Read/Write mechanisms
Release-behind-read and release-behind-write allow the file system to release the file pages from file system
buffer cache as soon as an application has read or written the file pages. This feature helps the
performance when an application performs a great deal of sequential reads or writes. Most often, these file
pages will not be reassessed after they are accessed.
Without this option, the memory will still be occupied with no benefit of reuse, which causes paging eventually
after a long run. When writing a large file without using release-behind, writes will go very fast as long as
pages are available on the free list. When the number of pages drops to minfree, VMM uses its LRU
algorithm to find candidate pages for eviction.
This feature can be configured on a file system basis. When using the mount command, enable releasebehind by specifying one of the three flags below:
– The release-behind sequential read flag (rbr)
– The release-behind sequential write flag (rbw)
– The release-behind sequential read and write flag (rbrw)
A trade-off of using the release-behind mechanism is that the application can experience an increase in CPU
utilization for the same read or write throughput rate (as compared to not using release-behind). This is
because of the work required to free the pages, which is normally handled at a later time by the LRU
daemon. Also note that all sequential IO file page accesses result in disk I/O because sequential IO file
data is not cached by VMM. However, applications (especially long-running applications) with the releasebehind mechanism applied are still likely to perform more optimally and with greater stability.
© Copyright IBM Corporation 2012
29
Appendix: AIXperfdataScript 2012Oct30.txt
(page 1)
#!/bin/ksh -x
#
#
#
#
#
#
#
#
#
Assumes IBM System P POWER4-->POWER7+ and AIX 5.3/6.1/7.1
Earl Jew -- Senior IT Management Consultant - IBM Power Systems and IBM Systems Storage
IBM Lab Services and Training - US Power Systems (group/dept)
IBM Certified Technical Sales Specialist - Power Systems with POWER7 and AIX - V1
IBM Certified Specialist - Midrange Storage Technical Support V2
IBM Certified Specialist - Enterprise Storage Technical Support V2
400 North Brand Blvd., c/o IBM 8th floor, Glendale, CA 91203
[email protected] (310) 251-2907 cell
Version: October 30, 2012
# Mundane Performance Data Collection script:
# NOTE: There is a subsection of rootuser commands in this script.
# Please execute for data-collection and send the collection to [email protected],
# and I will review and offer my findings by telephone/concall.
# Please execute this script when there is an active workload of concern.
# The script below collects 500kb-20mb of textdata per run.
#================================================================================
date
uname -a
id
oslevel -s
lparstat -i
uptime
vmstat -s
vmstat -v
vmstat -Iwt 1 80
ps -ekf | grep -v egrep | egrep "syncd|lrud|nfsd|biod|wait|getty|xmwlm“
…
© Copyright IBM Corporation 2012
30
Appendix: AIXperfdataScript 2012Oct30.txt
(page 2)
…
ipcs -bm
lsps -a
lsps -s
lssrad -av
mount
df -k
cat /etc/filesystems
cat /etc/xtab
showmount
prtconf
ps -el | wc
ps -elmo THREAD | wc
ps -kl | wc
ps -klmo THREAD | wc
nfsstat
##### BEGIN rootuser-privileges section
vmo -L
# requires root-user to execute; makes no changes
ioo -L
# requires root-user to execute; makes no changes
no -L
# requires root-user to execute; makes no changes
nfso -L
# requires root-user to execute; makes no changes
schedo -L
# requires root-user to execute; makes no changes
raso -L
# requires root-user to execute; makes no changes
lvmo -L
for VG in `lsvg`
do
lvmo -a -v $VG
echo
done
…
# requires root-user to execute; makes no changes
# requires root-user to execute; makes no changes
© Copyright IBM Corporation 2012
31
Appendix: AIXperfdataScript 2012Oct30.txt
…
uptime
sar -a 2 40
sar -b 2 40
sar -c 2 40
sar -k 2 40
sar -d 2 40
##### END rootuser-privileges section
# requires root-user
# requires root-user
# requires root-user
# requires root-user
# requires root-user
to execute;
to execute;
to execute;
to execute;
to execute;
(page 3)
makes no changes
makes no changes
makes no changes
makes no changes
makes no changes
aioo -a
lsdev
lscfg
lsconf
uptime
vmstat -Iwt 1 80
uptime
mpstat -w 2 40
uptime
mpstat -dw 2 40
uptime
mpstat -i 2 40
mpstat -w 2 40
vmstat -f
vmstat -i
nfso -a
lspv
for VG in `lsvg`
do
lsvg $VG ; echo
lsvg -p $VG ; echo ; echo ; echo
done
echo "\n\n============== ps -ef ==============================================================="
ps -ef
echo "\n\n============== ps -kf ==============================================================="
ps –kf
…
© Copyright IBM Corporation 2012
32
Appendix: AIXperfdataScript 2012Oct30.txt
(page 4)
echo "\n\n============== ps -el ==============================================================="
ps -el
echo "\n\n============== ps -kl ==============================================================="
ps -kl
echo "\n\n============== ps -elmo THREAD ======================================================"
ps -elmo THREAD
echo "\n\n============== ps -klmo THREAD ======================================================"
ps -klmo THREAD
echo "\n\n============== ps guww =============================================================="
ps guww
echo "\n\n============== ps gvww =============================================================="
ps gvww
echo "\n\n============================================================================="
echo "\n\n============================================================================="
ifconfig -a
netstat -ss
netstat -in
netstat -rn
netstat -m
netstat -v
netstat -c
netstat -C
netstat -D
netstat -s
netstat -M
netstat -A
iostat -a
iostat -s
uptime
iostat -aT 2 40 | grep -v "0
0.0
0
0
iostat -mT 2 40 | grep -v "0.0
0.0
0.0
iostat -AQ 2 40 | grep -v "
0
"
iostat -DRTl 60 4
0.0
0
0.0"
0" | grep -v "tm_act"
uptime
vmstat –s ; vmstat -v
vmstat -Iwt 1 80 ; date ; id ; uname -a
© Copyright IBM Corporation 2012
33
Session Evaluations
• ibmtechu.com/vp
Prizes will be
drawn from
Evals
© Copyright IBM Corporation 2012
34
© Copyright IBM Corporation 2012
35
2011
IBM Power Systems Technical University
October 10-14 | Fontainebleau Miami Beach | Miami, FL
򔻐򗗠򙳰
Thank you
Earl Jew ([email protected])
310-251-2907 cell
Senior IT Management Consultant - IBM Power Systems and IBM Systems Storage
IBM Lab Services and Training - US Power Systems (group/dept)
400 North Brand Blvd., c/o IBM 8th floor, Glendale, CA 91203
© Copyright IBM Corporation 2012
Materials may not be reproduced in whole or in part without the prior written permission of IBM.
5.3
Trademarks
The following are trademarks of the International Business Machines Corporation in the United States, other countries, or both.
Not all common law marks used by IBM are listed on this page. Failure of a mark to appear does not mean that IBM does not use the mark nor does it mean that the product is not
actively marketed or is not significant within its relevant market.
Those trademarks followed by ® are registered trademarks of IBM in the United States; all others are trademarks or common law marks of IBM in the United States.
For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml:
*, AS/400®, e business(logo)®, DBE, ESCO, eServer, FICON, IBM®, IBM (logo)®, iSeries®, MVS, OS/390®, pSeries®, RS/6000®, S/30, VM/ESA®, VSE/ESA,
WebSphere®, xSeries®, z/OS®, zSeries®, z/VM®, System i, System i5, System p, System p5, System x, System z, System z9®, BladeCenter®
The following are trademarks or registered trademarks of other companies.
Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.
Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom.
Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.
Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.
Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel
Corporation or its subsidiaries in the United States and other countries.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.
ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office.
IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency, which is now part of the Office of Government Commerce.
* All other products may be trademarks or registered trademarks of their respective companies.
Notes:
Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will
experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed.
Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here.
IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.
All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual
environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.
This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without
notice. Consult your local IBM business contact for information on the product or services available in your area.
All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.
Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance,
compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.
Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.
37
4-Apr-13
© Copyright IBM Corporation 2012
37
Disclaimers
No part of this document may be reproduced or transmitted in any form without written permission from IBM
Corporation.
Product data has been reviewed for accuracy as of the date of initial publication. Product data is subject to change
without notice. This information could include technical inaccuracies or typographical errors. IBM may make
improvements and/or changes in the product(s) and/or program(s) at any time without notice. Any statements
regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals
and objectives only.
The performance data contained herein was obtained in a controlled, isolated environment. Actual results that may be
obtained in other operating environments may vary significantly. While IBM has reviewed each item for accuracy in a
specific situation, there is no guarantee that the same or similar results will be obtained elsewhere. Customer
experiences described herein are based upon information and opinions provided by the customer. The same results
may not be obtained by every user.
Reference in this document to IBM products, programs, or services does not imply that IBM intends to make such
products, programs or services available in all countries in which IBM operates or does business. Any reference to
an IBM Program Product in this document is not intended to state or imply that only that program product may be
used. Any functionally equivalent program, that does not infringe IBM's intellectual property rights, may be used
instead. It is the user's responsibility to evaluate and verify the operation on any non-IBM product, program or
service.
THE INFORMATION PROVIDED IN THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER
EXPRESS OR IMPLIED. IBM EXPRESSLY DISCLAIMS ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE OR INFRINGEMENT. IBM shall have no responsibility to update this information. IBM
products are warranted according to the terms and conditions of the agreements (e.g. IBM Customer Agreement,
Statement of Limited Warranty, International Program License Agreement, etc.) under which they are provided. IBM
is not responsible for the performance or interoperability of any non-IBM products discussed herein.
© Copyright IBM Corporation 2012
38
Disclaimers (Continued)
Information concerning non-IBM products was obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not tested those products in connection with this
publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM
products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those
products.
The providing of the information contained herein is not intended to, and does not, grant any right or license under any
IBM patents or copyrights. Inquiries regarding patent or copyright licenses should be made, in writing, to:
IBM Director of Licensing
IBM Corporation
North Castle Drive
Armonk, NY 10504-1785
USA
IBM customers are responsible for ensuring their own compliance with legal requirements. It is the customer's sole
responsibility to obtain advice of competent legal counsel as to the identification and interpretation of any relevant
laws and regulatory requirements that may affect the customer's business and any actions the customer may need
to take to comply with such laws.
IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is
in compliance with any law.
The information contained in this documentation is provided for informational purposes only. While efforts were made
to verify the completeness and accuracy of the information provided, it is provided “as is” without warranty of any
kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related
to, this documentation or any other documentation. Nothing contained in this documentation is intended to, nor
shall have the effect of, creating any warranties or representations from IBM (or its suppliers or licensors), or altering
the terms and conditions of the applicable license agreement governing the use of IBM software.
© Copyright IBM Corporation 2012
39