1. No category

High
Availability
Continuous
Availability
Continuous
Operations
How to connect
Linux to a storage unit
PSSC - IBM Customer Center Montpellier
Guillaume CRESTA
1
France
Table of Contents
1
Introduction ........................................................................................................................ 3
Presentation of the infrastructure .......................................................3
2
How to configure................................................................................................................ 4
2.1
2.2
2.3
2.4
2.5
2.6
3
Gathering the World Wide Name..............................................4
Configuring the unit storage......................................................5
Configuration at the SAN level: zoning....................................9
The operating system: Linux...................................................12
Script........................................................................................20
Dscli .........................................................................................21
Special Thanks ................................................................................................................. 23
2
1 Introduction
In order to answer to some of customers’ main concerns and to provide them with answers
based on real testing, we are implementing a two sites long distance disaster recovery solution
between Montpellier (France) and Poughkeepsie (USA) that is 7000 km between the
production site and the backup site. Within the framework of this project, an integration of
System x has been effected. The purpose of this document is to describe the configuration of
the System x with RedHat Enterprise Linux 4.0 and the storage unit DS8000 o DS6000 based
on this real experience.
In order to connect these two technologies, we ran different steps explained all over the
document. The last part of this document is dedicated to the persons who help me.
Presentation of the infrastructure
The main components of the architecture are:
IBM System p servers
IBM System x servers
IBM DS8000 and DS6000 disk subsystems
MCDATA SAN switches
MCDATA iFCP router 2027-R16
The hardware environment we are using for this test is dispatched in 2 sites (one site in
Montpellier-France and one site in Poughkeepsie-US).
3
2 How to configure
Different steps have to be done in order to attach a Linux (RedHat Enterprise Linux release
4.0) to a DS8000 storage unit:
gather the World Wide Name, unique identifier of each fiber card and that permits
the identification of the System x on storage unit
configure the storage
make configuration at SAN level
install sdd service
to configure Linux in order to show disks
make starting script and at the end
install the dscli tool in order to communicate with storage unit.
2.1 Gathering the World Wide Name
First we have to see the compatibility matrix in order to verify if the Linux version installed is
in
agreement
to
the
storage
unit
(http://www-
03.ibm.com/servers/storage/disk/ds8000/pdf/ds8000-matrix.pdf). As far as that goes, we have
to know the Linux version and his kernel:
[root@halinux Desktop]# more /etc/redhat-release
Red Hat Enterprise Linux AS release 4 (Nahant Update 3)
[root@halinux Desktop]# uname -a
Linux halinux 2.6.9-34.ELsmp #1 SMP Fri Feb 24 16:54:53 EST 2006 i686 i686
i386 GNU/Linux
Then, in order to make zoning, we need the WWPN of fiber channel card of the System x:
[root@halinux Desktop]# cd /proc/scsi/qla2xxx/
[root@halinux qla2xxx]# ls
1
[root@halinux qla2xxx]# cat 1
QLogic PCI to Fibre Channel Host Adapter for QLA2310:
Firmware version 3.03.18 IPX, Driver version 8.01.02-d4
ISP: ISP2300, Serial# D76762
Request Queue = 0x1bc0000, Response Queue = 0x37a00000
Request Queue count = 2048, Response Queue count = 512
Total number of active commands = 0
Total number of interrupts = 2781
Device queue depth = 0x20
Number of free request entries = 784
Number of mailbox timeouts = 0
Number of ISP aborts = 0
Number of loop resyncs = 0
Number of retries for empty slots = 0
Number of reqs in pending_q= 0, retry_q= 0, done_q= 0, scsi_retry_q= 0
Host adapter:loop state = <READY>, flags = 0x1a03
4
Dpc flags = 0x4000000
MBX flags = 0x0
Link down Timeout = 000
Port down retry = 030
Login retry count = 030
Commands retried with dropped frame(s) = 0
Product ID = 4953 5020 2020 0001
SCSI Device Information:
scsi-qla0-adapter-node=200000e08b05babf;
scsi-qla0-adapter-port=210000e08b05babf;
scsi-qla0-target-0=50050763031080b7;
scsi-qla0-target-1=50050763031bc0b7;
FC Port Information:
scsi-qla0-port-0=5005076303ffc0b7:50050763031080b7:740d13:81;
scsi-qla0-port-1=5005076303ffc0b7:50050763031bc0b7:740f13:82;
SCSI LUN Information:
(Id:Lun) * - indicates lun is not registered with the OS.
( 0: 0): Total reqs 1674, Pending reqs 0, flags 0x0, 0:0:81 00
( 1: 0): Total reqs 1587, Pending reqs 0, flags 0x0, 0:0:82 00
[root@halinux qla2xxx]#
The LUNs mapped to the server can be verified by checking the contains of the file
/proc/scsi/scsi:
[root@halinux ~]# cat /proc/scsi/scsi
Host:SCSI0
Channel:00
Id:00
Lun:00
Vendor: IBM-ESXS
Model:CBR036C31210ESFN
Rev: DFQD
Type: Direct-Access
ANSI SCSI revision:04
Host:SCSI0
Channel:00
Id:08
Lun:00
Vendor: IBM
Model:FT1V1S2
Rev: 0
Type: Processor
ANSI SCSI revision:02
Host:SCSI1
Channel:00
Id:00
Lun:00
Vendor: IBM-ESXS
Model:2107900
Rev: 201
Type: Direct-Access
ANSI SCSI revision:05
Host:SCSI1
Channel:00
Id:01
Lun:00
Vendor: IBM-ESXS
Model:2107900
Rev: 201
Type: Direct-Access
ANSI SCSI revision:05
2.2 Configuring the unit storage
The configuration of the unit storage can be done by GUI (web interface) or by CLI
(commands line). I myself prefer working with commands line. The working is the same with
a DS6000 or with a DS8000. The below extracts correspond to the creation on DS8000.
5
To create host on storage unit permitting relation between Linux and storage unit, the World
Wide Port Name mentioned on Linux (210000E08B05BABF), the type of operating system
used (LinuxRHEL) and the DS ports used (all in order to mention all available):
dscli mkhostconnect -wwname 210000E08B05BABF
-hosttype LinuxRHEL -ioport
all 'HA1Linux' -cfg C:\Progra~1\IBM\dscli\profile\DS8000_HA.profile
Date/Time: 4 ao?t 2006 14:25:30 CEST IBM DSCLI Version: 5.1.600.260 DS:
IBM.2107-7506551
CMUC00012I mkhostconnect: Host connection 0013 successfully created.
Date/Time: 08 04, 2006 14:25:39
Execution time :
12 Seconds.
The configuration made can be verified:
dscli lshostconnect -l
-cfg
C:\Progra~1\IBM\dscli\profile\DS8000_HA.profile
Date/Time: 25 juillet 2006 18:42:57 CEST IBM DSCLI Version: 5.1.600.260 DS:
IBM.2107-7506551
Name
ID
WWPN
HostType LBS addrDiscovery
Profile
portgrp volgrpID atchtopo ESSIOport
===========================================================================
===========================================================================
===========================================================================
===========================================================================
==============================================================
BSBI_P53
0000 10000000C93EC6FE pSeries
512 reportLUN
IBM pSeries - AIX
1 FC-AL
all
[…]
HA1Linux
0013 210000E08B05BABF LinuxRHEL 512 LUNPolling
Intel - Linux RHEL
0 all
[…]
SVC_SpeederMan_EW
0042 50050768014003EA SVC
512 reportLUN
San Volume Controller
0 V2
all
Date/Time: 07 25, 2006 18:43:06
Execution time :
11 Seconds.
Moreover, a space on storage unit called Volumes is created, specifying the capacity (5 Gb),
the volume name (Vol1Halinux) and a number available (47000) and the extend pool on
which it’s going to be created (P3). Before, we need to verify that the extend pool has
allocated space.
dscli lsextpool -l
-cfg C:\Progra~1\IBM\dscli\profile\DS8000_HA.profile
Date/Time: 4 ao?t 2006 14:29:36 CEST IBM DSCLI Version: 5.1.600.260 DS:
IBM.2107-7506551
Name
ID stgtype rankgrp status availstor (2^30B) %allocated
available reserved numvols numranks
===========================================================================
=============================
extpool_ha0
P0 fb
0 below
650
16
650
0
8
1
extpool_ha1
P1 fb
1 below
679
12
679
0
4
1
extpool_ha2
P2 fb
0 below
859
5
859
0
2
1
6
extpool_ha3
P3 fb
1 below
729
0
2
1
[…]
Date/Time: 08 04, 2006 14:29:51
Execution time :
17 Seconds.
729
6
dscli mkfbvol -extpool P3 -cap 5 -name 'Vol1Halinux' 4700 -cfg
C:\Progra~1\IBM\dscli\profile\DS8000_HA.profile
Date/Time: 4 ao?t 2006 14:30:45 CEST IBM DSCLI Version: 5.1.600.260 DS:
IBM.2107-7506551
CMUC00025I mkfbvol: FB volume 4700 successfully created.
Date/Time: 08 04, 2006 14:30:55
Execution time :
12 Seconds.
When the volumes are established they are grouped in a volume group. In this case, there’s
only one volume. But, in order to connect the host to the volume, a volume group need to be
created. When the volume group is created by command line interface, the type SCSIMASK o
SCSIMap256 can be configured. By default, if you put SCSIMask, the relation between
volume group and host configured as a RedHat EL is impossible in this case. In order to make
a volume group, the type (scsimpa256 in case of a Linux RedHatEL), the volumes number
gathered in the volume group (4700) and volume group name (VolGrpe1HALinux) have to be
specified:
dscli mkvolgrp -type scsimap256 -volume 4700 'Volgrpe1HALinux' -cfg
C:\Progra~1\IBM\dscli\profile\DS8000_HA.profile
Date/Time: 4 ao?t 2006 14:31:45 CEST IBM DSCLI Version: 5.1.600.260 DS:
IBM.2107-7506551
CMUC00030I mkvolgrp: Volume group V4 successfully created.
Date/Time: 08 04, 2006 14:31:53
Execution time :
10 Seconds.
The volume establishment by command line can be verified:
dscli lsvolgrp -l
-cfg C:\Progra~1\IBM\dscli\profile\DS8000_HA.profile
Date/Time: 4 ao?t 2006 14:22:27 CEST IBM DSCLI Version: 5.1.600.260 DS:
IBM.2107-7506551
Name
ID Type
========================================
FC_OracleCode
V0 SCSI Mask
HAVG1
V1 SCSI Mask
SVCDEMO_EW
V2 SCSI Mask
MED4PC12
V3 SCSI Map 256
VolGrp1_HALINUX
V4 SCSI Map 256
sdvolume
V5 SCSI Map 256
VGPC2
V6 SCSI Map 256
blue_aix_VG
V7 SCSI Mask
Volumes_for_AIX
V8 SCSI Mask
Date/Time: 08 04, 2006 14:22:36
Execution time :
12 Seconds.
7
The volume establishment can be verified and so that it is associated to the volume group V4:
dscli lsfbvol -l -cfg C:\Progra~1\IBM\dscli\profile\DS8000_HA.profile
Date/Time: 17 mai 2006 18:40:59 CEST IBM DSCLI Version: 5.1.600.260 DS:
IBM.2107-7506551
Name
ID
accstate datastate configstate deviceMTM datatype
extpool sam
captype cap (2^30B) cap (10^9B) cap (blocks) volgrp
===========================================================================
================================================================
03000
0300 Online
Normal
Normal
2107-900 FB 512
P7
Standard
DS
2.0
4194304 V18
[...]
Vol1Halinux
4700 Online
Normal
Normal
2107-900 FB 512
P3
Standard
DS
5.0
10485760 V4
[...]
volAIXLaurent
FB00 Online
Normal
Normal
2107-900 FB 512
P7
Standard
DS
1.0
2097152 Date/Time: 05 17, 2006 18:41:08
Execution time :
12 Seconds.
We need to associate the volume group to the host previously established, specifying the
volume group number (V4) and the host one (0013):
dscli chhostconnect -volgrp V4 0013 -cfg
C:\Progra~1\IBM\dscli\profile\DS8000_HA.profile
Date/Time: 4 ao?t 2006 14:32:43 CEST IBM DSCLI Version: 5.1.600.260 DS:
IBM.2107-7506551
CMUC00013I chhostconnect: Host connection 0013 successfully modified.
Date/Time: 08 04, 2006 14:32:51
Execution time :
10 Seconds.
It can be verified that this operation has been realized by watching the host configuration
again:
dscli lshostconnect -l
-cfg
C:\Progra~1\IBM\dscli\profile\DS8000_HA.profile
Date/Time: 25 juillet 2006 18:42:57 CEST IBM DSCLI Version: 5.1.600.260 DS:
IBM.2107-7506551
Name
ID
WWPN
HostType LBS addrDiscovery
Profile
portgrp volgrpID atchtopo ESSIOport
===========================================================================
===========================================================================
===========================================================================
===========================================================================
==============================================================
BSBI_P53
0000 10000000C93EC6FE pSeries
512 reportLUN
IBM pSeries - AIX
1 FC-AL
all
[…]
HA1Linux
0013 210000E08B05BABF LinuxRHEL 512 LUNPolling
Intel - Linux RHEL
0 V4
all
[…]
SVC_SpeederMan_EW
0042 50050768014003EA SVC
512 reportLUN
San Volume Controller
0 V2
all
Date/Time: 07 25, 2006 18:43:06
Execution time :
11 Seconds.
8
2.3 Configuration at the SAN level: zoning
In order to administrate the switches, a web application is available. Thanks to EFCM Basic,
we can configure switches by accessing to the menu Configure > Zoning and the page
figuring next page can be seen.
In the first column, all nodes are listed. The zone name is notified in the second column.
The World Wide Port Names have to be added by using the button >.
9
In the zone S1_LINUX, the System x World Wide Port Names and the unit storage World
Wide Port Names can be verified to be there. When this inspection is done, this zone has to be
joined to the zone set. The button > is used to realized it.
10
Now, the zone appears in the column Zone set. Thus, the modification is saved by the button
Update and applied by Activate.
The created zones are editable using the button < and the steps above remade.
The port used is configured. In this case, System x is connected on switch port 0. The fields
associated to this port are informed.
11
Thereafter, we can access to the list of the switch ports (Product > Port List). Clicking on
port 0, information linked to this port appear, for example le World Wide Port Name and the
World Wide Node Name.
The zoning and the storage unit configuration can be done in parallel. The order done doesn’t
matter.
2.4 The operating system: Linux
The integration of sdd service
After the DS6/8000 configuration, the operating operation has to be restarted. It’s necessary to
verify that the disks are in relation with the operating system using the fdisk -l command:
login as: root
Sent username "root"
[email protected]'s password:
Last login: Wed Aug 16 11:04:10 2006 from 10.8.1.1
[root@halinux ~]# fdisk -l
Disk /dev/sda: 36.4 GB, 36401479680 bytes
255 heads, 63 sectors/track, 4425 cylinders
12
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot
/dev/sda1
*
/dev/sda2
Start
1
14
End
13
4425
Blocks
104391
35439390
Id
83
8e
System
Linux
Linux LVM
Disk /dev/sdb: 5368 MB, 5368709120 bytes
166 heads, 62 sectors/track, 1018 cylinders
Units = cylinders of 10292 * 512 = 5269504 bytes
Disk /dev/sdb doesn't contain a valid partition table
Disk /dev/sdc: 5368 MB, 5368709120 bytes
166 heads, 62 sectors/track, 1018 cylinders
Units = cylinders of 10292 * 512 = 5269504 bytes
Disk /dev/sdc doesn't contain a valid partition table
If this command doesn’t work, the disks mount through the Fiber Channel card can be forced
by the command line: echo "scsi-qlascan" > /proc/scsi/qla2xxx/1.
The System x linked to the DS8000 has got only one card, and when the zoning is made, two
links are put to the storage unit. The disk also appears in double: /etc/sdb and /etc/sdc above.
Whereas on System x linked to the DS6000 there are two cards each linked to the storage unit
by two fibers, four disks appear as notified below (/etc/sdb, /etc/sdc, /etc/sdd and /etc/sde):
login as: root
Sent username "root"
[email protected]'s password:
Last login: Wed Aug 16 11:05:28 2006 from 10.8.1.1
[root@halinux2 ~]# fdisk -l
Disk /dev/sda: 36.4 GB, 36401479680 bytes
255 heads, 63 sectors/track, 4425 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot
/dev/sda1
*
/dev/sda2
Start
1
14
End
13
4425
Blocks
104391
35439390
Id
83
8e
System
Linux
Linux LVM
Disk /dev/sdb: 5368 MB, 5368709120 bytes
166 heads, 62 sectors/track, 1018 cylinders
Units = cylinders of 10292 * 512 = 5269504 bytes
Disk /dev/sdb doesn't contain a valid partition table
Disk /dev/sdc: 5368 MB, 5368709120 bytes
166 heads, 62 sectors/track, 1018 cylinders
Units = cylinders of 10292 * 512 = 5269504 bytes
Disk /dev/sdc doesn't contain a valid partition table
Disk /dev/sdd: 5368 MB, 5368709120 bytes
166 heads, 62 sectors/track, 1018 cylinders
Units = cylinders of 10292 * 512 = 5269504 bytes
Disk /dev/sdd doesn't contain a valid partition table
13
Disk /dev/sde: 5368 MB, 5368709120 bytes
166 heads, 62 sectors/track, 1018 cylinders
Units = cylinders of 10292 * 512 = 5269504 bytes
Disk /dev/sde doesn't contain a valid partition table
Thus, the file IBM.sdd-1.6.1.0-3.i686.rhel4.rpm is installed.
[root@halinux ~]# rpm -ivh IBMsdd-1.6.0.1-11.1.i686.rhel4.rpm
Preparing....
####################################################[100%]
1:IBMsdd
####################################################[100%]
Added following line to /etc/inittab:
srv:345:respawn:/opt/IBMsdd/bin/sddsrv > /dev/null 2>&1
In order to verify if it is correctly installed, /etc/inittab can be verified:
[root@halinux ~]# cat /etc/inittab | grep sdd
#IBMsdd path recovery daemon:
srv:345:respawn:/opt/IBMsdd/bin/sddsrv > /dev/null 2>&1
When done, you can verify the content of the package:
[root@halinux ~]# rpm -ql IBMsdd
/etc/cron.hourly/sddrv_log.sh
/etc/logrotate.d/sddsrv_log.d
/etc/sddsrv.conf
/etc/vpath.conf
/opt/IBMsdd
/opt/IBMsdd/bin
/opt/IBMsdd/bin/cfgvpath
/opt/IBMsdd/bin/datapath
/opt/IBMsdd/bin/lsvpcfg
/opt/IBMsdd/bin/make_sddcrs
/opt/IBMsdd/bin/pathtest
/opt/IBMsdd/bin/rmvpath
/opt/IBMsdd/bin/sdd_rcscript
/opt/IBMsdd/bin/sddgetdata
/opt/IBMsdd/bin/sddsrv
/opt/IBMsdd/kernel_list
/opt/IBMsdd/sdd-mod.ko-2.6.9-11.ELhugemen
/opt/IBMsdd/sdd-mod.ko-2.6.9-11.ELsmp
/opt/IBMsdd/sdd-mod.ko-2.6.9-22.0.1.ELhugemen
/opt/IBMsdd/sdd-mod.ko-2.6.9-22.0.1.ELsmp
/opt/IBMsdd/sdd-mod.ko-2.6.9-22.0.2.ELhugemen
/opt/IBMsdd/sdd-mod.ko-2.6.9-22.0.2.ELsmp
/opt/IBMsdd/sdd-mod.ko-2.6.9-22.ELhugemen
/opt/IBMsdd/sdd-mod.ko-2.6.9-22.ELsmp
/opt/IBMsdd/sdd-mod.ko-2.6.9-34.ELhugemen
/opt/IBMsdd/sdd-mod.ko-2.6.9-34.ELsmp
/usr/share/doc/IBMsdd-1.6.1.0/LICENSE
/usr/share/doc/IBMsdd-1.6.1.0/README
/usr/share/doc/IBMsdd-1.6.1.0/rd-linux.txt
This file is the last version of sdd for RedHat 4.0. When the file is installed, the status has to
be verified:
14
[root@halinux ~]# sdd status
IBMsdd status:
You have new mail in /var/spool/mail/root
[FAILED]
If the status is FAILED, the sdd service has to be started:
[root@halinux ~]# sdd start
Starting IBMsdd driver load:
[ OK ]
Issuing killall sddsrv to trigger respawn...
Starting IBMsdd configuration:
[ OK ]
[root@halinux ~]# sdd status
IBMsdd status:
[ OK ]
000 vpatha ( 252,
0) 75065514700 = 6005076303ffc0b70000000000004700 =
/dev/sdb /dev/sdc
In order to make multipathing, sdd make a vpath going to take into account the two disks and
manage the traffic between the two links.
The visualization of the disks
When an operating system Linux and especially sdd is configured, the configuration of the
file /etc/lvm/lvm.conf has to be modified. Indeed, if you don’t modify it, you can’t create a
vpath in order to establish multipathing. In order to resolve this problem two lines in this file
have to be modified:
filter = ["a/.*/"]
types = ["fd", 16]
by the lines:
filter = ["a|/dev/sda|", "r|/dev/sd|", "a/.*/"]
types = ["vpath", 16]
These lines permit Linux server not to filter the other disk as sda (principal disk which is
installed Linux). When this step is done, we can also show that the path is created:
[root@halinux ~]#cfgvpath query
/dev/sdb ( 8, 16) host=1 ch=0 id=0 lun=0
vid=IBM
pid=2107900
serial=75065514700
lun_id=6005076303ffc0b70000000000004700 ctlr_flag=0 ctlr_nbr=0 df_ctlr=0
/dev/sdc ( 8, 32) host=1 ch=0 id=1 lun=0
vid=IBM
pid=2107900
serial=75065514700
lun_id=6005076303ffc0b70000000000004700 ctlr_flag=0 ctlr_nbr=0 df_ctlr=0
When the scan disk is made, we can notice that a new disk is present and correspond to the
path group:
15
[root@halinux ~]# fdisk -l
……………………………
Disk /dev/vpatha: 5368 MB, 5368709120 bytes
166 heads, 62 sectors/track, 1018 cylinders
Units = cylinders of 10292 * 512 = 5269504 bytes
Disk /dev/vpatha doesn't contain a valid partition table
Then, we need to make a partition:
[root@halinux ~]# fdisk /dev/vpatha
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF
disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by
w(rite)
Command (m for help): n
Command action
e
extended
p
primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1018, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1018, default 1018):
Using default value 1018
Command (m for help): t
Selected partition 1
Hex code (type L to list codes): l
0 Empty
boot
1 FAT12
2 XENIX root
(FAT3 XENIX usr
(FAT4 FAT16 <32M
(FAT5 Extended
6 FAT16
7 HPFS/NTFS
/ .
8 AIX
Utility
9 AIX bootable
a OS/2 Boot Manag
b W95 FAT32
c W95 FAT32 (LBA)
e W95 FAT16 (LBA)
f W95 Ext'd (LBA)
10 OPUS
12/16/
1e
Hidden W95 FAT1 75
PC/IX
24
39
NEC DOS
Plan 9
80
81
Old Minix
bf
Minix / old Lin c1
Solaris
DRDOS/sec
3c
PartitionMagic
82
Linux swap
c4
DRDOS/sec
40
Venix 80286
83
Linux
c6
DRDOS/sec
41
42
4d
PPC PReP Boot
SFS
QNX4.x
84
85
86
OS/2 hidden C: c7
Linux extended da
NTFS volume set db
Syrinx
Non-FS data
CP/M / CTOS
4e
QNX4.x 2nd part 87
NTFS volume set de
Dell
4f
50
51
52
53
54
55
QNX4.x 3rd part
OnTrack DM
OnTrack DM6 Aux
CP/M
OnTrack DM6 Aux
OnTrackDM6
EZ-Drive
Linux LVM
df
Amoeba
e1
Amoeba BBT
e3
BSD/OS
e4
IBM Thinkpad hi eb
FreeBSD
ee
OpenBSD
ef
BootIt
DOS access
DOS R/O
SpeedStor
BeOS fs
EFI GPT
EFI (FAT-
8e
93
94
9f
a0
a5
a6
be
Solaris
16
11 Hidden FAT12
RISC b
12 Compaq diagnost
14 Hidden FAT16 <3
16 Hidden FAT16
secondary
17 Hidden HPFS/NTF
auto
18 AST SmartSleep
1b Hidden W95 FAT3
1c Hidden W95 FAT3
Hex code (type L to
Changed system type
56
Golden Bow
5c
61
63
a7
NeXTSTEP
f0
Linux/PA-
Priam Edisk
a8
SpeedStor
a9
GNU HURD or Sys ab
Darwin UFS
NetBSD
Darwin boot
f1
f4
f2
SpeedStor
SpeedStor
DOS
64
Novell Netware
BSDI fs
fd
Linux raid
65
70
Novell Netware b8
DiskSecure Mult bb
b7
BSDI swap
fe
Boot Wizard hid ff
LANstep
BBT
list codes): 8e
of partition 1 to 8e (Linux LVM)
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
Thanks to the new sdd service, the new commands are accessible which permit to see the
LUN, the relation between LSS and the volumes:
[root@halinux ~]# datapath query essmap
Disk
Path
P Adapter
LUN SN
Type
Vol
Rank
C/A S
Connection
Port RaidMode
--------- - ------------- ----------- ----------------- ----------- ---- -------vpatha
sdb
Host1Channel0 75065514700 IBM 2107900
00
0000
17
Y
R1-B3-H1-ZC 202
RAID5
vpatha
sdc
Host1Channel0 75065514700 IBM 2107900
00
0000
17
Y
R1-B4-H3-ZD 333
RAID5
Size
----
LSS
---
-
1.0GB
71
1.0GB
71
Paths
2
Active
0
[root@halinux ~]# datapath query adapter
Active Adapters :1
Adpt#
0
Name
Host1Channel0
State
NORMAL
Mode
ACTIVE
Select
136
Errors
0
[root@halinux ~]# datapath query device
Total Devices : 1
DEV#:
0 DEVICE NAME: vpatha TYPE: 2107900 POLICY: Optimized Sequential
SERIAL: 75065514700
===========================================================================
Path#
Adapter/Hard Disk
State
Mode
Select
Errors
0
Host1Channel0/sdb
CLOSE
NORMAL
53
0
1
Host1Channel0/sdc
CLOSE
NORMAL
83
0
To create physical volumes, volumes group, logical volumes, a serial of command has to be
made before mounting the disk on /halinux1:
[root@halinux ~]# pvcreate /dev/vpatha1
Physical volume "/dev/vpatha1" successfully created
[root@halinux Desktop]# pvscan
17
PV /dev/sda2
VG VolGroup00
lvm2 [33.78 GB / 64.00 MB free]
PV /dev/vpatha1
VG VGHALINUX1
lvm2 [4.99 GB / 4.00 MB free]
Total: 2 [38.77 GB] / in use: 2 [38.77 GB] / in no VG: 0 [0
]
[root@halinux ~]# vgcreate VGHALINUX1 /dev/vpatha1
Volume group "VGHALINUX1" successfully created
[root@halinux Desktop]# vgscan
Reading all physical volumes. This may take a while...
Found volume group "VolGroup00" using metadata type lvm2
Found volume group "VGHALINUX1" using metadata type lvm2
[root@halinux Desktop]#
--- Volume group --VG Name
System ID
Format
Metadata Areas
Metadata Sequence No
VG Access
VG Status
MAX LV
Cur LV
Open LV
Max PV
Cur PV
Act PV
VG Size
PE Size
Total PE
Alloc PE / Size
Free PE / Size
VG UUID
vgdisplay VGHALINUX1
VGHALINUX1
lvm2
1
2
read/write
resizable
0
1
1
0
1
1
4.99 GB
4.00 MB
1278
1277 / 4.99 GB
1 / 4.00 MB
5P3JKI-ztYt-eQ4p-zzAp-4tVF-jrJW-4ZK7PN
[root@halinux ~]# pvscan
PV /dev/vpatha1
VG VGHALINUX1
lvm2 [4.99 GB / 4.99 GB free]
Total: 1 [4.99 GB] / in use: 1 [4.99 GB] / in no VG: 0 [0
]
[root@halinux ~]# lvcreate VGHALINUX1 -n
Logical volume "lv_halinux1" created
lv_halinux1 -l 1277
[root@halinux Desktop]# lvscan
ACTIVE
'/dev/VolGroup00/LogVol00' [31.78 GB] inherit
ACTIVE
'/dev/VolGroup00/LogVol01' [1.94 GB] inherit
ACTIVE
'/dev/VGHALINUX1/lv_halinux1' [4.99 GB] inherit
[root@halinux /]# mkdir /halinux1
[root@halinux /]# ls
bin
etc
initrd
boot halinux1 lib
dev
home
lost+found
media
misc
mnt
opt
proc
root
sbin
selinux
srv
sys
tftpboot
tmp
usr
var
[root@halinux /]# mount
/dev/mapper/VolGroup00-LogVol00 on / type ext3 (rw)
none on /proc type proc (rw)
none on /sys type sysfs (rw)
none on /dev/pts type devpts (rw,gid=5,mode=620)
usbfs on /proc/bus/usb type usbfs (rw)
/dev/sda1 on /boot type ext3 (rw)
none on /dev/shm type tmpfs (rw)
18
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
[root@halinux ~]# lvscan
ACTIVE
'/dev/VolGroup00/LogVol00' [31.78 GB] inherit
ACTIVE
'/dev/VolGroup00/LogVol01' [1.94 GB] inherit
inactive
'/dev/VGHALINUX1/lv_halinux1' [4.99 GB] inherit
[root@halinux ~]# lvchange /dev/VGHALINUX1/lv_halinux1 -a y
[root@halinux ~]# lvscan
ACTIVE
'/dev/VolGroup00/LogVol00' [31.78 GB] inherit
ACTIVE
'/dev/VolGroup00/LogVol01' [1.94 GB] inherit
ACTIVE
'/dev/VGHALINUX1/lv_halinux1' [4.99 GB] inherit
[root@halinux ~]# mount
/dev/mapper/VolGroup00-LogVol00 on / type ext3 (rw)
none on /proc type proc (rw)
none on /sys type sysfs (rw)
none on /dev/pts type devpts (rw,gid=5,mode=620)
usbfs on /proc/bus/usb type usbfs (rw)
/dev/sda1 on /boot type ext3 (rw)
none on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
[root@halinux ~]# mkfs.ext3 /dev/VGHALINUX1/lv_halinux1
mke2fs 1.35 (28-Feb-2004)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
654080 inodes, 1307648 blocks
65382 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1342177280
40 block groups
32768 blocks per group, 32768 fragments per group
16352 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736
Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 30 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
[root@halinux ~]# mount /dev/VGHALINUX1/lv_halinux1 /halinux1/
[root@halinux ~]# cd /halinux1
[root@halinux halinux1]# ll
total 16
drwx------ 2 root root 16384 Aug 18 19:12 lost+found
19
2.5 Script
When path problem is resolved and Linux and unit storage configured, a starting script has to
be made. If you restart the system without made this step (in order to see the elements which
have been take into account), you can see that sdd service is not started by default. Then, the
administrator’s intervention on the system in order to start it, it is needed. However, the
logical volume created on Linux appeared after each starting with a status Inactive, and
needed a user’s operation in order to give it a status Active and then he had to make a mount.
You can realize a starting script permitting to restart the operating system without reconfigure
and without root’s intervention. Here is the script:
#!/bin/bash
#
# script_sdd
#
#
# description:
#
#
# processname:
This shell script takes care of starting and stopping
sdd and configurating Linux.
sdd is a service which is the program
that permits to make multipathing ; and lvchange to change
the logical volume status.
sdd
# Stop sdd
. /etc/init.d/sdd stop
# Start sdd
. /etc/init.d/sdd start
# Source sdd configuration.
# Physical volume scan
pvscan
# Volume Groupe scan
vgscan
# Logical volume scan
lvscan
# Change logical volume lv_halinux1 to active
lvchange /dev/VGHALINUX1/lv_halinux1 -a y
# Logical volume scan
lvscan
mount /dev/VGHALINUX1/lv_halinux1 /halinux1
exit 0
20
This script is present on the two sites in order to automate configuration and in order the user
not to have to intervene after each restart. Once the script is written, you have to configure
Linux to take it into account each start. That is the procedure to realize: you have to place the
script in the directory /etc/init.d, put it in executable mode (that is to say -rwx-r-xr-x) and then
make links to the different rc.d by the command ln-s (cf. below) in order to permit its taking
into account at each restart.
[root@halinux
[root@halinux
[root@halinux
[root@halinux
[root@halinux
[root@halinux
[root@halinux
~]#
~]#
~]#
~]#
~]#
~]#
~]#
chmod 755 script_sdd
ln
ln
ln
ln
-s
-s
-s
-s
../init.d/script_sdd
../init.d/script_sdd
../init.d/script_sdd
../init.d/script_sdd
/etc/rc2.d/S81script_sdd_site1
/etc/rc3.d/S81script_sdd_site1
/etc/rc4.d/S81script_sdd_site1
/etc/rc5.d/S81script_sdd_site1
2.6 Dscli
When the configuration above and the starting script are finished, Linux is integrated on the
platform and copy services can be tested thanks to the service dscli permitting to
communicate with the unit storage.
First you have to access to the contains of the directory you have downloaded:
[root@halinux
[root@halinux
[root@halinux
[root@halinux
[root@halinux
~]# cd /opt
opt]# mkdir dscli
~]# mount DSCLI-5.1.720.8.iso /opt/dscli -t iso9660 -o loop
~]#
~]#
Then, you have to uninstall old dscli versions before running this install. You need to have
Java Runtime Environment (JRE) installed on your target machine before starting the install.
IBM Java 1.4.2 is the preferred JRE on Linux and strongly recommended, although dscli can
support any Java 1.4.1.
[root@halinux dscli]# rpm -ivh IBMJava2-JRE-1.4.2-0.0.i386.rpm
[root@halinux dscli]#
[root@halinux dscli]#
Thus you can run setupgenericunix.sh. The installer lets you choose where dscli gets installed.
For instance, on Linux, the default is /opt/ibm/dscli. Then you run setuplinux.bin.
[root@halinux dscli]# ./setupgenericunix.sh
[root@halinux dscli]# ./setuplinux.bin
21
In order to have installation in text mode than in GUI, you have to precise -console as
mentioned below:
[root@halinux dscli]# ./setupgenericunix.sh -console
[root@halinux dscli]# ./setuplinux.bin -console
The command dscli is only available in the directory /opt/bin/dscli. In order to access it all
over the directory, you need to modify .bashrc:
[root@halinux dscli]# vi ~/.bashrc
# .bashrc
# User specific aliases and functions
alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
PATH=$PATH:.:/opt/ibm/dscli/
[root@halinux dscli]# dscli
22
3 Special Thanks
I would like to thank the many people from PSSC who helped me spending time, energy and
expertise:
Christian Casanova for the SAN and Network cabling and configuration.
Christophe Majek and Said Kemiche for the DS8000 setup and support.
Alexandre Chabrol for his help on System x.
Special thanks too, for Christine O’Sullivan and Emmanuel Tong-Viet who help me all over
my research.
23