Front cover
Implementing the IBM System Storage SAN Volume Controller V4.3 Install, use, and troubleshoot the SAN Volume Controller Learn how to implement block virtualization Create space-efficient VDisks
Jon Tate Sameer Dhulekar Juerg Hossli Dan Koeck Suad Musovich
ibm.com/redbooks
International Technical Support Organization Implementing the IBM System Storage SAN Volume Controller V4.3 October 2008
SG24-6423-06
Note: Before using this information and the product it supports, read the information in “Notices” on page xv.
Seventh Edition (October 2008) This edition applies to Version 4 Release 3 of the IBM System Storage SAN Volume Controller. © Copyright International Business Machines Corporation 2003-2008. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
Contents Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii The team that wrote this book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi Summary of changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii October 2008, Seventh Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii Chapter 1. Introduction to storage virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 The need for storage virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 In-band virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Out-of-band virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.1 IBM Global Parallel File System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1 2 3 6 7 8
Chapter 2. IBM System Storage SAN Volume Controller overview . . . . . . . . . . . . . . . . 9 2.1 Maximum supported configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.2 Glossary of commonly used terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.3 Virtualization overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.4 Compass architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.4.1 SAN Volume Controller clustering. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.4.2 SAN Volume Controller virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.4.3 SAN Volume Controller multipathing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.4.4 SAN Volume Controller logical configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.5 Software licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.6 New with SVC V4.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.6.1 Space-efficient VDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.6.2 FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.6.3 VDisk mirroring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.6.4 IPv6 addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.6.5 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.6.6 Licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.6.7 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.6.8 Additional interoperability support since SVC 4.2.1 . . . . . . . . . . . . . . . . . . . . . . . 22 2.6.9 SVC 4.3 Interoperability enhancements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Chapter 3. Planning and configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 General planning rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Physical planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Preparing your UPS environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Physical rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.3 Cable connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 SAN planning and configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 SAN definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.2 Fibre Channel switches, fabrics, interswitch links, and hops . . . . . . . . . . . . . . . . 3.3.3 General design considerations with the SVC . . . . . . . . . . . . . . . . . . . . . . . . . . . .
© Copyright IBM Corp. 2003-2008. All rights reserved.
25 26 27 29 30 31 32 33 35 35
iii
3.3.4 Boot support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.5 Configuration saving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.6 High availability SAN design and configuration rules with SVC . . . . . . . . . . . . . . 3.4 Zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Naming conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.1 Dual room high availability configuration with the SVC. . . . . . . . . . . . . . . . . . . . . 3.5.2 Local and remote SAN fabrics with SVC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.3 Technologies for extending the distance between two SVC clusters . . . . . . . . . . 3.6 SVC disk subsystem planning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.1 Block virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.2 MDGs, I/O groups, virtual disks, and managed disks . . . . . . . . . . . . . . . . . . . . . . 3.6.3 Extents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.4 Image mode virtual disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.5 Managed mode virtual disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.6 Space-efficient Virtual Disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.7 Extent allocation and size rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.8 MDisk group planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.9 Planning a virtual disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.10 Planning for operations on virtual disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.11 Host considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.12 Quorum disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.13 Expanding an SVC cluster configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 Storage subsystem planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.1 Adding DS8000 storage to the SVC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.2 Adding DS4000 storage to the SVC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7.3 LUN layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
39 39 40 40 41 44 44 45 46 46 46 48 48 49 52 55 55 56 59 61 62 64 65 67 72 76
Chapter 4. Performance and capacity planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Performance considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.1 SAN. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.2 Disk subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.3 SVC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.4 Host. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Performance modeling and sizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Performance monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 Collecting performance statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.2 Cluster wide statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.3 Per node statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
77 78 78 79 80 87 88 88 88 88 90
Chapter 5. SVC Console . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 5.1 Systems Storage Productivity Center overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 5.1.1 SSPC hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 5.1.2 Example hardware configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 5.2 SVC Console software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 5.3 Installation planning information for the SSPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 5.4 Secure Shell overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 5.4.1 Generating public and private SSH key pairs using PuTTY . . . . . . . . . . . . . . . . . 98 5.5 Basic installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 5.5.1 Creating the cluster (first time) using the service panel . . . . . . . . . . . . . . . . . . . 102 5.6 Completing the initial cluster setup using the SAN Volume Controller Console GUI . 106 5.6.1 Configuring the GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 5.6.2 Uploading the SSH public key to the SVC cluster. . . . . . . . . . . . . . . . . . . . . . . . 115 5.6.3 Uploading SSH public key(s) sample scenarios . . . . . . . . . . . . . . . . . . . . . . . . . 121
iv
Implementing the IBM System Storage SAN Volume Controller V4.3
5.6.4 Configuring the PuTTY session for the CLI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.6.5 Starting the PuTTY CLI session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.7 Using IPv6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.7.1 Migrating a cluster from IPv4 to IPv6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.7.2 Migrating a cluster from IPv6 to IPv4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.8 Upgrading the SVC Console software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.9 E-mail error notification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
123 127 131 131 141 141 146
Chapter 6. Quickstart configuration using the command-line interface . . . . . . . . . . 6.1 Adding nodes to the cluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Setting the cluster time zone and time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Checking the license features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Creating host definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5 Displaying managed disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6 Creating managed disk groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.7 Creating a virtual disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.8 Assigning a VDisk to a host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
157 158 161 162 162 164 165 167 171
Chapter 7. Quickstart configuration using the GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 Adding nodes to the cluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.1 Installing certificates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Setting the cluster time zone and time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 Checking the license status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.4 Creating host definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5 Displaying managed disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.6 Creating managed disk groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.7 Creating a VDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.7.1 Creating a space-efficient VDisk (SEV Disk) . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.7.2 Creating a mirrored VDisk. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.8 Assigning a VDisk to a host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
173 174 178 183 184 185 187 188 192 198 203 207
Chapter 8. Host configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 SVC setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.1 Switch zoning recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.2 Using port masking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 AIX-specific information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.1 Configuring the AIX host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.2 Operating system versions and maintenance levels. . . . . . . . . . . . . . . . . . . . . . 8.2.3 HBAs for IBM System p hosts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.4 Configuring for fast fail and dynamic tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.5 Subsystem Device Driver (SDDPCM or SDD) . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.6 Discovering the assigned VDisk using SDD and AIX 5L V5.3 . . . . . . . . . . . . . . 8.2.7 Using SDD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.8 Creating and preparing volumes for use with AIX 5L V5.3 and SDD . . . . . . . . . 8.2.9 Discovering the assigned VDisk using AIX V6.1 and SDDPCM . . . . . . . . . . . . . 8.2.10 Using SDDPCM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.11 Creating and preparing volumes for use with AIX V6.1 and SDDPCM. . . . . . . 8.2.12 Expanding an AIX volume. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.13 Removing an SVC volume on AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.14 Running SVC commands from an AIX host system . . . . . . . . . . . . . . . . . . . . . 8.3 Windows-specific information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.1 Configuring Windows 2000, Windows 2003, and Windows 2008 hosts . . . . . . . 8.3.2 Configuring Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.3 Hardware lists, device driver, HBAs and firmware levels . . . . . . . . . . . . . . . . . .
209 210 211 212 212 213 213 213 214 215 218 222 223 223 227 228 228 232 232 233 233 233 233
Contents
v
8.3.4 Host adapter installation and configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.5 Changing the disk timeout on Microsoft Windows Server. . . . . . . . . . . . . . . . . . 8.3.6 SDD driver installation on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3.7 SDDDSM driver installation on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4 Discovering the assigned VDisk in Windows 2000 / 2003 . . . . . . . . . . . . . . . . . . . . . 8.4.1 Extending a Windows 2000 or 2003 volume. . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5 Example configuration of attaching an SVC to a Windows 2008 host . . . . . . . . . . . . 8.5.1 Installing SDDDSM on a Windows 2008 host . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.2 Installing SDDDSM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.3 Attaching SVC VDisks to Windows 2008 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.4 Extending a Windows 2008 Volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.5.5 Removing a disk on Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.6 Using the SVC CLI from a Windows host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.7 Microsoft Volume Shadow Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.7.1 Installation overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.7.2 System requirements for the IBM System Storage hardware provider . . . . . . . . 8.7.3 Installing the IBM System Storage hardware provider . . . . . . . . . . . . . . . . . . . . 8.7.4 Verifying the installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.7.5 Creating the free and reserved pools of volumes . . . . . . . . . . . . . . . . . . . . . . . . 8.7.6 Changing the configuration parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.8 Linux (on Intel) specific information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.8.1 Configuring the Linux host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.8.2 Configuration information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.8.3 Disabling automatic Linux system updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.8.4 Setting queue depth with QLogic HBAs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.8.5 Multipathing in Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.8.6 Creating and preparing SDD volumes for use . . . . . . . . . . . . . . . . . . . . . . . . . . 8.8.7 Using the operating system MPIO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.8.8 Creating and preparing MPIO volumes for use. . . . . . . . . . . . . . . . . . . . . . . . . . 8.9 VMware configuration information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.9.1 Configuring VMware hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.9.2 Operating system versions and maintenance levels. . . . . . . . . . . . . . . . . . . . . . 8.9.3 Guest operating systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.9.4 HBAs for hosts running VMware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.9.5 Multipath solutions supported . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.9.6 VMware storage and zoning recommendations . . . . . . . . . . . . . . . . . . . . . . . . . 8.9.7 Setting the HBA timeout for failover in VMware . . . . . . . . . . . . . . . . . . . . . . . . . 8.9.8 Multipathing in ESX. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.9.9 Attaching VMware to VDisks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.9.10 VDisk naming in VMware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.9.11 Setting the Microsoft guest operating system timeout . . . . . . . . . . . . . . . . . . . 8.9.12 Extending a VMFS volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.9.13 Removing a datastore from an ESX host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.10 SUN Solaris support information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.10.1 Operating system versions and maintenance levels. . . . . . . . . . . . . . . . . . . . . 8.10.2 SDD dynamic pathing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.11 HP-UX configuration information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.11.1 Operating system versions and maintenance levels. . . . . . . . . . . . . . . . . . . . . 8.11.2 Multipath solutions supported . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.11.3 Co-existence of SDD and PV Links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.11.4 Using an SVC VDisk as a cluster lock disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.11.5 Support for HP-UX greater than eight LUNs . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.12 Using SDDDSM, SDDPCM, and SDD Web interface . . . . . . . . . . . . . . . . . . . . . . . . vi
Implementing the IBM System Storage SAN Volume Controller V4.3
234 236 236 238 240 244 249 249 252 254 260 260 263 264 265 265 265 269 270 271 274 274 274 274 275 275 280 282 282 287 287 287 287 287 288 289 290 291 291 294 295 295 297 298 298 298 299 299 299 299 300 300 300
8.13 Calculating the queue depth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301 8.14 Further sources of information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302 Chapter 9. SVC configuration and administration using the CLI . . . . . . . . . . . . . . . . 9.1 Managing users using the CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.1.1 Maintaining SSH keys using the CLI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.1.2 Managing user roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2 Managing the cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.1 Organizing on screen content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.2 Viewing cluster properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.3 Changing cluster settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.4 Maintaining cluster passwords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.5 Modifying IP addresses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.6 Setting the cluster time zone and time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.7 Start statistics collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.8 Stopping a statistics collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.9 Audit Log commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.10 Status of discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.11 Status of copy operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.12 Shutting down a cluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3 Working with nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.1 I/O groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.2 Viewing I/O group details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.3 Renaming an I/O group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.4 Adding and removing hostiogrp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.5 Listing I/O groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4 Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4.1 Viewing node details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4.2 Adding a node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4.3 Renaming a node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4.4 Deleting a node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4.5 Shutting down a node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5 Working with managed disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.1 Disk controller systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.2 MDisk information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.3 Renaming an MDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.4 Discovering MDisks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.5 Setting up a quorum disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.6 Including an MDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.7 Showing the MDisk group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.5.8 Showing a VDisk on an MDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.6 Managed Disk Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.6.1 Creating an MDisk group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.6.2 Renaming an MDisk group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.6.3 Deleting an MDisk group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.6.4 Adding MDisks to an MDisk group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.6.5 Removing MDisks from MDisk group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.6.6 Showing MDisks in a MDisk group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.6.7 Showing VDisks using a MDisk group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.7 Hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.7.1 Host information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.7.2 Creating a host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.7.3 Modify a host. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Contents
303 304 304 305 307 308 309 311 314 314 315 316 317 318 319 319 320 321 321 321 321 322 323 323 323 324 325 326 326 327 327 328 329 330 330 332 333 333 334 334 335 336 336 336 337 337 338 338 338 340
vii
viii
9.7.4 Deleting a host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.7.5 Adding ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.7.6 Deleting ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.8 SAN debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.9 Working with virtual disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.9.1 VDisk information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.9.2 Creating a VDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.9.3 Creating a VDisk in image mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.9.4 Adding a mirrored VDisk copy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.9.5 Splitting a VDisk Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.9.6 Deleting a VDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.9.7 Expanding a VDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.9.8 Mapping a VDisk to a host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.9.9 Deleting a VDisk-to-host mapping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.9.10 Showing the VDisks mapped to a host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.9.11 Modifying a VDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.9.12 Migrating a VDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.9.13 Migrate a VDisk to an image mode VDisk. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.9.14 Shrinking a VDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.9.15 Showing the MDisks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.9.16 Showing the MDisk group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.9.17 Showing the host to which the VDisk is mapped . . . . . . . . . . . . . . . . . . . . . . . 9.9.18 Showing the VDisk to which the host is mapped . . . . . . . . . . . . . . . . . . . . . . . 9.9.19 Tracing a host disk back to its source physical disk . . . . . . . . . . . . . . . . . . . . . 9.10 Service and maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.10.1 Upgrading software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.10.2 Running maintenance procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.10.3 Setting up error notification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.10.4 Analyzing the error log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.10.5 License settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.10.6 Viewing the feature log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.11 SVC cluster configuration backup and recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.11.1 Backing up the SVC cluster configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.11.2 Restoring the SVC cluster configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.11.3 Deleting configuration backup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.12 Listing dumps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.12.1 Error or event dump . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.12.2 Featurization log dump . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.12.3 I/O trace dump . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.12.4 I/O statistics dump . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.12.5 Software dump . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.12.6 Application abends dump . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.12.7 Other node dumps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.13 T3 recovery process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.14 Scripting and its usage under CLI for SVC task automation . . . . . . . . . . . . . . . . . . .
341 341 343 344 346 346 348 349 351 357 358 359 361 362 362 363 367 368 369 371 372 373 373 374 375 375 381 384 385 387 388 389 390 393 393 394 394 395 395 395 396 396 397 399 399
Chapter 10. SVC configuration and administration using the GUI. . . . . . . . . . . . . . . 10.1 Managing users. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.1.1 Creating a user using the GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.1.2 Modifying a user role. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.1.3 Deleting a user role . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2 Managing the cluster using the GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2.1 Organizing on screen content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
401 402 402 404 406 407 407
Implementing the IBM System Storage SAN Volume Controller V4.3
10.2.2 Viewing cluster properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2.3 Maintain cluster passwords. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2.4 Modifying IP Addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2.5 Setting the cluster time zone and time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2.6 Starting the statistics collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2.7 Stopping the statistics collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.2.8 Shutting down a cluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3 Working with nodes using the GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3.1 I/O groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3.2 Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.4 Viewing progress. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5 Working with managed disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.1 Disk controller systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.2 Discovery status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.3 Managed disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.4 MDisk information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.5 Renaming an MDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.6 Discovering MDisks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.7 Setting up a quorum disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.8 Including an MDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.9 Showing an MDisk group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.10 Showing a VDisk for an MDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.11 Creating a VDisk in image mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.5.12 Creating an image mode mirrored VDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6 Managed disk groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6.1 Viewing MDisk group information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6.2 Creating an MDisk group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6.3 Renaming an MDisk group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6.4 Deleting an MDisk group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6.5 Adding MDisks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6.6 Removing MDisks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6.7 Showing MDisks in this group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.6.8 Showing VDisks using this group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.7 Working with hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.7.1 Hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.7.2 Host information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.7.3 Creating a host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.7.4 Modifying a host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.7.5 Deleting a host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.7.6 Adding ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.7.7 Deleting ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.7.8 Fabrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8 Working with virtual disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.1 Using the Virtual Disks window for VDisks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.2 VDisk information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.3 Creating a VDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.4 Creating a space-efficient VDisk with auto-expand. . . . . . . . . . . . . . . . . . . . . . 10.8.5 Deleting a VDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.6 Deleting a VDisk-to-host mapping. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.7 Expanding a VDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.8 Mapping a VDisk to a host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.9 Modifying a VDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.10 Creating a VDisk Mirror from an existing VDisk . . . . . . . . . . . . . . . . . . . . . . .
413 414 415 418 419 420 421 423 423 424 430 430 431 433 433 434 434 435 436 436 437 438 440 444 450 450 451 453 454 456 457 458 459 460 460 461 463 467 468 470 472 473 474 474 475 477 484 489 490 491 492 493 499
Contents
ix
x
10.8.11 Migrating to a space-efficient VDisk using VDisk mirroring. . . . . . . . . . . . . . . 10.8.12 Splitting a VDisk Copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.13 Shrinking a VDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.14 Showing the MDisks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.15 Showing the MDisk group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.16 Showing the host to which the VDisk is mapped . . . . . . . . . . . . . . . . . . . . . . 10.8.17 Showing capacity information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.18 Showing VDisks mapped to a host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.8.19 Deleting VDisks from a host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.9 Managing Copy Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.10 Service and maintenance using the GUI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.11 Upgrading software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.11.1 Package numbering and version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.11.2 Upgrade status utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.11.3 Precautions before upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.11.4 SVC software upgrade test utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.11.5 Running maintenance procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.11.6 Setting up error notification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.11.7 Analyzing the error log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.11.8 License settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.11.9 Viewing the license settings log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.11.10 Listing dumps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.12 Backing up the SVC configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.12.1 Backup procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.12.2 Restoring the SVC configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.12.3 Deleting the configuration backup files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
501 504 505 506 507 507 508 509 509 510 510 511 511 511 511 512 519 522 524 528 530 531 535 536 537 537
Chapter 11. Copy Services: FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 SVC FlashCopy features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3 How it works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4 Implementation of SVC FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.1 FlashCopy mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.2 Multiple Target FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.3 Consistency groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.4 FlashCopy indirection layer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.5 Interaction and dependency between MTFC . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.6 Summary of the FlashCopy indirection layer algorithm. . . . . . . . . . . . . . . . . . . 11.4.7 Interaction with the cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.8 FlashCopy rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.9 FlashCopy and image mode disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.10 FlashCopy mapping events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.11 FlashCopy mapping states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.12 Space-efficient FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.13 Background copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.14 Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.15 Serialization of I/O by FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.16 Error handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.17 Asynchronous notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.4.18 Interoperation with Metro Mirror and Global Mirror . . . . . . . . . . . . . . . . . . . . . 11.4.19 Recovering data from FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5 Using the command line to perform FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5.1 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
539 540 541 542 543 543 544 544 547 549 550 551 551 552 553 555 558 559 559 560 560 562 562 562 563 563
Implementing the IBM System Storage SAN Volume Controller V4.3
11.5.2 Creating a FlashCopy consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5.3 Creating a FlashCopy mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5.4 Preparing (pre-triggering) the FlashCopy mapping. . . . . . . . . . . . . . . . . . . . . . 11.5.5 Preparing (pre-triggering) the FlashCopy consistency group . . . . . . . . . . . . . . 11.5.6 Starting (triggering) FlashCopy mappings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5.7 Starting (triggering) FlashCopy consistency group . . . . . . . . . . . . . . . . . . . . . . 11.5.8 Monitoring the FlashCopy progress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5.9 Stopping the FlashCopy mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5.10 Stopping the FlashCopy consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5.11 Deleting the FlashCopy mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5.12 Deleting the FlashCopy consistency group. . . . . . . . . . . . . . . . . . . . . . . . . . . 11.5.13 Migrate a VDisk to a space-efficient VDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6 Using the GUI to perform FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.1 Creating a FlashCopy consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.2 Creating a FlashCopy mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.3 Preparing (pre-triggering) the FlashCopy mapping. . . . . . . . . . . . . . . . . . . . . . 11.6.4 Preparing (pre-triggering) the FlashCopy consistency group . . . . . . . . . . . . . . 11.6.5 Starting (triggering) FlashCopy mappings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.6 Starting (triggering) a FlashCopy consistency group . . . . . . . . . . . . . . . . . . . . 11.6.7 Monitoring the FlashCopy progress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.8 Stopping the FlashCopy consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.9 Deleting the FlashCopy mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.10 Deleting the FlashCopy consistency group. . . . . . . . . . . . . . . . . . . . . . . . . . . 11.6.11 Migration from a fully allocated VDisk to SEV and vice versa using a GUI. . .
564 565 567 567 568 569 569 570 571 571 572 572 577 577 579 588 589 590 591 592 593 595 595 597
Chapter 12. Copy Services: Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1 Metro Mirror. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1.1 Metro Mirror overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1.2 Remote copy techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1.3 SVC Metro Mirror features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1.4 Metro Mirror relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1.5 How Metro Mirror works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1.6 Metro Mirror process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1.7 Methods of synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1.8 State overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1.9 Detailed states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1.10 Practical use of Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1.11 Metro Mirror configuration limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2 Metro Mirror commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.1 Listing available SVC cluster partners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.2 Creating SVC cluster partnership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.3 Creating a Metro Mirror consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.4 Creating a Metro Mirror relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.5 Changing a Metro Mirror relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.6 Changing a Metro Mirror consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.7 Starting a Metro Mirror relationship. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.8 Stopping a Metro Mirror relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.9 Starting a Metro Mirror consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.10 Stopping a Metro Mirror consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.11 Deleting a Metro Mirror relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.12 Deleting a Metro Mirror consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.13 Reversing a Metro Mirror relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.2.14 Reversing a Metro Mirror consistency group . . . . . . . . . . . . . . . . . . . . . . . . .
603 604 604 604 606 607 611 612 612 615 618 621 622 622 623 623 624 624 625 625 625 626 626 627 627 627 628 628
Contents
xi
xii
12.2.15 Background copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3 Metro Mirror scenario using the CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3.1 Setting up Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3.2 Starting Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3.3 Stopping and restarting Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.3.4 Changing copy direction for Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4 Metro Mirror scenario using the GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.1 Setting up Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.2 Starting Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.3 Stopping and restarting Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.4.4 Changing copy direction for Metro Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
628 629 630 634 637 640 642 643 657 660 666
Chapter 13. Copy Services: Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1 Global Mirror overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1.1 Intracluster Global Mirror. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1.2 Intercluster Global Mirror. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2 Remote copy techniques. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.1 Asynchronous remote copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.2 Synchronous remote copy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.3 SVC Global Mirror features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.4 Global Mirror relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.5 How Global Mirror works. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.6 Global Mirror process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.7 Methods of synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.8 State overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.9 Detailed states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.10 Practical use of Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.11 Global Mirror configuration limits. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3 Global Mirror commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.1 Listing the available SVC cluster partners . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.2 Creating an SVC cluster partnership. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.3 Creating a Global Mirror consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.4 Creating a Global Mirror relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.5 Changing a Global Mirror relationship. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.6 Changing a Global Mirror consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.7 Starting a Global Mirror relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.8 Stopping a Global Mirror relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.9 Starting a Global Mirror consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.10 Stopping a Global Mirror consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.11 Deleting a Global Mirror relationship. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.12 Deleting a Global Mirror consistency group . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.13 Reversing a Global Mirror relationship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3.14 Reversing a Global Mirror consistency group . . . . . . . . . . . . . . . . . . . . . . . . . 13.4 Global Mirror scenario using the CLI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.4.1 Setting up Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.4.2 Starting Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.4.3 Stopping and restarting Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.4.4 Changing direction for Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.5 Global Mirror scenario using the GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.5.1 Setting up Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.5.2 Starting Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.5.3 Stopping and restarting Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.5.4 Changing copy direction for Global Mirror. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
669 670 670 670 670 670 671 672 673 677 678 678 681 684 687 688 689 689 690 691 691 692 692 692 693 693 694 694 694 695 695 695 696 702 704 708 710 711 728 731 737
Implementing the IBM System Storage SAN Volume Controller V4.3
Chapter 14. Migration to and from the SAN Volume Controller . . . . . . . . . . . . . . . . . 14.1 Migration overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.2 Migration operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.2.1 Migrating multiple extents (within an MDG) . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.2.2 Migrating extents off an MDisk that is being deleted. . . . . . . . . . . . . . . . . . . . . 14.2.3 Migrating a VDisk between MDGs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.2.4 Migrating the VDisk to image mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.2.5 Migrating a VDisk between I/O groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.2.6 Monitoring the migration progress. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.3 Functional overview of migration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.3.1 Parallelism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.3.2 Error handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.3.3 Migration algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.4 Migrating data from an image mode VDisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.4.1 Image mode VDisk migration concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.4.2 Migration tips. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.5 Data migration for Windows using the SVC GUI. . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.5.1 SVC added between the host system and the DS4700 . . . . . . . . . . . . . . . . . . 14.5.2 Put the migrated disks on a Windows 2008 host online . . . . . . . . . . . . . . . . . . 14.5.3 Migrating the VDisk from image mode to managed mode . . . . . . . . . . . . . . . . 14.5.4 Migrating the VDisk from managed mode to image mode . . . . . . . . . . . . . . . . 14.5.5 Migrating the VDisk from image mode to image mode . . . . . . . . . . . . . . . . . . . 14.5.6 Free the data from the SVC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.5.7 Put the disks online in Windows 2008 that have been freed from SVC . . . . . . 14.6 Migrating Linux SAN disks to SVC disks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.6.1 Connecting the SVC to your SAN fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.6.2 Prepare your SVC to virtualize disks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.6.3 Move the LUNs to the SVC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.6.4 Migrate the image mode VDisks to managed MDisks . . . . . . . . . . . . . . . . . . . 14.6.5 Preparing to migrate from the SVC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.6.6 Migrate the VDisks to image mode VDisks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.6.7 Remove the LUNs from the SVC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.7 Migrating ESX SAN disks to SVC disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.7.1 Connecting the SVC to your SAN fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.7.2 Prepare your SVC to virtualize disks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.7.3 Move the LUNs to the SVC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.7.4 Migrate the image mode VDisks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.7.5 Preparing to migrate from the SVC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.7.6 Migrate the managed VDisks to image mode VDisks . . . . . . . . . . . . . . . . . . . . 14.7.7 Remove the LUNs from the SVC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.8 Migrating AIX SAN disks to SVC disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.8.1 Connecting the SVC to your SAN fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.8.2 Prepare your SVC to virtualize disks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.8.3 Move the LUNs to the SVC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.8.4 Migrate image mode VDisks to VDisks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.8.5 Preparing to migrate from the SVC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.8.6 Migrate the managed VDisks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.8.7 Remove the LUNs from the SVC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.9 Using SVC for storage migration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
741 742 742 742 743 744 746 746 748 749 749 750 750 751 751 754 754 758 765 768 771 775 779 781 783 785 786 790 793 796 799 800 803 804 806 810 813 816 818 819 822 824 825 830 832 834 837 838 841
Appendix A. Copy Services and open systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 843 AIX specifics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 844 AIX and FlashCopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 844
Contents
xiii
AIX and Metro Mirror and Global Mirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Making updates to the LVM information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Windows NT and 2000/2003 specifics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Windows NT and Copy Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Copy Services with Windows NT Disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Windows 2000/2003 and Copy Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
858 859 860 860 860 865
Appendix B. DS4000 and DS8000 migration scenarios. . . . . . . . . . . . . . . . . . . . . . . . Initial considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Device recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scenario 1: DS4000 Total number of LUNs is less than maximum LUNs per partition . . . Scenario 2: DS4000 total number of LUNs is more than the maximum LUNs per partition Scenario 3: Migrating DS8000 Storage to SVC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
885 886 887 888 892 896
Appendix C. Scripting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scripting structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Automated VDisk creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . SVC tree. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scripting alternatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
901 902 903 906 913
Appendix D. Node replacement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Replacing nodes nondisruptively . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Expanding an existing SVC cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Moving VDisks to a new I/O group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Replacing nodes disruptively (rezoning the SAN) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
915 916 920 922 923
Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Other resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Referenced Web sites. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . How to get Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
925 925 925 926 926 927
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 929
xiv
Implementing the IBM System Storage SAN Volume Controller V4.3
Notices This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs.
© Copyright IBM Corp. 2003-2008. All rights reserved.
xv
Trademarks IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. These and other IBM trademarked terms are marked on their first occurrence in this information with the appropriate symbol (® or ™), indicating US registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at http://www.ibm.com/legal/copytrade.shtml The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both: AIX 5L™ AIX® BladeCenter® Chipkill™ DB2® developerWorks® DS4000™ DS6000™ DS8000™
Enterprise Storage Server® FlashCopy® GPFS™ IBM® Power Systems™ Redbooks® Redbooks (logo) ® RS/6000® System i®
System p® System Storage™ System Storage DS® System x™ System z® Tivoli® TotalStorage® WebSphere®
The following terms are trademarks of other companies: Disk Magic, and the IntelliMagic logo are trademarks of IntelliMagic BV in the United States, other countries, or both. Data ONTAP, NetApp, and the NetApp logo are trademarks or registered trademarks of NetApp, Inc. in the U.S. and other countries. Novell, SUSE, the Novell logo, and the N logo are registered trademarks of Novell, Inc. in the United States and other countries. Oracle, JD Edwards, PeopleSoft, Siebel, and TopLink are registered trademarks of Oracle Corporation and/or its affiliates. QLogic, and the QLogic logo are registered trademarks of QLogic Corporation. SANblade is a registered trademark in the United States. VMotion, VMware, the VMware "boxes" logo and design are registered trademarks or trademarks of VMware, Inc. in the United States and/or other jurisdictions. JNI, Solaris, Sun, and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Active Directory, Internet Explorer, Microsoft, MS-DOS, Windows NT, Windows Server, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Intel Xeon, Intel, Itanium-based, Itanium, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others.
xvi
Implementing the IBM System Storage SAN Volume Controller V4.3
Preface This IBM® Redbooks® publication is a detailed technical guide to the IBM System Storage™ SAN Volume Controller (SVC), a virtualization appliance solution that maps virtualized volumes visible to hosts and applications to physical volumes on storage devices. Each server within the SAN has its own set of virtual storage addresses, which are mapped to physical addresses. If the physical addresses change, the server continues running using the same virtual addresses that it had before. This means that volumes or storage can be added or moved while the server is still running. The IBM virtualization technology improves management of information at the “block” level in a network, enabling applications and servers to share storage devices on a network. Successful businesses require real-time responsiveness to change, whether because of new customer needs, changes in the supply chain, unexpected competitive moves, external threats, or changes in the economic climate. Rapid response to change requires an IT infrastructure that can turn information into a competitive advantage; the IT infrastructure must provide the maximum benefit at an affordable cost, and must have the flexibility to support changes in business processes. An on demand operating environment provides a cost effective and flexible IT environment. With information at the heart of competitiveness, storage becomes an ever more critical component of an on demand operating environment. The IBM System Storage strategy addresses some of the most pressing needs currently facing Chief Information Officers (CIO) and IT managers. As part of its strategy, IBM intends to deliver industry leading technologies that will help dramatically reduce the total cost of ownership (TCO) for storage, and help turn fixed costs into variable costs that scale with business volume. Success in the on demand world will depend on the ability to leverage information technology. A greater dependence on information means a greater dependence on storage. What distinguishes an on demand business is the ability to quickly sense and rapidly respond to a dynamic marketplace; to do this, there are challenges that an on demand business must overcome. At the business level, customers are faced with three major storage challenges: Managing storage growth: Storage needs to continue to grow at over 50% per year. Managing storage is becoming more complex than ever, because we now have to deal with multiple server platforms and different operating systems, which may be connected to a storage area network (SAN) with multiple and diverse storage platforms. Increasing complexity: Although the declining cost of storage per megabyte makes it attractive to add additional disks, the increasing complexity of managing this storage results in overutilized staff and underutilized IT resources. Combining this with the shortage of skilled storage administrators, it is possible to add significant cost and introduce risk to storage management. Maintaining availability: The added complexity of 24x7 environments significantly reduces, for example, the efficiency of conducting routine maintenance, scheduling backups, data migration, and introducing new software and hardware. This problem is compounded by the fact that as availability increases, so does the cost inherent with making it so. Variety of information: Information technology holds the promise of bringing a variety of new types of information to the people who need it. Volume of data: Data is growing exponentially. Estimates show a continued 60% yearly growth of new disk in petabytes shipped. © Copyright IBM Corp. 2003-2008. All rights reserved.
xvii
Velocity of change: IT organizations are under tremendous pressure to deliver the right IT services. Approximately 85% of problems are caused by IT staff changing something, and 80% of problems are not detected by IT staff until reported. These challenges still exist, although large SANs do offer desirable and tangible benefits, for example, better connectivity, improved performance, distance flexibility, and scalability. However, even these benefits may be outweighed by the added complexity that they introduce. As an example, large enterprise SANs often contain different types of storage devices. These differences could be in the types of disk deployed, their level of performance, or the functionality provided, such as RAID or mirroring. Often, customers have different vendor storage devices as the result of mergers or consolidations. The result, however, is that storage and SAN administrators need to configure storage to servers, and then keep track of which servers own or have access to that storage. The storage administrative tasks can become daunting as the SAN grows and as the storage administrators manually attempt to manage the SAN. Furthermore, the complexity of having different file systems in the same SAN requires that storage administrators know how to administer each client operating system (OS) platform. The management interfaces for each may be different, since there is no common standard that all vendors adhere to. Lastly, since the file systems are tied to each of the servers, storage management functions potentially have to be run on hundreds of servers. It is easy to see why manageability and interoperability are the top areas for concern, especially in a SAN where the number of possible storage and OS platform permutations is considerable. These challenges are at odds with the commonly held belief that storage is decreasing in cost per megabyte. It is clear that the cost of managing storage is greater than the initial purchase price. A product is needed to address storage manageability, while at the same time addressing the need for interoperability. This product is the focus of this book.
The team that wrote this book This book was produced by a team of specialists from around the world working at the International Technical Support Organization, San Jose Center. Jon Tate is a Project Manager for IBM System Storage SAN Solutions at the International Technical Support Organization, San Jose Center. Before joining the ITSO in 1999, he worked in the IBM Technical Support Center, providing Level 2 support for IBM storage products. Jon has 23 years of experience in storage software and management, services, and support, and is both an IBM Certified IT Specialist and an IBM SAN Certified Specialist. Jon also serves as the UK Chair of the Storage Networking Industry Association. Sameer Dhulekar is a Senior IT Specialist for Systems Solutions Center, at the System and Technology Group, India. Sameer has 13 years of experience in Storage, Storage Area Networks, UNIX®, database and software management, services, and support. Sameer specializes in benchmark, performance, virtualization, and high availability solutions. He has participated in multiple storage seminars and forums. Juerg Hossli is an Advisory Technical Services Professional at Global Technology Services, Integrated Technology Delivery in Switzerland. He has 15 years of experience in the IT field and has worked at IBM for eleven years. He holds a Swiss Federal Certificate in Computer Science. His areas of expertise include planning, implementation, and maintenance of high-end storage solutions, storage virtualization, the SAN environment, and Tivoli® Storage products. xviii
Implementing the IBM System Storage SAN Volume Controller V4.3
Dan Koeck is a Storage and IBM System x™ Specialist in IBM Maintenance and Technical Support (MTS) in Austria. He has a graduate degree in applied computer science and has many industry certifications. In the very near future, he will join the IBM STG pre-sales group in Austria and is working towards a second degree in IT Security. He has worked at IBM for six years, and his areas of expertise include disk storage, SAN, IBM System x, BladeCenter®, server and storage virtualization, high-availability solutions, data center migration, and software engineering. He previously authored Tuning IBM System x Servers for Performance, SG24-5287 and IBM System Storage DS3000: Introduction and Implementation Guide, SG24-7065. Suad Musovich is a Senior IT Specialist for IBM Global Technology Services in New Zealand. He has seven years of experience working with IBM Storage Systems, and in his current role he is involved with the planning, design, implementation, management, and problem analysis of IBM storage solutions. His areas of expertise is the SAN infrastructure, disk storage, UNIX systems, and IBM Tivoli Storage Manager backup solutions.
Figure 1 Left to right: Dan, Juerg, Sam, Suad, and Jon
We extend our thanks to the following people for their contributions to this project. There are many people that contributed to this book. In particular, we thank the development and PFE teams in Hursley. Matt Smith was also instrumental in moving any issues along and ensuring that they maintained a high profile.
Preface
xix
In particular, we thank the authors of the previous editions of this book: Matt Amanat Angelo Bernasconi Steve Cody Sean Crawford Katja Gebuhr Deon George Amarnath Hiriyannappa Thorsten Hoss Philippe Jachimczyk Kamalakkannan J Jayaraman Bent Lerager Craig McKenna Andy McManus Joao Marcos Leite Barry Mellish Massimo Rosati Fred Scholten Robert Symons Marcus Thordal Xiao Peng Zhao We would also like to thank the following people for their contributions: John Agombar Alex Ainscow Trevor Boardman Chris Canto Peter Eccles Carlos Fuente Alex Howell Colin Jewell Paul Mason Paul Merrison Jon Parkes Steve Randle Lucy Raw Bill Scales Dave Sinclair Matt Smith Steve White Barry Whyte IBM Hursley Bill Wiegand IBM Advanced Technical Support Timothy Crawford Ross Hagglund IBM Beaverton Dorothy Faurot IBM Raleigh Marci Nagel IBM Rochester
xx
Implementing the IBM System Storage SAN Volume Controller V4.3
Chris Saul IBM San Jose Glen Routley IBM Australia Sharon Wang IBM Chicago Deanna Polm Sangam Racherla IBM ITSO
Become a published author Join us for a two- to six-week residency program! Help write a book dealing with specific products or solutions, while getting hands-on experience with leading-edge technologies. You will have the opportunity to team with IBM technical professionals, Business Partners, and Clients. Your efforts will help increase product acceptance and customer satisfaction. As a bonus, you will develop a network of contacts in IBM development labs, and increase your productivity and marketability. Find out more about the residency program, browse the residency index, and apply online at: ibm.com/redbooks/residencies.html
Comments welcome Your comments are important to us! We want our books to be as helpful as possible. Send us your comments about this book or other IBM Redbooks in one of the following ways: Use the online Contact us review Redbooks form found at: ibm.com/redbooks Send your comments in an e-mail to:
[email protected] Mail your comments to: IBM Corporation, International Technical Support Organization Dept. HYTD Mail Station P099 2455 South Road Poughkeepsie, NY 12601-5400
Preface
xxi
xxii
Implementing the IBM System Storage SAN Volume Controller V4.3
Summary of changes This section describes the technical changes made in this edition of the book and in previous editions. This edition may also include minor corrections and editorial changes that are not identified. Summary of Changes for SG24-6423-06 for Implementing the IBM System Storage SAN Volume Controller V4.3 as created or updated on October 24, 2008.
October 2008, Seventh Edition This revision reflects the addition, deletion, or modification of new and changed information described below.
New information
SVC Console Space-efficient VDisks VDisk Mirroring IPv6 Additional host support
© Copyright IBM Corp. 2003-2008. All rights reserved.
xxiii
xxiv
Implementing the IBM System Storage SAN Volume Controller V4.3
1
Chapter 1.
Introduction to storage virtualization In this chapter, we describe the need for storage virtualization and the IBM approach to both in-band and out-of-band storage virtualization. The fundamental differences between the two architectures are articulated to explain why IBM has chosen to use in-band virtualization for the IBM System Storage SAN Volume Controller (the focus of the remainder of this book).
© Copyright IBM Corp. 2003-2008. All rights reserved.
1
1.1 The need for storage virtualization At the business level, clients are faced with three major storage challenges: Managing storage growth: Storage needs continue to grow at a rate that is normally higher than what has been planned for each year. As an example, storage subsystems can be purchased to last for 3 to 5 years; however, organizations are finding that they are filling to capacity much earlier than that. To fill the growth, customers are then either extending their current storage subsystems in chunks, or buying different types of storage subsystems to match their storage needs and budget. Increasing complexity: As storage needs grow, this need can be filled by more than one disk subsystem, which might not even be from the same vendor. Together with the variety of server platforms and operating systems in a customer’s environment, customers can have storage area networks (SAN) with multiple and diverse storage subsystems and host platforms. Combining this with the shortage of skilled storage administrators, the cost and risk of storage increases as the environment becomes more complex. Maintaining availability: With the increased range of storage options available, the storage growth rate, and no similar increase in storage budget, customers are having to manage more storage with minimal or no additional staff. Thus, with the complexity highlighted above, and with business requirements on IT resources demanding higher business system availability, the room for errors increases as each new storage subsystem is added to the infrastructure. Additionally, making changes to the storage infrastructure to accommodate storage growth traditionally leads to outages that might not be acceptable by the business. Storage needs are rising, and the challenge of managing disparate storage systems is growing. The IBM System Storage SAN Volume Controller brings storage devices together in a virtual pool to make all storage appear as: One “logical” device to centrally manage and to allocate capacity as needed One solution to help achieve the most effective use of key storage resources on demand Virtualization solutions can be implemented in the storage network, in the server, or in the storage device itself. The IBM storage virtualization solution is SAN-based, which helps allow for a more open virtualization implementation. Locating virtualization in the SAN, and therefore in the path of input/output (I/O) activity, helps provide a solid basis for policy-based management. The focus of IBM on open standards means its virtualization solution supports freedom of choice in storage-device vendor selection. The IBM System Storage SAN Volume Controller solution is designed to: Simplify storage management Reduce IT data storage complexity and costs while enhancing scalability Extend on demand flexibility and resiliency to the IT infrastructure Increase application availability by making changes in the infrastructure without having to shut down hosts
2
Implementing the IBM System Storage SAN Volume Controller V4.3
1.2 In-band virtualization In a conventional SAN, the logical unit numbers (LUNs) that are defined within the storage subsystem are directly presented to the host or hosts. In-band virtualization, otherwise known as block aggregation, essentially means having an appliance in the data path that can take physical storage from one or more storage subsystems and offer it to hosts in the form of a virtual disk (VDisk). The Storage Networking Industry Association (SNIA) block aggregation model (Figure 1-1) specifies that block aggregation can be performed within hosts (servers), in the storage network (storage routers and storage controllers), or in storage devices (intelligent disk arrays).
Application
Storage devices (disks, …)
Capacity planning
Device-based block aggregation
High availability (fail-over, …)
SN-based block aggregation
Redundancy mgmt (backup, …)
Block aggregation
Security, billing
Host-based block aggregation
Discovery, monitoring
File system (FS)
Services subsystem
Storage domain
Database (dbms)
Resource mgmt, configuration
File/record subsystem
Block subsystem Copyright 2000, Storage Network Industry Association
Figure 1-1 SNIA block aggregation model
While each of these approaches has pros and cons and all are available in various forms from various vendors, IBM chose to develop its latest block aggregation product (IBM System Storage SAN Volume Controller) within the storage network. Block aggregation within the storage network provides four significant benefits to clients: Increased storage administrator productivity: Administrators can manage, add, and migrate physical disks non-disruptively from an application server point of view. This is accomplished by providing insulation between the server’s view of the logical disks and the disks as presented by the storage subsystem. Productivity is improved by allowing administrators to perform management functions when convenient rather than waiting for ever decreasing maintenance windows. Downtime requirements are almost eliminated.
Chapter 1. Introduction to storage virtualization
3
Providing a common platform for advanced functions: By providing a logical view of physical storage, advanced functions like disaster recovery can be done at a single point in the SAN in a consistent way regardless of the underlying physical storage. FlashCopy®, Metro Mirror (formerly referred to as Peer-to-Peer Remote Copy (PPRC)) and data migration can also be performed in a consistent way. This common platform is used to provide other advanced functions over time, such as advanced security and quality of service (QoS) capabilities. Improved capacity utilization: Spare capacity on underlying physical disks can be reallocated non-disruptively from an application server point of view irrespective of the server operating system or platform type. Logical disks can be created from any of the physical disks being managed by the virtualization device (that is, vendor agnostic). Simplification of connectivity: Each vendor storage subsystem would traditionally require a vendor’s device driver on the host to access the subsystem. Where there are many subsystems in the environment, regardless of whether any one host is accessing more than one vendor’s storage subsystems, then managing the range of device drivers is unnecessarily complex. The IBM approach means that only one device driver, the IBM System Storage Subsystem Device Driver (SDD), is required to access any virtualized storage on the SAN regardless of the vendor storage subsystem. Figure 1-2 shows the IBM approach to block aggregation.
Figure 1-2 IBM plan for block aggregation
4
Implementing the IBM System Storage SAN Volume Controller V4.3
In addition to the four major benefits outlined above, abstracting the hosts from directly accessing the storage subsystem or subsystems has many other benefits over other methods of block aggregation, including these: It provides the ability to add advanced functions and apply them to the entire storage infrastructure. The first release of the product offered these functions: – Copy Services (Metro Mirror (formerly referred to as PPRC) and FlashCopy) – Data migration – Read and Write Caching Later releases of the product offer such functions as: – Quality of Service – Performance based data migration – Performance optimization in the data path – Advanced security – Copy Services: Global Mirror – Space-efficiency
It does not lock a client into a particular storage hardware vendor. It is not intrusive on the hosts. It can offload functionality from the hosts. It can support storage management from multiple ISVs. It offers superior scalability.
The IBM virtualization product provides redundant, modular, and scalable solutions. It is based on a clustered IBM SAN appliance running a Linux® kernel to support high availability and performance. Additional nodes are capable of being added non-disruptively, providing enterprise class scalability. IBM’s long history of storage controller development has enabled us to develop systems where, in the exceptionally rare case that a failure occurs, the virtualization device can fail and recover gracefully.
Chapter 1. Introduction to storage virtualization
5
Figure 1-3 shows a representation of the IBM System Storage SAN Volume Controller.
Figure 1-3 Conceptual diagram of the IBM SAN Volume Controller
In summary, enterprise class block aggregation functionality is added to the storage network. The IBM solution improves storage administrator productivity, provides a common base for advanced functions, and provides for more efficient use of storage. The IBM product is designed to be delivered as a horizontally scalable, integrated solution based on the IBM SAN appliance, and Linux, using a fault tolerant clustered architecture.
1.3 Out-of-band virtualization Out-of-band virtualization, otherwise known as file aggregation, is when the virtualization appliance is not in the data path. Typically, out-of-band virtualization is more geared towards file sharing across the SAN. To this end, it typically involves a single file system in a single name space. File aggregation is a similar technique to block aggregation. However, rather than dealing with blocks of data, file aggregation addresses the needs of accessing and sharing files in a storage network. In the SNIA model, hosts get file metadata from file system or Network Attached Storage (NAS) controllers, and then access the data directly. File aggregation can be used in conjunction with or independent from block aggregation. Figure 1-4 on page 7 shows the SNIA file aggregation model.
6
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 1-4 SNIA file aggregation model
1.3.1 IBM Global Parallel File System The IBM approach to file virtualization is through the use of a common file system based on the IBM Global Parallel File System (GPFS™). GPFS is a high-performance shared-disk cluster file system. GPFS provides concurrent high-speed file access to applications executing on multiple nodes of an AIX® 5L™ cluster, a Linux cluster, or a heterogeneous cluster of AIX 5L and Linux nodes. In addition to providing file system storage capabilities, GPFS provides tools for management and administration of the GPFS cluster and allows for shared access to file systems from remote GPFS clusters. Note: Effective April 20, 2007, IBM withdrew from marketing the SAN File System. Its replacement is GPFS. GPFS provides scalable, high-performance data access from a single node to 2,000 nodes or more. Up to 512 Linux nodes or 128 AIX 5L nodes with access to one or more file systems are supported as a general statement and larger configurations exist by special arrangements with IBM. A GPFS file system is built from a collection of disks that contain the file system data and metadata. A file system can be built from a single disk or contain thousands of disks, each up to 2 terabytes in size, storing petabytes of data. GPFS provides file level virtualization through a rule based policy engine. Data from files in a in a single directory can reside in one or across several storage pools. Determination of which storage pool the file data is initially written to, and how it is migrated, mirrored, or deleted over its life span, is based on a set of business rules in an administrator defined policy.
Chapter 1. Introduction to storage virtualization
7
GPFS enables file level virtualization. This allows clients to reap the benefit of better application business responsiveness, maximized storage utilization, dynamic resource allocation, improved storage administration utilization, and reduced storage outage. For more details about the technology and implementation of GPFS, see the white paper, An Introduction to GPFS, at: http://www-03.ibm.com/systems/clusters/software/whitepapers/gpfs_intro.html
1.4 Conclusion In conclusion, the IBM System Storage SAN Volume Controller enables storage virtualization. This allows clients to reap the benefit of better application business responsiveness, maximized storage utilization, dynamic resource allocation, improved storage administration utilization, and reduced storage outage. In-band and out-of-band virtualization provide two very distinct yet complementary approaches to virtualization. IBM will extol the virtues of each in two separate products. Both products fulfill different requirements, and therefore use different approaches to virtualization. The rest of this book is dedicated to the IBM System Storage SAN Volume Controller and its method of in-band virtualization.
8
Implementing the IBM System Storage SAN Volume Controller V4.3
2
Chapter 2.
IBM System Storage SAN Volume Controller overview In this chapter, we describe the major concepts behind the IBM System Storage SAN Volume Controller to provide the framework for the discussion of the remainder of this book.
© Copyright IBM Corp. 2003-2008. All rights reserved.
9
2.1 Maximum supported configurations For a list of the maximum supported configurations, visit the SVC support site at: http://www.ibm.com/storage/support/2145
2.2 Glossary of commonly used terms Before providing an overview of the IBM System Storage SAN Volume Controller, we begin this chapter with a short glossary of terms (in alphabetical order) most commonly used throughout this book.
Configuration node While the cluster is operational, a single node in the cluster is appointed to provide configuration and service functions over the network interface. This node is termed the configuration node. This configuration node manages a cache of the configuration information that describes the cluster configuration and provides a focal point for configuration commands. If the configuration node fails, another node in the cluster will assume the role.
Extent To track the space that is available on an MDisk, the SAN Volume Controller divides each MDisk into chunks of equal size. These chunks are called extents and are indexed internally
Front end and back end The SAN Volume Controller takes managed disks and presents these to application servers (hosts). The managed disks are looked after by the “back-end” application of the SAN Volume Controller. The virtual disks presented to hosts are looked after by the “front-end” application in the SAN Volume Controller.
Grain A grain is the unit of data represented by a single bit in a FlashCopy bitmap (64 KB / 256 KB) in the SAN Volume Controller. It is also the unit to extend the real size of a space-efficient VDisk (32,64,128 or 256 KB).
I/O group An input/output (I/O) group contains two SAN Volume Controller nodes defined by the configuration process. Each SAN Volume Controller node is associated with exactly one I/O group. The nodes in the I/O group provide access to the VDisks in the I/O group.
LU and LUN Strictly speaking, there is a difference between a logical unit (LU) and a logical unit number (LUN). A LUN is a unique identifier used on a SCSI bus that enables it to differentiate between devices (each of which is a logical unit). Each of the LUs in the SVC is an MDisk. In practice, the two terms are used interchangeably. In this book, when we refer to a LUN, we refer to the unit of storage that is defined in a storage subsystem such as an IBM System Storage Enterprise Storage Server® (ESS), IBM System Storage DS3000,DS4000™, DS6000™, and DS8000™ series Storage Server, or storage servers from other vendors.
Managed disk A managed disk (MDisk) is a logical disk (typically a RAID or partition thereof) that a storage subsystem has exported to the SAN fabric to which the nodes in the cluster are attached.
10
Implementing the IBM System Storage SAN Volume Controller V4.3
Managed disk group A managed disk (MDisk) group is a collection of MDisks.
Master console The master console is the platform on which the software used to manage the SAN Volume Controller runs. With Version 4.3. it is being replaced by SSPC. However, V4.3 GUI Console code is supported on existing master consoles.
Node A SAN Volume Controller node is a single processing unit, which provides virtualization, cache, and copy services for the SAN. Nodes are deployed in pairs called I/O groups. One node in the cluster is designated the configuration node, but each node in the cluster holds a copy of the cluster state information.
SAN Volume Controller The SAN Volume Controller is a SAN appliance designed for attachment to a variety of host computer systems, which carries out block level virtualization of disk storage.
SSPC IBM System Storage Productivity Center (SSPC) replaces the master console for new installations of SAN Volume Controller Version 4.3.0. For SSPC planning, installation, and configuration information, see the following Web site: http://publib.boulder.ibm.com/infocenter/tivihelp/v4r1/index.jsp
Virtual disk A virtual disk (VDisk) is a SAN Volume Controller device that appears to host systems attached to the SAN as a SCSI disk. Each VDisk is associated with exactly one I/O group.
2.3 Virtualization overview The SVC nodes are the hardware elements of the IBM System Storage SAN Volume Controller, a member of the IBM System Storage virtualization family of solutions. The SAN Volume Controller combines servers into a high availability cluster. Each of the servers in the cluster is populated with 8 GB of high-speed memory, which serves as the cluster cache. A management card is installed in each server to monitor various parameters that the cluster uses to determine the optimum and continuous data path. The cluster is protected against data loss by uninterruptible power supplies. The SAN Volume Controller nodes can only be installed in pairs to avoid a single point of failure. Storage virtualization addresses the increasing cost and complexity in data storage management. It addresses this increased complexity by shifting storage management intelligence from individual SAN disk subsystem controllers into the network through a virtualization cluster of nodes. The SAN Volume Controller solution is designed to reduce both the complexity and costs of managing your SAN-based storage. With the SAN Volume Controller, you can: Simplify management and increase administrator productivity by consolidating storage management intelligence from disparate disk subsystem controllers into a single view. Improve application availability by enabling data migration between disparate disk storage devices non-disruptively.
Chapter 2. IBM System Storage SAN Volume Controller overview
11
Improve disaster recovery and business continuance needs by applying and managing copy services across disparate disk storage devices within the Storage Area Network (SAN). These solutions include a Common Information Model (CIM) Agent, enabling unified storage management based on open standards for units that comply with CIM Agent standards. Provide advanced features and functions to the entire SAN, such as: – Large scalable cache – Copy Services – Space management (later releases to include Policy Based Management) – Mapping based on desired performance characteristics – Quality of Service (QoS) metering and reporting Simplify device driver configuration on hosts, so all hosts within your network use the same IBM device driver to access all storage subsystems through the SAN Volume Controller. Note: The SAN Volume Controller is not a RAID controller. The disk subsystems attached to SANs that have the SAN Volume Controller provide the basic RAID setup. The SAN Volume Controller uses what is presented to it as a managed disk to create virtual disks.
2.4 Compass architecture The IBM System Storage SAN Volume Controller is based on the Commodity Parts Storage System (Compass) architecture developed at the IBM Almaden Research Center. The overall goal of the Compass architecture is to create storage subsystem software applications that require minimal porting effort to leverage a new hardware platform. To meet this goal: Compass, although currently deployed on the Intel® hardware platform, can be ported to other hardware platforms. Compass, although currently deployed on a Linux kernel, can be ported to other Portable Operating System Interface (POSIX)-compliant operating systems. Compass uses commodity adapters and parts wherever possible. To the highest extent possible, it only uses functions in the commodity hardware that are commonly exercised by the other users of the parts. This is not to say that Compass software could not be ported to a platform with specialized adapters. However, the advantage in specialized function must be weighed against the disadvantage of future difficulty in porting and in linking special hardware development plans to the release plans for applications based on the Compass architecture. Compass is developed in such a way that it is as easy as possible to troubleshoot and correct software defects. Compass is designed as a scalable, distributed software application that can run in increasing sets of Compass nodes with near linear gain in performance while using a shared data model that provides a single pool of storage for all nodes. Compass is designed so that there is a single configuration and management view of the entire environment regardless of the number of Compass nodes in use.
12
Implementing the IBM System Storage SAN Volume Controller V4.3
The approach is to minimize the dependency on unique hardware, and to allow exploitation of or migration to new SAN interfaces simply by plugging in new commodity adapters. Performance growth over time is ensured by the ability to port Compass to just about any platform and remain current with the latest processor and chipset technologies on each. The SAN Volume Controller implementation of the Compass architecture has exploited Linux as a convenient development platform to deploy this function. This has enhanced and will continue to enhance the ability of IBM to deploy robust function in a timely way. SVC relies on the Compass architecture to provide high levels of fault tolerance and high availability. Extensive dump capabilities are provided to enable first failure capture of software defects. Fault tolerance and high levels of availability are achieved by:
The RAID capabilities of the underlying disk subsystems SVC clustering using the Compass architecture Auto-restart of hung nodes UPS units to provide memory protection in the event of a site power failure Host System Failover capabilities
High levels of serviceability are achieved by providing:
Cluster error logging Asynchronous error notification Dump capabilities to capture software detected failures Concurrent diagnostics Directed maintenance procedures Concurrent log analysis and dump data recovery tools Concurrent maintenance of all SVC components Concurrent upgrade of SVC software and microcode Concurrent addition or deletion of SVC nodes in a cluster Software recovery through a service panel push button Automatic software version correction when replacing a node Detailed status and error conditions displayed on the service panel Error and event notification through SNMP and e-mail
Support is provided for the end-to-end SAN problem determination through detailed fabric status reporting capabilities.
2.4.1 SAN Volume Controller clustering In simple terms, a cluster is a collection of servers that, together, provide a set of resources to a client. The key point is that the client has no knowledge of the underlying physical hardware of the cluster. This means that the client is isolated and protected from changes to the physical hardware, which brings a number of benefits. Perhaps the most important of these benefits is high availability. Resources on clustered servers act as highly available versions of unclustered resources. If a node (an individual computer) in the cluster is unavailable, or too busy to respond to a request for a resource, the request is transparently passed to another node capable of processing it, so clients are unaware of the exact locations of the resources they are using.
Chapter 2. IBM System Storage SAN Volume Controller overview
13
For example, a client can request the use of an application without being concerned about either where the application resides or which physical server is processing the request. The user simply gains access to the application in a timely and reliable manner. Another benefit is scalability: If you need to add users or applications to your system and want performance to be maintained at existing levels, additional systems can be incorporated into the cluster. The IBM System Storage SAN Volume Controller is a collection of up to eight cluster nodes, added in pairs. In future releases, the cluster size will be increased to permit further performance scalability. These nodes are managed as a set (cluster) and present a single point of control to the administrator for configuration and service activity. Note: Although the SAN Volume Controller code is based on a Linux kernel, the clustering feature is not based on Linux clustering code. The clustering failover and failback feature is part of the SAN Volume Controller application software. Within each cluster, one node is defined as the configuration node. This node is assigned the cluster IP address and is responsible for transitioning additional nodes into the cluster. During normal operation of the cluster, the nodes communicate with each other. If a node is idle for a few seconds, then a heartbeat signal is sent to ensure connectivity with the cluster. Should a node fail for any reason, the workload intended for it is taken over by another node until the failed node has been restarted and re-admitted to the cluster (which happens automatically). In the event that the microcode on a node becomes corrupted, resulting in a failure, the workload is transferred to another node. The code on the failed node is repaired, and the node is re-admitted to the cluster (again, all automatically). For I/O purposes, SAN Volume Controller nodes within the cluster are grouped into pairs, called I/O groups, with a single pair being responsible for serving I/O on a given VDisk. One node within the I/O group represents the preferred path for I/O to a given VDisk. The other node represents the non-preferred path. This preference alternates between nodes as each VDisk is created within an I/O group to balance the workload evenly between the two nodes. Note: The preferred node by no means signifies absolute ownership. The data can still be accessed by the partner node in the I/O group in the event of a failure. Beyond automatic configuration and cluster administration, the data transmitted from attached application servers is also treated in the most reliable manner. When data is written by the host, the preferred node within the I/O group stores a write in its own write cache and the write cache of its partner (non-preferred) node before sending an “I/O complete” status back to the host application. The write cache is automatically destaged to disk after two minutes of no writes to a VDisk. To ensure that data is written in the event of a node failure, the surviving node empties all of its remaining write cache and proceeds in write-through mode until the cluster is returned to a fully operational state. Note: Write-through mode is where the data is not cached in the nodes, but written directly to the disk subsystem instead. While operating in this mode, performance can be degraded. More importantly, it ensures that the data makes it to its destination without the risk of data loss. A single copy of data in cache would constitute exposure to data loss.
14
Implementing the IBM System Storage SAN Volume Controller V4.3
Another data protection feature that the SAN Volume Controller has is uninterruptible power supply units. In addition to voltage regulation to protect valuable electronic components within the SAN Volume Controller configuration, in the event of a main power outage, the uninterruptible power supply provides enough power to destage data to the SAN Volume Controller internal disk and shut down the nodes within the SAN Volume Controller cluster gracefully. This is a feature found in most high-end disk subsystems.
2.4.2 SAN Volume Controller virtualization The SAN Volume Controller provides block aggregation and volume management for disk storage within the SAN. In simpler terms, this means that the SAN Volume Controller manages a number of back-end disk subsystem controllers and maps the physical storage within those controllers to logical disk images that can be seen by application servers and workstations in the SAN. The SAN must be zoned in such a way that the application servers cannot see the same back-end LUNs seen by the SAN Volume Controller, preventing any possible conflict between the SAN Volume Controller and the application servers both trying to manage the same back-end LUNs. As described earlier, when an application server performs I/O to a VDisk assigned to it by the SAN Volume Controller, it can access that VDisk through either of the nodes in the I/O group. Each node can only be in one I/O group, and since each I/O group only has two nodes, the distributed redundant cache design in the SAN Volume Controller only needs to be two-way. The SAN Volume Controller I/O groups are connected to the SAN in such a way that all back-end storage and all application servers are visible to all of the I/O groups. The SAN Volume Controller I/O groups see the storage presented to the SAN by the back-end controllers as a number of disks, known as managed disks. Because the SAN Volume Controller does not attempt to provide recovery from physical disk failures within the back-end controllers, MDisks are recommended, but not necessarily required, for a RAID array. The application servers must not see the MDisks at all. Instead, they should see a number of logical disks, known as virtual disks or VDisks, which are presented to the SAN by the SAN Volume Controller. MDisks are collected into groups, known as managed disk groups (MDGs). The MDisks that are used in the creation of a particular VDisk must all come from the same MDG. Each MDisk is divided into a number of extents. The minimum extent size is 16 MB, and the maximum extent size is 2048 MB, based on the definition of its MDG. These extents are numbered sequentially from the start to the end of each MDisk. Conceptually, this is represented as shown in Figure 2-1.
VDISK 1
Figure 2-1 Extents being used to create a virtual disk
Chapter 2. IBM System Storage SAN Volume Controller overview
15
The virtualization function in the SAN Volume Controller maps the VDisks seen by the application servers to the MDisks presented by the back-end controllers. I/O traffic for a particular VDisk is, at any time, handled exclusively by the nodes of a single I/O group. Although a cluster can have several pairs of nodes, the nodes handle I/O in independent pairs. This means that the I/O capability of the SAN Volume Controller scales well (almost linearly), since additional throughput can be obtained by simply adding additional I/O groups. Figure 2-2 summarizes the various relationships that bridge the physical disks through to the virtual disks within the SAN Volume Controller architecture.
Figure 2-2 The relationship between physical and virtual disks
Virtualization mappings Several different mapping functions are provided by the SAN Volume Controller: Striped: Here a VDisk is mapped to a number of MDisks in an MDG. The extents on the VDisk are striped over the MDisks. Therefore, if the VDisk is mapped to five MDisks, then the first, sixth, eleventh (and so on) extents come from the first MDisk, the second, seventh, twelfth (and so on) extents come from the second MDisk, and so on. This is the default mapping. Sequential: Here a VDisk is mapped to a single MDisk in an MDG. There is no guarantee that sequential extents on the MDisk map to sequential extents on the VDisk, although this might be the case when the VDisk is created.
16
Implementing the IBM System Storage SAN Volume Controller V4.3
Note: There are no ordering requirements in the MDisk to VDisk extent mapping function for either striped or sequential VDisks. This means that if you examine the extents on an MDisk, it is quite possible for adjacent extents to be mapped to different VDisks. It is also quite possible for contiguous extents on the MDisk to be mapped to widely separated extents on the same VDisk, or to close-by extents on the VDisk. In addition, the position of the extents on the MDisks is not fixed by the initial mapping, and can be varied by the user performing data migration operations. Image: Image mode sets up a one-to-one mapping of extents on an MDisk to the extents on the VDisk. Because the VDisk has exactly the same extent mapping as the underlying MDisk, any data already on the disk is still accessible when migrated to a SAN Volume Controller environment. Within the SAN Volume Controller environment, the data can (optionally) be seamlessly migrated off the image mode VDisk to a striped or sequential VDisk within the same or another MDG.
Virtual Disk Copy Every VDisk is associated to at least one VDisk Copy. The VDisk itself is a logical entity whereas the VDisk Copy is a physical entity. The VDisk Copy represents the physical capacity occupied by the VDisk on the MDisks. A second copy can be created as a VDisk Copy Mirror, as shown in Figure 2-3.
Figure 2-3 VDisk and VDisk copy
2.4.3 SAN Volume Controller multipathing Each SAN Volume Controller node presents a VDisk to the SAN through multiple paths. We recommend that a VDisk be seen in the SAN by four paths. In normal operation, two nodes provide redundant paths to the same storage. This means that, depending on zoning and SAN architecture, a single host might see eight paths, to each LUN presented by the SAN Volume Controller. Because most operating systems cannot resolve multiple paths back to a single physical device, IBM provides a multipathing device driver. The multipathing driver supported by the SAN Volume Controller is the IBM Subsystem Device Driver (SDD). SDD groups all available paths to a virtual disk device and presents it to the operating system. SDD performs all the path handling and selects the active I/O path(s).
Chapter 2. IBM System Storage SAN Volume Controller overview
17
2.4.4 SAN Volume Controller logical configuration Figure 2-4 shows an example of a SAN Volume Controller configuration.
Figure 2-4 SAN Volume Controller logical view
Cluster There is a maximum of four I/O groups (node pairs) per cluster. The performance of a cluster increases almost linearly with additional I/O groups.
I/O Group Because there is a minimum of two nodes per I/O group, there is no single point failure. Even if one node breaks away, data in the cache is not lost because the cache is mirrored between the two nodes. Every virtual disk is assigned to a single I/O group. For load balancing, a virtual disk is owned by alternating nodes in a I/O Group.
Configuration notes Here are some basic characteristics and recommendations in regard to the configuration: To provide high availability, we strongly recommend that the SAN Volume Controller nodes be configured in redundant SAN fabrics. The Fibre Channel switches need to be zoned to permit the hosts to see the SAN Volume Controller nodes and the SAN Volume Controller nodes to see the RAID Controllers. The SAN Volume Controller nodes within a cluster must be able to see each other. In addition, if there are two SAN Volume Controller clusters with Metro Mirror (formerly referred to as 18
Implementing the IBM System Storage SAN Volume Controller V4.3
Peer-to-Peer Remote Copy (PPRC)) and Global Mirror services between them, zoning must be set so that all the nodes of both clusters can see each other. In addition to Fibre Channel connections, each device has an Ethernet connection for configuration and error reporting. However, only one of the nodes, the configuration node, binds an IP address to its Ethernet connection. There is also a service IP address for a cluster that can be used by any node when in “service mode”.
2.5 Software licensing SVC V4.3.0 brings several changes to the licensing scheme. However, the scheme utilized in V4.2.1 will still be used until you add additional licenses to the cluster using svctask chlicense. Refer to the SVC V4.3.0 announcement letter for the definitive statement at the following link: http://tinyurl.com/4zgam3
2.6 New with SVC V4.3 For the most up-to-date information for the SVC, go to: http://www.ibm.com/storage/support/2145
2.6.1 Space-efficient VDisk A VDisk can now be created as a space-efficient VDisk. This means that a different virtual and real capacity is defined. The real capacity defines the space actually allocated to a VDisk. The VDisk capacity shown on the host is the virtual capacity. A directory maps the virtual address space to the real address space. Directory and user data share the real capacity, as shown in Figure 2-5.
Figure 2-5 Space-efficient VDisk
This enables advanced planning for future growth, as SVC administrators can expand the physical capacity without reconfiguration or downtime on the host. However, proper storage provisioning planning is still required, regardless of whether the configuration is space-efficient or fully provisioned. In SVC, it is possible to set warnings if a VDisk runs out of its capacity. These warnings should be configured sensibly to provide adequate time to provision more physical storage, and they should not be ignored! If required, more storage can be allocated and the real capacity can gradually be extended by grains up to the virtual capacity. The virtual capacity can also be expanded, but this is a manual process.
Chapter 2. IBM System Storage SAN Volume Controller overview
19
Note: If the used capacity reaches the real capacity, then the VDisk will go offline and application I/O will fail. To get the VDisk online again, the real capacity must be expanded. To avoid exhausting the real capacity, alerts can be sent to administrators to provide space, or by using autoextend to increase the real capacity automatically. An existing VDisk / Image mode VDisk cannot be converted to a space-efficient VDisk. It can be migrated afterwards to a space-efficient VDisk.
Read and write I/O process The different scenarios of read and write processes from or to a space-efficient VDisk is explained in the following summary: Write to an unallocated region When the directory lookup indicates that the region to be written to is unallocated, the SVC allocates space and updates the directory. Space is formatted while data and directory is written to the disk. Write to an allocated region When the directory lookup indicates that the region to be written to is already allocated. Data is written to the disk in this case. Read to an unallocated region (unusual) When the directory lookup indicates that the region to be read from is unallocated, the SVC returns a buffer of 0x00s. Read to an allocated region When the directory lookup indicates that the region to be read from is already allocated, data is read from the disk in this case.
2.6.2 FlashCopy Space-efficient FlashCopy. – Copy of a space-efficient source VDisk to a space-efficient target VDisk – Copy of a fully-allocated source VDisk to a space-efficient target VDisk The background copy process does not copy unallocated regions. The incremental feature allows you to refresh FC mapping once a fully copy of the actually regions is complete.
FlashCopy now supports up to 256 target copies from a single source virtual disk. Auto-delete of consistency group. Up to 4096 FlashCopy mappings. Licensing. – Virtual capacity of space-efficient source VDisk used in calculations – Multiple target mappings from the same source consume no extra license quota.
2.6.3 VDisk mirroring Allows creation of a single VDisk with one or two copies. Copies can be in different MDGs on different disk controllers. Can be used for migration. – Migrate VDisk between MDisk groups with different extent size – Migrate from space-efficient to fully-allocated VDisk – Control copy rate of migration
20
Implementing the IBM System Storage SAN Volume Controller V4.3
2.6.4 IPv6 addresses IPv6 addresses are now supported for: – Cluster IP – Service IP – SNMP server – E-mail server – ICAT address Can have both IPv4 and IPv6 addresses for a cluster and both can used at the same time.
2.6.5 Limitations 2048 VDisk per I/O group (double of existent maximum)
2.6.6 Licensing Metro Mirror/Global Mirror is licensed by capacity. – The Metro Mirror/Global Mirror license is calculated by the capacity needed on a cluster. – If the remote copy relationship affects two clusters, the license is calculated for each cluster individually. – If space-efficient virtual disks are part of the Metro Mirror/Global Mirror relationship, the virtual capacity is used for license calculations. Note: After upgrading, the licensing system will behave as in V4.2.1. until license values are added with the svctask chlicense command.
2.6.7 Miscellaneous New software upgrade status query. – Uses the svcinfo lssoftwareupgradestatus command. – Indicated states: inactive, upgrading, downgrading, committing, and stalled. The code level is displayed on the front panel. – The node level as well as the cluster level is shown on the front panel, so you can see a code upgrade as it happens. 1625 (controller misconfiguration) errors do not call home any more. – 1626 generates warning only. – Persistent unfixed 1625 errors cause a new 1695 error that does call home. – DS3K and DS4K misconfiguration calls home with a new 1624 error. Allow more time for back-end configuration changes and re-zoning changes to occur before taking MDisk offline. Report new node error 454 for internal disk going write-only. – The service action will be disk replacement. The Master Console is now replaced by SSPC. – The V4.3. GUI console code is supported on existing master consoles.
Chapter 2. IBM System Storage SAN Volume Controller overview
21
NetApp® gateway support for V4.2 and V4.3. – Ifix is available through RPQ for V4.2 in order to disable FNR and address an issue with the NetApp driver. – No ifix is planned for V4.3. – Full support of V4.2 and FNR will be available when Data ONTAP® V7.3 becomes generally available from NetApp. – Support of V4.3 will follow in 90 days and hopefully sooner. – The NSeries gateway support will follow later when it supports Data ONTAP V7.3. Change in support for split I/O group configurations. – An RPQ is needed with a review of a solution design by ATS or development is required. – No change is required for existing working split cluster environments. – Nodes in the I/O group must be physically within 100 meters of each other. – The node and its UPS must be in the same rack. – No ISLs can be used for node-to-node communication within an I/O group. – Currently, there are no LW GBIC/SFPs for SVC HBA, so the distance is limited through RPQ to 300 meters for 4 Gbs SAN or 600 meters for a 2 Gbs connection. – Requires IBM storage for quorum disks and they must be located in a third site.
2.6.8 Additional interoperability support since SVC 4.2.1 Microsoft® Virtual Server Host Environment Progressions: – NetWare V6.5 SP2 for 4 Gb QLogic® HBA – Tru64 V5.1 B4 – OpenVMS V8.3 – RHEL V4.6, including SAN boot – RHEL V5 Itanium® (IA64) – SLES 9 SP2 – SLES 10 for System z® – VMware® ESX Server V3.5, 3i Clustering Updates: – HP-UX11iV2, PVLinks, ServiceGuard 11.18 – 2 node MSCS, x32/x64, on BladeCenter HS21 – Steeleye Lifekeeper for Linux – SUN Cluster 3.2 – SLES 10 Native Clustering – RHEL 5 Native Clustering Oracle® 10g RAC/ASM support for: – Solaris™ 10 – HPUX11iV2 – SLES 10 – RHEL 4 – RHEL 5 Veritas Storage Foundation 5.x support for: – AIX 5L V5.2 – AIX 5L V5.3 – Solaris 10 x86 – Windows® 2003 with 2 Gb HBAs
22
Implementing the IBM System Storage SAN Volume Controller V4.3
Blade Updates: – JS22 Blade Support – JS12 Blade Support – HP Blade Server support QLogic 6140 Router Brocade-McDATA interop Mode Brocade DCX Director class SAN768B Fujitsu Eternus 8000 Model 2100 Disk System
2.6.9 SVC 4.3 Interoperability enhancements Host support – Microsoft Windows 2008, including Enterprise x64 Edition, SAN boot, 32-bit support, and clustering – Microsoft Windows 2008 Enterprise Edition for Itanium-based™ systems, including SAN boot – HP-UX 11i V3 for PA-RISC and Itanium-based systems, including clustering with HP ServiceGuard – Apple Mac OS X Server 10.5.2 with ATTO Celerity FC-42ES HBA – Storage system support – Pillar Axiom Models 300 and 500
Chapter 2. IBM System Storage SAN Volume Controller overview
23
24
Implementing the IBM System Storage SAN Volume Controller V4.3
3
Chapter 3.
Planning and configuration In this chapter, we describe the steps required when planning the installation of an IBM System Storage SAN Volume Controller (SVC) in your storage network. We look at the implications for your storage network.
© Copyright IBM Corp. 2003-2008. All rights reserved.
25
3.1 General planning rules To achieve the most benefit from SAN Volume Controller (SVC), pre-installation planning should include several important steps. These steps ensure that SVC provides the best possible performance, reliability, and ease of management for your application needs. Proper configuration also helps minimize downtime by avoiding changes to SVC and the storage area network (SAN) environment to meet future growth needs. Tip: The IBM System Storage SAN Volume Controller: Planning Guide, GA32-0551, contains comprehensive information that goes into greater depth regarding the topics we discuss here. Planning the SVC requires that you follow these steps: 1. Document the number of hosts (application servers) to attach to the SVC, the traffic profile activity (read or write, sequential or random), and the performance requirements (input/output (I/O) per second). 2. Document the storage requirements and capacities. – The total back-end storage to be provisioned on the SVC. – The required virtual storage capacity (space-efficient Virtual Disk) and its associated real capacity. – The required storage capacity for local mirror copy (Virtual Disk Mirroring). – The required storage capacity for point-in-time copy (FlashCopy). – The required storage capacity for remote copy (Metro and Global Mirror). – Per host: Storage capacity, the host logical unit number (LUN) quantity, and sizes. 3. Define the local and remote SAN fabrics and clusters, if a remote copy or a secondary site are needed. 4. Define the number of clusters and the number of pairs of nodes (between 1 and 4) for each cluster. Each pair of nodes (an I/O group) is the container for the virtual disks. How many I/O groups are needed depends on the overall performance requirements. 5. Design the SAN according to the requirement for high availability and best performance. Consider the total number of ports and the bandwidth needed between the host and the SVC, the SVC and the disk subsystem, between the SVC nodes, and for the ISL between the local and remote fabric. 6. Define the managed disks (MDisks) in the disk subsystem. 7. Define the managed disk groups (MDGs). This depends on the disk subsystem in place and the data migration needs. 8. Create and re-partition the VDisks between the different I/O groups and the different MDGs in such a way as to optimize the I/O load between the hosts and the SVC. This can be an equal re-partition of all the VDisks between the different nodes, or a re-partition that takes into account the expected load from the different hosts. 9. Plan for the physical location of the equipment in the rack. 10.Determine the IP addresses for the SVC Cluster, the SVC service IP address, and SSPC (SVC console). 11. Define the number of FlashCopy required per hosts. 12.Define the cluster configuration backup and business data backup.
26
Implementing the IBM System Storage SAN Volume Controller V4.3
3.2 Physical planning There are several main factors to take into account when carrying out the physical planning of an SVC installation. The physical site must have the following characteristics: Power, cooling, and location requirements are present for the SVC and uninterruptible power supplies. SVC nodes and uninterruptible power supplies must be in the same rack as its associated SVC node. Plan for two different power sources if you have ordered a redundant AC power switch (available as an optional feature). An SVC node is one EIA unit high. Each of the uninterruptible power supplies (UPSs) that comes with SVC V4.3 is one EIA unit high; the UPS shipped with the earlier version of the SVC is two EIA units high. The SSPC (SVC console) is two EIA units high: one for the server and one for the keyboard and monitor. Other hardware devices can be in the rack, such as IBM System Storage DS4000, IBM System Storage DS6000, SAN switches, Ethernet switch, and others. Consider the maximum power rating of the rack; this must not be exceeded.
Chapter 3. Planning and configuration
27
In Figure 3-1, we show the SVC in a rack shared with other storage equipment.
Figure 3-1 SVC in its rack
28
Implementing the IBM System Storage SAN Volume Controller V4.3
3.2.1 Preparing your UPS environment Ensure that your physical site meets the installation requirements for the uninterruptible power supply (UPS). Note: The 2145 UPS-1U is a Powerware 5115 and the 2145 UPS is a Powerware 5125.
2145 UPS-1U The 2145 uninterruptible power supply-1U (2145 UPS-1U) is one EIA unit high and is shipped, and can only operate, on the following node types: SAN Volume Controller 2145-8G4 SAN Volume Controller 2145-8F2 SAN Volume Controller 2145-8F4 It was also shipped and will operate with SAN Volume Controller 2145-4F2. When you configure the 2145 UPS-1U, the voltage that is supplied to it must be 200 – 240 V, single phase. Note: The 2145 UPS-1U has an integrated circuit breaker and does not require external protection.
2145 UPS The 2145 uninterruptible power supply (2145 UPS) is two EIA units high and was only shipped with the SAN Volume Controller 2145-4F2 prior to SVC V2.1. Be aware of the following considerations when configuring the 2145 uninterruptible power supply (2145 UPS): Each 2145 UPS must be connected to a separate branch circuit. A UL-listed 15 A circuit breaker must be installed in each branch circuit that supplies power to the 2145 UPS. The voltage that is supplied to the 2145 UPS must be single phase 200 – 240 V with a supplied frequency at 50 or 60 Hz.
Heat output The maximum heat output parameters are as follows: 40 watts (135 Btu per hour) during normal operation 150 watts (510 Btu per hour) when power has failed and the UPS is supplying power to the nodes of the SAN Volume Controller Ensure that you comply with the following requirements for UPSs: If the UPS is cascaded from another UPS, the source UPS must have at least three times the capacity per phase, and the total harmonic distortion must be less than 5% with any single harmonic being less than 1%. The UPS must also have input voltage capture that has a slew rate faster than 3 Hz per second and 1 msec glitch rejections. For more detailed information, refer to the IBM System Storage SAN Volume Controller Planning Guide, GA32-0551.
Chapter 3. Planning and configuration
29
3.2.2 Physical rules The SVC must be installed in pairs to provide high availability, and each node in an I/O group must be connected to different UPSs, as shown in Figure 3-2.
Figure 3-2 Node uninterruptible power supply setup
In SVC versions prior to SVC V2.1, the Powerware 5125 UPS was shipped with the SVC; in SVC V4.2, Powerware 5115 UPS is shipped with the SVC. You can upgrade an existing SVC cluster to V4.1 and still use the UPS Powerware 5125 that was delivered with the SVC prior to V2.1. Each SVC node of an I/O group must be connected to a different UPS. Each UPS shipped with SVC V3.1, V4.1, V4.2, and V4.3 supports one node only, but each UPS in earlier versions of SVC supports up to two SVC nodes (in distinct I/O groups). Each UPS pair that supports a pair of nodes must be connected to a different power domain (if possible) to reduce the chances of input power loss. The UPSs must be installed in the lowest available position in the rack. If necessary, move lighter units toward the top. A cluster can contain up to eight SVC nodes. The power and serial connection from a node must be connected to the same UPS, otherwise the node will not boot. The 5115 and 5125 UPS can be mixed with UPSs that were supplied with earlier SVC versions, but the UPS rules above have to be followed, and SVC nodes in the same I/O group must be attached to the same type of UPSs, though not the same UPS. 8G4, 8F2, and 8F4 hardware models must be connected to 5115 UPS. They will not boot with a 5125 UPS. Important: Do not share the SVC UPS with any other devices.
30
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 3-3 shows a layout sample within a rack.
Figure 3-3 Sample rack layout
3.2.3 Cable connections Complete a cable connection table to document all of the connections required for the setup:
Nodes UPS Ethernet Fibre Channel ports SSPC (SVC Console)
Chapter 3. Planning and configuration
31
Parts of a typical planning chart are shown in Figure 3-4 and Figure 3-5.
Figure 3-4 Cable connection table
Figure 3-5 SVC (Master) Console
3.3 SAN planning and configuration SAN storage systems using the SVC can be configured with two, or up to eight, SVC nodes, arranged in an SVC cluster. These are attached to the SAN fabric, along with disk subsystems and host systems. The SAN fabric is zoned to allow the SVCs to “see” each other’s nodes and the disk subsystems, and for the hosts to “see” the SVCs. The hosts are not able to directly “see” or operate LUNs on the disk subsystems that are assigned to the SVC cluster. The SVC nodes within an SVC cluster must be able to see each other and all the storage assigned to the SVC cluster. The zoning capabilities of the SAN switch are used to create these distinct zones. The SVC in Release 4 supports 1 Gbps, 2 Gbps, or 4 Gbps Fibre Channel fabric. This depends on the hardware platform and on the switch where the SVC is connected. We recommend connecting the SVC and the disk subsystem to the switch operating at the highest speed, in an environment where you have a fabric with multiple speed switches. All SVC nodes in the SVC cluster are connected to the same SANs, and present virtual disks to the hosts. These virtual disks are created from managed disks presented by the disk subsystems. There are two distinct zones in the fabric: Host zones, to allow host ports to see and address the SVC nodes. There can be multiple host zones. See 3.3.3, “General design considerations with the SVC” on page 35 for more information. One disk zone in which the SVC nodes can see and address the LUNs presented by the disk subsystems. 32
Implementing the IBM System Storage SAN Volume Controller V4.3
Hosts are not permitted to operate on the disk subsystem LUNs directly if the LUNs are assigned to the SVC. All data transfer happens through the SVC nodes. Under some circumstances, a disk subsystem can present LUNs to both the SVC (as managed disks, which it then virtualizes to hosts) and to other hosts in the SAN. Figure 3-6 shows the data flow across the physical topology.
Host
Host
Host
Host Host Zone
SVC SVC
Disk Zone
Managed disks
SVC SVC RAID Ctrl
RAID Ctrl
RAID Ctrl
RAID Ctrl
Data Transfer
Figure 3-6 Data flows on a SVC physical topology
Logically, the two zones can be thought of as two separate logical SANs.
3.3.1 SAN definitions The following definitions are used in this section.
ISL hop An interswitch link (ISL) is a connection between two switches, and is counted as an “ISL hop.” The number of “hops” is always counted on the shortest route between two N-ports (device connections). In an SVC environment, the number of ISL hops is counted on the shortest route between the pair of nodes farthest apart. It measures distance only in terms of ISLs in the fabric.
Oversubscription Oversubscription is the ratio of the sum of the traffic on the initiator N-port connection, or connections to the traffic on the most heavily loaded ISL(s) where more than one is used between these switches. This assumes a symmetrical network, and a specific workload applied evenly from all initiators and directed evenly to all targets. A symmetrical network means that all the initiators are connected at the same level, and all the controllers are connected at the same level.
Chapter 3. Planning and configuration
33
As an example, on a 16-port switch, where there are 14 host connections going through two ISL connections, the oversubscription is 14:2, or 7:1 (14/2), with seven hosts “sharing” one ISL. In the SVC environment, each ISL link oversubscription cannot exceed six.
Redundant SAN A redundant SAN is a SAN configuration in which there is no single point of failure (SPoF), so no matter what component fails, data traffic will continue. Connectivity between the devices within the SAN is maintained, albeit possibly with degraded performance, when an error has occurred. A redundant SAN design is normally achieved by splitting the SAN into two independent counterpart SANs (two SAN fabrics), so even if one counterpart SAN is destroyed, the other counterpart SAN keeps functioning.
Counterpart SAN A counterpart SAN is a non-redundant portion of a redundant SAN. A counterpart SAN provides all the connectivity of the redundant SAN, but without the 100% redundancy. An SVC node is typically connected to a redundant SAN made out of two counterpart SANs. A counterpart SAN is often called a SAN fabric. For example, if you have only one switch, this is one fabric, or one counterpart SAN. However, if you have two switches, but they are not connected together, you have two fabrics, or two counterpart SANs, and one redundant SAN if the devices are connected to both SANs.
Local fabric Since the SVC supports remote copy, there might be significant distances between the components in the local cluster and those in the remote cluster. The local fabric is composed of those SAN components (switches, cables, and so on) that connect the components (nodes, hosts, and switches) of the local cluster together.
Remote fabric Since the SVC supports remote copy, there might be significant distances between the components in the local cluster and those in the remote cluster. The remote fabric is composed of those SAN components (switches, cables, and so on) that connect the components (nodes, hosts, and switches) of the remote cluster together.
Local and remote fabric interconnect These are the SAN components that are used to connect the local and remote fabrics. They might simply be single mode optical fibers driven by high-power GBICs or SFPs, or they might be other more sophisticated components, such as channel extenders or special SFP modules. This can be used to extend the distance to thousands of kilometers. Performance may degrade as distance increases.
Fibre Channel port logins This is the number of hosts that can see any one SVC node port. Some disk subsystems, such as the IBM DS8000, recommend limiting the number of hosts that use each port, to prevent excessive queuing at that port. Clearly, if the port fails or the path to that port fails, the host might fail over to another port and the fan-in criteria might be exceeded in this degraded mode.
Channel extender A channel extender is a device for long distance communication connecting other SAN fabric components. Generally, these can involve protocol conversion to asynchronous transfer mode (ATM), Internet Protocol (IP), or some other long distance communication protocol.
34
Implementing the IBM System Storage SAN Volume Controller V4.3
3.3.2 Fibre Channel switches, fabrics, interswitch links, and hops The local or remote fabric must not contain more than three hops in each fabric. Any configuration that causes this to be exceeded is unsupported. When a local fabric is connected to a remote fabric for Metro Mirror, the hop count between a local node and a remote node must not exceed seven. For example, node A in fabric A wishes to connect to node B in fabric B. Within fabric A, node A takes two hops to reach the ISL that connects fabric A to fabric B. The hop count is at two at this point in time. Traversing the ISL between fabric A and fabric B takes the hop count to three. Once it has reached fabric B, it takes three hops to reach node B. The hop count total is six (2+1+3), which is within our limits. Another example would be where three hops have been used within fabric A, and three hops have been used in fabric B. This means that there can only be one hop between fabric A and fabric B, otherwise the supported hop count limit of seven will be exceeded. Alternatively, where, for example, fabric A consisted of one hop, and fabric B also consisted of one hop, then this would leave up to five hops that could be used to interconnect switches between fabric A and fabric B. If multiple ISLs are available between switches, we recommend that these ISLs be trunked. Follow the switch vendor's recommendations for trunking. Note: The SVC supports the use of distance extender technology to increase the overall distance between local and remote clusters; this includes DWDM and FCIP extenders. If this extender technology involves a protocol conversion, then the local and remote fabrics should be regarded as independent fabrics, limited to three hops each. The only restriction on the interconnection between the two fabrics is the maximum latency allowed in the distance extender technology. For the latest information relating to distance limitations, visit the following Web site: http://www.ibm.com/storage/support/2145
3.3.3 General design considerations with the SVC Note: The SVC system in and of itself does not support disk redundancy, so to maintain access in the case of a disk failure requires redundancy within the disk subsystem below the SVC. To ensure high availability in SVC installations, keep the following considerations in mind when you design a SAN with the SVC.
For any SVC cluster The following general guidelines apply: An SVC node, in this case, 4F2 and 8F2, always contains two host bus adapters (HBAs), each of which has two Fibre Channel (FC) ports. If an HBA fails, this remains a valid configuration, and the node operates in degraded mode. If an HBA is physically removed from an SVC node, then the configuration is unsupported. The 8G4 and 8F4 has one HBA with four ports. All nodes in a cluster must be in the same LAN segment. This is because the nodes in the cluster must be able to assume the same cluster, or service IP, address. Make sure that the network configuration will allow any of the nodes to use these IP addresses. Chapter 3. Planning and configuration
35
To maintain application uptime in the unlikely event of an individual SVC node failing, SVC nodes are always deployed in pairs (I/O groups). If a node fails or is removed from the configuration, the remaining node operates in a degraded mode, but is still a valid configuration. The remaining node operates in write through mode, meaning the data is written directly to the disk subsystem (the cache is disabled for the write). The UPS must be in the same rack as the node it supplies, and each UPS can only have one node connected. Nodes shipped pre-SVC V2.1, with the 2145 UPS (Powerware 5125), can have two nodes connected per UPS. The Fibre Channel SAN connections between the SVC node and the switches are optical fiber. These connections can run at either 1 Gbps, 2 Gbps, or 4 Gbps, depending on your SVC and switch hardware. The 8G4 and 8F4 SVC nodes auto-negotiate the connection speed with the switch. The 4F2 and 8F2 nodes are capable of a maximum of 2 Gbps, which is determined by the cluster speed. SVC node ports must be connected to the Fibre Channel fabric only. Direct connections between SVC and host, or disk subsystem, are unsupported. We recommend that the two nodes within an I/O group be co-located, and co-location is a recommendation even for an SVC cluster (all nodes in a cluster should be located close to one another (within the same set of racks), and within the same room or adjacent rooms for ease of service and maintenance). An SVC cluster can be connected (through the SAN fabric switches) to application hosts, disk subsystems, or other SVC clusters, through short wave only optical FC connections (long wave connections are no longer supported). These can be distances of up to 150 m (short wave 4 Gbps), 300 m (short wave 2 Gbps), or 500 m (short wave 1 Gbps) between the cluster and the host, and between the cluster and the disk subsystem. Longer distances are supported between SVC clusters when using intercluster Metro or Global Mirror. IBM has tested up to 10km SFP long wave links, but we actually support whatever the manufacturer of that fabric supports in terms of long distance links. Refer to 3.5.3, “Technologies for extending the distance between two SVC clusters” on page 45 for more information. A cluster should be regarded as a single entity for disaster recovery purposes. This includes the disk subsystem that is providing the quorum disks for that cluster. This means that the cluster and the quorum disks should be co-located. We do not recommend locating the components of a single cluster in different physical locations for the purpose of disaster recovery, because this might lead to issues over maintenance, service, and quorum disk management.
For multiple SVC clusters Two SVC clusters cannot share the same disk subsystem. The consequences of sharing the same disk subsystem can result in data loss. If the same MDisk becomes visible on two different SVC clusters, then this is an error that can cause data corruption.
For the SAN fabric The following guidelines apply: The Fibre Channel switch must be zoned to permit the hosts to see the SVC nodes, and the SVC nodes to see the disk subsystems. The SVC nodes within a cluster must be able to see each other, the disk subsystems, and the front-end host HBAs. We recommend that you have a zone for the SVC nodes within a cluster, another zone between the SVC nodes and the disk subsystems, and a zone between each front-end host HBA and the SVC. Mixed speeds are permitted within the fabric, but not for intracluster communication. You can use lower speeds to extend the distance.
36
Implementing the IBM System Storage SAN Volume Controller V4.3
Uniform SVC port speed for 4F2 and 8F2 nodes: The optical fiber connections between Fibre Channel switches and all 4F2 or 8F2 SVC nodes in a cluster must run at one speed, either 1 Gbps or 2 Gbps. Operating 4F2 or 8F2 nodes with different speeds running on the node to switch connections in a single cluster is an unsupported configuration (and is impossible to configure anyway). This rule does not apply to 8F4 and 8G4 nodes because the Fibre Channel ports on these nodes auto-negotiate their speed independently of one another and can run at 1 Gbps, 2 Gbps, or 4 Gbps Each of the local or remote fabrics should not contain more than three ISL hops within each fabric. An operation with more ISLs is unsupported. When a local and a remote fabric are connected together for remote copy purposes, there should only be one ISL hop between the two SVC clusters. This means that some ISLs can be used in a cascaded switch link between local and remote clusters, provided that the local and remote cluster internal ISL count is less than three. This gives a maximum of seven ISL hops in an SVC environment with both local and remote fabrics. The switch configuration in an SVC fabric must comply with the switch manufacturer’s configuration rules. This can impose restrictions on the switch configuration. For example, a switch manufacturer might limit the number of supported switches in a SAN. Operation outside the switch manufacturer’s rules is not supported. The SAN contains only supported switches as listed on the Web at: http://www.ibm.com/storage/support/2145 Operation with other switches is unsupported. Host HBAs in dissimilar hosts or dissimilar HBAs in the same host need to be in separate zones. For example, if you have AIX and Microsoft hosts, they need to be in separate zones. Here dissimilar means that the hosts are running different operating systems or use different hardware platforms. Therefore, different levels of the same operating system are regarded as similar. This is a SAN interoperability issue rather than an SVC requirement. We recommend that the host zones contain only one initiator (HBA) each, and as many SVC node ports as you need, depending on the high availability and performance you want to have from your configuration. Note: In SVC Version 3.1 and later, the command svcinfo lsfabric generates a report that displays the connectivity between nodes and other controllers and hosts. This is particularly helpful in diagnosing SAN problems.
For the disk subsystem The following guidelines apply: In the SAN, disk subsystems are always connected to SAN switches and nothing else. Multiple connections are allowed from the redundant controllers in the disk subsystem to improve data bandwidth performance. It is not mandatory to have a connection from each redundant controller in the disk subsystem to each counterpart SAN. For example, in a DS4000 configuration in which the DS4000 contains two redundant controllers, only two controller minihubs are normally used. This means that Controller A in the DS4000 is connected to counterpart SAN A, and controller B in the DS4000 is connected to counterpart SAN B. Operation with direct connections between host and controller is unsupported. Split controller configurations are supported with certain rules and configuration guidelines. Refer to IBM System Storage SAN Volume Controller Planning Guide, GA32-0551 for more information.
Chapter 3. Planning and configuration
37
The SVC is configured to manage LUNs exported only by disk subsystems, as listed at: http://www.ibm.com/storage/support/2145 Operation with other disk subsystems is unsupported. All SVC nodes, in an SVC cluster, must be able to see the same set of disk subsystem ports on each disk subsystem controller. An operation in a mode where two nodes see a different set of ports on the same controller becomes degraded. The system logs errors requesting a repair action. This can occur if inappropriate zoning was applied to the fabric. It can also occur if inappropriate LUN masking is used. This has important implications for a disk subsystem, such as DS4000, which imposes exclusivity rules on which HBA world wide names (WWNs) a storage partition can be mapped to. It is up to you to check that the planned configuration is supported. You can find the supported hardware list at: http://www.ibm.com/storage/support/2145
For the host and application servers The following guidelines apply: Each SVC node presents a virtual disk (VDisk) to the SAN through four paths. Since in normal operation two nodes are used to provide redundant paths to the same storage, this means that a host with two HBAs can see eight paths to each LUN presented by the SVC. We suggest using zoning to limit the pathing from a minimum of two paths to the maximum available of eight paths, depending on the kind of high availability and performance you want to have in your configuration. We recommend using zoning to limit the pathing to four paths. The hosts must run a multipathing device driver to resolve this back to a single device. The multipathing driver supported and delivered by SVC is the IBM Subsystem Device Driver (SDD). Native multipath I/O (MPIO) drivers on selected hosts are supported. For operating system specific information about MPIO support, see: http://www.ibm.com/storage/support/2145 The number of paths to a VDisk from a host to the nodes in the I/O group that owns the VDisk must not exceed eight, even if this is not the maximum number of paths supported by the multi-path driver (SDD supports up to 32). To restrict the number of paths to a host VDisk, the fabric(s) should be zoned such that each host Fibre Channel port is zoned with one port from each SVC node in the I/O group that owns the VDisk. Note: The recommended number of VDisk paths is 4. If a host has multiple HBA ports, then each port should be zoned to a different set of SVC ports to maximize high availability and performance. In order to configure greater than 256 hosts, you will need to configure the host to iogrp mappings on the SVC. Each iogrp can contain a maximum of 256 hosts, so it is possible to create 1024 host objects on an eight node SVC cluster. The mappings can be configured using the svctask mkhost, svctask addhostiogrp, and svctask rmhostiogrp commands. The mappings can be viewed using the svcinfo lshostiogrp and svcinfo lsiogrphost commands. VDisks can only be mapped to a host that is associated with the I/O Group to which the VDisk belongs. The SAN Volume Controller supports connection to the Cisco MDS 9000 SAN-OS Software Release 3.2 for the Cisco MDS 9000 Family platform with the attached iSCSI hosts (in single path mode only). See the following Web site for the latest support information: http://www.ibm.com/storage/support/2145
38
Implementing the IBM System Storage SAN Volume Controller V4.3
For management The following guidelines apply: In addition to a Fibre Channel connection, each node has an Ethernet connection for configuration and error reporting. These connections are aggregated together through an Ethernet switch. All nodes in an SVC cluster must be in the same IP subnet. This is because the node in the SVC cluster must be able to assume the same SVC cluster IP address or SVC service IP address. IBM supports the option of having two master consoles to provide redundancy. Multiple master consoles can access a single cluster, but when multiple master consoles access one cluster, you cannot concurrently perform configuration and service tasks. The master console can be either preinstalled hardware or software supplied to and installed by the user.
3.3.4 Boot support The SVC supports SAN boot for AIX and Windows 2003 using MPIO, HP-UX by using PVLinks as the multipathing software for the boot device, and Solaris 9 running Veritas Volume Manager/DMP, but the SAN boot support could change from time to time, so we recommend regularly checking the following Web site: http://www.ibm.com/systems/storage/software/virtualization/svc/interop.html
3.3.5 Configuration saving We recommend that you save the configuration externally, when changes such as adding new nodes, disk subsystems, and so on have been done in the cluster.
Chapter 3. Planning and configuration
39
3.3.6 High availability SAN design and configuration rules with SVC Figure 3-7 shows a basic two node configuration. To provide high availability, the SVC should be configured in redundant SAN fabrics. Our configuration, as shown in Figure 3-7, is a redundant fabric made up of two 16-port switches.
W2K3_1 P1
AIX_270
P2
P1
P2
Linux P1
W2K3_2
P2
P1
P2
SSPC
P1 (Master Console)
P2
p4
p3
p3 0 3 4 8 9 12 14 SW11 SAN switch
SVC1 node1
14 12 9 8 4 3 0 SW21 SAN switch
1 2 5
p4 SVC1 node2
5 2 1
p2
p1
p1
p2
DS4300_A
DS4300_B
Figure 3-7 Simple two-node SVC high availability configuration
3.4 Zoning To manage the zoning in the fabric, port zoning or WWN zoning can be used. For the latest information, regularly check whether your configuration is supported at the following Web site: http://www.ibm.com/storage/support/2145 Figure 3-8 on page 41 shows an example of a host zone where each host adapter is zoned to two SVC I/O group HBA ports, one from each node in the I/O group. The numbers 11, 12, 13, and 14 are for node1 FC port 1, 2, 3, and 4. The blue zone consists of A1, 13, and 22, meaning host adapter FC port 1, node1 FC port 3, and node2 FC port 2. The position in the switch where the FC port is connected is used when making the zone definitions. For example, 13 is connected to the switch with domain ID 11 port 3, (11,3); remember that the first port in the switch starts numbering at zero.
40
Implementing the IBM System Storage SAN Volume Controller V4.3
Host
HOST-A1 Zone (21,1;11,2;11,3)
HOST-A2 Zone (22,1;12,2;12,3)
A1 A2 ID 21
A1 Fabric 1
Fabric 2 ID 11
11 22 13 24 Host port A1 and one SVC port per SVC node
ID 22
A2
11 12 13 14 SVC node1
21 12 23 14
ID 12
21 22 23 24 Host port A2 and SVC node2
Note: No storage subsystem ports!
one SVC port per SVC node
Figure 3-8 Host zoning example
With this zoning configuration, each VDisk has four paths to the host. A new host will be configured with zoning that uses the remaining, not yet used SVC nodes ports 11 , 14 , 21 , and 24 , and in this way we will perform manual load balancing on the SVC nodes in the SVC cluster. More hosts will use the same route in a round robin fashion.
3.5 Naming conventions Naming conventions in the open systems environment have always been a challenge. The challenges come from finding naming conventions that will continue to be steady as changes occur to the environment. Everyone has their own way of naming equipment in an IT infrastructure. When working in a SAN environment where an SVC cluster is installed, we recommend assigning names that help in locating and identifying equipment, and that provide information about connections so that any changes and troubleshooting are easier. One way to do this is to include site name, equipment name, and adapter information in the naming convention. As an example, in a two-site solution, site A and site B, all equipment in site A is identified by odd numbers, while all equipment at site B is identified by even numbers. Such an example could look like this: SVC1N1P1, where SVC1 is the name of the equipment, number 1 indicates that it is located in site A, N1 is the node name for the SVC cluster, and the P1 is the SVC FC port number. On site B, the name would have been SVC2N1P1. (Note that names stemming from popular culture can be amusing, but do not always give any meaningful information about what the equipment is or where it is located, which has several disadvantages.) Chapter 3. Planning and configuration
41
Figure 3-9 shows an example of a naming convention.
Naming convention
SW11
SW12
SVC1N2
SVC2N2
SVC1N1
SVC2N1
SW21
FAST1-A
DS4301_A
SW22
FAST1-B
DS4301_B
FAST2-A
DS4302_A
FAST2-B
DS4302_B
Figure 3-9 An example of a name convention in a dual SVC setup
Below, we list examples of names in an SVC cluster setup, and we document those which you can use as examples to build your own naming convention. SVC naming convention examples: SVC1N1 = SVC cluster 1, node 1 SVC2N1 = SVC cluster 2, node 1 SVC2N2P3 = SVC cluster 2, node 2, FC port 3 Disk subsystem name convention examples:
DS4301_A DS4302_B EVA301_A DS801B3A1
Here is an explanation of names for the disk subsystem: DS4301_A_1, where DS43 tells you the type of storage back end (in our example, a DS4300). DS4301_A_1, where 01 is the number of this DS4300 in your installation, and also gives you the information that it is placed at site A (1 is an odd number). DS4301_A_1, where _A is the name of the controller in the DS4300. DS4301_A_1, where _1 is the FC port number on controller A, but port number is only used in SAN zoning information, not on the SVC. Here we would only recommend DS4301 as the name. DS4302_B will then mean a DS4300, located at site B, controller B. EVA301_A will then mean a EVA3000, located at site A, controller A.
42
Implementing the IBM System Storage SAN Volume Controller V4.3
Host Fibre Channel ports FC0 and FC1: HOSTNAMExx_FC0 HOSTNAMExx_FC1 Here, xx is a number used to identify the server and give information as to its location. For example, AIX01_fc0, AIX01_fc1 gives you the type of server, what server it is, and where it is located, in this example, at site A. SAN switch names: SW11, where SW indicates a switch, the first number gives a fabric ID, and the last is the location, and even combining the last two numbers together is also useful as the domain ID of the switch. SW12 is then SAN switch 2 in fabric 1, located at site B, domain id 12. SW21 is then SAN switch 1 in fabric 2, located at site A, domain id 21. When using these kinds of names, we have a limit of 10 switches in a fabric, so if we need more switches, we will use two digits for the switch number, for example, SW101 and so on. SVC zone names: SVC1, which includes SVC1N1P1, SVC1N1P2, SVC1N1P3, SVC1N1P4, SVC1N2P1, and so on for all SVC1 nodes. SVC2, which includes all SVC2 node ports. Storage zone names: SVC1_DS4301 SVC2_DS4302 Host zone names: HOSTNAME_FC0_SVC1 HOSTNAME_FC1_SVC1 Metro or Global Mirror zone name: SVC1_SVC2 Changing the domain IDs can affect your zoning configuration and the existing setup. Therefore, you must change them first before you change your zoning information. Note: A change of the domain ID or core PID disrupts some UNIX operating systems. Make sure that you check first before you attempt this when storage is already defined and in use.
Chapter 3. Planning and configuration
43
3.5.1 Dual room high availability configuration with the SVC Figure 3-10 shows a high availability configuration of the SVC when two SVC clusters are in two different rooms/locations. We recommend this configuration for maximum availability.
Hostnn
Host01
Up to 64 hosts in each SVC cluster
SVC cluster1
SVC1N1
Host02
Hostnn
SVC cluster2
I/O group 0
SVC2N1
I/O group 0
SVC1N2
SVC2N2
SVC1N3
SVC2N3 I/O group 1
I/O group 1
SVC1N4
SVC2N4
SW11
UPS1A
UPS2A
Up to 256 ports in each fabric
UPS1B
DS4301_A
SW12
SW21
DS4301_B
UPS2B
SW22
Up to 64 controllers per cluster Site B Site A
DS4302_A
DS4302_B
Figure 3-10 High availability SAN Volume Controller cluster in a two-site configuration
3.5.2 Local and remote SAN fabrics with SVC The SVC supports both intracluster and intercluster Metro and Global Mirror. From the intracluster point of view, the only reasonable candidate for a Metro or Global Mirror operation is the other node in the same I/O group. Intercluster operation needs a pair of clusters, separated by a number of moderately high bandwidth links, which means that the bandwidth should be large enough to handle the writes from the Metro or Global Mirror process, but no reads are done over the links. Here we describe the configuration that is shown in Figure 3-10: In the SVC, the local and remote fabric interconnect supported is a single ISL hop between a switch in the local fabric and a switch in the remote fabric, if it is a single mode fiber, up to 10 km in length. Check the support Web site for any changes to this configuration at: http://www.ibm.com/storage/support/2145 SVC V4.3 supports a operation with Fibre Channel DWDM extenders and SAN Routers. The supported distances depend on the SAN fabric vendor, but the latency must be better than an 80 ms round-trip delay (40 ms one way). In Metro or Global Mirror configurations, additional zones are required that contain only the local nodes and the remote nodes. It is unsupported to create a zone that presents both the local and remote disk subsystem and either local and remote nodes or both. 44
Implementing the IBM System Storage SAN Volume Controller V4.3
3.5.3 Technologies for extending the distance between two SVC clusters Technologies for extending the distance between two SVC clusters can be divided into two categories: Fibre Channel Extenders SAN Routers
Fibre Channel extenders Fibre Channel extenders simply extend a Fibre Channel link by transmitting Fibre Channel packets across long distance links without changing the contents of those packets. Here is a list of examples: FCIP extenders implemented in CISCO MDS 9500 series switches CNT Ultranet Edge Storage Router DWDM, CWDM, and longwave SFP extenders Any Multiprotocol router (for example, Brocade Multiprotocol Routers) only when used in FCIP tunnelling mode. The maximum supported one way latency is 34 ms. Any Fibre Channel technology is supported that meets the following requirements: The one-way latency between sites must not exceed 34 ms. Note that 1 ms equates to approximately 100 km, but this will depend on the quality of circuit, the type of equipment used, and the configuration. The bandwidth between sites must be sized to meet peak workload requirements while maintaining the maximum latency of 34 ms. If the link between sites is configured with redundancy so that it can tolerate single failure, then the link must be sized so that the bandwidth and the latency statements continue to hold true even during such single failure conditions. A channel extender can be used only for inter-cluster links (intra-cluster is not supported). The entire configuration must be tested with the expected peak workload. The configuration is tested to simulate a failure of a primary site, and eventually a failback from the secondary site to the primary site (to test recovery procedures). The configuration is tested to confirm that any failover mechanism in the inter-cluster links interoperate with the SVC. Particular attention must be taken for the compatibility between switches if they are from different vendors, and also between the switches and extender. Measurements about latency and bandwidth must be made during installation and the records must be kept. Testing should be repeated before and after any significant change in the infrastructure providing the inter-cluster links.
SAN Routers SAN Routers extend the scope of a SAN by providing “virtual nPorts” on two or more SANs. The router arranges it so that traffic at one virtual nPort is propagated to the other virtual nPort, but the two Fibre Channel fabrics are independent of one another. Thus, nPorts on each of the fabrics cannot directly log into each other.
Chapter 3. Planning and configuration
45
At the time of writing, some of the systems IBM supported are: McDATA 1620 and 2640: These are supported up to a one way latency of 10 ms. Note that 1 ms equates to approximately 100 km to 150 km, but this will depend on the type of equipment used, links, and the configuration. Cisco MDS 9000 series Inter VSAN Routing: The use of Inter VSAN Routing in the configuration using MD 9000 series switches is supported with latency up to 10 ms. Latency is approximately about 1 ms for 100-150 km, but this will depend on the type of equipment used, links, and the configuration. You can find the detailed supported list of inter-cluster extenders and routers on the Web at: http://www-03.ibm.com/systems/storage/software/virtualization/svc/interop.html Important: It is the latency that IBM will support, not the distance. The above distances are provided for illustrative purposes only.
3.6 SVC disk subsystem planning This section covers planning SVC disk subsystem components.
3.6.1 Block virtualization The managed disk group (MDG) is at the center of the many-to-many relationship between managed disks and virtual disks. It acts as a container into which managed disks contribute chunks of disk blocks, known as extents, and from which virtual disks consume these extents of storage. MDisks in the SVC are LUNs assigned from the underlying disk subsystems to the SVC, and can be either managed or unmanaged. A managed MDisk is an MDisk assigned to an MDG. MDGs are collections of managed disks. A managed disk is contained within exactly one MDG. An SVC supports up to 128 MDGs. There is no limit to the number of virtual disks that can be in an MDG other than the limit per cluster. MDGs are collections of virtual disks. Under normal circumstances, a virtual disk is associated with exactly one MDG. The exception to this is when a virtual disk is migrated, or mirrored, between MDGs.
3.6.2 MDGs, I/O groups, virtual disks, and managed disks Figure 3-11 on page 47 shows three disk subsystems that were configured to provide a number of LUNs. In SVC terminology, each of these logical units (LUNs) is a managed disk. Disk subsystem A contains two managed disks, known as M1 and M2. Disk subsystem B contains managed disks M3 and M4. Disk subsystem C contains managed disks M5 and M6.
46
Implementing the IBM System Storage SAN Volume Controller V4.3
I/O Group RS
I/O Group PQ SVC Node
SVC Node
P
Q
SVC Node
Some other I/O group
Some other I/O group
V3
V2
V1
Managed Disk Group Y
Managed Disk Group X
M1
SVC Node S
R
M2
Disk Subsystem A
M3
M4
Disk Subsystem B
V5
V4
V6
V7
Managed Disk Group Z
M5
M6
Disk Subsystem C
Figure 3-11 Disk relationships
Figure 3-11 also shows three MDGs: X, Y, and Z. Each managed disk is contained within a single MDG, and that one MDG can span disk subsystems. SVC supports an arbitrary relationship between disk subsystems and MDGs. The MDG simply contains a collection of managed disks from the set of available disk subsystems. Note: For LUNS to be a part of same MDG, we recommend that all the LUNs that are part of that MDG come from: Disk subsystems having similar characteristic, for example, IOPS, cache, throughput, and so on. Hard disks having the same characteristics, for example, type, speed, and RPM. For further details, refer to SAN Volume Controller: Best Practices and Performance Guidelines, SG24-7521. Figure 3-11 also shows virtual disks numbered V1 to V7. Each virtual disk is contained entirely within an MDG. This is a normal situation. The only exception to this is during migration. Virtual disks are also members of another collection, namely I/O groups. Figure 3-11 shows two I/O groups named PQ and RS. An I/O group contains an arbitrary set of virtual disks and exactly two SVC nodes (unless one has failed). The I/O group defines which nodes support I/O request from hosts.
Chapter 3. Planning and configuration
47
There is no fixed relationship between I/O groups and MDGs. An individual virtual disk is normally a member of one MDG and one I/O group: The MDG defines virtual disk is carved using which managed disks, which are coming from the underlying disk subsystem. The I/O group defines which SVC nodes provide I/O access to the virtual disk.
3.6.3 Extents A virtual disk occupies an integer number of extents. Its length does not need to be an integer multiple of the extent size, but must be an integer multiple of the block size. Any space left over between the last logical block in the virtual disk and the end of the last extent in the virtual disk is unused. You can define a VDisk with the smallest granularity of 512 bytes (a block). However, an entire extent is reserved even if it is partially used. An extent is never shared between virtual disks. Each extent on a managed disk is contained with at the most one virtual disk. Free extents are associated with no virtual disk. SVC supports extent sizes of 16, 32, 64, 128, 256, 512, 1024, and 2048 MB. The extent size is a property of the MDG, which is set when the MDG is created. It cannot be changed and all managed disks, which are contained in the MDG, have the same extent size, so all virtual disks associated with the MDG must also have the same extent size. Table 3-1 shows the relationship between the extent size and the maximum capacity of the cluster. Table 3-1 Extent size and maximum cluster capacities Extent size
Maximum cluster capacity
16 MB
64 TB
32 MB
128 TB
64 MB
256 TB
128 MB
512 TB
256 MB
1 PB
512 MB
2 PB
1024 MB
4 PB
2048 MB
8 PB
3.6.4 Image mode virtual disk Image mode provides a direct block-for-block translation from the managed disk to the virtual disk with no virtualization. This mode is intended to allow virtualization of managed disks, which already contain data that was written directly, not through an SVC node from a pre-virtualized disk subsystem. When an image mode virtual disk is created, it directly corresponds to the managed disk it is created from. This allows you to insert an SVC into the data path of an existing storage environment with minimal downtime. After the SVC is inserted into the data path using image mode, you can use the migration facilities to migrate the data to managed mode and rearrange the data while an application is accessing the data.
48
Implementing the IBM System Storage SAN Volume Controller V4.3
When you create an image mode virtual disk, the managed disk specified must not be a member of an MDG. The managed disk is made a member of the specified MDG as a result of the creation of the image mode virtual disk. Image mode provides direct mapping from managed disk to virtual disk. You can think of it as a property of both virtual disks and managed disks. The capacity specified must be less than or equal to the size of the managed disk. If it is less than the size of the managed disk, then the unused space in the managed disk is not available for use in any other virtual disk. There is no facility to specify an offset. Therefore, logical block address (LBA) “N” on the resulting image mode virtual disk maps directly to LBA “N” on the image mode managed disk. Image mode virtual disks have a minimum size of one block (512 bytes) and always occupy at least one extent. Image mode managed disks are members of an MDG, but do not contribute free extents to the pool of free extents. Therefore, an image mode managed disk can have at most one virtual disk associated with it.
3.6.5 Managed mode virtual disk Disks operating in managed mode provide a full set of virtualization functions. Within an MDG, the SVC supports an arbitrary relationship between extents on (managed mode) virtual disks and extents on managed disks. Subject to the constraint that each managed disk extent is contained in at most one virtual disk, each virtual disk extent maps to exactly one managed disk extent (except when in the progress of migrating).
Chapter 3. Planning and configuration
49
Figure 3-12 shows virtual disk V, which is made up of a number of extents. Each of these extents is mapped to an extent on one of the managed disks A, B, or C. The mapping table stores the details of this indirection. You can see that some of the managed disk extents are unused, that is, there is no virtual disk extent that maps to them. These unused extents are available for creating new virtual disks, migration, expansion, and so on.
Figure 3-12 Simple view of block virtualization
Creating managed mode virtual disks When a virtual disk is created, the SVC needs to know the policy to apply to create the initial assignment of managed disk extents to virtual disk extents. The supported policies are listed in the following sections. These policies are only used for the creation of a new virtual disk. After the virtual disk is created, the policy has no effect and is not considered when making decisions during migration operations.
Striped When a virtual disk is created using a striped policy, its extents are allocated from the specified ordered list of managed disks. The allocation algorithm starts with the first managed disk in the ordered list and attempts to allocate an extent from it, then moves to the next disk, and so on, for each managed disk in turn. If the specified managed disk has no free extents, then it misses its turn, and the turn passes to the next managed disk in the list. When the end of the list is reached, the algorithm loops back to the first disk in the list. Allocation proceeds until all the required extents have been allocated. When selecting which extent to allocate from the chosen managed disk, the policy followed is as described in 3.6.7, “Extent allocation and size rules” on page 55. This allocation policy leads to coarse grained striping. The granularity of the striping is at the extent level. This coarse grained striping is unlikely to result in large bandwidth for sequential transfers, but is
50
Implementing the IBM System Storage SAN Volume Controller V4.3
likely to spread the workload caused by random small transactions across the managed disks from which the extents are allocated. Wide striping increases the probability that the data on the virtual disk will be lost due to the failure of one of the managed disks across which the virtual disk is striped. It is acceptable for the list to contain only one disk. In this case, extents are allocated from a single disk as described in 3.6.7, “Extent allocation and size rules” on page 55. Contrast this with the allocation scheme for the sequential policy.
Sequential When a virtual disk is created using a sequential policy, its extents are allocated from a single specified managed disk. The SVC searches for regions of the target managed disk that contain free extents that are sequential so that the region is large enough to allocate the virtual disk from completely sequential extents. If it finds more than one such region, it chooses the smallest region that satisfies this condition. If it does not find any suitable regions, creation of the virtual disk fails.
Cache modes and cache disabled VDisks Prior to SVC V3.1, enabling any copy services function in a RAID array controller for a LUN that was being virtualized by SVC was not supported because the behavior of the write-back cache in SVC would have led to data corruption. With the advent of cache disabled VDisks, it becomes possible to enable copy services in the underlying RAID array controller for LUNs that are virtualized by SVC. Note: Wherever possible, we recommend using SVC copy services in preference to the underlying controller copy services.
Using underlying controller remote copy with SVC cache disabled VDisks Where synchronous or asynchronous remote copy is used in the underlying storage controller, the controller LUNS at both the source and destination must be mapped through the SVC as image mode disks with the SVC cache disabled. Note that of course it is possible to access either the source or the target of the remote copy from a host directly, rather than through the SVC. The SVC copy services can be usefully employed with the image mode virtual disk representing the primary of the controller remote copy relationship, but it would not make sense to use SVC copy services with the VDisk at the secondary site. This is because the SVC does not see the data flowing to this LUN through the controller.
Using underlying controller FlashCopy with SVC cache disabled VDisks Where FlashCopy is used in the underlying storage controller, the controller LUNS for both the source and target must be mapped through the SVC as image mode disks with the SVC cache disabled. Note that it is possible to access either the source or the target of the FlashCopy from a host directly, rather than through SVC. The SVC copy services can be used with the VDisk representing the source for the controller FlashCopy relationship, but it would not make sense to use SVC copy services with the VDisk representing the controller FlashCopy target. This is because the SVC does not see the data flowing to this LUN through the controller.
Controlling copy services on the underlying storage controller Where a storage controller has a copy services interface that is accessed over an IP connection (out-of-band), there will be little difference in the way that the copy services are controlled when the SVC is added between the controller and the servers. Where a storage controller has a copy services interface that is accessed in-band, it might still be possible to control the copy services from the hosts through the in-band interface. This should be addressed on a controller by controller basis.
Chapter 3. Planning and configuration
51
As stated, with SVC Version 3.1, you can choose if you want read and write operations to be stored in cache by specifying a cache mode. You must specify the cache mode when you create the VDisk. After the VDisk is created, you cannot change the cache mode. Table 3-2 describes two different types of cache modes for a VDisk. Table 3-2 Cache mode parameters Cache modes
Description
readwrite
All read and write I/O operations that are performed by the VDisk are stored in cache. This is the default cache mode for all VDisks.
none
All read and write I/O operations that are performed by the VDisk are not stored in cache.
3.6.6 Space-efficient Virtual Disk In release 4.3.0, a new feature has been introduced known as space-efficient VDisk (SEV). This feature enables the VDisk size (virtual capacity) to be larger than the space occupied by VDisk Copy 0 (zero), physical capacity, on MDisk. The real capacity defines the amount space that is actually allocated to the VDisk, and the virtual capacity defines the size of the VDisk as it appears to the host. The mapping between the virtual address space to the real address space is implemented using the directory map. This directory map is implemented as B-Tree and stored on back-end storage. This is shown in Figure 3-13 on page 53. The SVC consumes less than 1% of used capacity to store the metadata for the directory map. Important: In an event where the real capacity is fully utilized, either due to a lack of available physical storage on auto-expand, or because of a disk not marked for auto-expand, and it has not been expanded manually, this will result in the VDisk entering an offline state. This is not a data integrity or loss issue, as the data is maintained in the SVC cache until additional storage space is provisioned. However, the SVC’s cache cannot be treated or used as a backup mechanism, and the necessary steps for the provisioning of additional storage space should be immediately implemented.
52
Implementing the IBM System Storage SAN Volume Controller V4.3
Space-Efficient VDisk Host Server
Virtual Capacity
Managed Disk Group
Directory
Controller
Real Capacity
Figure 3-13 Space-efficient VDisk mapping
The VDisk is allocated sufficient extents to make up the VDisk real capacity. The host’s write I/O on to the VDisk at any LBA utilizes the space in the real capacity, resulting in an increase of the used capacity. Again, if the real capacity is fully utilized, then the VDisk goes offline and application I/O to that VDisk fails. To make this VDisk online again, the administrator must provision for more storage to expand the real capacity of the VDisk.
Chapter 3. Planning and configuration
53
Figure 3-14 shows the relationship between virtual capacity, real capacity, and used capacity.
Virtual Capacity 2GB Virtual LBA 0
Virtual LBA Max
Real Capacity 1GB Real LBA Max
Real LBA 0
Free Capacity
400MB
Used Capacity
600MB
Warning Level for Used Capacity
800MB
Figure 3-14 Space-efficient VDisk’s virtual capacity, real capacity, and used capacity relationship
SVC provides the ability to avoid exhaustion of real capacity by allowing alerts to be sent to the administrator for additional capacity or for automatically increasing the real capacity. The allocation unit for the real capacity is known as the grain size and can be 32 KB, 64 KB, 128 KB, or 256 KB. The space is allocated when data is first returned to a virtual LBA. Once allocated, the space cannot be de-allocated and it is the responsibility of the volume manager to reuse the space that is freed up due to deletion of data or files on that VDisk. The space-efficient VDisk is a feature that helps an application or group of users that require a higher storage space capacity, but the actual utilization is much lower. For example, some of the applications require a higher amount of free space at the time of installation, but it is not actually utilized until data is written to that part of the disk. This gives the storage administrator the ability to meet the higher demand of storage space, but at the same time efficiently utilize the valuable storage space, and support a pay as you grow strategy for disk subsystems. However, space-efficient VDisk requires proper planning, as shown in “Space-efficient virtual disk considerations” on page 58.
54
Implementing the IBM System Storage SAN Volume Controller V4.3
3.6.7 Extent allocation and size rules Migration operations and some of the virtualization operations require the allocation of a specific number of extents from a specific set of managed disks. The algorithm used to achieve this task is described in the following sections.
Choosing the managed disk to allocate from Where the set of managed disks to allocate extents from contains more than one disk, extents are allocated from managed disks in a round robin fashion. If a managed disk has no free extents when its turn arrives, then its turn is missed and the round robin moves to the next managed disk in the set that has a free extent. As the algorithm progresses, disks with no free extents on the previous pass of the round robin are queried for free extents on each turn of the algorithm in case extents become free.
Choosing the extent to allocate from a specific managed disk When an extent is to be allocated from a specific managed disk, the allocation policy is to allocate the next free extent from a list of free extents held by the SVC cluster for the specific managed disk.
Size of extent If you want to migrate a VDisk from one MDisk group to another MDisk group, the extent size must be the same between the two MDisk groups. Because of this, it can be useful to set a common extent size for all the MDisk groups. A value of 32 MB (corresponding to a maximum cluster capacity of 128 TB) or 64 MB (corresponding to a maximum cluster capacity of 256 TB) can be the best trade-off between performance and capacity. If you need to migrate a VDisk from one MDisk group to another MDisk group that has another extent size, you need to use Metro or Global Mirror so the VDisks have to belong to the same I/O group if they are in the same SVC cluster.
3.6.8 MDisk group planning There are several guidelines or rules that you must follow when creating an MDisk group.
Number of MDGs The number of MDisk groups depends on the following factors: The need for image mode virtual disks (data migration) The need for managed mode virtual disks with sequential policy The models of the disk subsystem controller (disk subsystem with cache or without, DS4000, ESS, and so on) that have different properties of performance, availability, response time, and so on The models of the hard disks in the disk subsystem controller (hard disk type, RPM, speed, and so on) that have different properties on performance, availability, response time, and so on. It is possible to have a common MDG for the SVC cluster. However, a virtual disk (VDisk) is offline if any managed disk in the MDG is offline, even if that managed disk does not contribute any extents to the virtual disk in question, or the managed Disk has no allocated extents. The more managed disks there are in an MDG, the more the VDisk (host LUN) is striped and the better the performance is.
Chapter 3. Planning and configuration
55
We recommend that you: Create at least one separate MDG for all the image mode virtual disks. Create one separate MDG for each array (or RAID) type presented from a disk subsystem, or one separate MDG for each subsystem when the RAID protection is the same for all the subsystems or types of the hard disks on the disk subsystem. So the MDGs are characterized by performance, RAID level, reliability, vendor, and so on. Keep in mind that more MDG granularity reduces the possibility to take VDisks offline due to MDisk problems or subsystem maintenance procedures, but this level of granularity increases the management activity required. Note: It could be wise to keep each disk subsystem in a separate MDisk group. This prevents a failure in storage subsystem A from affecting VDisks in an MDisk group from storage subsystem B. If a VDisk is composed of MDisks from both A and B, then a failure in either A or B causes the VDisk to be unavailable. Name them in such a way that it is easy for you (when you create a virtual disk) to associate a virtual disk with an MDG that has the appropriate level of performance and reliability needed, for example, pool1_high_perf_high_rela, pool2_low_perf_low_rela, mDisk_grp_ESS1, mDisk_grp_DS40002, mDisk_grp_raid10, and so on.
Creating a managed disk First, you need to create the logical disks (LUNs) in your disk subsystem that will be made available as MDisks in the SVC. We recommend that you use the maximum LUN size to be presented as an MDisk. The discovery of the managed disk is automatically done by the SVC. The managed disk is in unmanaged mode until you include it in an MDG. You need at least one managed disk for the support of the quorum disk used in the cluster. All SVC nodes must have access at any time to all the managed disks. The size of the managed disk can be up to 2 TB. You can use some common sizes for all the managed disks (16 GB or 32 GB). This helps simplify things and ensures that, as much as possible, all the MDisks are used in the striping process for a managed disk with striped policy. If you have three managed disks, two of 4 GB and one of 210 GB, then only the disk of 210 GB is used in the striping process. If you want to migrate data from an existing back-end LUN, you do not have to create an MDisk (LUNs in the disk subsystem) of that size, since you already have that LUN. Simply make an image mode vdisk without specifying the size, which will then default to the size of the existing LUN or unmanaged MDisk.
3.6.9 Planning a virtual disk An individual virtual disk is a member of one MDG and one I/O group.
Selecting MDGs There is only one question you might want to ask regarding the selection of MDGs: From which MDisk group should I create my VDisk? The answer to this question is that you need to keep in mind that an individual virtual disk is a member of one MDG and one I/O group: The MDG defines which managed disks provided by the disk subsystem make up the virtual disk. The I/O group (two nodes make an I/O group) defines which SVC nodes provide I/O access to the virtual disk.
56
Implementing the IBM System Storage SAN Volume Controller V4.3
Note: There is no fixed relationship between I/O groups and MDGs. Therefore, you could define the virtual disks using the following considerations: Optimize the performance between the hosts and SVC by distributing the VDisks between the different nodes of the SVC cluster. This means spreading the load equally on the nodes in the SVC cluster. Get the level of performance, reliability, and capacity you require by using the MDG that corresponds to your needs (you can access any MDG from any node), that is, choose the MDG that fulfils the demands for your VDisk, with respect to performance, reliability, and capacity.
I/O handling and offline conditions For a virtual disk to be online, all managed disks in the MDG or MDGs associated with the virtual disk must be online. A virtual disk is offline if any managed disk in the MDG is offline, even if that managed disk does not contribute any extents to the virtual disk in question, or the managed disk has no allocated extents. Note: Normally, a virtual disk is associated with just one MDG. However, for the duration of a migration between MDGs, the virtual disk is associated with two MDGs. In this case, the offline rules apply to both MDGs for the duration of the migration only. Referring back to Figure 3-11 on page 47, this means that if managed disk M1 is taken offline by disk subsystem A, virtual disks V1 and V2 are taken offline by the SVC. This notion of offline and online is managed on a node basis. Therefore, if a condition arises that causes one SVC node to see a managed disk offline, then the affected virtual disks are taken offline on that node only, but may still be online at the other node. For example, refer again to Figure 3-11 on page 47. If the SAN connection between disk subsystem B and SVC node P were to fail, then node P would lose contact with managed disks M3 and M4. Since M3 is in MDG X and M4 is in MDG Y, this takes all the virtual disks in MDGs X and Y on node P offline. Therefore, hosts accessing node P see virtual disk V2 go offline. Hosts accessing V2 through node Q continue to see the virtual disk as online. When using SDD, the paths to node P show offline, while the paths to node Q show online, and the host still has access to the virtual disks.
I/O group considerations When you create a VDisk, it is associated to one node of an I/O group. By default, every time you create a new VDisk, it is associated to the next node using a round robin algorithm. For example, you might have four hosts (host1, host2, host3, and host4) with 100 VDisks for each host of the same size with the same level of I/O activity, and a four node (two I/O groups) cluster. The result is 100 VDisks on each node (25 VDisks from host1, 25 VDisks from host2, and so on). You can specify a preferred access node. This is the node through which you send I/O to the VDisk instead of using the round robin algorithm. For example, in one host with four VDisks (VD1, VD2, VD3, and VD4). VD1 and VD3 have a high level of I/O activity. VD2 and VD4 have a low level of I/O activity. If you use the round robin algorithm, VD1 and VD3 are on the same node 1 of the I/O group, and VD2 and VD4 are on the same node 2 of the I/O group. To avoid this, use the preferred node feature to specify VD1 on node 1 and VD3 on node 2.
Chapter 3. Planning and configuration
57
A virtual disk is defined for an I/O group, which provides the following benefits: The VDisk is “exported” by the two nodes of the I/O group to the host through eight paths (four paths for each node). We use zoning to limit it to four paths from each node. Each write is copied into the cache of the two nodes before acknowledgment is sent to the host. Even if you have eight paths for each virtual disk, all I/O traffic flows only towards one node (the preferred node). Therefore, only four paths are really used by SDD. The other four are used only in case of a failure of the preferred node or when CCU is running. Before you create a virtual disk, you can check the amount of space that is available in the MDG. You can determine the free capacity for an MDisk or an MDisk group using the svcinfo lsmdiskgrp <MDG_Name> command.
Creating image mode virtual disks Use image mode virtual disks when a managed disk already has data on it, from a pre-virtualized disk subsystem. When an image mode virtual disk is created, it directly corresponds to the managed disk from which it is created. Therefore, virtual disk LBA x = managed disk LBA x. The capacity of image mode VDisks defaults to the capacity of the supplied MDisk, but can be reduced if desired. When you create an image mode disk, the managed disk must have a mode of unmanaged and therefore does not belong to any MDG. A capacity of 0 is not allowed. Image mode virtual disks can be created in sizes with a minimum granularity of 512 bytes, and must be at least one block (512 bytes) in size. The SVC can reserve an integer number of extents to hold the image mode disk. It effectively rounds up its size to the nearest whole number of extents.
Creating managed mode virtual disks with sequential policy When creating a managed mode virtual disk with sequential policy, you must use a managed disk containing free extents that are sequential and of a size that is equal or greater than the size of the virtual disk you want to create. It may be the case that there are sufficient extents available on the managed disk, but that there is no contiguous block large enough to satisfy the request.
Space-efficient virtual disk considerations The space-efficient VDisk is great feature, but it requires proper planning for effective utilization of it. While creating the space-efficient volume, it is necessary to understand the utilization patterns by the applications or group users accessing this volume. Items such as the actual size of the data, rate of creation of new data, modifying or deletion of existing data, and so on all need to be taken into consideration. Depending on the initial size for the real capacity, the grain size and warning level can be set. If a disk goes offline, either through a lack of available physical storage on auto-expand, or because a disk marked as non-expand has not been expanded, then there is a danger of data being left in the cache until some storage is made available. This is not a data integrity or data loss issue, but you should not be relying on the SVC cache as a backup storage mechanism.
58
Implementing the IBM System Storage SAN Volume Controller V4.3
Recommendation: We highly recommend to keeping Warning Level on Used Capacity such that it provides adequate time for provision of more physical storage. Warnings should not be ignored by administrator. Use the auto-expand feature of SEV. The grain size allocation unit for the real capacity in VDisk can be set as 32 KB, 64 KB, 128 KB, or 256 KB. A smaller grain size utilizes space more effectively, but it results in a larger directory map, which may reduce performance. Once allocated, space can never be just de-allocated. It is the responsibility of the volume manager to reuse the space that is freed up due to deletion of data or files on that VDisk. Therefore, to achieve the effective utilization of storage space, it is necessary for the administrator to perform regular maintenance on volume manager for SEV. In the case of running a data erasure type of application that writes to every LBA of the VDisk during the destruction of the data on the disk, consult IBM Technical Support prior to running such an application.
3.6.10 Planning for operations on virtual disks You can perform several operations on virtual disks.
Modifying a virtual disk You can change the I/O group with which a virtual disk is associated. This requires a flush of the cache within the nodes in the current I/O group to ensure that all data is written to disk. I/O should be suspended at the host level before you perform this operation, and an SDD reset or cfgmgr restart is likely to be needed if the target WWNNs change.
Expanding a virtual disk A virtual disk can be expanded. The granularity of expansion is one block (512 bytes). If the expansion requires the allocation of additional extents, then these are allocated to the virtual disk from the managed disks specified using the algorithm described in 3.6.7, “Extent allocation and size rules” on page 55. Expanding a virtual disk using the sequential policy forces the virtualization policy to be changed to striped. Image mode virtual disks cannot be expanded. They must first be migrated to managed mode. Warning: Not all volume managers can tolerate the expansion of a virtual disk. A reboot or remount of the disk might be needed to use the additional space.
Reducing a virtual disk A virtual disk can be shrunk. The granularity of shrinking is one block (512 bytes). If the shrink operation allows extents to be freed, then these are returned to the pool of free extents for allocation by later virtualization and migration operations. Image mode virtual disks cannot be reduced in size. They must first be migrated to managed mode.
Chapter 3. Planning and configuration
59
Warning: Not all volume managers can tolerate a virtual disk being reduced in size. You must be cautious and know where data resides, or data loss can occur.
Deleting a virtual disk A virtual disk can be deleted. When a virtual disk is deleted, all host mappings are deleted and any cached read or write data is discarded. Also, any FlashCopy mappings or Metro or Global Mirror relationships in which the disk is participating in are also deleted. If the virtual disk was operating in managed mode, then the extents are returned to the pool of free extents for allocation by later virtualization operations. If the virtual disk was an image mode virtual disk, deleting the virtual disk causes the managed disk to be ejected from the MDG. The mode of the managed disk is returned to “unmanaged” mode. This makes the delete operation the inverse of the create operation for image mode disks.
Migration This facility allows the mapping of virtual disk extents to managed disk extents to be changed, without interrupting a host’s access to that virtual disk. You can perform this for any virtual disk managed by the SVC. You can use this for:
Redistributing workload within a cluster across the disk subsystem Moving workload onto newly installed storage Moving workload off old or failing storage, ahead of decommissioning it Moving workload to rebalance a changed workload Migrating data from an older disk subsystem to SVC managed storage Migrating data from one disk subsystem to another VDisk Migration (from MDisk Group to another MDisk Group)
As mentioned above, you might want to migrate VDisks from one set of MDisks to another to retire an old disk subsystem, or to have better balance performance across your virtualized environment, or simply to migrate data into the SVC environment transparently using image mode. A sufficient amount of free space or extents must be available in the target MDG. If insufficient extents are available within your target MDG, you will receive an error message. Make sure the source and target MDG have the same extent size. The optional threads parameter allows you to assign a priority to the migration process. The default is 4, which is the highest priority setting. However, if you want the process to take a lower priority over other types of I/O, you can specify 3, 2, or 1.
Migrate a VDisk to an image mode VDisk To migrate a VDisk to an image mode VDisk, the following rules apply: The destination MDisk must be greater than or equal to the size of the VDisk. The MDisk specified as the target must be in an unmanaged state. Regardless of the mode that the VDisk starts in, it is reported as managed mode during the migration. Both of the MDisks involved are reported as being Image mode during the migration. If the migration is interrupted by a cluster recovery or by a cache problem, then the migration will resume after the recovery completes. For further details about the migration facility, see Chapter 14, “Migration to and from the SAN Volume Controller” on page 741.
60
Implementing the IBM System Storage SAN Volume Controller V4.3
Also with V4.3.0, there is one more possibility that exists for migration, from space-efficient to non-space-efficient VDisk and vice-versa. This migration is not supported with the CLI. The SVC will carry out migration, or copying, of data on a per-block basis. In the current release, the SVC is unable to identify the zero data block per OS perspective. Hence, migrating from a non space-efficient VDisk to a space-efficient VDisk requires the target VDisk to be the same size as the source and target VDisk that is being 100% utilized.
Quality of service on VDisk You can set the I/O governing rate, which is a cap on the amount of I/O that is accepted for this virtual disk. You can set it in terms of I/O per second or MBs per second. By default, no I/O governing is set when a virtual disk is created. An I/O threshold is expressed as a number of I/Os, or a number of MBs, over a minute. The threshold is evenly divided between all SVC nodes that service the VDisk, that is, between the nodes that form the I/O Group of which the VDisk is a member. The algorithm operates two levels of policing: while I/O is under threshold, and when threshold is reached. While a VDisk on each SVC node receives I/O at a rate below the governed level, then no governing is performed. A check is made every minute that the VDisk on each node is continuing to receive I/O below the threshold level. If this check shows that the host has exceeded its threshold on one or more nodes, then policing begins for new I/Os. While policing is in force: A threshold quantity is calculated for a one-second period. I/Os are counted over a period of a second. If I/Os are received in excess of the one-second threshold quantity on any node in the I/O, those I/Os are grouped and later I/Os are pended. When the second expires, a new threshold quantity is established, and any pended I/Os are re-driven under the new threshold quantity. If a host stays within its one second threshold quantity on all nodes in the I/O group for a period of one minute, then the policing is relaxed, and monitoring takes place over the one-minute period as it was before the threshold was reached.
3.6.11 Host considerations In this section, we discuss creating a host.
Port masking You can use a port mask to control the node target ports that a host can access. This satisfies two requirements: As part of a security policy, to limit the set of WWPNs that are able to obtain access to any VDisks through a given SVC port. As part of a scheme to limit the number of logins with mapped VDisks visible to a host multi-pathing driver (like SDD) and thus limit the number of host objects configured without resorting to switch zoning. The port mask is an optional parameter of the svctask mkhost and chhost commands. The port mask is four binary bits. Valid mask values range from 0000 (no ports enabled) to 1111 (all ports enabled). For example, a mask of 0011 enables port 1 and port 2. The default value is 1111 (all ports enabled). Chapter 3. Planning and configuration
61
Standard and persistent reserve On 4.1 code and above, you can use the svctask rmvdiskhostmap command to remove standard and persistent reservations that a host has on the VDisk.
3.6.12 Quorum disks A quorum disk is used to resolve tie-breaker situations, when the “voting set” of nodes disagree on the current cluster state. The voting set is an overview of the SVC cluster configuration running at a given point in time, and is the set of nodes and quorum disk that are responsible for the integrity of the SVC cluster. On cluster creation, the voting set consists of a single node with a unique ID of 1, which was used to create the cluster. When nodes are integrated into the SVC cluster, they get added to the voting set, and when a node is removed from the SVC cluster, it will also be removed from the voting set. A failed node is considered a removed node, and is removed from the voting set. When MDisks are added to the SVC cluster, it checks the MDisk to see if it can be used as a quorum disk. If the MDisk fulfils the demands, the SVC will assign the three first MDisks as quorum candidates, and one of them is selected as the active quorum disk. If possible, the SVC will place the quorum candidates on different disk subsystems. Once the quorum disk has been selected, however, no attempt is made to ensure that the other quorum candidates are presented through different disk subsystems. When the set of quorum disk candidates has been chosen, it is fixed. A new quorum disk candidate will only be chosen if: The administrator requests that a specific MDisk becomes a quorum disk using the svctask setquorum command. An MDisk that is a quorum disk is deleted from an MDG. An MDisk that is a quorum disk changes to image mode. An MDisk will not be replaced as a quorum disk candidate simply because it is offline. The cluster must contain at least half of its nodes in the voting set to function. A tie-breaker situation can occur if exactly half the nodes in the cluster fail at the same time, or if the cluster is divided so that exactly half the nodes in the cluster cannot communicate with the other half. For example, in a cluster of four nodes, if any two nodes fail at the same time, or any two cannot communicate with the other two, a tie-breaker condition exists and must be resolved. To resolve the tie-breaker condition, a quorum disk is used. The cluster automatically chooses three managed disks to be quorum disks. One of these disks is used to settle a tie-breaker condition. If a tie-breaker condition occurs, the first half of the cluster to access of the quorum disk after the split has occurred locks the disk and continues to operate. The other side stops. This action prevents both sides from becoming inconsistent with each other. In a two-site solution, we recommend using two SVC clusters, one at each site, and mirroring the data by using either host mirror software/functions or by using SVC Metro or Global Mirror. In a two-site solution with only one SVC cluster, you can have a situation where you will lose access to the data, and is not allowed. For example, in a four node SVC cluster, with two nodes at each location, the quorum will only be located at one of the sites, and if that site “dies,” the remaining two nodes cannot get access to the quorum disk, and will also shut down. As a result, the entire SVC cluster is shut down, even though only one site is out. The same applies in a two-node SVC cluster, if you put the two nodes in different locations or rooms. Important: A cluster should be regarded as a single entity for disaster recovery purposes. This means that the cluster and the quorum disk should be co-located.
62
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 3-15 and Figure 3-16 on page 64 show two different scenarios with respect to quorum disk and cluster co-location.
Bad One IO Group Design, if HA is key issue
I/O group 0
SVC1N1
SVC1N2
SVC cluster
UPS1A
DS4301_A
SW11
SW12
SW21
SW22
UPS1B
D S4302_A
DS4301_B
DS4302_B
Site B
Site A
=Quorum disk Figure 3-15 Bad scenario for quorum disk and cluster co-location
Chapter 3. Planning and configuration
63
HA Design
SVC cluster1
SVC1N1
SVC cluster2
I/O group 0
SVC2N1
I/O group 0
SVC1N2
SVC2N2
SVC1N3
SVC2N3 I/O group 1
I/O group 1
SVC1N4
SVC2N4
UPS1
DS4301_A
SW11
SW12
SW21
SW22
UPS2
DS4302_A
DS4301_B
DS4302_B
Site B
Site A
= Quorum disk Figure 3-16 Correct HA scenario for quorum disk and cluster co-location
3.6.13 Expanding an SVC cluster configuration You can expand an SVC cluster configuration.
Adding a node to a cluster SVC clusters of up to 8 nodes are supported. You can easily add new nodes, add new hosts, or redistribute workload.
Adding a new disk controller to an MDisk group MDisk groups can span disk subsystems. We recommend that you do not do this. Each MDisk group should, in normal circumstances, comprise disks from one disk subsystem. See 3.6.2, “MDGs, I/O groups, virtual disks, and managed disks” on page 46 for more information.
VDisk size increase and decrease The SVC allows you to increase and decrease the size of VDisks. Not all operating systems allow this scenario. See “Expanding a virtual disk” on page 59 for more information.
64
Implementing the IBM System Storage SAN Volume Controller V4.3
3.7 Storage subsystem planning For a list of the maximum configurations, go to: http://www.ibm.com/storage/support/2145 In the configuration shown in Figure 3-17, the disk subsystem presents a LUN to the SVC and another LUN to host B. The SVC presents the VDisks created from the MDisk to the host A. Since the disk subsystem is a DS4000, host B would have RDAC installed to support the direct attachment, and SDD is installed on host A to support the attachment to the SVC. This is a supported configuration.
Host A SDD
Host B RDAC
VDisk SVC
SAN
MDisk
LUN
Array
Array
RAID Controller Figure 3-17 Disk subsystem shared
Chapter 3. Planning and configuration
65
With the ESS, you can attach directly to an ESS LUN and to a VDisk from the SVC that comes from the ESS, as long as the same LUN is not assigned to both the host and the SVC. The host uses SDD to access the LUNs presented from both the ESS and the SVC. This is a supported configuration and is shown in Figure 3-18.
Host A SDD
VDisk SVC
SAN
MDisk
LUN
Array
Array
DS8000 Figure 3-18 Host connected to ESS and SVC
In the configuration shown in Figure 3-19 on page 67, the host needs to have both the RDAC driver installed for access to the DS4000 and SDD installed to access the SVC. The SVC supports the use of the IBM SDD, native Multi-path I/O (MPIO) drivers on selected operating systems, and some other operating system specific software. We recommend having different HBAs for RDAC and SDD with proper zoning. Check for supported configurations on the Web at: http://www.ibm.com/support/docview.wss?uid=ssg1S1003278
66
Implementing the IBM System Storage SAN Volume Controller V4.3
Host A SDD/RDAC
VDisk
SAN
SVC
MDisk
LUN
Array
Array DS4000
Figure 3-19 RDAC and SDD drivers installed
3.7.1 Adding DS8000 storage to the SVC Perform the following steps to add DS8000 storage using the DS command-line Interface: 1. Go to the Web site and check the prerequisites: http://www.ibm.com/storage/support/2145 2. You might need to upgrade the microcode level of the DS8000 to support this attachment. 3. Before the MDisks can be presented to the SVC, add the DS8000 ports to the storage zone to all SVC node ports. 4. Sign on to the DSCLI of DS8000 (Example 3-1). Example 3-1 Logon to DSCLI C:\Program Files\IBM\dscli\profile>dscli Date/Time: June 5, 2007 11:49:12 PM IST IBM DSCLI Version: 5.2.200.381 IBM.2107-7520331
Chapter 3. Planning and configuration
67
Display a list of array sites to verify if any of them are available (Example 3-2). Example 3-2 lsarraysite command dscli> lsarraysite -dev IBM.2107-7520331 Date/Time: June 6, 2007 12:01:27 AM IST IBM DSCLI Version: 5.2.200.381 DS: IBM.2107-7520331 arsite DA Pair dkcap (10^9B) State Array ============================================= S1 2 73.0 Assigned A0 S2 2 73.0 Assigned A1 S3 2 73.0 Assigned A2 S4 2 73.0 Assigned A3 S5 2 73.0 Assigned A38 S6 2 73.0 Unassigned S7 2 73.0 Unassigned S8 2 73.0 Assigned A23 S9 0 73.0 Assigned A4 S10 0 73.0 Assigned A10 S11 0 73.0 Assigned A16 S12 0 73.0 Assigned A22 S13 0 73.0 Assigned A28 S14 0 73.0 Assigned A35 S15 0 73.0 Assigned A17 S16 0 73.0 Unassigned S17 7 73.0 Assigned A32
5. Make a new array (Example 3-3). Example 3-3 mkarray command dscli> mkarray -raidtype 5 -arsite S6 -dev IBM.2107-7520331 Date/Time: June 6, 2007 12:04:10 AM IST IBM DSCLI Version: 5.2.200.381 DS: IBM.2107-7520331 CMUC00004I mkarray: Array A39 successfully created. dscli>
6. Display the new array (Example 3-4). Example 3-4 lsarray command (Truncated output) dscli> lsarray -dev IBM.2107-7520331 Date/Time: June 6, 2007 12:05:54 AM IST IBM DSCLI Version: 5.2.200.381 DS: IBM.2107-7520331 Array State Data RAIDtype arsite Rank DA Pair DDMcap (10^9B) ==================================================================== A0 Assigned Normal 5 (6+P+S) S1 R0 2 73.0 A1 Assigned Normal 5 (6+P+S) S2 R1 2 73.0 A2 Assigned Normal 5 (6+P+S) S3 R2 2 73.0 A3 Assigned Normal 5 (6+P+S) S4 R3 2 73.0 . . . A37 Assigned A38 Assigned A39 Unassigned A42 Assigned A43 Assigned dscli>
68
Normal Normal Normal Normal Normal
5 5 5 5 5
(7+P) (7+P) (7+P) (6+P+S) (6+P+S)
S22 S5 S6 S35 S36
R37 R38 R42 R43
7 2 2 5 5
Implementing the IBM System Storage SAN Volume Controller V4.3
73.0 73.0 73.0 73.0 73.0
7. Create the ranks (Example 3-5). Example 3-5 Create the rank mkrank dscli> mkrank -stgtype fb -array A39 -dev IBM.2107-7520331 Date/Time: June 6, 2007 12:22:57 AM IST IBM DSCLI Version: 5.2.200.381 DS: IBM.2107-7520331 CMUC00007I mkrank: Rank R39 successfully created. dscli>.
8. Display the ranks (Example 3-6). A truncated output is shown for brevity. Example 3-6 lsrank dscli> lsrank -dev IBM.2107-7520331 Date/Time: June 6, 2007 12:24:03 AM IST IBM DSCLI Version: 5.2.200.381 DS: IBM.2107-752033 ID Group State datastate Array RAIDtype extpoolID stgtype =============================================================== R0 0 Normal Normal A0 5 P0 ckd R1 1 Normal Normal A1 5 P1 ckd R2 0 Normal Normal A2 5 P2 fb R3 1 Normal Normal A3 5 P3 fb R4 0 Normal Normal A4 5 P4 ckd R5 1 Normal Normal A5 5 P15 fb R6 0 Normal Normal A6 5 P6 ckd R7 1 Normal Normal A7 5 P5 ckd . . . R35 R36 R37 R38 R39 R42 R43 dscli>
1 0 1
Unassigned Unassigned Normal Unassigned Unassigned Normal Normal
Normal Normal Normal Normal Normal Normal Normal
A35 A36 A37 A38 A39 A42 A43
5 5 5 5 5 5 5
P37 P42 P43
ckd ckd ckd fb fb fb fb
9. Create the extpool (Example 3-7). Example 3-7 Creating the extpool dscli> mkextpool -dev IBM.2107-7520331 -rankgrp 1 -stgtype fb ITSOSVC_extpool Date/Time: June 6, 2007 12:29:27 AM IST IBM DSCLI Version: 5.2.200.381 DS: IBM.2107-7520331 CMUC00000I mkextpool: Extent pool P22 successfully created.
10.Display the newly created extpool (Example 3-8). Example 3-8 Display the newly created extpool (truncated output) dscli> lsextpool -dev IBM.2107-7520331 Date/Time: June 6, 2007 12:32:03 AM IST IBM DSCLI Version: 5.2.200.381 DS: IBM.2107-7520331 Name ID stgtype rankgrp status availstor(2^30B) %allocated available reserved numvols =========================================================================================== ===== ITSOSVC_extpool P22 fb 0 below 0 100 0 0 0 dscli>
Chapter 3. Planning and configuration
69
11.Assign the rank to the newly created extpool (Example 3-9). Example 3-9 Assigning the rank dscli> chrank -dev IBM.2107-7520331 -extpool P22 R39 Date/Time: June 6, 2007 12:52:56 AM IST IBM DSCLI Version: 5.2.200.381 DS: IBM.2107-7520331 CMUC00008I chrank: Rank R39 successfully modified. dscli>
12.Display the extpool status (Example 3-10). Example 3-10 Displaying the extpool status (truncated output) dscli> lsextpool -dev IBM.2107-7520331 Date/Time: June 6, 2007 12:56:26 AM IST IBM DSCLI Version: 5.2.200.381 DS: IBM.2107-7520331 Name ID stgtype rankgrp status availstor (2^30B) %allocated available reserved numvols =========================================================================================== =========== ITSOSVC_extpool 0 0
P22 fb
0 below
452
0
452
13.Create an FB volume with IDs 2100, 2101, 2102, and 2103 (Example 3-11). Example 3-11 mkfbvol dscli> mkfbvol -dev IBM.2107-7520331 -extpool P22 -cap 1 2100-2103 Date/Time: June 6, 2007 1:05:04 AM IST IBM DSCLI Version: 5.2.200.381 DS: IBM.2107-7520331 CMUC00025I mkfbvol: FB volume 2100 successfully created. CMUC00025I mkfbvol: FB volume 2101 successfully created. CMUC00025I mkfbvol: FB volume 2102 successfully created. CMUC00025I mkfbvol: FB volume 2103 successfully created.
14.Create the volume group (Example 3-12). Example 3-12 mkvolgrp dscli> mkvolgrp -dev IBM.2107-7520331 -volume 2100-2103 -type scsimap256 SVC_VG Date/Time: June 6, 2007 1:10:28 AM IST IBM DSCLI Version: 5.2.200.381 DS: IBM.2107-7520331 CMUC00030I mkvolgrp: Volume group V6 successfully created.
15.Display the list of volume groups (Example 3-13). Example 3-13 lsvolgrp dscli> lsvolgrp -dev IBM.2107-7520331 Date/Time: June 6, 2007 1:11:29 AM IST IBM DSCLI Version: 5.2.200.381 DS: IBM.2107-7520331 Name ID Type ======================================= CraigAIX V0 SCSI Mask SVC4_VolGroup V1 SCSI Mask WorkshopVG_GM V2 SCSI Mask Dan_Systemi_Lpar2 V3 OS400 Mask WorkshopVG_MM V4 SCSI Mask Dan_pSeries_Lpar3 V5 SCSI Mask SVC_VG V6 SCSI Map 256 Dan_Systemi_Lpar3 V9 OS400 Mask All CKD V10 FICON/ESCON All
70
Implementing the IBM System Storage SAN Volume Controller V4.3
All Fixed Block-512 V20 SCSI All All Fixed Block-520 V30 OS400 All dscli>
16.Display the volume group you created with its volumes (Example 3-14). Example 3-14 showvolgrp dscli> showvolgrp -dev IBM.2107-7520331 V6 Date/Time: June 6, 2007 1:14:16 AM IST IBM DSCLI Version: 5.2.200.381 DS: IBM.2107-7520331 Name SVC_VG ID V6 Type SCSI Map 256 Vols -2100 2101 2102 2103
17.Configure the I/O port (Example 3-15). Example 3-15 setioport dscli> setioport -dev IBM.2107-7520331 -topology SCSI-FCP I0103 Date/Time: June 6, 2007 1:19:11 AM IST IBM DSCLI Version: 5.2.200.381 DS: IBM.2107-7520331 CMUC00011I setioport: I/O Port I0103 successfully configured. dscli>.
18.Display the list of I/O ports (Example 3-16). Example 3-16 lsioport dscli> lsioport -dev IBM.2107-7520331 Date/Time: June 6, 2007 1:20:03 AM IST IBM DSCLI Version: 5.2.200.381 DS: IBM.2107-7520331 ID WWPN State Type topo portgrp =============================================================== I0010 5005076303010194 Online Fibre Channel-SW SCSI-FCP 0 I0011 5005076303014194 Online Fibre Channel-SW SCSI-FCP 0 I0012 5005076303018194 Online Fibre Channel-SW SCSI-FCP 0 I0013 500507630301C194 Online Fibre Channel-SW SCSI-FCP 0 I0030 5005076303030194 Online Fibre Channel-LW SCSI-FCP 0 I0031 5005076303034194 Online Fibre Channel-LW SCSI-FCP 0 I0032 5005076303038194 Online Fibre Channel-LW SCSI-FCP 0 I0033 500507630303C194 Online Fibre Channel-LW SCSI-FCP 0 I0040 5005076303040194 Online Fibre Channel-LW SCSI-FCP 0 I0041 5005076303044194 Online Fibre Channel-LW SCSI-FCP 0 I0042 5005076303048194 Online Fibre Channel-LW SCSI-FCP 0 I0043 500507630304C194 Online Fibre Channel-LW SCSI-FCP 0 I0100 5005076303080194 Online Fibre Channel-LW SCSI-FCP 0 I0101 5005076303084194 Online Fibre Channel-LW SCSI-FCP 0 I0102 5005076303088194 Online Fibre Channel-LW SCSI-FCP 0 I0103 500507630308C194 Online Fibre Channel-LW SCSI-FCP 0
19.Configure the SVC attachment to the DS8000. You must define eight host connections in a two-node environment, one for each single Fibre Channel port of the SVC (Example 3-17). Example 3-17 mkhostconnect dscli> mkhostconnect -dev IBM.2107-7520331 -wwname 5005076801302C49 -profile "IBM SAN Volume Controller" -volgrp V1 -ioport all SVC4_Node1 Date/Time: June 6, 2007 1:18:12 AM IST IBM DSCLI Version: 5.2.200.381 DS: IBM.2107-7520331 CMUC00012I mkhostconnect: Host connection 0003 successfully created.
Chapter 3. Planning and configuration
71
20.Repeat these steps for all FC ports of your SVC. 21.Finally, display the host connections defined (Example 3-18). Example 3-18 lshostconnect dscli> lshostconnect -dev IBM.2107-7520331 Date/Time: June 6, 2007 1:21:11 AM IST IBM DSCLI Version: 5.2.200.381 DS: IBM.2107-7520331 Name ID WWPN HostType Profile portgrp volgrpID ESSIOport =========================================================================================== ====== SVC4_Node1 0011 5005076801302C49 SVC San Volume Controller 0 V1 all SVC4_Node1 0012 5005076801402C49 SVC San Volume Controller 0 V1 all SVC4_Node1 0013 5005076801202C49 SVC San Volume Controller 0 V1 all SVC4_Node1 0014 5005076801102C49 SVC San Volume Controller 0 V1 all SVC4_Node2 0015 500507680140376F SVC San Volume Controller 0 V1 all SVC4_Node2 0016 500507680130376F SVC San Volume Controller 0 V1 all SVC4_Node2 0017 500507680120376F SVC San Volume Controller 0 V1 all SVC4_Node2 0018 500507680110376F SVC San Volume Controller 0 V1 all
3.7.2 Adding DS4000 storage to the SVC To add DS4000 storage to the SVC, follow these steps: 1. Check the prerequisites on the Web at: http://www.ibm.com/storage/support/2145 2. Check the supported firmware levels and configurations before you connect to the SVC. You can see the firmware version for the DS4000 in the Storage Manager by choosing Storage Subsystem → View profile, as shown in Figure 3-20 on page 73, Figure 3-21 on page 73, and Figure 3-22 on page 74.
72
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 3-20 Where to find the Storage Subsystem Profile
Figure 3-21 The Storage Subsystem Profile showing the firmware version
Chapter 3. Planning and configuration
73
Figure 3-22 DS4000 mappings view
3. We defined one storage partition in the DS4000 with all of the SVC node ports defined, and one host partition with eight ports. See Figure 3-23.
Figure 3-23 Shows the array created
Figure 3-24 on page 75 shows the host type for the SVC.
74
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 3-24 Host type for storage partition
Figure 3-25 shows the port mapping. Now the volumes are presented to the SVC on both ports. You can find the MDisks on the SVC and rename them to a unique name identifying the origin. Consider the example of F1_Array_LUN - DS4000 number 1, _array number_LUN number. You can add the MDisk to an MDisk group called DS4000_14830, for example.
Figure 3-25 Port mapping
Chapter 3. Planning and configuration
75
3.7.3 LUN layout When assigning LUNs from your disk subsystem to the SVC, assign the LUN to all ports in the SVC, and spread the LUNs in the disk subsystem, so you get the best performance and reliability. In the DS4000, you should have an equal number of arrays and LUNs, equally spread on the two controllers. After having assigned spare disks in the DS4000, define your arrays with RAID protection, and create as few LUNs as you can in the array. If possible, make the LUNs the same size, so you can utilize the full capacity when striping VDisks on the SVC. In an MDG where the VDisks are all striped, we recommend that all the MDisks be the same size if possible, so you can utilize the full capacity when striping VDisks on the SVC.
76
Implementing the IBM System Storage SAN Volume Controller V4.3
4
Chapter 4.
Performance and capacity planning While storage virtualization with SVC improves flexibility and provides simpler management of a storage infrastructure, it can also provide a substantial performance advantage for a variety of workloads. The SVC’s caching capability and its ability to stripe VDisks across multiple disk arrays is the reason why performance improvement is significant when implemented with midrange disk subsystems, since this technology is often only provided with high-end enterprise disk subsystems. Note: Technically, almost all storage controllers provide both striping (RAID 1 or RAID 10) and some form of caching. The real advantage is the degree to which you can stripe the data, that is, across all MDisks in a group, and therefore have the maximum number of spindles active at once. The caching is a secondary reason and the point is that the SVC provides additional caching above what midrange controllers provide (usually a couple of GB), whereas enterprise systems have much larger caches. To ensure the desired performance and capacity of your storage infrastructure, we recommend that you do a performance and capacity analysis to reveal the business requirements of your storage environment. When this is done, you can use the guidelines in this chapter to design a solution that meets the business requirements.
© Copyright IBM Corp. 2003-2008. All rights reserved.
77
4.1 Performance considerations When discussing performance for a system, it always comes down to identifying the bottleneck, and thereby the limiting factor of a given system. At the same time, you must take into consideration the component for whose workload you do identify a limiting factor, since it might not be the same component that is identified as the limiting factor for different workloads. When designing a storage infrastructure using SVC, or implementing SVC in an existing storage infrastructure, you must therefore take into consideration the performance and capacity of the SAN, the disk subsystems, the SVC, and the known/expected workload.
4.1.1 SAN The SVC now has four models: 4F2, 8F2, 8F4, and 8G4. The first number indicates the memory size; the last number represents the maximum fabric speed. All of them can connect to 1 Gbps, 2 Gbps, or 4 Gbps switches. From a performance point of view, it is better to connect the SVC to 4 Gbps switches. There are some guidelines about designing a IBM TotalStorage: SAN Product, Design, and Optimization Guide, SG24-6384. It is available in PDF format at: http://www.redbooks.ibm.com/redbooks/pdfs/sg246384.pdf Correct zoning on the SAN switch will bring security (zoning is not designed to provide security, but is often used to provide a level of security) and performance together. We recommend to implement a dual HBA approach at the host to access the SVC, and we recommend that you use the zoning shown in Figure 4-1 on page 79. During host discovery, both the SVC host path definitions and the switch zoning affect the number of occurrences of a given VDisk that the host will identify. We recommend that the number of occurrences should be kept to four or fewer, because increasing the number to more than four does not improve either performance or reliability, and may cause more time and trouble for the administrator responsible for the OS.
78
Implementing the IBM System Storage SAN Volume Controller V4.3
Host
HOST-A1 Zone (21,1;11,2;11,3)
HOST-A2 Zone (22,1;12,2;12,3)
A1 A2 ID 21
A1
A2
Fabric 1
Fabric 2
11 22 13 24 Host port A1 and one SVC port per SVC node
ID 22
11 12 13 14 SVC node1
ID 11
21 12 23 14
ID 12
21 22 23 24 Host port A2 and SVC node2
one SVC port per SVC node
Figure 4-1 Host side zone recommendation
The configuration shown in Figure 4-1 is best for many tens of VDisks and where only one path will be used actively (the other is a failover path and SDD will only send to the preferred node). With this design, the number of VDisk occurrences that the host sees during a discovery will be equal to four (the VDisk will be seen twice through each side of the dual fabric, once on the preferred node and once on the alternate node). For smaller numbers of VDisks with greater throughput requirements, more than one HBA should be mapped again to a single SVC node port each.
4.1.2 Disk subsystem There are a number of IBM Redbooks that cover the topic of tuning IBM disk subsystems for performance: IBM TotalStorage DS8000 Series: Performance Monitoring and Tuning, SG24-7146 http://www.redbooks.ibm.com/redbooks/pdfs/sg247146.pdf IBM TotalStorage DS6000 Series: Performance Monitoring and Tuning, SG24-7145 http://www.redbooks.ibm.com/redbooks/pdfs/sg247145.pdf DS4000 Best Practices and Performance Tuning Guide, SG24-6363 http://www.redbooks.ibm.com/redbooks/pdfs/sg246363.pdf
Chapter 4. Performance and capacity planning
79
From the performance perspective, there are a few guidelines in connecting with SVC: 1. Connect all storage ports to the switch and zone them to all the SVC ports. You zone all ports on the disk back-end storage to all ports on the SVC nodes in a cluster. And you must also make sure to configure the storage subsystem LUN Masking settings to map all LUNs to all the SVC WWPNs in the cluster. The SVC is designed to handle large quantities of multiple paths from back-end storage. 2. Using as many as possible 15K RPM disks will improve performance considerably. 3. Creating one LUN per array will help in a sequential workload environment. 4. There are some settings that should be changed on the storage subsystem when working with the SVC. For example, with the DS4000, we recommended setting the start/stop flushing parameter to 80%/80%, set the Cache block size to 4 KB, and set the segment size according to the DataBase application behavior when working with the SVC. For further reference, see the IBM System Storage SAN Volume Controller: Software Installation and Configuration Guide, which is available at: http://www-1.ibm.com/support/docview.wss?rs=591&uid=ssg1S7002156 In most cases, the SVC will be able to improve the performance, especially on middle-low end disk subsystems or older disk subsystems with slow controllers or uncached disk systems. This improvement happens because: The SVC has the capability to stripe across disk arrays and it can do so across the entire set of supported physical disk resources. The SVC has a 4 to 8 GB cache and it has an advanced caching mechanism. The SVC’s large cache and advanced cache management algorithms also allow it to improve upon the performance of many types of underlying disk technologies. SVC’s capability to manage, in the background, the destaging operations incurred by writes (while still supporting full data integrity), has the potential to be particularly important in achieving good database performance. Depending upon the size, age, and technology level of the disk storage system, the total cache available in the SVC may be larger, smaller, or about the same as that associated with the disk storage. Since hits can occur in either the upper (SVC) or the lower (disk controller) level of the overall system, the system as a whole can take advantage of the larger amount of cache wherever it is located. Thus, if the storage control level of cache has the greater capacity, hits to this cache should be expected to occur, in addition to hits in the SVC cache. Also, regardless of their relative capacities, both levels of cache will tend to play an important role in allowing sequentially organized data to flow smoothly through the system. SVC cannot increase the throughput potential of the underlying disks in all cases. Its ability to do so depends upon both the underlying storage technology, as well as the degree to which the workload exhibits “hot spots” or sensitivity to cache size or cache algorithms. IBM SAN Volume Controller 4.2.1 Cache Partitioning, REDP-4426, shows the SVC’s cache partitioning capability: http://www.redbooks.ibm.com/abstracts/redp4426.html?Open
4.1.3 SVC The SVC cluster is scalable up to eight nodes, and the performance is almost linear when adding more nodes into an SVC cluster, until it becomes limited by other components in the storage infrastructure. While virtualization with the SVC provides a great deal of flexibility, it does not diminish the necessity to have a SAN and disk subsystems that can deliver the 80
Implementing the IBM System Storage SAN Volume Controller V4.3
desired performance. Essentially, SVC performance improvements are gained by having as many MDisks as possible, therefore creating a greater level of concurrent I/O to the back end without overloading a single disk or array. In the following sections, we discuss the performance of the SVC and assume that there are no bottlenecks in the SAN or on the disk subsystem.
Latency Latency is the delay added to the response time for an I/O operation. All in-band storage virtualization solutions add some latency to cache miss I/Os. This is not a unique characteristic of the SVC. However, the SVC latency is very low. For a 4 KB read operation, the SVC introduces approximately 60 µs (microseconds, millionths of a second) of additional delay. Considering that a typical cache miss response time is approximately 10 ms (milliseconds, thousandths of a second), the delay typically caused by the SVC is negligible (less than 1% of total response time). In the real world, the effect of latency is normally even less. All writes to the SVC are cache hits, so they add no latency. Because of the advanced cache algorithm in the SVC, many reads are also cache hits. Only cache miss I/Os add latency.
Performance increases SVC V4.2 posted the highest results in the SPC-1 and SPC-2 disk subsystem storage performance test by the Storage Performance Council. SPC-1 consists of a single workload designed to demonstrate the performance of a storage subsystem while performing the typical functions of business critical applications. Those applications are characterized by predominately random I/O operations and require both queries as well as update operations. Examples of those types of applications include OLTP, database operations, and mail server implementations. SPC-2 consists of three distinct workloads designed to demonstrate the performance of a storage subsystem during the execution of business critical applications that require the large-scale, sequential movement of data. Those applications are characterized predominately by large I/Os organized into one or more concurrent sequential patterns. Examples of those types of applications include large file processing, large database queries, and Video on Demand. The SPC Web site can be found at: http://www.storageperformance.org Ideas International keeps a list of results ranked by throughput at: http://www.ideasinternational.com/ In IBM internally, there is a white paper about SVC V4.2 performance that is available. Contact your IBM pre-sales contact to get it by going to: http://w3-03.ibm.com/sales/support/ShowDoc.wss?docid=B552897U42160U19&infotype=SK& infosubtype=M0 There is also an IBM Redbooks publication available that covers more detailed information about SVC performance called SAN Volume Controller: Best Practices and Performance Guidelines, SG24-7521, found at: http://www.redbooks.ibm.com/abstracts/sg247521.html?Open
Chapter 4. Performance and capacity planning
81
The areas to be considered when planning a storage infrastructure using SVC, or implementing the SVC in an existing storage infrastructure, are listed in the following sections.
Determine the number of SVC clusters If you plan to deploy the SVC in a geographically dispersed environment, for example, a dual site design for disaster recovery reasons, it is essential to use two SVC clusters. Due to the design of the SVC cluster, we do not recommend splitting I/O groups geographically (for the nodes in an I/O group), since this will not provide the resiliency needed for a disaster recovery solution. There is a specific limit on the distance between nodes due to latency; this can cause the cluster to “lease expire,” which will stop I/O. Note: IBM recommends that you use two or more cluster configurations for all production disaster recovery systems. Intracluster Metro Mirror consumes more cluster resources than intercluster Metro Mirror.
Guidelines in creating the managed disk group Here are some guidelines to follow when creating a managed disk group (MDG): All the managed disks in a single MDG should have the same (or similar) performance characteristics. If you mix managed disks with different performance characteristics, VDisks might exhibit uneven performance where I/O to different portions of the VDisk perform differently. The overall performance will be that of the slowest member; all I/O will be slowed to the slowest MDisk due to the way the cache works. For high availability, easier maintenance, and troubleshooting reasons, we recommend that you do not include multiple disk subsystems in the same MDG, since the failure of one disk subsystem will make the MDG go offline, and thereby all VDisks belonging to the MDG will go offline. For example, if you have IBM DS4000 and EMC CLARiiON CX3-80, put the DS4000 MDisks in one MDG and the CX3-80 MDisks in another MDG. If there are less than 20 MDisks from one back-end storage, put them into one MDG. Using a smaller extent size can help balance the workload onto different MDisks, different LUNs, and different physical disks, and this will help the performance. Note: The extent size does not have a great impact on the performance of an SVC installation, and the most important consideration is to be consistent across MDGs. This means that you should use the same extent size in all MDGs within an SVC cluster to avoid limitations when migrating VDisks from one MDG to another. Another consideration is the storage capacity that this SVC cluster will potentially manage. For example, a 16 MB extent size cluster can manage a 64 TB storage capacity, while a 512 MB extent size cluster can manage 2 PB.
Guidelines in creating a VDisk Here are some guidelines to follow when creating a VDisk: Always choose stripe as the VDisk mode. We recommend that you stripe a VDisk among all the MDisks in an MDG, unless you need to ensure that data is provided from one and only one disk. The striping will balance I/O across all the managed disks in the MDG, which tends to optimize overall performance and helps to reduce hot spots.
82
Implementing the IBM System Storage SAN Volume Controller V4.3
Note: There is an exception here. If you are setting up for a purely sequential workload, for example, large transfer video on demand type applications, then making SVC VDisks sequential will allow SVC and the back-end MDisks to detect the sequential nature of the workload, and thus double prefetch will kick in. This, however, will limit the ultimate performance to that data that can be read off the single MDisk providing the VDisk. This should only be used when a mid to high back-end controller is presenting the MDisk and it has a large cache with a good prefetch algorithm, for example, DS6000/8000 type disk subsystems. Set the cache mode to read/write whenever possible. By default, the SVC will alternate VDisks between the nodes of an I/O group in the order the VDisks are created, and this normally produces good results. This will generally balance the load well across both nodes of an I/O group. In cases where VDisks vary greatly in size, or where the I/O load to different VDisks varies greatly, then you can use the -node parameter when creating VDisks (or the equivalent graphical user interface parameter) to specify which node of an I/O group should be the preferred path, in order to balance the workload evenly for the I/O group.
Guidelines for creating space-efficient VDisks Space-efficient VDisks require more I/Os because of the increased directory access. For a 100% write workload the impact can be, in a worst case scenario, up to 50% (one directory I/O for every user I/O). For reads, the impact depends on whether the data is in the cache, or if the metadata directory is in the cache. This causes more CPU processing, so the performance per I/O group will be lower. However, the directory is two-way write-back cached (just like the SVC fastwrite cache), so some applications will perform better. Note: To get the best space-efficient VDisk performance, use striping to spread space-efficient VDisks across many back-end disks. Be careful when using space-efficient VDisks and virtualization to reduce the number of disks required, as this is likely to cause a performance problem.
I/O queue depth handling in large SANs The purpose of discussing I/O queue depth is to avoid situations where an SVC node reaches its maximum number of queued commands. Note: Unless you have an SVC configuration close to the maximum supported, and all your hosts are simultaneously busy, it is unlikely that you will encounter problems with I/O queue depth limitations for your SVC. As a guideline, you should consult your IBM representative in case your calculation shows that the I/O queue limit is less than 20; see “Homogeneous queue depth calculation” on page 85.
Chapter 4. Performance and capacity planning
83
The queuing of tasks consumes internal resources in the SVC node. Each SVC node can handle 10,000 concurrent commands, distributed across all hosts and all VDisks. Mechanisms are provided to ensure correct operation in the event that the I/O queue is full. Each host port will be guaranteed to be able to queue a single command on an SVC node (this is per node, not per VDisk). I/O governing can be used to restrict the I/Os a host can submit. If the SVC runs out of resources to queue an I/O that it has received, the algorithm shown in Figure 4-2 takes effect to handle the situation when the maximum of queued commands is reached.
Unable to enqueue an I/O
Has initiator already consumed its specially reserved command on this node
Yes
1. Set “Unit Attention - Commands cleared by another initiator” on the LUN for that initiator
2.
Discard the command
No
Has the initiator at least one task queued for that LUN on this port
If a Unit Attention is already set, then the command is simply discarded
Yes
Return Task Set Full status, using the specially reserved command
No
The initiator has no tasks queued for that LUN on this port
Then
Return Check Condition “Unit Attention - Commands aborted by another initiator” to the received command, using the specially reserved command
Figure 4-2 I/O queue depth algorithm
This algorithm allows the SVC to discard commands, and to give a valid reason to the host as to why this has happened. This algorithm is also used when internal recoveries within the SVC node mean that the SVC is unable to start new host I/Os immediately, and the SVC consequently runs out of resources to queue all the I/Os that are received. Unfortunately, many host operating systems do not have helpful recovery algorithms if this situation persists for more than 15 seconds, and the result will often be that one or more hosts present errors to applications that result in application failures. Following these recommendations will hopefully avoid this problem. Note: This issue is not in any way specific to the SVC. All controllers and operating systems have the same issues if the maximum queue depth is reached.
84
Implementing the IBM System Storage SAN Volume Controller V4.3
Calculating a queue depth limit When calculating the queue depth, consider the following factors: Although the maximum number of queued commands is per node and there are two nodes in an I/O group, the system must continue to function when one of the nodes in an I/O group is not available. Thus, you must consider an I/O group to have the same number of queued commands as a node. However, when a node fails, the number of paths to each disk is halved. In practice, this effect can be neglected, and you can count nodes rather than I/O groups in the calculation below. If a VDisk is mapped so that it can be seen by more than one host, then each host that it is mapped to can send a number of commands to it. Multipathing drivers on some hosts round robin I/Os among the available I/O paths. For hosts that do not currently do this, it is possible that this behavior might change in the future, and you need to avoid “breaking” customers’ configurations when this happens. If a device driver times out a command, it will typically re-issue that command almost immediately. The SVC will have both the original command and the retry in the command queue in addition to any Error Recovery Process (ERP) commands that are issued. In order for the maximum queue depth not to be reached, then, for all VDisks associated with the I/O group, and for all hosts that are mapped to be able to see each VDisk, and for all paths on each host, the sum of the queue depths must be less than 10,000. Because ERPs can consume some number of queued command slots, this number is reduced to 7000 to allow a safety margin.
Homogeneous queue depth calculation This calculation applies to systems where: The available queued commands are to be shared out among all paths, rather than giving some hosts additional resources. The VDisks are shared out evenly among the I/O groups in the cluster.
Chapter 4. Performance and capacity planning
85
Note: The queue depth for each VDisk should be set on the hosts using the following calculation: q = Round up ((n * 7000) / (v * p*c)1 ) q = Per device path q-depth setting n = Number of nodes in the cluster v = Number of VDisks configured in the cluster p = Number of paths per VDisk per host. A path is a route from a host FC port to an SVC FC port that is recognized by the host as giving access to the VDisk. c= The number of hosts that can concurrently access each VDisk. Very few applications support concurrent access from multiple hosts to a single VDisk. Examples where multiple hosts have concurrent access to a disk include cases where the SAN File System (SFS) is in use. Thus, typically c will be 1. For example, we calculate the I/O queue for a homogenous SVC configuration with eight SVC nodes and the maximum number of supported VDisks. n=8: An 8 node SVC cluster v=4096: The number of VDisks per SVC cluster is a maximum of 4096. c=1: One host is able to access each VDisk. p=4: Each host has four paths to each VDisk (Two HBAs each with two paths to the I/O group). Calculate the queue depth as follows: roundup q= (8*7000) / (4096*4*1) = 4 So, the queue depth in the operating systems should be set to four concurrent commands per path.
Non-homogeneous queue depth calculation In some cases, it could be appropriate to give favored hosts additional resources to allow them to queue additional commands, or the number of VDisks supported by each I/O group might be different. In these cases, the queue depth is calculated in the following way. Note: Consider each I/O group in turn: For each VDisk, consider each host to which that VDisk has a mapping. This gives a set of (host and VDisk) pairs. As long as the total sum of all (host and VDisk) pairs (queue depth) is less than 7000, the system should not experience problems due to queue full situations. The above calculation assumes that there is a significant probability that all of the hosts will initiate the number of concurrent commands that they are limited to, that is, each host is busy. If there are a large number of fairly idle hosts in the configuration, which are not going to be initiating very many concurrent commands, then it reasonable that the queue depth does not need to be limited, even if the calculation above says that it should be. If this is the case, we recommend that the queue depth be increased/not set.
86
Implementing the IBM System Storage SAN Volume Controller V4.3
How to limit the queue depth Once you have determined the appropriate queue depth limit, you must apply it. Each operating system has an OS/HBA specific way to limit the queue depth on a per device path basis. An alternative to setting a per path limit is to set a limit on the HBA. Note: For example, one host has two HBAs and four paths per VDisk. The host will access 40 VDisks, and its maximum concurrent I/O commands per path is five. The calculation is: Four paths per VDisk, with a limit of five concurrent I/O commands per path, equals 20 concurrent I/O commands per VDisk. 40 VDisks with 20 concurrent I/O commands per VDisk equals 800 concurrent I/O commands for the host. Therefore, it may be appropriate to place a queue depth limit of (40*(4*5))/2=400 on each adapter. This allows sharing of the queue depth allocation between VDisks. For a system that is already configured, the result (v*p*c) is actually the same number, as determined by issuing the datapath query device command on all hosts, and summing up the number of paths.
4.1.4 Host Here are some considerations when installing a host into the SVC environment.
Balanced load across HBA ports To obtain the best performance from a host with multiple FC ports, the zoning should ensure that each FC port of a host is zoned with a different group of SVC ports (see Figure 4-1 on page 79).
Balanced host load across SVC ports To obtain the best performance of the subsystem and to prevent overloading, the workload to each SVC port should be equal. This will typically involve zoning approximately the same number of host FC ports to each SVC FC port.
Optimize multipath for SDD SDD takes advantage of redundant connections between a host server and the disk storage server to optimize availability and distribute I/O activity. It distributes the workload among the multiple I/O paths for load balancing. SDD uses either the load balancing (default) or round robin policy to select paths for I/O. The preferred paths will do load balancing. The best configuration we recommend is four paths per physical LUN on the host (two preferred paths and two backup paths). An example configuration that addresses this point is shown in Figure 4-1 on page 79. More than four paths may degrade the IO performance, while two paths are not reliable enough. In the case of one host or storage HBA, or if the switch malfunctions, there is only one good path left. If for some reason the storage microcode failed, or an I/O request timed out on the good path, then there is a possibility that the application may fail. In an environment where multiple servers are sharing LUNs, the total number of paths from all the servers to a LUN should be considered. Configuring more than four paths per LUN potentially increases the risk of reaching the maximum number of queued commands for the SVC cluster.
Chapter 4. Performance and capacity planning
87
4.2 Performance modeling and sizing At the pre-sales stage, you can use IBM Disk Magic™ tool to assist you in estimating how many SVC nodes can satisfy the performance requirement in a typical environment. At the post-sales stage, you can use the IBM Disk Magic tool to model an existing configuration with SVC already installed. Here is the link to the too; you will need to contact your IBM account representative for access to it: http://w3-03.ibm.com/sales/support/ShowDoc.wss?docid=Q947558L63209Z65&infotype=SK& infosubtype=S0&campaign=&lng=&node=brands%2CB5000&ftext=disk+magic
4.3 Performance monitoring In this section, we discuss some performance monitoring techniques.
4.3.1 Collecting performance statistics By default, performance statistics are not collected. You can start performance collection by using the svctask startstats command, as described in 9.12, “Listing dumps” on page 394, and you can stop them using the svctask stopstats command, as described in 9.2.8, “Stopping a statistics collection” on page 317. You can list the files using the lsiostatsdumps command. Statistics gathering is enabled or disabled on a cluster basis. When gathering is enabled, all nodes in the cluster gather statistics. SVC supports sampling periods of the gathering of statistics from 1 to 60 minutes in steps of one minute. The gathering of this data is coordinated at a cluster level. There are two sets of performance statistics: Cluster wide statistics Per-node statistics Important: Enabling statistics collection with an interval less than 15 minutes will only enable per-node statistics. Cluster wide statistics will not be collected at an interval less than 15 minutes.
4.3.2 Cluster wide statistics A number of statistics are collected for every Virtual Disk and every Managed Disk known to the cluster. The statistics reported are on a per-cluster rather than a per-node basis. Thus, for example, the count of I/Os for a given managed disk are the aggregate of the I/Os for that managed disk across all of the nodes in the cluster. At the end of each sampling period, the statistics gathered during the sampling period are written to files on the configuration node. Each sampling period results in the creation of one file for Virtual Disk statistics, and one file for Managed Disk statistics.
88
Implementing the IBM System Storage SAN Volume Controller V4.3
Statistics file naming The files generated are written to the directory /dumps/iostats. The file name is in the following format: m_stats_
__ for MDisk statistics. v_stats___ for VDisk statistics. The node_front_panel_id is the node from which the statistics are collected: The panel ID is taken from the current configuration node. The date is in the form YYMMDD. The time is in the form HHMMSS. Example 4-1 shows some typical MDisk and the VDisk statistics file names. Example 4-1 Filename of per cluster wide statistics IBM_2145:ITSOSVC42A:admin>svcinfo lsiostatsdumps id iostat_filename 12 m_stats_104603_070528_133722 13 v_stats_104603_070528_133722
Tip: You can use pscp.exe, which is installed with PuTTY, from an MS-DOS® command line prompt to copy these files to local drives. WordPad can be used to open them. For example: C:\Program Files\PuTTY>pscp -load ITSO-CLS1 6 -unsafe [email protected] :/dumps/iostats/* c:\temp\iostats The -load parameter is used to specify the session defined in PuTTY.
Statistics collected For each virtual disk and for each managed disk, the following statistics are collected during the sample period:
Number of SCSI READ commands processed Number of SCSI WRITE commands processed Number of blocks of data read Number of blocks of data written
Contents of statistics files A cluster wide statistics file is a plain text file. The file contains one entry for every managed or virtual disk. In Example 4-2, we use WordPad to open the file m_stats_104603_070528_133722. We use the columns to get a count of reads, writes, block reads, and block writes. Example 4-2 MDisk per cluster statistics lun_id 0 1 2 3 4 5
: : : : : : :
num_reads 49153 49409 61440 61440 48679 45113
: : : : : : :
num_writes 1842 2814 61440 61440 289 618
: : : : : : :
block_reads 3145736 3156303 31457280 31457280 3114430 2885430
: block_writes : 117525 : 170733 : 31457280 : 31457280 : 16959 : 36547
: : : : : : :
Chapter 4. Performance and capacity planning
89
4.3.3 Per node statistics The collecting of per node statistics is enabled or disabled in the same way as cluster wide statistics, as described in 4.3, “Performance monitoring” on page 88. Each node maintains a number of counters, which are reset to zero when a node is booted or reset. Each of these counters is sampled at the end of each period. The sampled value is the absolute value of the counter, not the increase of the counter during the sample period. The file format for these statistics is XML.
Statistics file naming The files generated are written to the directory /dumps/iostats. The file name is in the following format: Nm_stats_<node_front panel_id>__ for MDisk statistics. Nv_stats_<node_front panel_id>__ for VDisk statistics. Nv_stats_<node_front panel_id>__ for Node statistics. The node_front_panel_id is the node from which the statistics are collected: The date is in the format of YYMMDD. The time is in the format of HHMMSS. Example 4-3 shows MDisk, VDisk, and the node statistics file names. Example 4-3 Node statistics filename IBM_2145:ITSO-CLS1:admin>svcinfo lsiostatsdumps id iostat_filename 0 Nm_stats_104603_070528_113203 1 Nv_stats_104603_070528_113203 2 Nn_stats_104603_070528_113203 3 Nm_stats_104603_070528_115707 4 Nv_stats_104603_070528_115707 5 Nn_stats_104603_070528_115707 6 Nm_stats_104603_070528_122211 7 Nv_stats_104603_070528_122211 8 Nn_stats_104603_070528_122211 9 Nm_stats_104603_070528_124714 10 Nv_stats_104603_070528_124714 11 Nn_stats_104603_070528_124714 12 Nn_stats_104603_070528_131218
A maximum number of 16 files for each type can be present in the directory, for example, 16 files for MDisk statistics,16 files for VDisk statistics, and 16 files for node statistics. Note: If you plan to collect these files manually, that is, without TPC, then you should make sure to collect the files from the config node and the other nodes in the cluster at regular intervals; otherwise, you may lose the older files due to this limit. These SVC counters give you an additional input for performance analysis. Analyzing Storage Performance is described in the IBM Redbooks publications listed in 4.1.2, “Disk subsystem” on page 79.
90
Implementing the IBM System Storage SAN Volume Controller V4.3
More information about using TPC to monitor your storage subsystem is covered in Monitoring Your Storage Subsystems with TotalStorage Productivity Center, SG24-7364, found at: http://www.redbooks.ibm.com/abstracts/sg247364.html?Open A good introduction about how to monitor host performance can be found in the following IBM Redbooks publications: Tuning IBM System x Servers for Performance, SG24-5287, found at: http://www.redbooks.ibm.com/abstracts/sg245287.html?Open AIX 5L Practical Performance Tools and Tuning Guide, SG24-6478, found at: http://www.redbooks.ibm.com/abstracts/sg246478.html?Open
Chapter 4. Performance and capacity planning
91
92
Implementing the IBM System Storage SAN Volume Controller V4.3
5
Chapter 5.
SVC Console In this chapter, we present an overview of the SAN Volume Controller console and the System Storage Productivity Center (SSPC), the software and hardware components, and the initial installation and configuration procedures for the IBM System Storage SAN Volume Controller (SVC) using the service panel and the cluster Web interface. Note: The service panel consists of the LCD display window and buttons on the front of each SVC node.
© Copyright IBM Corp. 2003-2008. All rights reserved.
93
5.1 Systems Storage Productivity Center overview The System Storage Productivity Center (SSPC) is an integrated hardware and software solution that provides a single management console for managing IBM SAN Volume controller, IBM DS8000, and other components of your data storage infrastructure. The current release of the SSPC V1.2 is made up three major components: SSPC hardware, IBM TPC V3.3.2 Basic Edition, and SAN Volume Controller V4.3.0 Console and CIM agent. This replaces the functionality of the SVC Master Console (MC), which was a dedicated management console for the SVC. The Master Console is still supported and will run the latest code levels of the SVC Console software components. The SSPC has all the software components pre-installed and tested on a System xTM machine model SSPC 2805-MC2 with Windows installed on it. All the software components installed on the SSPC can be ordered and installed on hardware that meets or exceeds minimum requirements. The SVC Console software components are also available on the Web. When using the SSPC with the SAN Volume Controller, you have to install it and configure it before configuring the SAN Volume Controller. For a detailed guide to the SSPC, we recommend that you refer to the IBM System Storage Productivity Center Software Installation and User’s Guide, SC23-8823. For information pertaining to physical connectivity, storage area network (SAN) zoning, and assigning disks to the SVC, see Chapter 3, “Planning and configuration” on page 25.
5.1.1 SSPC hardware The hardware used by the SSPC Solution is the IBM System Storage Productivity Center 2805-MC2. It is 1U rack mounted server. It has the following initial configuration:
1x Quad-Core Intel Xeon® Processor E5310 (1.60 GHz 8 MB L2) 4x 1 GB PC2-5300 CL5 ECC DDR2 Chipkill™ FBDIMM 667 MHz 2x Primary array hard disk drives: 146 GB 15K 3.5" SAS HDD RAID 1 Optical drive: CD-ROM Ultrabay Enhanced Drive Ethernet: Dual integrated 10/100/1000 Mbps Ethernet Microsoft Windows 2003 Enterprise Edition
It is designed to perform basic SSPC functions. If you plan to upgrade SSPC for more functions, you can purchase the Performance Upgrade Kit to add more capacity to your hardware. If an SSPC or SVC Master Console is not already installed, the customer must obtain a rack-mounted, high-performance, and highly-reliable Intel server (such as the IBM System x Model 3550 or equivalent) with the following options: One Quad Core Intel Xenon processor, minimum 1.6 GHz Minimum of 4 GB of system memory Two SATA hard disk drives, minimum 40 GB each; as a required step in the installation process, these drives must be configured as mirrored CD-ROM drive One 1 Gb port for Ethernet connections (FC or copper) Console connectivity (Screen, Keyboard, Mouse) 94
Implementing the IBM System Storage SAN Volume Controller V4.3
5.1.2 Example hardware configuration Here is a list of hardware required for an example configuration:
IBM System x3550 server (1U) A single 2.50 GHz Intel Xenon quad-core processor 4 GB of memory DIMM (4 x 1 GB) Two 146 GB SAS hard disk drives (arranged for mirroring) One 10/100/1000 Copper Ethernet ports on planar NetBay 1U Flat Panel Monitor Console Kit without keyboard or equivalent Keyboard, such as the Space Saver NLS keyboard
5.2 SVC Console software The SVC Console requires that you obtain the following software: Operating system: – The SSPC is shipped with Microsoft Windows Server® 2003 Enterprise Edition pre-installed. – The SVC Console requires that one of the following operating systems is provided on your hardware platform: • • •
Microsoft Windows 2000 Server 5.00.2195 Microsoft Windows Server 2003 Standard Edition Microsoft Windows Server 2003 Enterprise Edition
Microsoft Windows Internet Explorer® Version 7.0 (or Version 6.1 with Service Pack 1, for Windows 2000 Server). Antivirus software (not required but strongly recommended). PuTTY Version 0.60 (if not installed) You can obtain the latest copy of PuTTY by going to the following Web site and downloading the Windows installer in the Binaries section: http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html Consideration: If you want to use IPv6, then you must be running Windows 2003 Server. For a complete and current list of the supported software levels for the SVC Console, refer to the SVC Support page at: http://www.ibm.com/storage/support/2145
5.3 Installation planning information for the SSPC Take the following steps when planning the SSPC installation:
Verify that the hardware and software prerequisites have been met. Determine the location of the rack where the SSPC is to be installed. Verify that the SSPC will be installed in line of sight to the SVC Nodes. Verify that you have a Keyboard, Mouse, and Monitor available to use. Determine the cabling required. Determine the Network IP address. Determine the SSPC host name.
Chapter 5. SVC Console
95
For detailed installation guidance, see the IBM System Storage Productivity Center: Introduction and Planning Guide, SC23-8824, at: https://www-304.ibm.com/systems/support/supportsite.wss/supportresources?brandind= 5000033&familyind=5356448 and the IBM System Storage Volume Controller: Software Installation and Configuration Guide, SC23-6628 at: http://www.ibm.com/storage/support/2145
5.4 Secure Shell overview Secure Shell (SSH) is used to secure data flow between the SVC cluster configuration node (SSH server) and a client, either a command-line client through the command-line interface (CLI) or the CIMOM. The connection is secured by the means of a private key and public key pair: A public key and a private key are generated together as a pair. A public key is uploaded to the SSH server. A private key identifies the client and is checked against the public key during the connection. The private key must be protected. The SSH server must also identify itself with a specific host key. If the client does not have that host key yet, it is added to a list of known hosts. Secure Shell is the communication vehicle between the management system (usually the SSPC) and the SVC cluster. SSH is a client-server network application. The SVC cluster acts as the SSH server in this relationship. The SSH client provides a secure environment from which to connect to a remote machine. It uses the principles of public and private keys for authentication. When an SSH client (A) attempts to connect to a server (B), a key is needed to authenticate the connection. The key consists of two halves: the public and private keys. The public key is put onto (B). When (A) tries to connect, the private key on (A) can authenticate with its public half on (B). These mechanisms (public/private key pair and host key) are used so that each party is sure about the identity of the other one, as shown in Figure 5-1.
96
Implementing the IBM System Storage SAN Volume Controller V4.3
A
B
Uploading public key
SSH
SSH
Client
Server
A
B
private key
SSH
Client
My message
public key
SSH
Server
host key
Zdss5yXZCKgeQrhtKPqmXXqJYZ
My message
Encryption Figure 5-1 SSH client/server
The communication interfaces are shown in Figure 5-2.
Native SVC CLI over secured (SSH) IP
SVC Cluster SVC Native CLI linux-based kernel
ICAT Proxy / SSPC
Ethernet
2-4 node pairs
xmlCIM over HTTP
Native SVC CLI over secured (SSH) IP
ICAT CIM Agent
HTTP
ICAT GUI
CLI client
Web Browser ICAT GUI Client
Figure 5-2 Communication interfaces
SSH keys are generated by the SSH client software. This includes a public key, which is uploaded and maintained by the cluster, and a private key that is kept private to the workstation that is running the SSH client. These keys authorize specific users to access the administration and service functions on the cluster. Each key pair is associated with a user-defined ID string that can consist of up to 40 characters. Up to 100 keys can be stored on the cluster. New IDs and keys can be added and unwanted IDs and keys can be deleted. To use the CLI or SVC graphical user interface (GUI), an SSH client must be installed on that system, the SSH key pair must be generated on the client system, and the client’s SSH public key must be stored on the SVC cluster or clusters.
Chapter 5. SVC Console
97
The SSPC has the freeware implementation of SSH-2 for Windows called PuTTY pre-installed. This software provides the SSH client function for users logged into the SVC Console who want to invoke the CLI or GUI to manage the SVC cluster.
5.4.1 Generating public and private SSH key pairs using PuTTY Perform the following steps to generate SSH keys on the SSH client system (SSPC). Note: These keys will be used in the step documented in 5.6.4, “Configuring the PuTTY session for the CLI” on page 123. 1. Start the PuTTY Key Generator to generate public and private SSH keys. From the client desktop, select Start → Programs → PuTTY → PuTTYgen. 2. On the PuTTY Key Generator GUI window (Figure 5-3), generate the keys: a. Select the SSH2 RSA radio button. b. Leave the number of bits in a generated key value at 1024. c. Click Generate.
Figure 5-3 PuTTY key generator GUI
98
Implementing the IBM System Storage SAN Volume Controller V4.3
3. The message in the Key section of the window changes. Figure 5-4 shows this message.
Figure 5-4 PuTTY random key generation
Note: The blank area indicated by the message is the large blank rectangle on the GUI inside the section of the GUI labelled Key. Continue to move the mouse pointer over the blank area until the progress bar reaches the far right. This generates random characters to create a unique key pair.
Chapter 5. SVC Console
99
4. After the keys are generated, save them for later use as follows: a. Click Save public key, as shown in Figure 5-5.
Figure 5-5 Saving the public key
b. You are prompted for a name (for example, pubkey) and a location for the public key (for example, C:\Support Utils\PuTTY). Click Save. If another name or location is chosen, ensure that a record of them is kept, because the name and location of this SSH public key must be specified in the steps documented in 5.6.2, “Uploading the SSH public key to the SVC cluster” on page 115. Note: The PuTTY Key Generator saves the public key with no extension by default. We recommend that you use the string “pub” in naming the public key, for example, “pubkey”, to easily differentiate the SSH public key from the SSH private key. c. In the PuTTY Key Generator window, click Save private key. d. You are prompted with a warning message, as shown in Figure 5-6. Click Yes to save the private key without a passphrase.
Figure 5-6 Saving the private key without passphrase
100
Implementing the IBM System Storage SAN Volume Controller V4.3
e. When prompted, enter a name (for example, icat) and location for the private key (for example, C:\Support Utils\PuTTY). Click Save. If you choose another name or location, ensure that you keep a record of it, because the name and location of the SSH private key must be specified when the PuTTY session is configured in the steps documented in 5.6.4, “Configuring the PuTTY session for the CLI” on page 123. Note: The PuTTY Key Generator saves the private key with the PPK extension. 5. Close the PuTTY Key Generator GUI. 6. Using Windows Explorer on the SVC Console, navigate to the directory where the private key was saved (for example, C:\Support Utils\PuTTY). 7. Copy the private key file (for example, icat.ppk) to the C:\Program Files\IBM\svcconsole\cimom directory. Important: If the private key was named something other than icat.ppk, make sure that you rename it to icat.ppk in the C:\Program Files\IBM\svcconsole\cimom folder. The GUI (which will be used later) expects the file to be called icat.ppk and for it to be in this location.
Chapter 5. SVC Console
101
5.5 Basic installation This section provides step-by-step instructions for building the SVC cluster initially.
5.5.1 Creating the cluster (first time) using the service panel This section provides the step-by-step instructions needed to create the cluster for the first time using the service panel. Use Figure 5-7 as a reference for the SVC Node 8F2 and 8F4 model buttons to be pushed in the steps that follow, and Figure 5-8 for the SVC Node 8G4 model.
Figure 5-7 SVC 8F2 Node and SVC 8F4 Node front and operator panel
102
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 5-8 SVC 8G4 Node front and operator panel
Prerequisites Ensure that the SVC nodes are physically installed. Prior to configuring the cluster, ensure that the following information is available: License: The license indicates whether the customer is permitted to use FlashCopy, MetroMirror, or both. It also indicates how much capacity the customer is licensed to virtualize. For IPv4 addressing: – Cluster IPv4 addresses: These include one for the cluster and another for the service address – IPv4 Subnet mask – Gateway IPv4 Address Chapter 5. SVC Console
103
For IPv6 addressing: – Cluster IPv6 addresses: These include one for the cluster and another for the service address – IPv6 prefix – Gateway IPv6 address
Process After the hardware is physically installed into racks, complete the following steps to initially configure the cluster through the service panel: 1. Choose any node that is to become a member of the cluster being created. 2. At the service panel of that node, click and release the Up or Down navigation button continuously until Node: is displayed. Important: If a timeout occurs when entering the input for the fields during these steps, you must begin again from step 2. All the changes are lost, so be sure to have all the information on hand before beginning. 3. Click and release the Left or Right navigation button continuously until Create Cluster? is displayed. Click the Select button. 4. If IPv4 Address: is displayed on line 1 of the service display, go to step 5. If Delete Cluster? is displayed in line 1 of the service display, this node is already a member of a cluster. Either the wrong node was selected, or this node was already used in a previous cluster. The ID of this existing cluster is displayed in line 2 of the service display. a. If the wrong node was selected, this procedure can be exited by clicking the Left, Right, Up, or Down button (it cancels automatically after 60 seconds). b. If it is certain that the existing cluster is not required, follow these steps: i. Click and hold the Up button. ii. Click and release the Select button. Then release the Up button. This deletes the cluster information from the node. Go back to step 1 and start again. Important: When a cluster is deleted, all client data contained in that cluster is lost. 5. If you are creating the cluster with IPv4, then click the Select button, otherwise for IPv6 press the down arrow to display IPv6 Address: and click the Select button. 6. Use the Up or Down navigation button to change the value of the first field of the IP address to the value that has been chosen. Note: For IPv4, pressing and holding the Up or Down buttons will increment or decrease the IP address field by units of 10. The field value rotates from 0 to 255 with the Down button, and from 255 to 0 with the Up button. For IPv6, you do the same except that it is a 4 digit hexadecimal field and the individual characters will increment. 7. Use the Right navigation button to move to the next field. Use the Up or Down navigation buttons to change the value of this field. 8. Repeat step 7 for each of the remaining fields of the IP address.
104
Implementing the IBM System Storage SAN Volume Controller V4.3
9. When the last field of the IP address has been changed, click the Select button. 10.Click the Right button. a. For IPv4, IPv4 Subnet: is displayed. b. For IPv6, IPv6 Prefix: is displayed. 11.Click the Select button. 12.Change the fields for IPv4 Subnet in the same way that the IPv4 IP address fields were changed. There is only a single field for IPv6 Prefix. 13.When the last field of IPv4 Subnet/IPv6 Mask has been changed, click the Select button. 14.Click the Right navigation button. a. For IPv4, IPv4 Gateway: is displayed. b. For IPv6, IPv6 Gateway: is displayed. 15.Click the Select button. 16.Change the fields for the appropriate Gateway in the same way that the IPv4/IPv6 address fields were changed. 17.When changes to all Gateway fields have been made, click the Select button. 18.Click the Right navigation button. a. For IPv4, IPv4 Create Now? is displayed. b. For IPv6, IPv6 Create Now? is displayed. 19.When the settings have all been verified as accurate, click the Select navigation button. To review the settings before creating the cluster, use the Right and Left buttons. Make any necessary changes, return to Create Now?, and click the Select button. If the cluster is created successfully, Password: is displayed in line 1 of the service display panel. Line 2 contains a randomly generated password, which is used to complete the cluster configuration in the next section. Important: Make a note of this password now. It is case sensitive. The password is displayed only for approximately 60 seconds. If the password is not recorded, the cluster configuration procedure must be started again from the beginning. 20.When Cluster: is displayed in line 1 of the service display and the Password: display timed out, then the cluster was created successfully. Also, the cluster IP address is displayed on line 2 when the initial creation of the cluster is completed. If the cluster is not created, Create Failed: is displayed in line 1 of the service display. Line 2 contains an error code. Refer to the error codes that are documented in IBM System Storage SAN Volume Controller: Service Guide, GC26-7901, to find the reason why the cluster creation failed and what corrective action to take. Important: At this time, do not repeat this procedure to add other nodes to the cluster. Adding nodes to the cluster is accomplished in 6.1, “Adding nodes to the cluster” on page 158 and in 7.1, “Adding nodes to the cluster” on page 174.
Chapter 5. SVC Console
105
5.6 Completing the initial cluster setup using the SAN Volume Controller Console GUI After you have performed the activities in 5.5, “Basic installation” on page 102, you need to complete the cluster setup using the SAN Volume Controller Console. Our recommendation is that you follow 5.6.1, “Configuring the GUI” on page 106 to create the cluster. Note: Make sure that the SVC Cluster IP address (svcclusterip) can be reached successfully with a ping command from the SVC console.
5.6.1 Configuring the GUI If this is the first time that the SAN Volume Controller administration GUI is being used, you must configure it as explained here: 1. Open the GUI using one of the following methods: – Double-click the icon marked SAN Volume Controller Console on the SVC Console’s desktop. – Open a Web browser on the SVC Console and point to this address: http://localhost:9080/ica (We accessed the SVC Console using this method.) – Open a Web browser on a separate workstation and point to this address: http://svcconsoleipaddress:9080/ica 2. On the Signon page (Figure 5-9), type the user ID superuser and the default password of passw0rd. Click OK. Note: Passwords for the central administration GUI are separate from the passwords set for individual SVC clusters.
Figure 5-9 GUI Signon
3. The first time you sign on as the superuser, you will be prompted to and you must change the password for the superuser. The Change Password window is displayed, as shown in Figure 5-10. Enter the new password in the field labelled New Password and re-enter the same password in the field labelled Re-Enter New Password. Click OK.
106
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 5-10 Change the default password
Note: Like all passwords, this is case sensitive. 4. On the GUI Welcome window (Figure 5-11), click the Add SAN Volume Controller Cluster button in the center of the window. If you changed the GUI default password in step 3, this button might not be displayed. If so, click Clusters in the My Work window, then select Add Cluster from the drop-down menu.
Figure 5-11 Adding the SVC cluster for management Chapter 5. SVC Console
107
Important: If you followed the setup method https://svcclusterip/create in 5.6, “Completing the initial cluster setup using the SAN Volume Controller Console GUI” on page 106, do not select the Create (initialize) Cluster box in step 5. Doing so will invoke the initial cluster installation process. If it is selected, the cluster will be re-initialized and any configuration settings entered previously are lost. 5. On the Adding Clusters window (Figure 5-12), type the IP address of the SVC cluster and select Create (Initialize) Cluster, and then click OK.
Figure 5-12 Adding Clusters window
6. A Security Alert window will pop up, as shown in Figure 5-13. Click Yes to continue.
Figure 5-13 Security Alert
108
Implementing the IBM System Storage SAN Volume Controller V4.3
7. A pop-up window appears and prompts for the user ID and password of the SVC cluster, as shown in Figure 5-14. Enter the user ID admin and the cluster admin password that was set earlier in 5.5.1, “Creating the cluster (first time) using the service panel” on page 102 and click OK.
Figure 5-14 SVC cluster user ID and password sign-on window
8. The browser accesses the SVC and displays the Create New Cluster wizard window, as shown in Figure 5-15. Click Continue.
Figure 5-15 Create New Cluster wizard
Chapter 5. SVC Console
109
9. The Create New Cluster page (Figure 5-16) opens. Fill in the following details: – A new admin password to replace the random one that the cluster generated: The password is case sensitive and can consist of A to Z, a to z, 0 to 9, and the underscore. It cannot start with a number and has a minimum of one character, and a maximum of 15 characters. – A service password to access the cluster for service operation: The password is case sensitive and can consist of A to Z, a to z, 0 to 9, and the underscore. It cannot start with a number and has a minimum of one character and a maximum of 15 characters. – A cluster name: The cluster name is case sensitive and can consist of A to Z, a to z, 0 to 9, and the underscore. It cannot start with a number and has a minimum of one character and a maximum of 15 characters. – A service IP address to access the cluster for service operations. Choose between an automatically assigned IP address from DHCP or a static IP address. Note: The service IP address is different from the cluster IP address. However, because the service IP address is configured for the cluster, it must be on the same IP subnet. – The fabric speed of the Fibre Channel network for the SVC model types 4F2 or 8F2 nodes in a cluster must run one speed, either 1 Gbps or 2 Gbps. Operation with 4F2 or 8F2 nodes with different speeds running on the node to switch connections in a single cluster is not possible and also not configurable. – The SVC model types 8F4 or 8G4 will autonegotiate their speed independently of another, which can run at 1 Gbps, 2 Gbps, or 4 Gbps. – The Administrator Password Policy check box, if selected, enables a user to reset the password from the service panel (this is helpful, for example, if the password is forgotten). This check box is optional. Note: The SVC should be in a secure room if this function is enabled, because anyone who knows the correct key sequence can reset the admin password. The key sequence is as follows: a. From the Cluster: menu item displayed on the service panel, click the Left or Right button until Recover Cluster? is displayed. b. Click the Select button. Service Access? should be displayed. c. Click and hold the Up button and then click and release the Select button. This generates a new random password. Write it down. Important: Be careful, because clicking and holding the Down button, and clicking and releasing the Select button, places the node in service mode.
110
Implementing the IBM System Storage SAN Volume Controller V4.3
Click the Create New Cluster button (see Figure 5-16).
Figure 5-16 Cluster details
Important: Make sure you confirm and retain the Administrator and Service password for future use.
Chapter 5. SVC Console
111
10.A number of progress windows appear, as shown in Figure 5-17. Click Continue each time when prompted.
Figure 5-17 Maintaining SSH Keys Progress window
11.A new window with the confirmation that the password has been modified is displayed in Figure 5-18. To set up the Error Notification Settings, click Continue.
Figure 5-18 Password Change Confirmation window
Note: By this time, the service panel display on the front of the configured node should display the cluster name entered previously (for example, ITSO-CLS2). 12.The Error Notification Settings window is shown in Figure 5-19. This setting will be covered in more detail in 9.10.3, “Setting up error notification” on page 384. For now, click Update Settings and then go to the next window, and click Continue when prompted.
112
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 5-19 Error Notification Settings configuration window
13.The Error Notification confirmation window is displayed in Figure 5-20. To set up Licensing, click Continue.
Figure 5-20 Error Notification setup confirmation
Chapter 5. SVC Console
113
14.The Featurization Settings window (Figure 5-21) is displayed. To continue, at a minimum the Virtualization Limit (Gigabytes) field must be filled out. If you are licensed for FlashCopy and MetroMirror (the window reflects Remote Copy in this example), the Enabled radio buttons can also be selected here. Click the Set Features button. Click Continue when prompted.
Figure 5-21 Featurization Settings Configuration window
15.A confirmation window that state that the featurization settings have been set is shown in Figure 5-22. Click Continue to upload an SSH Key to the cluster.
Figure 5-22 Featurization Settings Confirmation window
114
Implementing the IBM System Storage SAN Volume Controller V4.3
16.When the changes are accepted, the cluster displays the Enter Network Password window again. Type the User Name admin and the new admin password you created under step 9 on page 110. 17.Log back in. Note: The SVC uses the standard of 1 GB = 1024 MB. Therefore, typing 10 GB in the Featurization Settings window provides you with 10240 MB rather than 10000 MB as with other disk subsystems. This window uses the previous term “Remote Copy” to refer to Metro Mirror.
5.6.2 Uploading the SSH public key to the SVC cluster After updating the featurization settings, the Add SSH Public Key page (Figure 5-23) opens. 1. Browse or type the fully qualified directory path and file name of the public key created and saved in 5.4.1, “Generating public and private SSH key pairs using PuTTY” on page 98. Then type the name of the user ID to be associated with this admin key pair (for example, admin) and click Add Key.
Chapter 5. SVC Console
115
Figure 5-23 Add SSH public key
2. On the next window (Figure 5-24), a message is displayed indicating that a new SSH administrator key associated with the ID admin was added. Click Continue.
116
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 5-24 Adding the SSH admin key successfully
3. The basic setup requirements for the SVC cluster using the SVC cluster Web interface have now been completed. Close the window shown in Figure 5-25.
Figure 5-25 Closing the window after successful cluster creation
Chapter 5. SVC Console
117
4. The next step is to complete the installation and configuration of the SVC cluster using either the CLI or CIM Agent and Console for SVC GUI. a. The Viewing Clusters window (Figure 5-26) is displayed. i. If it does not display automatically, click Clusters in the My Work window menu. ii. Click the Select box for the SVC cluster, highlight the option Launch the SAN Volume Controller Application from the drop-down menu to select it, and click Go.
Figure 5-26 Cluster selection window
Note: If the message “Invalid SSH Fingerprint” is shown in the Availability Status, to correct this situation, use the drop-down list options and choose Reset fingerprints, and click Go. Click OK when prompted. The status should change to “OK”. If this message persists, then you might have an SSH key problem. To correct the SSH key, follow the steps in 5.4.1, “Generating public and private SSH key pairs using PuTTY” on page 98, paying particular attention to the Important notes in that section. 5. If you need to maintain your SSH keys to add more keys, the Maintaining SSH Keys window is displayed, as shown in Figure 5-27. a. Browse to find the public key or paste the public key string in the Public Key (Direct Input) box. b. In the Access Level box: i. Select the user you want to assign to your key, Administrator or Service. The SVC Console Level is for creating a key pair between the SVC Console and the Cluster. ii. If you select Administrator level, you must select the role Administrator or Monitor. The role based content will be covered in 10.1, “Managing users” on page 402. c. Enter a key ID in the ID field. d. Click the Add Key button in the lower left of the window.
118
Implementing the IBM System Storage SAN Volume Controller V4.3
Note: Using the same ID for both access levels helps identify them both as coming from the same SSH client for potential later maintenance of SSH key pairs. Any descriptive string will suffice; the ID does not have to be admin or have to match the ID used earlier for administrative access level.
Chapter 5. SVC Console
119
Figure 5-27 Maintaining SSH Keys window
120
Implementing the IBM System Storage SAN Volume Controller V4.3
e. An Added message should be displayed. 6. Click the X in the upper right corner of the Maintaining SSH Keys window, as shown in Figure 5-27, and you will see the cluster selection window, as shown in Figure 5-28.
Figure 5-28 Using the Viewing Clusters window to Launch the SAN Volume Controller Application
7. You have now completed the tasks required to configure the GUI for SVC administration. a. Either close the browser session completely or leave it open on the Welcome window and continue to Chapter 6, “Quickstart configuration using the command-line interface” on page 157 or Chapter 7, “Quickstart configuration using the GUI” on page 173 to add the second node to the cluster. b. If SSH access from other workstations is desired, proceed to the next section(s). c. To continue with the SVC configuration, select your cluster with the check box and click Go.
5.6.3 Uploading SSH public key(s) sample scenarios For each SVC cluster to be managed by the SVC Console, the SSH public key must be uploaded from the SVC Console. A public key must also be uploaded from every other system that requires access to each new SVC cluster. Perform this task using a Web browser. This same information is included in IBM System Storage SAN Volume Controller: Software Installation and Configuration Guide, SC23-6628. Important: If the SSH public key from a specified server is not stored onto a particular SVC cluster, the SVC access software cannot connect to that particular cluster from that specified server. Here is a summary of the main steps: 1. Start the browser to access the SVC Console. 2. Log onto the SVC Console using the superuser account and password. 3. Identify the SVC cluster to the SVC Console. 4. Store the SSH public key on the SVC cluster.
Chapter 5. SVC Console
121
5. Launch the secondary browser window to manage the selected cluster. The detailed procedure follows: 1. Start a browser and log onto the server on which the SVC Console is installed by pointing to the uniform resource locator (URL): http://<svcconsoleipaddress>:9080/ica 2. Log onto the SAN Volume Controller Console using the superuser account and password. 3. Identify the SVC clusters to the SVC Console. The steps required depend on the current status of the cluster to be configured: – SVC cluster that has not yet been initialized: If an SVC cluster has not yet been created using the front panel of the SVC cluster, that phase of the cluster creation will need to be performed first. See 5.5.1, “Creating the cluster (first time) using the service panel” on page 102. A special password is displayed on the SVC front (service) panel for 60 seconds to be used in later steps to initialize the SVC Console. After completing the first phase to create the SVC cluster using the front panel of an SVC node, the next step required is to complete the creation of the cluster by using the SVC Console native Web interface, as described in 5.6, “Completing the initial cluster setup using the SAN Volume Controller Console GUI” on page 106. – Previously initialized SVC cluster: If the SVC cluster has completed the initialization (creation) process but is not yet registered with the SVC Console, log on with the superuser ID and password, and select Add Cluster from the list in the SVC Welcome page to add the cluster. Enter the IP address of the cluster to be added but do not select the Create (Initialize) Cluster check box, which is above the OK button. When you click the OK button, the system displays the page to provide the SSH public key for upload to the cluster. Step 4 continues with the SSH key input description. As part of this process, the program prompts you to enter the network password. Type the admin user name and the password that is configured for the cluster. 4. Store the SSH public key on the SAN Volume Controller cluster. Each key is associated with an ID string that is user-defined and can consist of up to 30 characters. Up to 100 keys can be stored on a cluster. Keys can be added to provide either administrator access or service access. 5. Launch the secondary browser window to manage the new cluster. Select the specific cluster to be managed and then launch the browser window specifically for that cluster. a. Under the My Work section of the browser window, click Clusters. A new view is displayed in the work area (main frame). b. In the Select column, select the check box to the left of the cluster to be managed. Select Launch the SAN Volume Controller Application from the drop-down menu in the work area and click Go. A secondary browser window opens to the SVC application to work with the specific SVC cluster that was selected. Notice the ClusterName parameter in the browser location URL, which identifies the IP address of the cluster currently being managed, as shown here: http://9.43.86.115:9080/svc/Console?Console.loginToken=-48368b3b:1126943cab9 :-7fb4&Console.ClusterName=9.43.86.116 There is an issue with Windows registration of an SSH key when a cluster is deleted and then recreated with the same IP address. When a cluster definition is added to the ICAT for management, SVC will send a host key to the SVC Console. This host key is saved in the Windows registry. If a cluster is deleted and another cluster is created with the same IP address, SVC will again send a host key to the SVC Console.
122
Implementing the IBM System Storage SAN Volume Controller V4.3
Since a key for this IP address is already saved, the Windows registry is not updated with the new key and the cluster cannot be managed from the ICAT. This is for security reasons, because the console erroneously detects that another device is attempting to access it. The workaround is to delete the host key from the registry after deleting the cluster and before the new cluster is recreated. There is a function (Reset SSH Fingerprint) provided in the drop-down list to correct this situation. This is not an issue with the command line SSH, since you are prompted to overwrite the host key. To establish a SSH connection to the cluster, the public key that was sent by SVC is stored in the following path at the SVC Console: \HKEY_USERS\.DEFAULT\Software\SimonTatham\PuTTY\SshHostKeys The name of the registry key is rsa2@22:cluster_IP_address. The reset function fixes the registry to use the correct public SSH key sent from SVC.
5.6.4 Configuring the PuTTY session for the CLI Before the CLI can be used, the PuTTY session must be configured using the SSH keys generated earlier in 5.4.1, “Generating public and private SSH key pairs using PuTTY” on page 98. Perform these steps to configure the PuTTY session on the SSH client system: 1. From the SSPC Windows desktop, select Start → Programs → PuTTY → PuTTY to open the PuTTY Configuration GUI window. 2. In the PuTTY Configuration window (Figure 5-29), from the Category pane on the left, click Session, if it is not selected. Note: The items selected in the Category pane affect the content that appears in the right pane.
Chapter 5. SVC Console
123
Figure 5-29 PuTTY Configuration window
3. In the right pane, under the “Specify the destination you want to connect to” section, select the SSH radio button. Under the “Close window on exit” section, select the Only on clean exit radio button. This ensures that if there are any connection errors, they will be displayed on the user’s screen.
124
Implementing the IBM System Storage SAN Volume Controller V4.3
4. From the Category pane on the left side of the PuTTY Configuration window, click Connection → SSH to display the PuTTY SSH Configuration window, as shown in Figure 5-30.
Figure 5-30 PuTTY SSH Connection Configuration window
5. In the right pane, in the section “Preferred SSH protocol version”, select radio button 2. 6. From the Category pane on the left side of the PuTTY Configuration window, select Connection → SSH → Auth. 7. In the right pane, in the “Private key file for authentication:” field under the Authentication Parameters section, either browse to or type the fully qualified directory path and file name
Chapter 5. SVC Console
125
of the SSH client private key file created earlier (for example, C:\Support Utils\PuTTY\icat.PPK). See Figure 5-31.
Figure 5-31 PuTTY Configuration: Private key location
8. From the Category pane on the left side of the PuTTY Configuration window, click Session.
126
Implementing the IBM System Storage SAN Volume Controller V4.3
9. In the right pane, follow these steps, as shown in Figure 5-32: a. Under the “Load, save, or delete a stored session” section, select Default Settings and click Save. b. For the Host Name (or IP address), type the IP address of the SVC cluster. c. In the Saved Sessions field, type a name (for example, SVC) to associate with this session. d. Click Save.
Figure 5-32 PuTTY Configuration: Saving a session
The PuTTY Configuration window can now either be closed or left open to continue. Tip: Normally, output that comes from the SVC is wider than the default PuTTY window size. We recommend that you change your PuTTY window appearance to use a font with a character size of 8. To do this, click the Appearance item in the Category tree, as shown in Figure 5-32, and then click Font. Choose a font with character size of 8.
5.6.5 Starting the PuTTY CLI session The PuTTY application is required for all CLI tasks. If it was closed for any reason, restart the session as detailed here: 1. From the SVC Console desktop, open the PuTTY application by selecting Start → Programs → PuTTY. 2. On the PuTTY Configuration window (Figure 5-33), select the session saved earlier (in our example, ITSO-SVC1) and click Load. 3. Click Open.
Chapter 5. SVC Console
127
Figure 5-33 Open PuTTY command-line session
4. If this is the first time the PuTTY application is being used since generating and uploading the SSH key pair, a PuTTY Security Alert window with a prompt pops up stating that there is a mismatch between the private and public keys, as shown in Figure 5-34. Click Yes, which invokes the CLI.
Figure 5-34 PuTTY Security Alert
128
Implementing the IBM System Storage SAN Volume Controller V4.3
5. At the Login as: prompt, type admin and press Enter (the user ID is case sensitive). As shown in Example 5-1, the private key used in this PuTTY session is now authenticated against the public key uploaded to the SVC cluster. Example 5-1 Authenticating login as: admin Authenticating with public key "rsa-key-20080617" Last login: Wed Jun 18 03:30:21 2008 from 9.43.86.111 IBM_2145:ITSO-CL2:admin>
You have now completed the tasks required to configure the CLI for SVC administration from the SVC Console. You can close the PuTTY session. Continue with the next section to configure the GUI on the SVC Console. Note: Starting with SVC Version 3.1, the CLI prompt has been changed to include the cluster name in the prompt.
Configuring SSH for non-SVC Console Windows clients The SVC cluster IP address must be able to be successfully reached using the ping command from the Windows workstation from which cluster access is desired. The software putty.exe and puttygen.exe can be downloaded from the following site: http://www.chiark.greenend.org.uk/~sgtatham/putty/ PuTTY can also be found on the SAN Volume Controller CD-ROM that was shipped with the SVC nodes. Generate and store the key pair as in the examples above. To upload the public key onto the SAN Volume Controller, follow these steps: 1. Browse to the SAN Volume Controller Console at: http://<SVCConsole_IP_Address>:9080/ica 2. Log in using the superuser account. 3. Click Clusters in the My Work pane on the left. 4. Click the Select box to the left of the cluster to which access is desired. 5. From the drop-down menu, select Maintain SSH Keys and click Go. 6. Type a descriptive ID for the workstation in the ID field. 7. Select Administrator or Service for the level of access. 8. Click Browse and locate the SSH public key on the workstation. 9. Click the Add key button.
Configuring SSH for AIX clients To configure SSH for AIX clients, follow these steps: 1. The SVC cluster IP address must be able to be successfully reached using the ping command from the AIX workstation from which cluster access is desired. 2. Open SSL must be installed for OpenSSH to work.
Chapter 5. SVC Console
129
3. Install OpenSSH on the AIX client: a. Installation images can be found at: https://www14.software.ibm.com/webapp/iwm/web/preLogin.do?source=aixbp http://sourceforge.net/projects/openssh-aix b. Follow the instructions carefully, as OpenSSL must be installed before using SSH. 4. Generate an SSH key pair: a. Run cd to go to the /.ssh directory. b. Run the command ssh-keygen -t rsa. c. The following message is displayed: Generating public/private rsa key pair. Enter file in which to save the key (//.ssh/id_rsa) d. Pressing Enter will use the default in parentheses above; otherwise, enter a file name (for example, aixkey) and press Enter. e. The following prompt is displayed: Enter a passphrase (empty for no passphrase) We recommend entering a passphrase when the CLI will be used interactively, as there is no other authentication when connecting through the CLI. After typing in the passphrase, press Enter. f. The following prompt is displayed: Enter same passphrase again: Type in the passphrase again and then press Enter again. g. A message is displayed indicating that the key pair has been created. The private key file will have the name entered above (for example, aixkey). The public key file will have the name entered above with an extension of .pub (for example, aixkey.pub). Note: If you are generating an SSH keypair so you can interactively use the CLI, we recommend that you use a passphrase so you will need to authenticate every time you connect to the cluster. It is possible to have a passphrase protected key for scripted usage, but you will have to use something like the expect command to have the passphrase parsed into the ssh command. 5. Upload the public key onto the SVC by browsing to the SVC Console at http://<SVCConsole_IP_Address>:9080/ica. 6. Log in under the superuser account. 7. Click Clusters in the My Work pane on the left. 8. Click the Select box to the left of the cluster to which access is desired. 9. From the drop-down menu, select Maintain SSH Keys and click Go. 10.Type a descriptive ID for the workstation in the ID field. 11.Select Administrator or Service for the user and role, or Monitor or Administrator if you selected Administrator. 12.Click Browse and locate the SSH public key on the AIX workstation. 13.Click the Add key button. To SSH from the AIX client to the SVC, type ssh admin@ on the AIX client type.
130
Implementing the IBM System Storage SAN Volume Controller V4.3
The private key to be used can be specified by typing: ssh -i admin@<svcclusterip>
5.7 Using IPv6 SVC V4.3 introduces IPv6 functionality to the console and clusters. You can use IPv4, or IPv6 in a dual stack configuration. Migrating to (or from) IPv6 can be done remotely and is nondisruptive, except you need to remove and re-define the cluster to the SVC Console. Note: To remotely access the SVC Console and clusters running IPv6, you are required to run Internet Explorer 7 and have IPv6 configured on your local workstation.
5.7.1 Migrating a cluster from IPv4 to IPv6 As a prerequisite, you should have IPv6 already enabled and configured on the SSPC/Windows server running SVC Console. We have configured an interface with IPv4 and IPv6 addresses on the SSPC, as shown in Example 5-2. Example 5-2 Output of ipconfig on SSPC C:\Documents and Settings\Administrator>ipconfig Windows IP Configuration
Ethernet adapter IPv6: Connection-specific IP Address. . . . . Subnet Mask . . . . IP Address. . . . . IP Address. . . . . Default Gateway . .
DNS . . . . . . . . . .
Suffix . . . . . . . . . . . . . . . . . . . .
. . . . . .
: : : : : :
10.0.1.115 255.255.255.0 2001:610::115 fe80::214:5eff:fecd:9352%5
Chapter 5. SVC Console
131
1. Select Manage Cluster → Modify IP Address, as shown in Figure 5-35.
Figure 5-35 Modify IP Addresses window
132
Implementing the IBM System Storage SAN Volume Controller V4.3
2. In the IPv6 section (Figure 5-36): a. Type an IPv6 prefix in the IPv6 Network Prefix field. The Prefix field can have a value of 0 to 127. b. Type an IPv6 address in the Cluster IP field. c. Type an IPv6 address in the Service IP address field. d. Type an IPv6 gateway in the Gateway field. e. Click the Modify Settings button.
Chapter 5. SVC Console
133
Figure 5-36 Modify IP Addresses - Adding IPv6 addresses
134
Implementing the IBM System Storage SAN Volume Controller V4.3
3. A confirmation window displays (Figure 5-37). You can click the X, on the top right hand corner, to close this tab.
Figure 5-37 Modify IP Addresses window
4. Before you remove the cluster from the SVC Console, you should test IPv6 connectivity using the ping command from a cmd.exe session on the SSPC (as shown in Example 5-3). Example 5-3 Testing IPv6 connectivity to SVC Cluster C:\Documents and Settings\Administrator>ping 2001:0610:0000:0000:0000:0000:0000:119 Pinging 2001:610::119 from 2001:610::115 with 32 bytes of data: Reply Reply Reply Reply
from from from from
2001:610::119: 2001:610::119: 2001:610::119: 2001:610::119:
time=3ms time<1ms time<1ms time<1ms
Ping statistics for 2001:610::119: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 0ms, Maximum = 3ms, Average = 0ms
5. In the Viewing Clusters pane, in the GUI Welcome window, select the radio button next to the cluster you want to remove. Select Remove a Cluster from the drop-down menu and click Go. (Figure 5-38).
Chapter 5. SVC Console
135
Figure 5-38 Removing a cluster
6. A confirmation box will appear (Figure 5-39). Click Yes to proceed.
Figure 5-39 Confirmation to remove cluster
136
Implementing the IBM System Storage SAN Volume Controller V4.3
7. The Viewing Clusters window will re-appear, without the cluster you have removed. Select Add a Cluster from the drop-down menu and click OK (Figure 5-40).
Figure 5-40 Adding a cluster
A pop-up window appears and prompts you for the user ID and password of the SVC cluster, as shown in Figure 5-41. Enter the user ID admin and the cluster admin password and click OK.
Figure 5-41 SVC Cluster user ID and password sign-on window
8. The maintaining SSH Keys window will display (Figure 5-42). As you have already registered the SSH key, you can click Skip SSH Keys.
Chapter 5. SVC Console
137
Figure 5-42 Maintaining SSH Keys window - Skip SSH Keys
9. The Viewing Clusters window will re-appear with the cluster displaying an IPv6 address, as shown in Figure 5-43. Launch the SAN Volume Controller Console for the cluster and go back into Modify IP Address, as you have done for step 1.
Figure 5-43 Viewing Clusters window - Displaying new cluster using IPv6 address
138
Implementing the IBM System Storage SAN Volume Controller V4.3
10.In the Modify IP addresses window, as shown in Figure 5-44, type in the IPv6 address of the SSPC/SVC Console in the IP address / IPv6 field in the SAN Volume Controller Console section.
Figure 5-44 Modify IP Addresses window: adding IPv6 SVC Console
Chapter 5. SVC Console
139
A confirmation window will appear. Close the window by clicking the X in the top right hand corner on the window and re-open the Modify IP Addresses tab, as shown in Figure 5-45.
Figure 5-45 Confirmation of IP Address change window
Click the Delete All IPv4 Settings button, in the IPv4 section of the Modify IP Addresses window, as shown in Figure 5-46, to remove the IPv4 addresses from the Cluster.
Figure 5-46 Delete all IPv4 Settings
11.A confirmation window will display, as shown in Figure 5-47. Click Confirm.
Figure 5-47 Confirm removing all IPv4 settings window
12.A second window (Figure 5-48) will display, confirming the IPv4 stack has been disabled and the associated addresses have been removed. Click Return.
Figure 5-48 IPv4 stack has been removed
140
Implementing the IBM System Storage SAN Volume Controller V4.3
5.7.2 Migrating a cluster from IPv6 to IPv4 The process of migrating a cluster from IPv6 to IPv4 is identical to the process described in 5.7.1, “Migrating a cluster from IPv4 to IPv6” on page 131, except you add IPv4 addresses and remove the IPv6 addresses.
5.8 Upgrading the SVC Console software This section takes you through the steps to upgrade your existing SVC Console GUI. You can also use these steps to install a new SVC Console on another server. It is assumed that you have already received your SVC SMIS Agent V4.3 installation CD-ROM or zip file. Proceed as follows: 1. When you insert the CD-ROM, the autorun should start and present you with the installation wizard shown in Figure 5-49. If you have the zip file, extract it and double-click the Launchpad.bat file in the W2K folder.
Figure 5-49 Inserting the Upgrade CD-ROM
2. Click Installation wizard to start the setup wizard. This first window (as shown in Figure 5-50) asks you to: – Shut down any running Windows programs. – Stop all SVC services – Review the README file on the installation CD/folder. Once you are ready, click Next.
Chapter 5. SVC Console
141
Figure 5-50 Launching the upgrade wizard
3. The installation should detect your existing SVC Console installation (if you are upgrading). If it does, it will ask you to: – Select Preserve Configuration if you want to keep your old configuration. (You should make sure that this is checked.) – Manually shut down the SVC Console services, namely: • • •
142
IBM System Storage SAN Volume Controller Pegasus Server Service Location Protocol IBM WebSphere® Application Server V6 - SVC
Implementing the IBM System Storage SAN Volume Controller V4.3
There may be differences in the existing services, depending on the version you are upgrading from. Follow the instructions on the dialog wizard for which services to shut down, as shown in Figure 5-51.
Figure 5-51 Product Installation Check
Important: If you want to keep your SVC configuration, then make sure you check the Preserve Configuration check box. If you omit this, you will lose your entire SVC Console setup, and you will have to reconfigure your console as though it were a fresh install. 4. The installation wizard will then check that appropriate services are shut down and take you to the Installation Confirmation window shown in Figure 5-52. If the wizard detects any problems, it first shows you a page detailing the possible problems, giving you time to fix them before proceeding.
Chapter 5. SVC Console
143
Figure 5-52 Installation Confirmation
5. The progress of the installation is shown in Figure 5-53. For our environment, it took approximately 10 minutes to complete.
Figure 5-53 Installation Progress
144
Implementing the IBM System Storage SAN Volume Controller V4.3
6. On successful completion, the wizard will either restart all the appropriate SVC Console processes or inform you that you will need to reboot, and then give you a summary of the installation. In this case, we were told we will need a reboot, as shown in Figure 5-54.
Figure 5-54 Installation summary
7. The wizard gives us the option to reboot on completion (Figure 5-55).
Figure 5-55 Installation finished: requesting reboot
8. And finally, to see what the new interface looks like, you can launch the SVC Console by using the icon on the desktop. Login and confirm that the upgrade was successful by
Chapter 5. SVC Console
145
noting the Console Version number on the right hand side of the window (under the graphic). See Figure 5-56.
Figure 5-56 Launching the upgraded SVC Console
You have now completed the upgrade of your SVC Console.
5.9 E-mail error notification The SVC can send all operational and error related data through e-mail to the IBM support center and to local accounts in your organization. The options are: Automated call raised Warning e-mails The e-mail address for Call Home for the United States and Canada is: mailto:[email protected] For all other countries, it is: mailto:[email protected]
146
Implementing the IBM System Storage SAN Volume Controller V4.3
To configure e-mail features for the SVC Call Home feature, complete these steps: 1. From the Welcome menu (shown on Figure 5-57), click the Set Email Features link. You should see a new wizard asking you to set up the e-mail server. Click the Create Email Server button.
Figure 5-57 Email Error Notification wizard: Introduction
2. You now need to enter in all the relevant setup information: the SMTP server address and port, local email reply address, your contact name, machine location, and phone contact details. After it is filled in, click the Create Email Server button, as shown in Figure 5-58.
Figure 5-58 Email Error Notification wizard: Create Email Server
Chapter 5. SVC Console
147
3. You will now see a confirmation that the e-mail settings have been updated. Click the Continue button (Figure 5-59).
Figure 5-59 Email Error Notification wizard: Settings committed
4. You now need to select the relevant Support Center option (Figure 5-60). Select one of the supplied call home addresses to use, depending on your geography. You may add your own custom support contact, if you have special requirements. Click Set Support Center to continue or click Start Without adding Support Center to skip this step.
Figure 5-60 Email Error Notification wizard: Select support address
148
Implementing the IBM System Storage SAN Volume Controller V4.3
5. The next window (Figure 5-61) requests confirmation of the Support Center/contact you chose. Click Confirm to continue.
Figure 5-61 Email Error Notification wizard: Confirm support address
6. The next window summarizes what information and to whom the support information is going to be sent to, and requests that you start the e-mail service. Click Start Email Service. (Figure 5-62).
Figure 5-62 Email Error Notification wizard: Summary and start e-mail service
7. The next window gives you the option of testing your e-mail service. Press Send Test Email to progress or Return to exit the wizard (Figure 5-63).
Chapter 5. SVC Console
149
Figure 5-63 Email Error Notification wizard: Send test e-mail - initial screen
8. This next window (Figure 5-64) shows all the enabled e-mail users for e-mail notification. You should only have a single entry, as you have only set up one. Press Test All Users to select this user.
Figure 5-64 Email Error Notification wizard: Send test e-mail - test all users.
150
Implementing the IBM System Storage SAN Volume Controller V4.3
9. Now you need to confirm that you wish to send e-mail. Click Confirm to send the e-mail (Figure 5-65).
Figure 5-65 Email Error Notification wizard: Send test e-mail Confirm to send email.
10.On successful completion, the wizard will confirm that the e-mail has been sent. Click Return to exit the wizard (Figure 5-66).
Figure 5-66 Email Error Notification wizard: Completion message
11.You will now be taken to the Manage Email Users window. In this window, you can manage support and local e-mail recipients for notifications and alerts. Add a user by clicking the Add User button (Figure 5-67).
Chapter 5. SVC Console
151
Figure 5-67 Manage Email Users: Add an e-mail user
12.Since we have already set up a support e-mail user, we can add a local e-mail user. Click the Add Local User button (Figure 5-68).
Figure 5-68 Manage Email Users: Add local e-mail user
152
Implementing the IBM System Storage SAN Volume Controller V4.3
13.We now add the e-mail user details (Figure 5-69): – – – –
An e-mail address. A user name. The user name must not have any spaces. What error types the e-mail user will be sent. Inventory Information. This will provide current SVC inventory information for better problem resolution.
Press the Add User button.
Figure 5-69 Manage Email Users: local user details and level of reporting
14.On successful completion, you will see the window shown in Figure 5-70. Click Continue to return to the main e-mail management window.
Figure 5-70 Manage Email Users: completion of adding a local user
Chapter 5. SVC Console
153
15.Back on the Manage Email window in Figure 5-71, you will see the local user in a different section. You can continue adding users, testing e-mail, or click Return to go to the Email Error Notification window.
Figure 5-71 Manage Email Users: Main window with local user added
154
Implementing the IBM System Storage SAN Volume Controller V4.3
The Email Error Notification window (Figure 5-72) will be displayed when you select the Set Email Features link from the My Work toolbar. You can go back into the e-mail user management, manage e-mail settings, manage regular inventory e-mails to users, or preview the e-mail messages.
Figure 5-72 Set Email features screen
The SVC can now be managed through the GUI or the CLI.
Chapter 5. SVC Console
155
156
Implementing the IBM System Storage SAN Volume Controller V4.3
6
Chapter 6.
Quickstart configuration using the command-line interface In this chapter, we describe the basic configuration procedures required to get the IBM System Storage SAN Volume Controller (SVC) environment up and running as quickly as possible using the command-line interface (CLI). Important: The CLI is case sensitive.
© Copyright IBM Corp. 2003-2008. All rights reserved.
157
6.1 Adding nodes to the cluster After cluster creation is completed through the service panel (the front panel of one of the SVC nodes) and cluster Web interface, only one node (the configuration node) is set up. (Refer to 5.5.1, “Creating the cluster (first time) using the service panel” on page 102”). To have a fully functional SVC cluster, you must add a second node to the configuration. To add a node to a cluster, gather the necessary information, as explained in these steps: 1. We need some information from the existing node. Gather this information by using the svcinfo lsnode node1 command, as shown in Example 6-1. Note: The name of the nodes already in the cluster are shown on the service panel displays on each node. By default, the first node is called node1. We show how to change this in a later topic. Example 6-1 svcinfo lsnode command IBM_2145:ITSO-CLS1:admin>svcinfo lsnode node1 id 1 name node1 UPS_serial_number 1000739007 WWNN 50050768010037E5 status online IO_group_id 0 IO_group_name io_grp0 partner_node_id partner_node_name config_node yes UPS_unique_id 20400001C3240007 port_id 50050768014037E5 port_status active port_speed 4Gb port_id 50050768013037E5 port_status active port_speed 4Gb port_id 50050768011037E5 port_status active port_speed 4Gb port_id 50050768012037E5 port_status active port_speed 4Gb hardware 8G4
The most important information to look for here is the IO_group_name, because we will use it when adding our second node to the SVC cluster configuration. Note: You can see in Example 6-1 that no partner_node_id or partner_node_name exists for node1 as of yet. 2. To see what nodes are available for inclusion in the SVC cluster configuration, enter the command shown in Example 6-2 on page 159.
158
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 6-2 svcinfo lsnodecandidate command IBM_2145:ITSO-CLS1:admin>svcinfo lsnodecandidate id panel_name UPS_serial_number 50050768010027E2 108283 100066C108 50050768010037DC 104603 1000739004 5005076801001D1C 107662 100066C107
UPS_unique_id 20400001864C1008 20400001C3240004 20400001864C1007
hardware 8G4 8G4 8G4
Important: You must also ensure that the nodes within an I/O group are attached to different UPSs. You can determine the UPS to which a node is attached from the output of svcinfo lsnodecandidate (UPS_serial_number column). If this command returns no information and your second node is powered on and the zones are correctly defined, then pre-existing cluster configuration data can be stored in it. If you are sure this node is not part of another active SVC cluster, you can use the service panel to delete the existing cluster information. After this is complete, re-issue the svcinfo lsnodecandidate command and you should see it listed. For information about how to delete an existing cluster configuration using the service panel, see 5.5.1, “Creating the cluster (first time) using the service panel” on page 102. For information about storage area network (SAN) zoning, see Chapter 3, “Planning and configuration” on page 25. 3. Using a combination of the information obtained in the previous steps, add the second node to the SVC cluster configuration. Enter the svctask addnode command. The full syntax of the command is: >>- svctask -- -- addnode -- -----------------------------------> >--+- -panelname -- -- panel_name -+-- -------------------------> '- -wwnodename -- -- wwnn_arg --' >--+----------------------------+-- ----------------------------> '- -name -- -- new_name_arg -' >-- -iogrp -- --+- iogroup_name -+----------------------------->< '- iogroup_id ---'
Note the following explanation: – – – –
panelname: Name of the node as it appears on the panel wwnodename: Worldwide node name (WWNN) of the node name: Name to be allocated to the node iogrp: I/O group to which the node is added Note: -wwnodename and -panelname are mutually exclusive; only one is required to uniquely identify the node.
Example 6-3 shows how to add a node. Example 6-3 Add node to a cluster IBM_2145:ITSO-CLS1:admin>svctask addnode -panelname 104603 -iogrp 0 Node, id [2], successfully added
Chapter 6. Quickstart configuration using the command-line interface
159
In this example: – 104603 is the panel name found using the svcinfo lsnodecandidate command. – io_grp0 is the name of the I/O group to which node1 belonged and was found using the svcinfo lsnode node1 command. Note: Because we did not provide the -name parameter, the SVC automatically generates the name nodeX (where X is the ID sequence number assigned by the SVC internally). In our case, this is node2. If you want to provide a name, you can use A to Z, a to z, 0 to 9, and the underscore. The name can be between one and 15 characters in length. However, it cannot start with a number or the word node, because this prefix is reserved for SVC assignment only. 4. If we display the node information for node1 again, as shown in Example 6-4, node1 now has a partner_node_id of 2 and a partner_node_name of node2. Example 6-4 svcinfo lsnode command IBM_2145:ITSO-CLS1:admin>svcinfo lsnode node1 id 1 name node1 UPS_serial_number 1000739007 WWNN 50050768010037E5 status online IO_group_id 0 IO_group_name io_grp0 partner_node_id 2 partner_node_name node2 config_node yes UPS_unique_id 20400001C3240007 port_id 50050768014037E5 port_status active port_speed 4Gb port_id 50050768013037E5 port_status active port_speed 4Gb port_id 50050768011037E5 port_status active port_speed 4Gb port_id 50050768012037E5 port_status active port_speed 4Gb hardware 8G4
Note: If you have more than two nodes, then you will have to add these nodes to new I/O groups, since each I/O group consists of exactly two nodes. Follow the foregoing directions for adding a node, changing the iogrp parameter whenever the current I/O group has reached its two-node limit. You have now completed the cluster configuration and you have a fully redundant SVC environment.
160
Implementing the IBM System Storage SAN Volume Controller V4.3
6.2 Setting the cluster time zone and time Perform the following steps to set the cluster time zone and time: 1. Find out for which time zone your cluster is currently configured. Enter the svcinfo showtimezone command, as shown in Example 6-5. Example 6-5 svcinfo showtimezone IBM_2145:ITSO-CLS1:admin>svcinfo showtimezone id timezone 522 UTC
2. To find the time zone code that is associated with your time zone, enter the svcinfo lstimezones command, as shown in Example 6-6. A truncated list is provided for this example. If this setting is correct (for example, 522 UTC), you can go to Step 4. If not, continue with Step 3. Example 6-6 svcinfo lstimezones IBM_2145:ITSO-CLS1:admin>svcinfo lstimezones id timezone . . 507 Turkey 508 UCT 509 Universal 510 US/Alaska 511 US/Aleutian 512 US/Arizona 513 US/Central 514 US/Eastern 515 US/East-Indiana 516 US/Hawaii 517 US/Indiana-Starke 518 US/Michigan 519 US/Mountain 520 US/Pacific 521 US/Samoa 522 UTC . .
3. Now that you know which time zone code is correct for you (in our example, 520), set the time zone by issuing the svctask settimezone (Example 6-7) command. Example 6-7 svctask settimezone IBM_2145:ITSO-CLS1:admin>svctask settimezone -timezone 520
4. Set the cluster time by issuing the svctask setclustertime command (Example 6-8). Example 6-8 svctask setclustertime IBM_2145:ITSO-CLS1:admin>svctask setclustertime -time 061718402008
The format of the time is MMDDHHmmYYYY. You have now completed the tasks necessary to set the cluster time zone and time.
Chapter 6. Quickstart configuration using the command-line interface
161
6.3 Checking the license features The lslicense command displays license settings for cluster features, including FlashCopy, Remote Copy, and Virtualization settings. The displayed output lists feature enablement and capacities (Example 6-9). Example 6-9 svcinfo lslicense used_flash 4.73 used_remote 0 used_virtualization 21.12 license_flash 5 license_remote 0 license_virtualization 32
Use the chlicense command to change the feature license settings. Because the feature license settings are entered when the cluster is first created, you must only update the settings if you have changed your license. The full syntax of the command is: >>- svctask -- -- chlicense -- ---------------------------------> >--+- -flash capacity_TB ----------+-------------------------->< +- -remote capacity_TB ---------+ '- -virtualization capacity_TB -'
For further details about software licensing, see 2.6, “New with SVC V4.3” on page 19.
6.4 Creating host definitions Perform the following steps to create host definitions within the SVC: 1. To determine which hosts ports are eligible for definition, issue the svcinfo lshbaportcandidate command, as shown in Example 6-10. Example 6-10 svcinfo lshbaportcandidate command IBM_2145:ITSO-CLS1:admin>svcinfo lshbaportcandidate id 210000E08B18D48F 210000E08B89B8C0 210000E08B89C1CD 210000E08B892BCD 210000E08B18FF8A 210000E08B054CAA
This command shows all WWPNs that are visible to the SVC that were not already defined to a host. If your WWPN does not appear, verify that the host is logged into the switch and that zoning is updated to allow SVC and host ports to see each other, as explained in Chapter 3, “Planning and configuration” on page 25. If you are working with an AIX host and do not see your adapter listed in the output of svcinfo lshbaportcandidate, then rerun the cfgmgr command to encourage the host HBAs to communicate with the SVC.
162
Implementing the IBM System Storage SAN Volume Controller V4.3
Note: There are situations when the information presented can include host HBA ports that are no longer logged in or even part of the SAN fabric. For example, a host HBA port is unplugged from a switch, but svcinfo lshost still shows the WWPN logged in to all SVC nodes. The incorrect entry will be removed when another device is plugged into the same switch port that previously contained the removed host HBA port. 2. The output from this command shows that we have six QLogic ports (21xxx) and two Emulex ports (10xxx). By checking the hosts and confirming with the switch nameserver, you determine that the 10xxx WWPNs belong to the AIX host. Therefore, you have everything necessary to create a host definition. You can add WWPN port definitions to a host one at a time using the mkhost and addhostport commands, as shown in Example 6-11. Example 6-11 svctask mkhost commands IBM_2145:ITSO-CLS1:admin>svctask mkhost -name Palau -hbawwpn 210000E08B89C1CD:210000E08B054CAA -iogrp 0 Host, id [0], successfully created
The -name parameter is used to name the host (in our case, Palau) and the -hbawwpn parameter is filled in using data retrieved from the lshbaportcandidate command. Note: The -name is optional. If you do not specify a -name, the default is hostX, where X is the ID sequence number assigned by the SVC internally. The -hbawwpn is mandatory. If you do not specify the -hbawwpn parameter, this will cause the command to fail. The -iogrp is optional. If you do not specify an iogrp, the host is associated with all I/O groups. Because of the limitation of 256 hosts per I/O group, we recommend specifying an I/O group to each host. I/O groups are specified using their names or IDs, separated by a colon. Names and IDs can be mixed in the list.
Tip: Some HBA device drivers will not log in to the fabric until they can see target LUNs. As they do not log in, their WWPNs will not be known as candidate ports. You can specify the force flag (-force) with this command to stop the validation of the WWPN list. Check that the host definitions were correctly created using the svcinfo lshost command, as shown in Example 6-12. Example 6-12 svcinfo lshost commands IBM_2145:ITSO-CLS1:admin>svcinfo lshost id name port_count 0 Palau 2 1 Nile 2 IBM_2145:ITSO-CLS1:admin>svcinfo lshost Palau id 0 name Palau port_count 2 type generic mask 1111 iogrp_count 1
iogrp_count 1 1
Chapter 6. Quickstart configuration using the command-line interface
163
WWPN 210000E08B054CAA node_logged_in_count 2 state inactive WWPN 210000E08B89C1CD node_logged_in_count 2 state inactive
You have now completed the tasks required to add host definitions to your SVC configuration.
6.5 Displaying managed disks Perform the following steps to display managed disks (MDisks): 1. First, see which MDisks are available. Enter the svcinfo lsmdiskcandidate command, as shown in Example 6-13. This displays all detected MDisks that are not currently part of a managed disk group (MDG). Example 6-13 svcinfo lsmdiskcandidate command IBM_2145:ITSO-CLS1:admin>svcinfo lsmdiskcandidate id 0 1 2 . .
Alternatively, you can list all MDisks (managed or unmanaged) by issuing the svcinfo lsmdisk command, as shown in Example 6-14. Example 6-14 svcinfo lsmdisk command IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk -delim , id,name,status,mode,mdisk_grp_id,mdisk_grp_name,capacity,ctrl_LUN_#,controller_name,UID 0,mdisk0,online,unmanaged,,,36.0GB,0000000000000000,controller0,600a0b8000174431000000eb 47139cca00000000000000000000000000000000 1,mdisk1,online,unmanaged,,,36.0GB,0000000000000001,controller0,600a0b8000174431000000ef 47139e1c00000000000000000000000000000000 2,mdisk2,online,unmanaged,,,36.0GB,0000000000000002,controller0,600a0b8000174431000000f1 47139e7200000000000000000000000000000000 3,mdisk3,online,unmanaged,,,36.0GB,0000000000000003,controller0,600a0b8000174431000000e4 4713575400000000000000000000000000000000 4,mdisk4,online,unmanaged,,,36.0GB,0000000000000004,controller0,600a0b8000174431000000e6 4713576000000000000000000000000000000000 5,mdisk5,online,unmanaged,,,36.0GB,0000000000000000,controller1,600a0b800026b28200003ea3 4851577c00000000000000000000000000000000 6,mdisk6,online,unmanaged,,,36.0GB,0000000000000005,controller0,600a0b8000174431000000e7 47139cb600000000000000000000000000000000 7,mdisk7,online,unmanaged,,,36.0GB,0000000000000001,controller1,600a0b80002904de00004188 485157a400000000000000000000000000000000 8,mdisk8,online,unmanaged,,,36.0GB,0000000000000006,controller0,600a0b8000174431000000ea 47139cc400000000000000000000000000000000 . .
From this output, you can see additional information about each MDisk (such as current status). For the purpose of our current task, we are only interested in the unmanaged disks because they are candidates for MDGs (all MDisks in our case).
164
Implementing the IBM System Storage SAN Volume Controller V4.3
Tip: The -delim, parameter collapses output instead of wrapping text over multiple lines. 2. If not all the MDisks that you expected are visible, rescan the available Fibre Channel network by entering the svctask detectmdisk command, as in Example 6-15. Example 6-15 svctask detectmdisk IBM_2145:ITSO-CLS1:admin>svctask detectmdisk
3. If you run the svcinfo lsmdiskcandidate command again and your MDisk or MDisks are still not visible, check that the logical unit numbers (LUNs) from your subsystem have been properly assigned to the SVC and that appropriate zoning is in place (for example, the SVC can see the disk subsystem). See Chapter 3, “Planning and configuration” on page 25 for details about how to set up your SAN fabric. Note: If you have assigned a large number of LUNs to your SVC, the discovery process could take a while. Check several times using the svcinfo lsmdisk command to see if all the MDisks you were expecting are present. If not, take the appropriate correct action as suggested above.
6.6 Creating managed disk groups Perform the following steps to create managed disk groups (MDGs): 1. From the information obtained in the previous section, add MDisks to MDGs using one of the following methods: – Issue the svctask mkmdiskgrp command, where you can add multiple MDisks to the MDG at the same time, as shown in Example 6-16. Example 6-16 svctask mkmdiskgrp IBM_2145:ITSO-CLS1:admin>svctask mkmdiskgrp -name MDG_DS45 -ext 512 -mdisk 0:1 MDisk Group, id [0], successfully created
This command creates an MDG called MDG_DS45. The extent size used within this group is 512 MB, and two MDisks (0 and 1) are added to the group. Note: The -name and -mdisk parameters are optional. If you do not enter a -name, the default is MDiskgrpX, where X is the ID sequence number assigned by the SVC internally. If you do not enter the -mdisk parameter, an empty MDG is created. If you want to provide a name, you can use letters A to Z, a to z, numbers 0 to 9, and the underscore. It can be between one and 15 characters in length, but it cannot start with a number or the word mDiskgrp because this prefix is reserved for SVC assignment only.
Chapter 6. Quickstart configuration using the command-line interface
165
By running the svcinfo lsmdisk command again, you should now see the MDisks (mdisk0 and mdisk2) as “managed” and part of the MDG0, as shown in Example 6-17. Example 6-17 svcinfo lsmdisk command IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk -delim , id,name,status,mode,mdisk_grp_id,mdisk_grp_name,capacity,ctrl_LUN_#,controller_name, UID 0,mdisk0,online,managed,0,MDG_DS45,36.0GB,0000000000000000,controller0,600a0b8000174 431000000eb47139cca00000000000000000000000000000000 1,mdisk1,online,managed,0,MDG_DS45,36.0GB,0000000000000001,controller0,600a0b8000174 431000000ef47139e1c00000000000000000000000000000000
– If you want to add an MDisk to an existing MDG or want to add MDisks one at a time, use the mkmdiskgrp command to create the initial MDG and then use the addmdisk command, as shown in Example 6-18, to add other MDisks to it. Example 6-18 Add mdisk to existing mdiskgrp IBM_2145:ITSO-CLS1:admin>svctask mkmdiskgrp -name MDG_DS47 -ext 512 MDisk Group, id [1], successfully created IBM_2145:ITSO-CLS1:admin>svctask addmdisk -mdisk 5 MDG_DS47
The first command in this example creates an MDisk group called MDG_DS47. The extent size used within this group is 512 MB. No MDisk is added to the group. The second command adds a second MDisk (mdisk id 5) to the same MDG. By running the svcinfo lsmdisk command again, you now see the MDisks (mdisk5) as “managed” and part of the MDG_DS47 (see Example 6-19). Example 6-19 svcinfo lsmdisk IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk -delim , id,name,status,mode,mdisk_grp_id,mdisk_grp_name,capacity,ctrl_LUN_#,controller_name, UID 0,mdisk0,online,managed,0,MDG_DS45,36.0GB,0000000000000000,controller0,600a0b8000174 431000000eb47139cca00000000000000000000000000000000 1,mdisk1,online,managed,0,MDG_DS45,36.0GB,0000000000000001,controller0,600a0b8000174 431000000ef47139e1c00000000000000000000000000000000 2,mdisk2,online,managed,0,MDG_DS45,36.0GB,0000000000000002,controller0,600a0b8000174 431000000f147139e7200000000000000000000000000000000 3,mdisk3,online,managed,0,MDG_DS45,36.0GB,0000000000000003,controller0,600a0b8000174 431000000e44713575400000000000000000000000000000000 4,mdisk4,online,managed,0,MDG_DS45,36.0GB,0000000000000004,controller0,600a0b8000174 431000000e64713576000000000000000000000000000000000 5,mdisk5,online,managed,1,MDG_DS47,36.0GB,0000000000000000,controller1,600a0b800026b 28200003ea34851577c00000000000000000000000000000000
For information about other tasks, such as adding MDisks to MDGs, renaming MDGs, or deleting MDGs, see Chapter 9, “SVC configuration and administration using the CLI” on page 303. You have now completed the tasks required to create an MDG.
166
Implementing the IBM System Storage SAN Volume Controller V4.3
6.7 Creating a virtual disk The mkvdisk command creates sequential, striped, or image mode virtual disk objects. When they are mapped to a host object, these objects are seen as disk drives with which the host can perform I/O operations. When creating a virtual disk (VDisk), you must enter several parameters (some mandatory, some optional) at the CLI. The full command syntax is: >>- svctask -- -- mkvdisk -- -----------------------------------> >-- -mdiskgrp --+- mdisk_group_id_list ---+-- ------------------> '- mdisk_group_name_list -' >-- -iogrp --+- io_group_id ---+-- -- -size -- disk_size -- ----> '- io_group_name -' >--+------------+-- --+-------------------------+---------------> '- -fmtdisk -' '- -vtype ---- striped ---' >--+------------------------------------------------... '- -rsize -- disk_size | disk_size_percentage -- ... ...-------------------------------------------------------------------... ...| auto--+-------------------------------------------------------+--... '- -warning disk_size | disk_size_percentage% | off-' ...------------------------------------------------------+--> ...+---------------+--+-----------------------------------+-' '- -autoexpand -' '- -grainsize 32 | 64 | 128 | 256 -' >--+-------------+--+------------------------+------------------> '- -import-- -' '- -copies-- num_copies -' >--+--------------------------+--+--------------+-- ------------> '- -syncrate-- percentage -' '- -createsync-' >--+-------------------------+-- -------------------------------> '- -udid -- vdisk_udid -' >--+--------------------------+-- --+-------------------+-- ----> '- -node --+- node_name -+-' '- -unit --+- b --+-' '- node_id ---' +- kb -+ +- mb -+ +- gb -+ +- tb -+ '- pb -' >--+---------------------------------+-- -----------------------> '- -mdisk --+- mdisk_id_list ---+-' '- mdisk_name_list -' >--+-------------------------+-- -------------------------------> '- -name -- new_name_arg -' >--+------------------------------+---------------------------->< '- -cache -- readwrite | none -'
Chapter 6. Quickstart configuration using the command-line interface
167
The parameters are defined as follows: -mdiskgrp mdisk_group_id_list | mdisk_group_name_list (Required) Specifies one or more managed disk groups to use when you are creating this virtual disk. If you are creating multiple copies, you must specify one managed disk group per copy. The primary copy is allocated from the first managed disk group in the list. -iogrp io_group_id | io_group_name (Required) Specifies the I/O group (node pair) with which to associate this virtual disk. -udid vdisk_udid (Optional) Specifies the unit number (udid) for the disk. The udid is an identifier that is required to support OpenVMS hosts; no other systems use this parameter. Valid options are a decimal number 0 - 32 767, or a hexadecimal number 0 - 0x7FFF. A hexadecimal number must be preceded by 0x (for example, 0x1234). -size disk_size (Required for sequential [seq] or striped VDisk creation) (Optional for image VDisk creation) Specifies the capacity of the virtual disk, which is used with the value of the unit. All capacities, including changes, must be in multiples of 512 bytes. An error occurs if you specify a capacity that is not a multiple of 512, which can only happen when byte units (-b) are used. However, an entire extent is reserved even if it is only partially used. The default capacity is in MB. You can specify a capacity of 0. Specify the size in bytes in multiples of logical block address (LBA) sizes. Note: If you do not specify the -size parameter when you create an image mode disk, the entire MDisk capacity is used. -rsize disk_size | disk_size_percentage% | auto (Optional) Makes the VDisk space-efficient; otherwise, the VDisk is fully allocated. Specify the disk_size | disk_size_percentage value using an integer, or an integer immediately followed by the percent character (%). Specify the units for a disk_size integer using the -unit parameter; the default is MB. The -rsize value can be greater than, equal to, or less than the size of the VDisk. The auto option creates a VDisk copy that uses the entire size of the MDisk; if you specify the -rsize auto option, you must also specify the -vtype image option. -warning disk_size | disk_size_percentage% (Optional) Requires that the -rsize parameter also be specified. Specifies a threshold at which a warning error log is generated for VDisk copies. A warning is generated when the used disk capacity on the space-efficient copy first exceeds the specified threshold. You can specify a disk_size integer, which defaults to MBs unless the -unit parameter is specified, or you can specify a disk_size%, which is a percentage of the virtual disk size. If -autoexpand is enabled, the default value for -warning is 80% of the virtual disk capacity. If -autoexpand is not enabled, the default value for warning is 80% of the real capacity. To disable warnings, specify 0 or 0%. -autoexpand (Optional) Specifies that space-efficient copies automatically expand their real capacities by allocating new extents from their managed disk group. Requires that the -rsize parameter also be specified. If the -autoexpand parameter is specified, the -rsize parameter specifies a capacity that is reserved by the copy. This protects the copy from going offline when its managed disk group runs out of space by allowing the managed disk group to consume this reserved space first. The parameter has no immediate effect on
168
Implementing the IBM System Storage SAN Volume Controller V4.3
image mode copies. However, if the image mode copy is subsequently migrated to managed mode, the copy is then automatically expanded. -grainsize 32 | 64 | 128 | 256 (Optional) Sets the grain size (KB) for a space-efficient VDisk. This parameter also requires that the -rsize parameter also be specified. The default is 32 KB. If you are using the space-efficient VDisk in a FlashCopy map, use the same grain size as the map grain size for best performance. If you are using the space-efficient VDisk directly with a host system, use a small grain size. -import (Optional) Imports a space-efficient VDisk from the MDisk. Requires that the -rsize parameter also be specified. -copies num_copies (Optional) Specifies the number of copies to create. The num_copies value can be 1 or 2. Setting the value to 2 creates a mirrored VDisk. The default value is 1. -syncrate percentage (Optional) Specifies the copy synchronization rate, as a percentage of the peak synchronization rate. A value of zero (0) prevents synchronization. The default value is 50. -createsync (Optional) Creates copies in sync. Use this parameter if you have already formatted the MDisks, or when read stability to unwritten areas of the VDisk is not required. -fmtdisk (Optional) Specifies that the virtual disk be formatted before it can be used. The -fmtdisk parameter formats (sets to all zeros) the extents that make up this VDisk after it is created. If this parameter is used, the command completes asynchronously; you can query the status using the svcinfo lsvdiskprogress command. The -fmtdisk parameter is not required when creating space-efficient virtual disks. space-efficient VDisks return zeros for extents that have not been written to.The -fmtdisk parameter synchronizes mirrored copies by default. Note: You cannot specify this parameter with the -vtype image parameter. -vtype seq | striped | image (Optional) Specifies the virtualization type. When creating sequential or image mode VDisks, you must also specify the -mdisk parameter. The default virtualization type is striped. -node node_id | node_name (Optional) Specifies the preferred node ID or the name for I/O operations to this virtual disk. You can use the -node parameter to specify the preferred access node. Note: This parameter is required for the subsystem device driver (SDD). The cluster chooses a default if you do not supply this parameter. -unit b | kb | mb | gb | tb | pb (Optional) Specifies the data units to use in conjunction with the capacity that is specified by the -size parameter.
Chapter 6. Quickstart configuration using the command-line interface
169
-mdisk mdisk_id_list | mdisk_name_list (Optional) Specifies one or more managed disks. For sequential and image mode VDisks, the number of MDisks must match the number of copies. For sequential mode VDisks, each MDisk must belong to the specified MDisk group. For striped VDisks, you cannot specify the -mdisk parameter if the -copies value is greater than 1. When creating a single copy striped VDisk, you can specify a list of VDisks to stripe across. -name new_name_arg (Optional) Specifies a name to assign to the new virtual disk. -cache readwrite | none (Optional) Specifies the caching options for the VDisk. Valid entries are readwrite or none. The default is readwrite. If you do not specify the -cache parameter, the default value (readwrite) is used. Perform the following steps to create VDisks: 1. Create a striped VDisk using the svctask mkvdisk command (we cover sequential and image mode VDisks in a later section). See Example 6-20. This command creates a 10 GB, striped VDisk with VDisk id0 within the MDG MDG_DS47 and assigns it to the I/O group iogrp_0. Example 6-20 svctask mkvdisk commands IBM_2145:ITSO-CLS1:admin>svctask mkvdisk -mdiskgrp MDG_DS45 -iogrp io_grp0 -size 10 -unit gb -name vdisk_A Virtual Disk, id [0], successfully created
2. Create the VDisks (four in this example) using the previous command several times, changing the -name parameter for each VDisk. The result can be displayed using the svcinfo lsvdisk command, as shown in Example 6-21. Example 6-21 svcinfo lsvdisk command IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk -delim , id,name,IO_group_id,IO_group_name,status,mdisk_grp_id,mdisk_grp_name,capacity,type,FC_id,FC _name,RC_id,RC_name,vdisk_UID,fc_map_count,copy_count 0,vdisk_A,0,io_grp0,online,0,MDG_DS45,10.0GB,striped,,,,,60050768018301BF2800000000000008,0 ,1 1,vdisk_B,1,io_grp1,online,1,MDG_DS47,100.0GB,striped,,,,,60050768018301BF2800000000000001, 0,1 2,vdisk_C,1,io_grp1,online,0,MDG_DS45,40.0GB,striped,,,,,60050768018301BF2800000000000002,0 ,1 3,vdisk_D,1,io_grp1,online,0,MDG_DS45,80.0GB,striped,,,,,60050768018301BF2800000000000003,0 ,1
To display more information about a specific VDisk, enter a variant of the svcinfo lsvdisk command, as shown in Example 6-22. Example 6-22 svcinfo lsvdisk command IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk 2 id 2 name vdisk_C IO_group_id 1 IO_group_name io_grp1 status online mdisk_grp_id 0 mdisk_grp_name MDG_DS45 capacity 40.0GB
170
Implementing the IBM System Storage SAN Volume Controller V4.3
type striped formatted no mdisk_id mdisk_name FC_id FC_name RC_id RC_name vdisk_UID 60050768018301BF2800000000000002 throttling 0 preferred_node_id 3 fast_write_state empty cache readwrite udid fc_map_count 0 sync_rate 50 copy_count 1 copy_id 0 status online sync yes primary yes mdisk_grp_id 0 mdisk_grp_name MDG_DS45 type striped mdisk_id mdisk_name fast_write_state empty used_capacity 0.41MB real_capacity 12.00GB free_capacity 12.00GB overallocation 333 autoexpand off warning 23 grainsize 32
For more detailed information about space-efficient VDisks (SEV) and tasks, such as deleting, renaming, or expanding a VDisk, see Chapter 9, “SVC configuration and administration using the CLI” on page 303. You have now completed the tasks required to create a VDisk.
6.8 Assigning a VDisk to a host Using the VDisk and host definition created in the previous sections, assign VDisks to hosts ready for their use. To do this, use the svctask mkvdiskhostmap command (see Example 6-23). Example 6-23 svctask mkvdiskhostmap IBM_2145:ITSO-CLS1:admin>svctask mkvdiskhostmap -host Nile vdisk_B Virtual Disk to Host map, id [2], successfully created IBM_2145:ITSO-CLS1:admin>svctask mkvdiskhostmap -host Nile vdisk_C Virtual Disk to Host map, id [1], successfully created
Chapter 6. Quickstart configuration using the command-line interface
171
This command assigns vdisk_B and vdisk_C to host Nile as shown in Example 6-24. Example 6-24 svcinfo lshostvdiskmap -delim, IBM_2145:ITSO-CLS1:admin>svcinfo lshostvdiskmap -delim , id,name,SCSI_id,vdisk_id,vdisk_name,wwpn,vdisk_UID 1,Nile,2,1,vdisk_B,210000E08B892BCD,60050768018301BF2800000000000001 1,Nile,1,2,vdisk_C,210000E08B892BCD,60050768018301BF2800000000000002
Note: The optional parameter -scsi scsi_num can help assign a specific LUN ID to a VDisk that is to be associated with a given host. The default (if nothing is specified) is to increment based on what is already assigned to the host. For information about other tasks, such as deleting a VDisk to host mapping, see Chapter 6, “Quickstart configuration using the command-line interface” on page 157. You have now completed all the tasks required to assign a VDisk to an attached host. You are ready to proceed to Chapter 8, “Host configuration” on page 209 and begin to use the assigned VDisks.
172
Implementing the IBM System Storage SAN Volume Controller V4.3
7
Chapter 7.
Quickstart configuration using the GUI In this chapter, we describe the basic configuration procedures required to get your IBM System Storage SAN Volume Controller (SVC) environment up and running as quickly as possible using the Master Console and its associated Graphical User Interface (GUI). See Chapter 10, “SVC configuration and administration using the GUI” on page 401 for more information about these and other configuration and administration procedures. Important: Data entries made through the GUI are case sensitive.
© Copyright IBM Corp. 2003-2008. All rights reserved.
173
7.1 Adding nodes to the cluster After cluster creation is completed through the service window (the front window of one of the SVC nodes) and cluster Web interface, only one node (the configuration node) is set up. To be a fully functional SVC cluster, at least a second node must be added to the configuration. Perform the following steps to add nodes to the cluster: 1. Open the GUI using one of the following methods: – Double-click the SAN Volume Controller Console icon on your SSPC desktop. – Open a Web browser on the SSPC console and point to this address: http://localhost:9080/ica – Open a Web browser on a separate workstation and point to this address: http://sspcconsoleipaddress:9080/ica On the Signon window (Figure 7-1), type the user ID superuser and the password passw0rd. These are the default user ID and password. Click OK.
Figure 7-1 GUI Signon window
2. The GUI Welcome window appears, as shown in Figure 7-2 on page 175. This window has several links: My Work (top left), a Recent Tasks list (bottom left), the GUI version and build level information (right, under the main graphic), and a hypertext link to the SVC download page: http://www.ibm.com/storage/support/2145
174
Implementing the IBM System Storage SAN Volume Controller V4.3
Under My Work on the left, click the Clusters link (Figure 7-2).
Figure 7-2 GUI Welcome window
3. On the Viewing Clusters window (Figure 7-3), select the radio button next to the cluster on which you want to perform actions (in our case, ITSOSVC42). In Master Console Version 4.2, the Launch the SAN Volume Controller application is automatically highlighted, so click Go.
Figure 7-3 Launch the SAN Volume Controller application
Chapter 7. Quickstart configuration using the GUI
175
4. The SAN Volume Controller Console Application launches in a separate browser window (Figure 7-4). In this window, as with the Welcome window, you can see several links under My Work (top left), a Recent Tasks list (bottom left), the SVC Console version and build level information (right, under main graphic), and a hypertext link that will bring you to the SVC download page: http://www.ibm.com/storage/support/2145 Under My Work, click the Work with Nodes option and then the Nodes link.
Figure 7-4 SVC Console Welcome window
5. The Viewing Nodes window (Figure 7-5) opens. Note the input/output (I/O) group name (for example, io_grp0). Select the node you want to add. Ensure that Add a node is selected from the drop-down list and click Go.
Figure 7-5 Viewing Nodes window
Note: You can rename the existing node to your own naming convention standards (we show how to do this later). In your window, it should appear as node1 by default.
176
Implementing the IBM System Storage SAN Volume Controller V4.3
6. The next window (Figure 7-6) displays the available nodes. Select the node from the Available Candidate Nodes drop-down list. Associate it with an I/O group and provide a name (for example, SVCNode2). Click OK.
Figure 7-6 Adding a Node to a Cluster window
Note: If you do not provide a name, the SVC automatically generates the name nodeX, where X is the ID sequence number assigned by the SVC internally. If you want to provide a name, you can use letters A to Z, a to z, numbers 0 to 9, and the underscore. It can be between one and 15 characters in length, but cannot start with a number or the word node, since this prefix is reserved for SVC assignment only. In our case, we only have enough nodes to complete the formation of one I/O group. Therefore, we added our new node to the I/O group that node1 was already using, namely io_grp0 (you can rename from the default of iogrp0 using your own naming convention standards). If this window does not display any available nodes (indicated by the message “CMMVC1100I There are no candidate nodes available”), check if your second node is powered on and that zones are appropriately configured in your switches. It is also possible that a pre-existing cluster’s configuration data is stored on it. If you are sure this node is not part of another active SVC cluster, use the service window to delete the existing cluster information. When this is complete, return to this window and you should see the node listed. For information about zoning requirements, see Chapter 3, “Planning and configuration” on page 25. For information about how to delete an existing cluster configuration using the service window, see 5.5, “Basic installation” on page 102.
Chapter 7. Quickstart configuration using the GUI
177
7. Return to the Viewing Nodes window (Figure 7-7). It shows the status change of the node from Adding to Online.
Figure 7-7 Node added successfully
Note: This window does not automatically refresh. Therefore, you continue to see the Adding status only until you click the Refresh button.
You have completed the cluster configuration and now you have a fully redundant SVC environment.
7.1.1 Installing certificates As we continue with setting up the SVC cluster, we will come across many instances where we are prompted with security warnings regarding unrecognized certificates. The security warning window (Figure 7-8 on page 179) shows three options.
178
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 7-8 Security Alert window
These security options are: Yes: Clicking Yes accepts the certificate for this task. This option allows you to proceed using the unrecognized certificate. Each time you select a task which transmits secure information, you are prompted to accept another certificate. In most cases, you are prompted multiple times due to the two-way data exchange, which occurs between the management workstation and the SVC cluster. In some cases, this can cause your browser to crash. No (default): Clicking this option rejects the certificate for this task and does not allow you to proceed.
Chapter 7. Quickstart configuration using the GUI
179
View Certificate: Clicking this option launches the Certificate window (Figure 7-9), from where you can install the certificate. If you do not want to be prompted repeatedly to accept or reject certificates, we recommend that you choose this option.
Figure 7-9 Certificate Information
Follow these steps to install a certificate: 1. From the Security Alert window (Figure 7-8 on page 179), select View Certificate. 2. The Certificate window opens (see Figure 7-9 below). Click Install Certificate.
180
Implementing the IBM System Storage SAN Volume Controller V4.3
3. The Welcome to the Certificate Import Wizard information window (Figure 7-10) opens. Click Next.
Figure 7-10 Certificate Import Wizard
4. In the Certificate Store window (Figure 7-11), click Next.
Figure 7-11 Certificate Store window
Chapter 7. Quickstart configuration using the GUI
181
5. In the Completion window (Figure 7-12), click Finish.
Figure 7-12 Completion window
6. You might be prompted with the Root Certificate Store confirmation window (Figure 7-13). If you are prompted, click Yes.
Figure 7-13 Root Certificate Store
7. You should see a message stating that the import was successful (Figure 7-14). Click OK. l
Figure 7-14 Certificate Import successful
8. You return to the Certificate Information window (Figure 7-9 on page 180) that you saw earlier. Click OK. From this point, you should no longer be asked to accept or reject certificates from the SVC cluster.
182
Implementing the IBM System Storage SAN Volume Controller V4.3
Note: Future code upgrades could result in new certificate IDs, so you might have to go through this process again.
7.2 Setting the cluster time zone and time Perform the following steps to set the cluster time zone and time: 1. From the SVC Welcome window (Figure 7-4 on page 176), select the Manage Cluster option and the Set Cluster Time link. 2. The Cluster Date and Time Settings window opens (Figure 7-15). At the top of the window, you see the existing settings. If necessary, make adjustments and ensure that the Update cluster data and time and Update cluster time zone check boxes are selected. Click Update.
Figure 7-15 Cluster Date and Time Settings window
Note: You might be prompted for the cluster user ID and password. If you are, enter admin and the password you set earlier.
Chapter 7. Quickstart configuration using the GUI
183
3. You see the messages “The cluster time zone setting has been updated” and “The cluster date and time setting have been updated”. You have now completed the tasks necessary to set the cluster time zone and time.
7.3 Checking the license status Perform the following steps to check the status of the license you have bought along with the SVC. 1. From the SVC Welcome window (Figure 7-4 on page 176), select Service and Maintenance and then the License Settings link. 2. Figure 7-16 shows your currently installed license. 3. Click the Close button, as shown in Figure 7-16, to close this window.
Figure 7-16 Checking the license settings
184
Implementing the IBM System Storage SAN Volume Controller V4.3
7.4 Creating host definitions Perform the following steps to create host objects within the SVC: 1. From the SVC Welcome window (Figure 7-4 on page 176), select the Working with Hosts option and then the Hosts link. 2. The Filtering Hosts window (not shown) should appear. Click the Bypass filter button at the top of this window. 3. The Viewing Hosts window opens (see Figure 7-17). Select Create a host from the list and click Go.
Figure 7-17 Viewing Hosts window
4. In the Creating Hosts window (Figure 7-18 on page 186), follow these steps: a. Type a name for your host (for example, aix_test). Note: If you do not provide a name, the SVC automatically generates the name hostX, where X is the ID sequence number assigned by the SVC internally. If you want to provide a name (as we have), you can use the letters A to Z, a to z, numbers 0 to 9, and the underscore. It can be between one and 15 characters in length. However, it cannot start with a number or the word host, because this prefix is reserved for SVC assignment only. b. Select a Type: •
Generic: Most hosts will use this. this is the default
•
HPUX: For HP UNIX
•
TPGS: For a Sun™ host using MPxIO
c. Define the Port Mask. Here we specify the SVC FC ports that the host can access on each node. (The rightmost bit is associated with Fibre Channel port 1 on each node. The leftmost bit is associated with port 4.) For example: 0111 prevents access by the host using port 4 on each SVC node; 1100 allows access on ports 3 and 4, but not on ports 1 and 2. d. Select the I/O Groups. e. From the Available Port list, select the WWN or WWNs, one at a time, and click the Add button.
Chapter 7. Quickstart configuration using the GUI
185
f. If the WWNs are not shown (the host has not been zoned and so on), you can manually add them in the Additional Ports box. g. When you are done adding the WWNs, click OK.
Figure 7-18 Creating Hosts window
Note: This window shows all WWNs that are visible to the SVC and that have not already been defined to a host. If your WWN does not appear, check that the host has logged into the switch and that zoning in the switches is updated to allow SVC and host ports to see each other. This is described in Chapter 3, “Planning and configuration” on page 25. Also note that if you are working with an AIX host and do not see your adapter listed in the Creating Hosts window, then rerun the cfgmgr command to encourage the host HBAs to communicate with the SVC and refresh this window.
186
Implementing the IBM System Storage SAN Volume Controller V4.3
5. You will return to the Viewing Hosts window (Figure 7-19) where you should see your newly created host. (We defined all our hosts connected to the SAN.) In SVC V4.3, the host window shows the WWPN of the hosts.
Figure 7-19 Host added successfully
For information about other tasks, such as adding host ports, deleting host ports, or deleting hosts, see Chapter 10, “SVC configuration and administration using the GUI” on page 401. You have now completed the tasks required to add host definitions to your SVC configuration.
7.5 Displaying managed disks Perform the following steps to display MDisks: 1. From the SVC Welcome window (Figure 7-4 on page 176), select the Work with Managed Disks option and then the Managed Disks link.
Chapter 7. Quickstart configuration using the GUI
187
2. In the Viewing Managed Disks window (Figure 7-20), if your MDisks are not displayed, rescan the Fibre Channel network. Select Discover MDisks from the list and click Go.
Figure 7-20 Discover MDisks
Note: If your MDisks are still not visible, check that the logical unit numbers (LUNs) from your subsystem are properly assigned to the SVC (for example, using storage partitioning with a DS4000) and that appropriate zoning is in place (for example, the SVC can see the disk subsystem). See Chapter 3, “Planning and configuration” on page 25 for more details about how to set up your storage area network (SAN) fabric.
7.6 Creating managed disk groups Perform the following steps to create a managed disk group (MDG): 1. From the SVC Welcome window (Figure 7-4 on page 176), select the Work with Managed Disks option and then the Managed Disks Groups link. 2. The Viewing Managed Disks Groups window opens (see Figure 7-21 on page 189). Select Create an MDisk Group from the list and click Go.
188
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 7-21 Selecting the option to create an MDisk group
3. In the window Create Managed Disk Group, the wizard will give you an overview of what will be done. Click Next. 4. While in the window “Name the group and select the managed disks” (Figure 7-22 on page 190), follow these steps: a. Type a name for the MDG. Note: If you do not provide a name, the SVC automatically generates the name MDiskgrpX, where X is the ID sequence number assigned by the SVC internally. If you want to provide a name (as we have), you can use the letters A to Z, a to z, numbers 0 to 9, and the underscore. It can be between one and 15 characters in length and is case sensitive, but cannot start with a number or the word MDiskgrp, because this prefix is reserved for SVC assignment only. b. From the MDisk Candidates box, one at a time, select the MDisks to put into the MDG. Click Add to move them to the Selected MDisks box. There may be more than one page of disks; you may navigate between the windows (the MDisks you selected will be preserved). c. You can specify a threshold to send a warning to the error log when the capacity is first exceeded. It can either be a percentage or a specific amount. d. Click Next.
Chapter 7. Quickstart configuration using the GUI
189
Figure 7-22 Name the group and select the managed disks window
5. From the list shown in Figure 7-23, select the extent size to use. When you select a specific extent size, it will display the total cluster size in TB. Click Next.
Figure 7-23 Select Extent Size window
190
Implementing the IBM System Storage SAN Volume Controller V4.3
6. In the window Verify Managed Disk Group (Figure 7-24), verify that the information specified is correct. Click Finish.
Figure 7-24 Verify MDG wizard
7. Return to the Viewing Managed Disk Groups window (Figure 7-25) where the MDG is displayed.
Figure 7-25 MDG added successfully
For information about other tasks, such as adding MDisks to MDGs and renaming MDGs or deleting MDGs, see Chapter 10, “SVC configuration and administration using the GUI” on page 401. You have now completed the tasks required to create an MDG.
Chapter 7. Quickstart configuration using the GUI
191
7.7 Creating a VDisk Perform the following steps to create VDisks: 1. From the SVC Welcome window (Figure 7-4 on page 176), select the Work with Virtual Disks option and then the Virtual Disks link. 2. The Viewing Virtual Disks window opens (see Figure 7-26). Select Create VDisk from the list and click Go.
Figure 7-26 Viewing Virtual Disks window
3. The Create Virtual Disks wizard will be displayed. Click the Next button. 4. In the window, choose an I/O Group and a Preferred Node (Figure 7-27 on page 193) and follow these steps: a. Select the I/O group to associate the VDisk with from the list. In our case, we only have one, io_grp0, so we must select it from the list. Note: You can let the system choose the preferred node and I/O group. b. Optionally, choose a preferred node. The default (if nothing is selected) is to alternate between nodes in the I/O group. c. Click Next.
192
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 7-27 Choose an I/O group and preferred node window
5. In the Set Attributes window (Figure 7-28 on page 194), follow these steps: a. Select the type of VDisk to create (striped or sequential) from the list. b. Select the cache mode (read/write, none) from the list. c. Select a Unit device identifier (numerical number) for this VDisk. d. Select the number of VDisks to create. e. Click Next. The Space-efficient check box option is shown in 7.7.1, “Creating a space-efficient VDisk (SEV Disk)” on page 198. The Mirrored Disk check box option is shown in 7.7.2, “Creating a mirrored VDisk” on page 203. The Format VDisk before use check box option will write zeros to all the managed disk extents. This is useful to remove any references to older data that may remain on the managed storage.
Chapter 7. Quickstart configuration using the GUI
193
Figure 7-28 Select the Type of VDisk window
6. In the window Select Attributes (Figure 7-29 on page 195), follow these steps: a. Select the Managed Disk Group. b. Optionally, choose the Managed Disk Candidates upon which to create the VDisk. Click Add to move them to the Managed Disks Striped in this Order box. Striped VDisks, by default, use all MDisks within an MDG. Therefore, it is not necessary to select anything here. However, you might want to select from the list, for example, if you want to specify that the VDisk only uses a subset of the MDisks available within an MDG. For image and sequential VDisks, we do not see the managed disk candidates or managed disks striped in the Order box. Instead, we see the Managed Disk Used to Create VDisk and the top one in the list selected by default. c. Type the capacity of the VDisk. Select the unit of capacity from the list. Remember, capacity is calculated based on 1 GB = 1024 MB. Therefore, an entry of 10 GB actually provides 10240 MB instead of 10000 MB as with other disk subsystems. d. After completing all the necessary entry fields, click Next.
194
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 7-29 Select MDisks and Size
7. In the window Name the VDisk(s) (Figure 7-30 on page 196), type a name for the virtual disk you are creating. In this case, we used the prefix Vdisk_, as we are creating multiple VDisks. Click Next. Note: If you do not provide a name, the SVC automatically generates the name VDiskX, where X is the ID sequence number assigned by the SVC internally. If you want to provide a name (as we have), you can use letters A to Z, a to z, numbers 0 to 9, and the underscore. It can be between one and 15 characters in length and is case sensitive. However, it cannot start with a number or the word VDisk because this prefix is reserved for SVC assignment only.
Chapter 7. Quickstart configuration using the GUI
195
Figure 7-30 Name the VDisk(s) window
8. In the Verify VDisk window (Figure 7-31), verify the selections. We can select the Back button at any time to make changes.
Figure 7-31 Verify VDisk Attributes
196
Implementing the IBM System Storage SAN Volume Controller V4.3
9. After selecting the Finish option, we are presented with a window (Figure 7-32) that tells us the result of the action.
Figure 7-32 VDisk creation success
10.We click Close again and see a list (Figure 7-33) of all created VDisks.
Figure 7-33 List of all created VDisks
For information about other tasks, such as deleting a VDisk, renaming a VDisk, or expanding a VDisk, see Chapter 10, “SVC configuration and administration using the GUI” on page 401. You have now completed the tasks required to create a VDisk.
Chapter 7. Quickstart configuration using the GUI
197
7.7.1 Creating a space-efficient VDisk (SEV Disk) In this section, we are going to create a space-efficient VDisk (SEV disk) step-by-step. This will allow you to create VDisks with much higher capacity than is physically available (this is called thin provisioning). If you want more detailed information, see 3.6.6, “Space-efficient Virtual Disk” on page 52 for a full discussion about space-efficient VDisks. See 7.7, “Creating a VDisk” on page 192, perform steps 1 to 4, and then do the following: 1. In the Set Attributes window (Figure 7-34 on page 199), follow these steps: a. Select the type of VDisk to create (striped or sequential) from the list. b. Select the cache mode (read/write, none) from the list. c. Select a Unit device identifier (numerical number) for this VDisk. d. Select the number of VDisks to create. e. Select the Space-efficient check box. This will expand the section with the following options: i. Type the size of the VDisk (remember, this is the virtual size). ii. Type in a percentage or select a specific size for the usage threshold warning. iii. Optionally, you can select the Autoexpand check box. This will allow the real disk size to grow as required. This is covered in greater detail in 10.8.4, “Creating a space-efficient VDisk with auto-expand” on page 484. iv. Select the Grain size (choose 32 KB normally, but you would match the FlashCopy grain size if the VDisk is being used for FlashCopy, which is 256 KB).
198
Implementing the IBM System Storage SAN Volume Controller V4.3
f. Click Next.
Figure 7-34 Select the type of VDisk window
Chapter 7. Quickstart configuration using the GUI
199
2. In the window Select MDisk(s) and Size for a <modetype>-Mode VDisk, shown in Figure 7-35, follow these steps: a. Select the managed disk group from the list. b. Optionally, choose the Managed Disk Candidates upon which to create the VDisk. Click Add to move them to the Managed Disks Striped in this Order box. c. Type in the Real size you wish to allocate. It can either be a percentage of the virtual size or a specific number. This is how much disk will actually be allocated. If you selected Autoexpand in the previous step, this will grow as the existing allocation is actually used. 3. Click Next.
Figure 7-35 Select the MDisk/Size for a VDisk window
4. In the window Name the VDisk(s) (Figure 7-36 on page 201), type a name for the virtual disk you are creating. In our case, we used vdisk_sev1. Click Next.
200
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 7-36 Name the VDisk(s) window
5. In the Verify space-efficient VDisk window (Figure 7-37), verify the selections. We can select the Back button at any time to make changes.
Figure 7-37 Verifying space-efficient VDisk Attributes window
Chapter 7. Quickstart configuration using the GUI
201
6. After selecting the Finish option, we are presented with a window (Figure 7-38) that tells us the result of the action.
Figure 7-38 Space-efficient VDisk creation success
7. We click Close again and see a listing (Figure 7-39) of the created space-efficient VDisk.
Figure 7-39 List of created space-efficient VDisks
Attention: If the used capacity reaches the real capacity then the VDisk will go offline, the volume mapped on the host will freeze, and I/Os will start to fail. You must then provision more storage to expand the real capacity to get the VDisk back online.
202
Implementing the IBM System Storage SAN Volume Controller V4.3
7.7.2 Creating a mirrored VDisk In this section, we are going to create a mirrored VDisk step-by-step. This provides a highly available VDisk. If you want more detailed information see 7.7.2, “Creating a mirrored VDisk” on page 203 Refer to 7.7, “Creating a VDisk” on page 192, perform steps 1 to 4, and then do the following: 1. In the Set Attributes window (Figure 7-40), follow these steps: a. Select the type of VDisk to create (striped or sequential) from the list. b. Select the cache mode (read/write, none) from the list. c. Select a Unit device identifier (numerical number) for this VDisk. d. Select the number of VDisks to create. e. Select the Mirrored Disk check box. Some mirror disk options will appear. f. Type the Mirror Synchronization rate, in percent. It is set to 50% by default. g. Optionally, you can check the Synchronized check box. Select this option when MDisks are already formatted or when read stability to unwritten areas of the VDisk is not required. h. Click Next.
Figure 7-40 Select the Type of VDisk window
Chapter 7. Quickstart configuration using the GUI
203
2. In the window Select MDisk(s) and Size for a Striped-Mode VDisk (Copy 0), shown in Figure 7-41, follow these steps: a. Select the managed disk group from the list. b. Type the capacity of the VDisk. Select the unit of capacity from the list. c. Click Next.
Figure 7-41 Select MDisk(s) and Size for a Striped-Mode VDisk (Copy 0) window
3. In the window Select MDisk(s) and Size for a Striped-Mode VDisk (Copy 1), shown in Figure 7-42, select a managed disk group for Copy 1 of the mirror. This can be defined within the same or on a different MDG. Click Next.
Figure 7-42 Select MDisk(s) and Size for a Striped-Mode VDisk (Copy 1) window
4. In the window Name the VDisk(s) (Figure 7-43 on page 205), type a name for the virtual disk you are creating. In this case, we used MirrorVDisk1. Click Next.
204
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 7-43 Name the VDisk(s) window
5. In the Verify Mirrored VDisk window (Figure 7-44), verify the selections. We can select the Back button at any time to make changes.
Figure 7-44 Verifying Mirrored VDisk Attributes window
Chapter 7. Quickstart configuration using the GUI
205
6. After selecting the Finish option, we are presented with the window shown in Figure 7-45, which tells us the result of the action.
Figure 7-45 Mirrored VDisk creation success
We click Close again and see a listing (Figure 7-46) of the created mirrored VDisk.
Figure 7-46 List of created mirrored VDisks
206
Implementing the IBM System Storage SAN Volume Controller V4.3
7.8 Assigning a VDisk to a host Perform the following steps to map a VDisk to a host: 1. From the SVC Welcome window (Figure 7-4 on page 176), select the Work with Virtual Disks option and then the Virtual Disks link. 2. In the Viewing Virtual Disks window (Figure 7-47), from the drop-down menu, select Map VDisks to a host from the list and click Go.
Figure 7-47 Assigning VDisks to a host
3. In the window Creating Virtual Disk-to-Host Mappings (Figure 7-48), select the target host. In V4.2, we have a new option to specify the SCSI LUN ID. (This field is optional. Use this field to specify an ID for the SCSI LUN. If you do not specify an ID, the next available SCSI LUN ID on the host adapter is automatically used.) Click OK.
Figure 7-48 Creating VDisk-to-Host Mappings window
Chapter 7. Quickstart configuration using the GUI
207
4. You are presented with an information window that displays the status, as shown in Figure 7-49.
Figure 7-49 VDisk to host mapping successful
5. You now return to the Viewing Virtual Disks window (Figure 7-47 on page 207). For information about other tasks such as deleting a VDisk to host mapping, see Chapter 10, “SVC configuration and administration using the GUI” on page 401. You have now completed all the tasks required to assign a VDisk to an attached host. You are ready to proceed to Chapter 8, “Host configuration” on page 209 and begin using the assigned VDisks.
208
Implementing the IBM System Storage SAN Volume Controller V4.3
8
Chapter 8.
Host configuration In this chapter, we describe the basic host configuration procedures required to connect supported hosts to the IBM System Storage SAN Volume Controller (SVC).
© Copyright IBM Corp. 2003-2008. All rights reserved.
209
8.1 SVC setup Figure 8-1 shows a basic configuration with multiple heterogeneous hosts connected to a two node SVC cluster through two switches.
Figure 8-1 SAN Volume controller environment
Even if there are 16 possible paths, only four paths should be available per VDisk to the host. Four paths per VDisk to the host are the recommended balance between performance and redundancy. Refer to Chapter 4, “Performance and capacity planning” on page 77 for more information. To accomplish this task, you can use either SAN Switch Zoning (see 8.1.1, “Switch zoning recommendations” on page 211) or Port Masking (see 8.1.2, “Using port masking” on page 212). The number of available paths for the host must not exceed eight.
210
Implementing the IBM System Storage SAN Volume Controller V4.3
8.1.1 Switch zoning recommendations Even if there are 16 possible paths (two server ports x eight SVC node ports), only four paths exist because of the switch zoning. For a two node cluster, a zone consists of one port of the server, and one port of each SVC node, as shown in Figure 8-2 (Zone 1: blue lines; Zone 2: green lines).
Figure 8-2 SVC zoning two nodes
Chapter 8. Host configuration
211
If you are using a four node cluster, then Figure 8-3 shows you the recommended zoning configuration. The host now has four paths to IO Group 1 (green lines) and also four paths to IO Group 2 (red lines). But the VDisks always reside in just the one IO Group. So this zoning also fulfills the four paths per VDisk recommendation. This scenario can be increased up to eight nodes.
Figure 8-3 SVC zoning four nodes
8.1.2 Using port masking From SVC V4.1 onwards, it is possible to configure which node ports a host HBA can access using port masking on the SVC. In this case, the host port can be zoned to all SVC node ports, and load balancing across node ports is configured on the SVC. Refer to “Port masking” on page 61 for more information.
8.2 AIX-specific information The following section details specific information that relates to the connection of AIX-based hosts into an SVC environment. Note: In this section, the IBM System p® information applies to all AIX hosts that are listed on the SAN Volume Controller interoperability support site, including IBM System i® partitions and IBM JS blades.
212
Implementing the IBM System Storage SAN Volume Controller V4.3
8.2.1 Configuring the AIX host To configure the AIX host, follow these steps: 1. Install the HBAs into the AIX host system. 2. Ensure that you have installed the correct operating systems and version levels on your host, including any updates and APARs (Authorized Program Analysis Reports) for the operating system. 3. Connect the AIX host system to the Fibre Channel switches. 4. Configure the Fibre Channel switches (zoning) if needed. 5. Install and configure the 2145 and SDD drivers. 6. Configure the host, VDisks, and host mapping on the SAN Volume Controller. 7. Run the cfgmgr command to discover the VDisks created on the SVC. The following sections detail the current support information. It is vital that you check the Web sites listed regularly for any updates.
8.2.2 Operating system versions and maintenance levels At the time of writing, the following AIX levels are supported: AIX V4.3.3 AIX 5L V5.1 AIX 5L V5.2 AIX 5L V5.3 AIX V6.1.3 For the latest information, and device driver support, always refer to this site: http://www-1.ibm.com/support/docview.wss?rs=591&uid=ssg1S1003278#_AIX
8.2.3 HBAs for IBM System p hosts Ensure that your IBM System p AIX hosts use the correct host bus adapters (HBAs). The following IBM Web site provides current interoperability information about supported HBAs and firmware: http://www-1.ibm.com/support/docview.wss?rs=591&uid=ssg1S1003277#_pSeries Note: The maximum number of Fibre Channel ports that are supported in a single host (or logical partition) is four. This can be four single-port adapters or two dual-port adapters or a combination, as long as the maximum number of ports that are attached to the SAN Volume Controller does not exceed four.
Installing the host attachment script on IBM System p hosts To attach an IBM System p AIX host, you must install the AIX host attachment script. Perform the following steps to install the host attachment scripts: 1. Access the following Web site: http://www.ibm.com/servers/storage/support/software/sdd/downloading.html
Chapter 8. Host configuration
213
2. Select Host Attachment Scripts for AIX. 3. Select either Host Attachment Script for SDDPCM or Host Attachment Scripts for SDD from the options, depending on your multipath device driver. 4. Download the AIX host attachment script for your multipath device driver. 5. Follow the instructions that are provided on the Web site or any readme files to install the script.
8.2.4 Configuring for fast fail and dynamic tracking For hosts systems that run an AIX 5L V5.2 or later operating system, you can achieve the best results by using the fast fail and dynamic tracking attributes. Perform the following steps to configure your host system to use the fast fail and dynamic tracking attributes: 1. Issue the following command to set the Fibre Channel SCSI I/O Controller Protocol Device to each Adapter: chdev -l fscsi0 -a fc_err_recov=fast_fail The previous command was for adapter fscsi0. Example 8-1 shows the command for both adapters on our test system running AIX 5L V5.3. Example 8-1 Enable Fast Fail
#chdev fscsi0 #chdev fscsi1
-l fscsi0 -a fc_err_recov=fast_fail changed -l fscsi1 -a fc_err_recov=fast_fail changed
2. Issue the following command to enable dynamic tracking for each Fibre Channel device: chdev -l fscsi0 -a dyntrk=yes The previous example command was for adapter fscsi0. Example 8-2 shows the command for both adapters on our test system running AIX 5L V5.3. Example 8-2 Enable dynamic tracking
#chdev fscsi0 #chdev fscsi1
-l fscsi0 -a dyntrk=yes changed -l fscsi1 -a dyntrk=yes changed
Host adapter configuration settings You can check the availability of the FC Host Adapters by using the command shown in Example 8-3. Example 8-3 FC Host Adapter availability
#lsdev -Cc adapter |grep fcs fcs0 Available 1Z-08 FC Adapter fcs1 Available 1D-08 FC Adapter
214
Implementing the IBM System Storage SAN Volume Controller V4.3
You can also find the worldwide port name (WWPN) of your FC Host Adapter and check the firmware level, as shown in Example 8-4. The Network Address is the WWPN for the FC adapter. Example 8-4 FC Host Adapter settings and WWPN
#lscfg -vpl fcs0 fcs0
U0.1-P2-I4/Q1
FC Adapter
Part Number.................00P4494 EC Level....................A Serial Number...............1E3120A68D Manufacturer................001E Device Specific.(CC)........2765 FRU Number.................. 00P4495 Network Address.............10000000C932A7FB ROS Level and ID............02C03951 Device Specific.(Z0)........2002606D Device Specific.(Z1)........00000000 Device Specific.(Z2)........00000000 Device Specific.(Z3)........03000909 Device Specific.(Z4)........FF401210 Device Specific.(Z5)........02C03951 Device Specific.(Z6)........06433951 Device Specific.(Z7)........07433951 Device Specific.(Z8)........20000000C932A7FB Device Specific.(Z9)........CS3.91A1 Device Specific.(ZA)........C1D3.91A1 Device Specific.(ZB)........C2D3.91A1 Device Specific.(YL)........U0.1-P2-I4/Q1
PLATFORM SPECIFIC Name: fibre-channel Model: LP9002 Node: fibre-channel@1 Device Type: fcp Physical Location: U0.1-P2-I4/Q1
8.2.5 Subsystem Device Driver (SDDPCM or SDD) SDD is a pseudo device driver designed to support the multipath configuration environments within IBM products. It resides on a host system along with the native disk device driver and provides the following functions:
Enhanced data availability Dynamic input/output (I/O) load balancing across multiple paths Automatic path failover protection Concurrent download of licensed internal code
Chapter 8. Host configuration
215
SDD works by grouping each physical path to an SVC LUN, represented by individual hdisk devices within AIX, into a vpath device (for example, if you have four physical paths to an SVC LUN, this produces four new hdisk devices within AIX). From this moment onwards, AIX uses this vpath device to route I/O to the SVC LUN. Therefore, when making an LVM volume group using mkvg, we specify the vpath device as the destination and not the hdisk device. The SDD support matrix for AIX is available at: http://www-1.ibm.com/support/docview.wss?rs=591&uid=ssg1S1003278#_AIX
SDD / SDDPCM installation After downloading the appropriate version of SDD, install it using the standard AIX installation procedure. The currently supported SDD Levels are available at: http://www-304.ibm.com/systems/support/supportsite.wss/supportresources?brandind=5 000033&familyind=5329528&taskind=2 Check the driver readmefile and make sure your AIX system fulfills all the prerequisites.
SDD installation In Example 8-5, we show the appropriate version of SDD downloaded into the /tmp/sdd directory. From here we extract it and initiate the inutoc command, which generates a dot.toc (.toc) file that is needed by the installp command prior to installing SDD. Finally, we initiate the installp command, which installs SDD onto this AIX host. Example 8-5 Installing SDD on AIX
#ls -l total 3032 -rw-r----1 root system 1546240 Jun 24 15:29 devices.sdd.53.rte.tar #tar -tvf devices.sdd.53.rte.tar -rw-r----0 0 1536000 Oct 06 11:37:13 2006 devices.sdd.53.rte #tar -xvf devices.sdd.53.rte.tar x devices.sdd.53.rte, 1536000 bytes, 3000 media blocks. # inutoc . #ls -l total 6032 -rw-r--r-1 root system 476 Jun 24 15:33 .toc -rw-r----1 root system 1536000 Oct 06 2006 devices.sdd.53.rte -rw-r----1 root system 1546240 Jun 24 15:29 devices.sdd.53.rte.tar # installp -ac -d . all Example 8-6 checks the installation of SDD. Example 8-6 Checking SDD device driver
#lslpp -l | grep -i sdd devices.sdd.53.rte devices.sdd.53.rte
1.7.0.0 1.7.0.0
COMMITTED COMMITTED
IBM Subsystem Device Driver IBM Subsystem Device Driver
Note: There no longer exists a specific “2145” devices.fcp file. The standard devices.fcp now has combined support for SVC / ESS / DS8000 / DS6000. We can also check that the SDD server is operational, as shown in Example 8-7 on page 217.
216
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 8-7 SDD server is operational
#lssrc -s sddsrv Subsystem sddsrv
Group
#ps -aef | grep sdd root 135174 41454 root 168430 127292 /usr/sbin/sddsrv
PID 168430
0 15:38:20 0 15:10:27
pts/1 -
Status active
0:00 grep sdd 0:00
Enabling the SDD or SDDPCM Web interface is shown in 8.12, “Using SDDDSM, SDDPCM, and SDD Web interface” on page 300.
SDDPCM installation In Example 8-8, we show the appropriate version of SDDPCM downloaded into the /tmp/sddpcm directory. From here we extract it and initiate the inutoc command, which generates a dot.toc (.toc) file that is needed by the installp command prior to installing SDDPCM. Finally, we initiate the installp command, which installs SDDPCM onto this AIX host. Example 8-8 Installing SDDPCM on AIX
# ls -l total 3232 -rw-r----1 root system 1648640 Jul 15 13:24 devices.sddpcm.61.rte.tar # tar -tvf devices.sddpcm.61.rte.tar -rw-r----- 271001 449628 1638400 Oct 31 12:16:23 2007 devices.sddpcm.61.rte # tar -xvf devices.sddpcm.61.rte.tar x devices.sddpcm.61.rte, 1638400 bytes, 3200 media blocks. # inutoc . # ls -l total 6432 -rw-r--r-1 root system 531 Jul 15 13:25 .toc -rw-r----1 271001 449628 1638400 Oct 31 2007 devices.sddpcm.61.rte -rw-r----1 root system 1648640 Jul 15 13:24 devices.sddpcm.61.rte.tar # installp -ac -d . all Example 8-9 checks the installation of SDDPCM. Example 8-9 Checking SDDPCM device driver
# lslpp -l | grep sddpcm devices.sddpcm.61.rte devices.sddpcm.61.rte
2.2.0.0 2.2.0.0
COMMITTED COMMITTED
IBM SDD PCM for AIX V61 IBM SDD PCM for AIX V61
Enabling the SDD or SDDPCM Web interface is shown in 8.12, “Using SDDDSM, SDDPCM, and SDD Web interface” on page 300.
Chapter 8. Host configuration
217
8.2.6 Discovering the assigned VDisk using SDD and AIX 5L V5.3 Before adding a new volume from the SVC, the AIX host system “Kanga” had a “vanilla” configuration, as shown in Example 8-10. Example 8-10 Status of AIX host system ‘Kanaga’
#lspv hdisk0 hdisk1 hdisk2 #lsvg rootvg
0009cddaea97bf61 0009cdda43c9dfd5 0009cddabaef1d99
rootvg rootvg rootvg
active active active
In Example 8-11, we show SVC configuration information relating to our AIX host, specifically, the host definition, the VDisks created for this host, and the VDisk-to-host mappings for this configuration. Using the SVC CLI, we can check that the host WWPNs, listed in Example 8-4 on page 215, are logged into the SVC for the host definition “aix_test”, by entering: svcinfo lshost aix_test We can also find the serial numbers of the VDisks using the following command: svcinfo lshostvdiskmap Example 8-11 SVC definitions for host system aix_test’
IBM_2145:ITSO-CLS1:admin>svcinfo lshost Kanaga id 2 name Kanaga port_count 2 type generic mask 1111 iogrp_count 2 WWPN 10000000C932A7FB node_logged_in_count 2 state active WWPN 10000000C932A800 node_logged_in_count 2 state active IBM_2145:ITSO-CLS1:admin>svcinfo lshostvdiskmap Kanaga id name SCSI_id vdisk_id vdisk_name wwpn vdisk_UID 2 Kanaga 0 13 Kanaga0001 10000000C932A7FB 60050768018301BF2800000000000015 2 Kanaga 1 14 Kanaga0002 10000000C932A7FB 60050768018301BF2800000000000016 2 Kanaga 2 15 Kanaga0003 10000000C932A7FB 60050768018301BF2800000000000017 2 Kanaga 3 16 Kanaga0004 10000000C932A7FB 60050768018301BF2800000000000018 IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk Kanaga0001 id 13 name Kanaga0001
218
Implementing the IBM System Storage SAN Volume Controller V4.3
IO_group_id 0 IO_group_name io_grp0 status offline mdisk_grp_id 0 mdisk_grp_name MDG_DS45 capacity 5.0GB type striped formatted yes mdisk_id mdisk_name FC_id FC_name RC_id RC_name vdisk_UID 60050768018301BF2800000000000015 throttling 0 preferred_node_id 1 fast_write_state empty cache readwrite udid 0 fc_map_count 0 sync_rate 50 copy_count 1 copy_id 0 status offline sync yes primary yes mdisk_grp_id 0 mdisk_grp_name MDG_DS45 type striped mdisk_id mdisk_name fast_write_state empty used_capacity 5.00GB real_capacity 5.00GB free_capacity 0.00MB overallocation 100 autoexpand warning grainsize
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdiskhostmap Kanaga0001 id name SCSI_id host_id host_name wwpn vdisk_UID 13 Kanaga0001 0 2 Kanaga 10000000C932A7FB 60050768018301BF2800000000000015 13 Kanaga0001 0 2 Kanaga 10000000C932A800 60050768018301BF2800000000000015
Chapter 8. Host configuration
219
We need to run cfgmgr on the AIX host to discover the new disks and enable us to start the vpath configuration; if we run the config manager (cfgmgr) on each FC adapter, it will not create the vpaths, only the new hdisks. To configure the vpaths, we need to run the cfallvpath command after issuing the cfgmgr command on each of the FC adapters: # cfgmgr -l fcs0 # cfgmgr -l fcs1 # cfallvpath Alternatively, use the cfgmgr -vS command to check the complete system. This command will probe the devices sequentially across all FC adapters and attached disks; however, it is very time intensive: # cfgmgr -vS The raw SVC disk configuration of the AIX host system now appears as shown in Example 8-12. We can see the multiple hdisk devices, representing the multiple routes to the same SVC LUN, and we can see the vpath devices available for configuration. Example 8-12 VDisks from SVC added with multiple different paths for each VDisk
#lsdev -Cc disk hdisk0 Available hdisk1 Available hdisk2 Available hdisk3 Available hdisk4 Available hdisk5 Available hdisk6 Available hdisk7 Available hdisk8 Available hdisk9 Available hdisk10 Available hdisk11 Available hdisk12 Available hdisk13 Available hdisk14 Available hdisk15 Available hdisk16 Available hdisk17 Available hdisk18 Available vpath0 Available vpath1 Available vpath2 Available vpath3 Available
1S-08-00-8,0 1S-08-00-9,0 1S-08-00-10,0 1Z-08-02 1Z-08-02 1Z-08-02 1Z-08-02 1D-08-02 1D-08-02 1D-08-02 1D-08-02 1Z-08-02 1Z-08-02 1Z-08-02 1Z-08-02 1D-08-02 1D-08-02 1D-08-02 1D-08-02
16 Bit LVD SCSI Disk Drive 16 Bit LVD SCSI Disk Drive 16 Bit LVD SCSI Disk Drive SAN Volume Controller Device SAN Volume Controller Device SAN Volume Controller Device SAN Volume Controller Device SAN Volume Controller Device SAN Volume Controller Device SAN Volume Controller Device SAN Volume Controller Device SAN Volume Controller Device SAN Volume Controller Device SAN Volume Controller Device SAN Volume Controller Device SAN Volume Controller Device SAN Volume Controller Device SAN Volume Controller Device SAN Volume Controller Device Data Path Optimizer Pseudo Device Data Path Optimizer Pseudo Device Data Path Optimizer Pseudo Device Data Path Optimizer Pseudo Device
Driver Driver Driver Driver
To make a volumegroup (for example, itsoaixvg) to host the vpath1 device, we use the mkvg command passing the vpath device as a parameter instead of the hdisk device. This is shown in Example 8-13. Example 8-13 Running the mkvg command
#mkvg -y itsoaixvg vpath1 0516-1254 mkvg: Changing the PVID in the ODM. itsoaixvg
220
Implementing the IBM System Storage SAN Volume Controller V4.3
Now, by running the lspv command, we can see that vpath1 has been assigned into the itsoaixvg volume group, as shown in Example 8-14. Example 8-14 Showing the vpath assignment into the volume group
#lspv hdisk0 hdisk1 hdisk2 vpath1
0009cddaea97bf61 0009cdda43c9dfd5 0009cddabaef1d99 0009cddabce27ba5
rootvg rootvg rootvg itsoaixvg
active active active active
The lsvpcfg command also displays the new relationship between vpath1 and the itsoaixvg volume group, but also shows each hdisk associated to vpath1, as shown in Example 8-15. Example 8-15 Displaying the vpath to hdisk to volume group relationship #lsvpcfg vpath0 (Avail vpath1 (Avail (Avail ) vpath2 (Avail vpath3 (Avail
) 60050768018301BF2800000000000015 = hdisk3 (Avail ) hdisk7 (Avail ) pv itsoaixvg) 60050768018301BF2800000000000016 = hdisk4 (Avail ) hdisk8 ) 60050768018301BF2800000000000017 = hdisk5 (Avail ) hdisk9 (Avail ) ) 60050768018301BF2800000000000018 = hdisk6 (Avail ) hdisk10 (Avail )
In Example 8-16, running the command lspv vpath1 shows a more verbose output for vpath1. Example 8-16 Verbose details of vpath1
#lspv vpath1 PHYSICAL VOLUME: vpath1 VOLUME GROUP: PV IDENTIFIER: 0009cddabce27ba5 VG IDENTIFIER 0009cdda00004c000000011abce27c89 PV STATE: active STALE PARTITIONS: 0 ALLOCATABLE: PP SIZE: 8 megabyte(s) LOGICAL VOLUMES: TOTAL PPs: 639 (5112 megabytes) VG DESCRIPTORS: FREE PPs: 639 (5112 megabytes) HOT SPARE: USED PPs: 0 (0 megabytes) MAX REQUEST: FREE DISTRIBUTION: 128..128..127..128..128 USED DISTRIBUTION: 00..00..00..00..00
itsoaixvg
yes 0 2 no 256 kilobytes
Chapter 8. Host configuration
221
8.2.7 Using SDD Within SDD, we are able to check the status of the adapters and devices now under SDD control with the use of the datapath command set. In Example 8-17, we can see the status of both HBA cards as NORMAL and ACTIVE. Example 8-17 SDD commands used to check the availability of the adapters
#datapath query adapter Active Adapters :2 Adpt# 0 1
Name State fscsi0 NORMAL fscsi1 NORMAL
Mode ACTIVE ACTIVE
Select 0 56
Errors 0 0
Paths 4 4
Active 1 1
In Example 8-18, we see detailed information about each vpath device. Initially, we see that vpath1 is the only vpath device in an open status. This is because it is the only vpath currently assigned to a volume group. Additionally, for vpath1, we see that only path #1 and path #3 have been selected (used) by SDD. This is because these are the two physical paths that connect to the preferred node of the I/O group of this SVC cluster. The remaining two paths within this vpath device are only accessed in a failover scenario. Example 8-18 SDD commands used to check the availability of the devices
#datapath query device Total Devices : 4
DEV#: 0 DEVICE NAME: vpath0 TYPE: 2145 POLICY: Optimized SERIAL: 60050768018301BF2800000000000015 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk3 CLOSE NORMAL 0 0 1 fscsi1/hdisk7 CLOSE NORMAL 0 0 2 fscsi0/hdisk11 CLOSE NORMAL 0 0 3 fscsi1/hdisk15 CLOSE NORMAL 0 0 DEV#: 1 DEVICE NAME: vpath1 TYPE: 2145 POLICY: Optimized SERIAL: 60050768018301BF2800000000000016 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk4 OPEN NORMAL 0 0 1 fscsi1/hdisk8 OPEN NORMAL 28 0 2 fscsi0/hdisk12 OPEN NORMAL 32 0 3 fscsi1/hdisk16 OPEN NORMAL 0 0 DEV#: 2 DEVICE NAME: vpath2 TYPE: 2145 POLICY: Optimized SERIAL: 60050768018301BF2800000000000017 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk5 CLOSE NORMAL 0 0 1 fscsi1/hdisk9 CLOSE NORMAL 0 0 2 fscsi0/hdisk13 CLOSE NORMAL 0 0 3 fscsi1/hdisk17 CLOSE NORMAL 0 0 222
Implementing the IBM System Storage SAN Volume Controller V4.3
DEV#: 3 DEVICE NAME: vpath3 TYPE: 2145 POLICY: Optimized SERIAL: 60050768018301BF2800000000000018 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk6 CLOSE NORMAL 0 0 1 fscsi1/hdisk10 CLOSE NORMAL 0 0 2 fscsi0/hdisk14 CLOSE NORMAL 0 0 3 fscsi1/hdisk18 CLOSE NORMAL 0 0
8.2.8 Creating and preparing volumes for use with AIX 5L V5.3 and SDD The volume group itsoaixvg is created using vpath1, A logical volume is created using the volume group and then the file system created, testlv1, and mounted on the mount point /testlv1, as seen in Example 8-19. Example 8-19 Host system new volume group and file system configuration
#lsvg -o itsoaixvg rootvg #lsvg -l itsoaixvg itsoaixvg: LV NAME TYPE loglv01 jfs2log fslv00 jfs2 fslv01 jfs2 #df -g Filesystem GB blocks /dev/hd4 0.03 /dev/hd2 9.06 /dev/hd9var 0.03 /dev/hd3 0.12 /dev/hd1 0.03 /proc /dev/hd10opt 0.09 /dev/lv00 0.41 /dev/fslv00 2.00 /dev/fslv01 2.00
LPs 1 128 128
PPs 1 128 128
Free %Used 0.01 62% 4.32 53% 0.03 10% 0.12 7% 0.03 2% 0.01 86% 0.39 4% 2.00 1% 2.00 1%
PVs 1 1 1
LV STATE open/syncd open/syncd open/syncd
MOUNT POINT N/A /teslv1 /teslv2
Iused %Iused Mounted on 1357 31% / 17341 2% /usr 137 3% /var 31 1% /tmp 11 1% /home - /proc 1947 38% /opt 19 1% /usr/sys/inst.images 4 1% /teslv1 4 1% /teslv2
8.2.9 Discovering the assigned VDisk using AIX V6.1 and SDDPCM Before adding a new volume from the SVC, the AIX host system “Atlantic” had a “vanilla” configuration, as shown in Example 8-20. Example 8-20 Status of AIX host system ‘Kanaga’
# lspv hdisk0 hdisk1 hdisk2 # lsvg rootvg
0009cdcaeb48d3a3 0009cdcac26dbb7c 0009cdcab5657239
rootvg rootvg rootvg
active active active
Chapter 8. Host configuration
223
In Example 8-22 on page 225, we show SVC configuration information relating to our AIX host, specifically the host definition, the VDisks created for this host, and the VDisk-to-host mappings for this configuration. Our example host is named “Atlantic”. Example 8-21 shows the HBA information of our example host. Example 8-21 HBA information example host “Atlantic”
## lsdev -Cc adapter | grep fcs fcs1 Available 1H-08 FC Adapter fcs2 Available 1D-08 FC Adapter # lscfg -vpl fcs1 fcs1 U0.1-P2-I4/Q1 FC Adapter Part Number.................00P4494 EC Level....................A Serial Number...............1E3120A644 Manufacturer................001E Customer Card ID Number.....2765 FRU Number.................. 00P4495 Network Address.............10000000C932A865 ROS Level and ID............02C039D0 Device Specific.(Z0)........2002606D Device Specific.(Z1)........00000000 Device Specific.(Z2)........00000000 Device Specific.(Z3)........03000909 Device Specific.(Z4)........FF401411 Device Specific.(Z5)........02C039D0 Device Specific.(Z6)........064339D0 Device Specific.(Z7)........074339D0 Device Specific.(Z8)........20000000C932A865 Device Specific.(Z9)........CS3.93A0 Device Specific.(ZA)........C1D3.93A0 Device Specific.(ZB)........C2D3.93A0 Device Specific.(ZC)........00000000 Hardware Location Code......U0.1-P2-I4/Q1
PLATFORM SPECIFIC Name: fibre-channel Model: LP9002 Node: fibre-channel@1 Device Type: fcp Physical Location: U0.1-P2-I4/Q1 ## lscfg -vpl fcs2 fcs2 U0.1-P2-I5/Q1 FC Adapter Part Number.................80P4383 EC Level....................A Serial Number...............1F5350CD42 Manufacturer................001F Customer Card ID Number.....2765 FRU Number.................. 80P4384 Network Address.............10000000C94C8C1C
224
Implementing the IBM System Storage SAN Volume Controller V4.3
ROS Level and ID............02C03951 Device Specific.(Z0)........2002606D Device Specific.(Z1)........00000000 Device Specific.(Z2)........00000000 Device Specific.(Z3)........03000909 Device Specific.(Z4)........FF401210 Device Specific.(Z5)........02C03951 Device Specific.(Z6)........06433951 Device Specific.(Z7)........07433951 Device Specific.(Z8)........20000000C94C8C1C Device Specific.(Z9)........CS3.91A1 Device Specific.(ZA)........C1D3.91A1 Device Specific.(ZB)........C2D3.91A1 Device Specific.(ZC)........00000000 Hardware Location Code......U0.1-P2-I5/Q1
PLATFORM SPECIFIC Name: fibre-channel Model: LP9002 Node: fibre-channel@1 Device Type: fcp Physical Location: U0.1-P2-I5/Q1 # Using the SVC CLI, we can check that the host WWPNs, as listed in Example 8-22, are logged into the SVC for the host definition “Atlantic”, by entering: svcinfo lshost Atlantic We can also find the serial numbers of the VDisks using the following command: svcinfo lshostvdiskmap Atlantic Example 8-22 SVC definitions for host system “Atlantic”
IBM_2145:ITSO-CLS2:admin>svcinfo lshost Atlantic id 8 name Atlantic port_count 2 type generic mask 1111 iogrp_count 4 WWPN 10000000C94C8C1C node_logged_in_count 2 state active WWPN 10000000C932A865 node_logged_in_count 2 state active IBM_2145:ITSO-CLS2:admin>svcinfo lshostvdiskmap Atlantic id name SCSI_id vdisk_id wwpn vdisk_UID 8 Atlantic 0 14 10000000C94C8C1C 6005076801A180E90800000000000060 8 Atlantic 1 22 10000000C94C8C1C 6005076801A180E90800000000000061
vdisk_name Atlantic0001 Atlantic0002
Chapter 8. Host configuration
225
8 Atlantic 2 23 10000000C94C8C1C 6005076801A180E90800000000000062 IBM_2145:ITSO-CLS2:admin>
Atlantic0003
We need to run cfgmgr on the AIX host to discover the new disks and enable us to use the disks: # cfgmgr -l fcs1 # cfgmgr -l fcs2 Alternatively, use the cfgmgr -vS command to check the complete system. This command will probe the devices sequentially across all FC adapters and attached disks; however, it is very time intensive: # cfgmgr -vS The raw SVC disk configuration of the AIX host system now appears as shown in Example 8-23. We can see the multiple MPIO FC 2145 devices, representing the SVC LUN. Example 8-23 VDisks from SVC added with multiple different paths for each VDisk
# lsdev -Cc disk hdisk0 Available hdisk1 Available hdisk2 Available hdisk3 Available hdisk4 Available hdisk5 Available
1S-08-00-8,0 1S-08-00-9,0 1S-08-00-10,0 1D-08-02 1D-08-02 1D-08-02
16 Bit LVD SCSI Disk Drive 16 Bit LVD SCSI Disk Drive 16 Bit LVD SCSI Disk Drive MPIO FC 2145 MPIO FC 2145 MPIO FC 2145
To make a volumegroup (for example, itsoaixvg) to host the LUNs, we use the mkvg command passing the device as a parameter. This is shown in Example 8-24. Example 8-24 Running the mkvg command
# mkvg -y itsoaixvg hdisk3 0516-1254 mkvg: Changing the PVID in the ODM. itsoaixvg # mkvg -y itsoaixvg1 hdisk4 0516-1254 mkvg: Changing the PVID in the ODM. itsoaixvg1 # mkvg -y itsoaixvg2 hdisk5 0516-1254 mkvg: Changing the PVID in the ODM. itsoaixvg2 Now, by running the lspv command, we can see the disks and the assigned volume groups, as shown in Example 8-25. Example 8-25 Showing the vpath assignment into the volume group
# lspv hdisk0 hdisk1 hdisk2 hdisk3 hdisk4 hdisk5
226
0009cdcaeb48d3a3 0009cdcac26dbb7c 0009cdcab5657239 0009cdca28b589f5 0009cdca28b87866 0009cdca28b8ad5b
Implementing the IBM System Storage SAN Volume Controller V4.3
rootvg rootvg rootvg itsoaixvg itsoaixvg1 itsoaixvg2
active active active active active active
In Example 8-26, we show that running the command lspv hdisk3 shows a more verbose output for one of the SVC LUNs. Example 8-26 Verbose details of vpath1
# lspv hdisk3 PHYSICAL VOLUME: hdisk3 VOLUME GROUP: PV IDENTIFIER: 0009cdca28b589f5 VG IDENTIFIER 0009cdca00004c000000011b28b58ae2 PV STATE: active STALE PARTITIONS: 0 ALLOCATABLE: PP SIZE: 8 megabyte(s) LOGICAL VOLUMES: TOTAL PPs: 511 (4088 megabytes) VG DESCRIPTORS: FREE PPs: 511 (4088 megabytes) HOT SPARE: USED PPs: 0 (0 megabytes) MAX REQUEST: FREE DISTRIBUTION: 103..102..102..102..102 USED DISTRIBUTION: 00..00..00..00..00 #
itsoaixvg
yes 0 2 no 256 kilobytes
8.2.10 Using SDDPCM Within SDD, we are able to check the status of the adapters and devices now under SDDPCM control with the use of the pcmpath command set. In Example 8-27, we can see the status of both HBA cards as NORMAL and ACTIVE. Example 8-27 SDDPCM commands used to check the availability of the adapters
# pcmpath query adapter Active Adapters :2 Adpt# 0 1
Name fscsi1 fscsi2
State NORMAL NORMAL
Mode ACTIVE ACTIVE
Select 407 425
Errors 0 0
Paths 6 6
Active 6 6
From Example 8-28, we see detailed information about each MPIO device. The * next to the path numbers show which paths have been selected (used) by SDDPCM. This is because these are the two physical paths that connect to the preferred node of the I/O group of this SVC cluster. The remaining two paths within this MPIO device are only accessed in a failover scenario. Example 8-28 SDDPCM commands used to check the availability of the devices
# pcmpath query device Total Devices : 3
DEV#: 3 DEVICE NAME: hdisk3 TYPE: 2145 ALGORITHM: Load Balance SERIAL: 6005076801A180E90800000000000060 ========================================================================== Path# Adapter/Path Name State Mode Select Errors 0 fscsi1/path0 OPEN NORMAL 152 0 1* fscsi1/path1 OPEN NORMAL 48 0 2* fscsi2/path2 OPEN NORMAL 48 0 Chapter 8. Host configuration
227
3
fscsi2/path3
OPEN
NORMAL
160
0
DEV#: 4 DEVICE NAME: hdisk4 TYPE: 2145 ALGORITHM: Load Balance SERIAL: 6005076801A180E90800000000000061 ========================================================================== Path# Adapter/Path Name State Mode Select Errors 0* fscsi1/path0 OPEN NORMAL 37 0 1 fscsi1/path1 OPEN NORMAL 66 0 2 fscsi2/path2 OPEN NORMAL 71 0 3* fscsi2/path3 OPEN NORMAL 38 0 DEV#: 5 DEVICE NAME: hdisk5 TYPE: 2145 ALGORITHM: Load Balance SERIAL: 6005076801A180E90800000000000062 ========================================================================== Path# Adapter/Path Name State Mode Select Errors 0 fscsi1/path0 OPEN NORMAL 66 0 1* fscsi1/path1 OPEN NORMAL 38 0 2* fscsi2/path2 OPEN NORMAL 38 0 3 fscsi2/path3 OPEN NORMAL 70 0 #
8.2.11 Creating and preparing volumes for use with AIX V6.1 and SDDPCM The volume group itsoaixvg is created using hdisk3, A logical volume is created using the volume group and then the file system is created, testlv1, and mounted on the mount point /testlv1, as seen in Example 8-29. Example 8-29 Host system new volume group and file system configuration
# lsvg -o itsoaixvg2 itsoaixvg1 itsoaixvg rootvg # crfs -v jfs2 -g itsoaixvg -a size=3G File system created successfully. 3145428 kilobytes total disk space. New File System size is 6291456 # lsvg -l itsoaixvg itsoaixvg: LV NAME TYPE LPs loglv00 jfs2log 1 fslv00 jfs2 384 #
-m /itsoaixvg -p rw -a agblksize=4096
PPs 1 384
PVs 1 1
LV STATE closed/syncd closed/syncd
MOUNT POINT N/A /itsoaixvg
8.2.12 Expanding an AIX volume It is possible to expand a VDisk in the SVC cluster, even if it is mapped to a host. Some operating systems such as AIX 5L Version 5.2 and higher versions can handle the volumes being expanded, even if the host has applications running. In the examples below, we show the procedure with AIX 5L V5.3 and SDD, but the procedure is the also the same using AIX V6 or SDDPCM. The volume group where the VDisk is assigned, if it is assigned to any, must not be a concurrent accessible volumegroup. A VDisk that is defined in a FlashCopy, Metro Mirror, or Global Mirror mapping on the SVC cannot be expanded, unless the mapping is 228
Implementing the IBM System Storage SAN Volume Controller V4.3
removed, which means the FlashCopy, Metro Mirror, or Global Mirror on that VDisk has to be stopped before it is possible to expand the VDisk. The following steps show how to expand a volume on an AIX host, where the volume is a VDisk from the SVC: 1. To list a VDisk size, use the command svcinfo lsvdisk . Example 8-30 shows the VDisk Kanga0002 that we have allocated to our AIX server before we expand it. Here, the capacity is 5 GB, and the vdisk_UID is 60050768018301BF2800000000000016. Example 8-30 Expanding a VDisk on AIX
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk Kanaga0002 id 14 name Kanaga0002 IO_group_id 0 IO_group_name io_grp0 status online mdisk_grp_id 0 mdisk_grp_name MDG_DS45 capacity 5.0GB type striped formatted yes mdisk_id mdisk_name FC_id FC_name RC_id RC_name vdisk_UID 60050768018301BF2800000000000016 throttling 0 preferred_node_id 2 fast_write_state not_empty cache readwrite udid 0 fc_map_count 0 sync_rate 50 copy_count 1 copy_id 0 status online sync yes primary yes mdisk_grp_id 0 mdisk_grp_name MDG_DS45 type striped mdisk_id mdisk_name fast_write_state empty used_capacity 5.00GB real_capacity 5.00GB free_capacity 0.00MB overallocation 100 autoexpand warning grainsize
Chapter 8. Host configuration
229
2. To identify which vpath this VDisk is associated to on the AIX host, we use the SDD command datapath query device, as shown in Example 8-19 on page 223. Here we can see that the VDisk with vdisk_UID 60050768018301BF2800000000000016 is associated to vpath1 as the vdisk_UID matches the SERIAL field on the AIX host. 3. To see the size of the volume on the AIX host, we use the lspv command, as shown in Example 8-31. This shows that the volume size is 5112 MB, equal to 5 GB, as shown in Example 8-30 on page 229. Example 8-31 Finding the size of the volume in AIX
#lspv vpath1 PHYSICAL VOLUME: vpath1 VOLUME GROUP: PV IDENTIFIER: 0009cddabce27ba5 VG IDENTIFIER 0009cdda00004c000000011abce27c89 PV STATE: active STALE PARTITIONS: 0 ALLOCATABLE: PP SIZE: 8 megabyte(s) LOGICAL VOLUMES: TOTAL PPs: 639 (5112 megabytes) VG DESCRIPTORS: FREE PPs: 0 (0 megabytes) HOT SPARE: USED PPs: 639 (5112 megabytes) MAX REQUEST: FREE DISTRIBUTION: 00..00..00..00..00 USED DISTRIBUTION: 128..128..127..128..128
itsoaixvg
yes 2 2 no 256 kilobytes
4. To expand the volume on the SVC, we use the command svctask expandvdisksize to increase the capacity on the VDisk. In Example 8-32, we expand the VDisk by 1GB. Example 8-32 Expanding a VDisk
IBM_2145:ITSO-CLS1:admin>svctask expandvdisksize -size 1 -unit gb Kanaga0002 5. To check that the VDisk has been expanded, use the svcinfo lsvdisk command. Here we can see that the VDisk Kanaga0001 has been expanded to 6 GB in capacity (Example 8-33). Example 8-33 Verifying that the VDisk has been expanded
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk Kanaga0002 id 14 name Kanaga0002 IO_group_id 0 IO_group_name io_grp0 status online mdisk_grp_id 0 mdisk_grp_name MDG_DS45 capacity 6.0GB type striped formatted yes mdisk_id mdisk_name FC_id FC_name RC_id RC_name vdisk_UID 60050768018301BF2800000000000016 throttling 0
230
Implementing the IBM System Storage SAN Volume Controller V4.3
preferred_node_id 2 fast_write_state empty cache readwrite udid 0 fc_map_count 0 sync_rate 50 copy_count 1 copy_id 0 status online sync yes primary yes mdisk_grp_id 0 mdisk_grp_name MDG_DS45 type striped mdisk_id mdisk_name fast_write_state empty used_capacity 6.00GB real_capacity 6.00GB free_capacity 0.00MB overallocation 100 autoexpand warning grainsize 6. AIX has not yet recognized a change in the capacity of the vpath1 volume, because no dynamic mechanism exists within the operating system to provide a configuration update communication. Therefore, to encourage AIX to recognize the extra capacity on the volume without stopping any applications, we use the chvg -g fc_source_vg command, where fc_source_vg is the name of the volumegroup which vpath1 belongs to. If AIX does not return anything, this means that the command was successful and the volume changes in this volume group have been saved. If AIX cannot see any changes in the volumes, it will return a message indicating this. 7. To verify that the size of vpath0 has changed, we use the lspv command again, as shown in Example 8-34. Example 8-34 Verify that AIX can see the newly expanded VDisk
#lspv vpath1 PHYSICAL VOLUME: vpath1 VOLUME GROUP: PV IDENTIFIER: 0009cddabce27ba5 VG IDENTIFIER 0009cdda00004c000000011abce27c89 PV STATE: active STALE PARTITIONS: 0 ALLOCATABLE: PP SIZE: 8 megabyte(s) LOGICAL VOLUMES: TOTAL PPs: 767 (6136 megabytes) VG DESCRIPTORS: FREE PPs: 128 (1024 megabytes) HOT SPARE: USED PPs: 639 (5112 megabytes) MAX REQUEST: FREE DISTRIBUTION: 00..00..00..00..128 USED DISTRIBUTION: 154..153..153..153..26
itsoaixvg
yes 2 2 no 256 kilobytes
Chapter 8. Host configuration
231
Here we can see that the volume now has a size of 6136 MB, equal to 6 GB. After this we can expand the file systems in this volumegroup to use the new capacity.
8.2.13 Removing an SVC volume on AIX Before we remove a VDisk assigned to an AIX host, we have to make sure that there is no data on it, and that no applications are dependent upon the volume. This is a standard AIX procedure. We move all data off the volume, remove the volume in the volumegroup, and delete the vpath and the hdisks associated to the vpath. Then we remove the vdiskhostmap on the SVC for that volume, and that VDisk is no longer needed. Then we delete it so the extents will be available when we create a new VDisk on the SVC.
8.2.14 Running SVC commands from an AIX host system To issue CLI commands, you must install and prepare the SSH client system on the AIX host system. For AIX 5L V5.1 and later, you can get OpenSSH from the Bonus Packs. You also need its prerequisite, OpenSSL, from the AIX toolbox for Linux applications for Power Systems™. For AIX V4.3.3, the software is available from the AIX toolbox for Linux applications. The AIX installation images from IBM developerWorks® are available at this Web site: http://sourceforge.net/projects/openssh-aix Do the following steps: 1. To generate the key files on AIX, issue the following command: ssh-keygen -t rsa -f filename The -t parameter specifies the type of key to generate: rsa1, rsa2 or dsa. The value for rsa2 is just rsa, while for rsa1 the type needs to be rsa1. When creating the key to the SVC, use type rsa2. The -f parameter specifies the file names of the private and public keys on the AIX server (the public key gets the extension .pub after the file name). 2. Next, you have to install the public key on the SVC, which can be done by using the master console. Copy the public key to the master console, and install the key to the SVC, as described in Chapter 5, “SVC Console” on page 93. 3. On the AIX server, make sure that the private key and the public key is in the .ssh directory, and in the home directory of the user. 4. To connect to the SVC and use a CLI session from the AIX host, issue the following command: ssh -l admin -i filename svc 5. You can also issue the commands directly on the AIX host, and this is useful when making scripts. To do this, add the SVC commands to the previous command. For example, to list the hosts defined on the SVC, enter the following command: ssh -l admin -i filename svc svcinfo lshost In this command, -l admin is the user on the SVC we will connect to, -i filename is the filename of the private key generated, and svc is the name or IP address of the SVC.
232
Implementing the IBM System Storage SAN Volume Controller V4.3
8.3 Windows-specific information In the following sections, we detail specific information about the connection of Windows 2000 based hosts to the SVC environment.
8.3.1 Configuring Windows 2000, Windows 2003, and Windows 2008 hosts This section provides an overview of the requirements for attaching the SVC to a host running the Windows 2000 Server, Windows 2003 Server, or Windows 2008 Server. Before you attach the SVC to your host, make sure that all requirements listed below are fulfilled: For Windows Server 2003 x64 Edition operating system, you must install the Hotfix from KB 908980. If you do not install it before operation, preferred pathing is not available. You can find the Hotfix at: http://support.microsoft.com/kb/908980 Check LUN limitations for your host system. Ensure that there are enough Fibre Channel adapters installed in the server to handle the total LUNs you want to attach.
8.3.2 Configuring Windows To configure the Windows hosts, follow these steps: 1. Make sure that the latest OS Hotfixes are applied to your Microsoft server. 2. Use the latest firmware and driver levels on your host system. 3. Install the HBA or HBAs on the Windows server, as shown in 8.3.4, “Host adapter installation and configuration” on page 234. 4. Connect the Windows 2000/2003/2008 server FC Host adapters to the switches, as shown in Figure 8-1 on page 210. 5. Configure the switches (zoning). 6. Install the FC Host adapter driver, as described in 8.3.3, “Hardware lists, device driver, HBAs and firmware levels” on page 233. 7. Configure the HBA for hosts running Windows, as described in 8.3.4, “Host adapter installation and configuration” on page 234. 8. Check the HBA driver readme for the required Windows registry settings, as described in 8.3.3, “Hardware lists, device driver, HBAs and firmware levels” on page 233. 9. Check the disk timeout on Microsoft Windows Server, as described in 8.3.5, “Changing the disk timeout on Microsoft Windows Server” on page 236. 10.Install and configure SDD/SDDDSM. 11.Restart the Windows 2000/2003/2008 host system. 12.Configure the host, VDisks, and host mapping in the SVC. 13.Use Rescan disk in Computer Management of the Windows server to discover the VDisks created on the SAN Volume Controller.
8.3.3 Hardware lists, device driver, HBAs and firmware levels The latest information about supported hardware, device driver, and firmware is available at: http://www-1.ibm.com/support/docview.wss?rs=591&uid=ssg1S1003277#_Windows
Chapter 8. Host configuration
233
There you will also find the hardware list for supported host bus adapters and the driver levels for Windows. Check the supported firmware and driver level for your host bus adapter and follow the manufacturers instructions to upgrade the firmware and driver levels for each type of HBA. In most manufacturers’ driver readmes, you will find instructions for the Windows registry parameters that have to be set for the HBA driver. For the Emulex HBA driver, SDD requires the port driver, not the miniport port driver. For the QLogic HBA driver, SDDDSM requires the storport version of the miniport driver. For the QLogic HBA driver, SDD requires the scsiport version of the miniport driver.
8.3.4 Host adapter installation and configuration Install the host adapter(s) into your system. Refer to the manufacturer’s instructions for installation and configuration of the HBAs. In IBM System x servers, the HBA should always be installed in the first slots. This means that if you install, for example, two HBAs and two network cards, the HBAs should be installed in slot 1 and slot 2, and the network cards can be installed in the remaining slots.
Configure the QLogic HBA for hosts running Windows After you have installed the HBA in the server, and have applied the HBA firmware and device driver, you have to configure the HBA. To do this, perform the following steps: 1. Restart the server. 2. When you see the QLogic banner, press the Ctrl-Q keys to open the FAST!UTIL menu panel. 3. From the Select Host Adapter menu, select the Adapter Type QLA2xxx. 4. From the Fast!UTIL Options menu, select Configuration Settings. 5. From the Configuration Settings menu, click Host Adapter Settings. 6. From the Host Adapter Settings menu, select the following values: a. Host Adapter BIOS: Disabled b. Frame size: 2048 c. Loop Reset Delay: 5 (minimum) d. Adapter Hard Loop ID: Disabled e. Hard Loop ID: 0 f. Spinup Delay: Disabled g. Connection Options: 1 - point to point only h. Fibre Channel Tape Support: Disabled i. Data Rate: 2 7. Press the Esc key to return to the Configuration Settings menu. 8. From the Configuration Settings menu, select Advanced Adapter Settings. 9. From the Advanced Adapter Settings menu, set the following parameters: a. Execution throttle: 100 b. Luns per Target: 0 c. Enable LIP Reset: No d. Enable LIP Full Login: Yes
234
Implementing the IBM System Storage SAN Volume Controller V4.3
e. Enable Target Reset: No Note: If you are using subsystem device driver (SDD) lower than 1.6, set Enable Target Reset to Yes. f. Login Retry Count: 30 g. Port Down Retry Count: 15 h. Link Down Timeout: 30 i. Extended error logging: Disabled (might be enabled for debugging) j. RIO Operation Mode: 0 k. Interrupt Delay Timer: 0 10.Press Esc to return to the Configuration Settings menu. 11.Press Esc. 12.From the Configuration settings modified window, select Save changes. 13.From the Fast!UTIL Options menu, select Select Host Adapter if more than one QLogic adapter was installed in your system. 14.Select the other Host Adapter and repeat all steps from point 4 to 12. 15.You have to repeat this for all installed Qlogic adapters in your system. When you are done press “Esc” to exit the Qlogic BIOS and restart the server.
Configuring the Emulex HBA for hosts running Windows After you have installed the Emulex HBA and driver, you must configure your HBA. For the Emulex HBA StorPort driver, accept the default settings and set topology to 1 (1 = F Port Fabric). For the Emulex HBA FC Port driver, use the default settings and change the parameters to those given in Table 8-1. Table 8-1 FC Port driver changes Parameters
Recommended Settings
Query name server for all N-ports (BrokenRSCN)
Enabled
LUN mapping (MapLuns)
Enabled (1)
Automatic LUN mapping (MapLuns)
Enabled (1)
Allow multiple paths to SCSI target (MultipleSCSIClaims)
Enabled
Scan in device ID order (ScanDeviceIDOrder)
Disabled
Translate queue full to busy (TransleteQueueFull)
Enabled
Retry timer (RetryTimer)
2000 milliseconds
Maximum number of LUNs (MaximumLun)
Equal or greater than the number of the SVC LUNs that are available to the HBA
Note: The parameters shown correspond to the parameters in HBAnywhere.
Chapter 8. Host configuration
235
8.3.5 Changing the disk timeout on Microsoft Windows Server The section describes how to change the disk I/O timeout value on Windows 2000, 2003, and 2008 Server operating systems. On your Windows Server hosts, change the disk I/O timeout value to 60 in the Windows registry, as follows: 1. In Windows, click the Start button and select Run. 2. In the dialog text box, type regedit and press Enter. 3. In the registry browsing tool, locate the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Disk\TimeOutValue key. 4. Confirm that the value for the key is 60 (decimal value) and, if necessary, change the value to 60, as shown in Figure 8-4.
Figure 8-4 Regedit
8.3.6 SDD driver installation on Windows At the time of writing, the SDD levels in Table 8-2 are supported. Table 8-2 Currently supported SDD levels Windows operating system
SDD level
NT 4
1.5.1.1
2000 / 2003 SP2 (32-bit) / 2003 SP2 (IA-64)
1.6.3.0-2
2000 with MSCS and Veritas Volume Manager / 2003 SP2 (32-bit) with MSCS and Veritas Volume Manager
Not available
See the following Web site for the latest information about SDD for Windows: http://www-1.ibm.com/support/docview.wss?rs=540&context=ST52G7&dc=DA400&uid=ssg1S7 001350&loc=en_US&cs=utf-8&lang=en Note: We recommend that you use SDD only on existing systems where you do not want to change from SDD to SDDDSM. New operating systems will only be supported with SDDDSM.
236
Implementing the IBM System Storage SAN Volume Controller V4.3
Before installing the SDD driver, the HBA driver has to be installed on your system. SDD requires the HBA SCSI port driver. After downloading the appropriate version of SDD from the Web site, extract the file and run setup.exe to install SDD. A command line will appear. Answer “Y” (Figure 8-5) to install the driver.
Figure 8-5 Confirm SDD installation
After the setup has completed, answer “Y” again to reboot your system (Figure 8-6).
Figure 8-6 Reboot system after installation
To check if your SDD installation is complete, open the Windows Device Manager, expand SCSI and RAID Controllers, right-click Subsystem Device Driver Management, and click Properties (see Figure 8-7).
Figure 8-7 Subsystem Device Driver Management
Chapter 8. Host configuration
237
The Subsystem Device Driver Management Properties window will appear. Select the Driver tab and make sure that you have installed the correct driver version (see Figure 8-8).
Figure 8-8 Subsystem Device Driver Management Properties Driver tab
8.3.7 SDDDSM driver installation on Windows The following sections show how to install the SDDDSM driver on Windows.
Windows 2003, 2008, and MPIO Microsoft Multi Path Input Output (MPIO) solutions are designed to work in conjunction with device specific modules (DSMs) written by vendors, but the MPIO driver package does not, by itself, form a complete solution. This joint solution allows the storage vendors to design device specific solutions that are tightly integrated with the Windows operating system. MPIO is not shipped with the Windows operating system; storage vendors must pack the MPIO drivers with their own DSM. IBM Subsystem Device Driver DSM (SDDDSM) is the IBM multipath IO solution based on Microsoft MPIO technology; it is a device specific module specifically designed to support IBM storage devices on Windows 2003 and 2008 servers. The intention of MPIO is to get a better integration of multipath storage solution with the operating system, and allows the use of multipaths in the SAN infrastructure during the boot process for SAN boot hosts.
Subsystem Device Driver Device Specific Module (SDDDSM) for SVC Subsystem Device Driver Device Specific Module (SDDDSM) installation is a package for the SVC device for the Windows Server 2003 and 2008 operating systems. SDDDSM is the IBM multipath IO solution based on Microsoft MPIO technology, and it is a device specific module specifically designed to support IBM storage devices. Together with MPIO, it is designed to support the multipath configuration environments in the IBM System Storage SAN Volume Controller. It resides in a host system with the native disk device driver and provides the following functions: Enhanced data availability Dynamic I/O load-balancing across multiple paths Automatic path failover protection 238
Implementing the IBM System Storage SAN Volume Controller V4.3
Concurrent download of licensed internal code Path-selection policies for the host system No SDDDSM support for Windows 2000 For the HBA driver, SDDDSM requires the StorPort version of HBA miniport driver
Table 8-3 shows, at the time of writing, the supported SDDDSM driver levels. Table 8-3 Currently supported SDDDSM driver levels Windows operating system
SDD level
2003 SP2(32bit) / 2003 SP2(x64)
2.2.0.0-11
2008 (32bit) / 2008 (x64)
2.2.0.0-11
To check which levels are available, go to the Web site: http://www-1.ibm.com/support/docview.wss?rs=540&context=ST52G7&dc=DA400&uid=ssg1S7 001350&loc=en_US&cs=utf-8&lang=en#WindowsSDDDSM To download SDDDSM, go to the Web site: http://www-1.ibm.com/support/docview.wss?rs=540&context=ST52G7&dc=D430&uid=ssg1S40 00350&loc=en_US&cs=utf-8&lang=en The installation procedure for SDDDSM and SDD are the same, but remember that you have to use the StorPort HBA driver instead of the SCSI driver. The SDD installation is described in 8.3.6, “SDD driver installation on Windows” on page 236. After completing the installation, you will now see the Microsoft MPIO in device manager (Figure 8-9).
Figure 8-9 Windows Device Manager - MPIO
The SDDDSM installation for Windows 2008 is described in 8.5, “Example configuration of attaching an SVC to a Windows 2008 host” on page 249.
Chapter 8. Host configuration
239
8.4 Discovering the assigned VDisk in Windows 2000 / 2003 In this section, we describe how to discover assigned VDisks in Windows 2000 and Windows 2003. The screen captures will show a Windows 2003 host with SDDDSM installed, but discovering the disks in Windows 2000 or with SDD is the same procedure. Before adding a new volume from the SAN Volume Controller, the Windows 2003 host system had the configuration shown in Figure 8-10, with only local disks.
Figure 8-10 Windows 2003 host system before adding a new volume from SVC
We can check that the WWPN is logged into the SAN Volume Controller for the host “Senegal” by entering the following command (Example 8-35): svcinfo lshost Senegal Example 8-35 Host info - Senegal
IBM_2145:ITSO-CLS2:admin>svcinfo lshost Senegal id 1 name Senegal port_count 2 type generic mask 1111 iogrp_count 4 WWPN 210000E08B89B9C0 node_logged_in_count 2 state active WWPN 210000E08B89CCC2 node_logged_in_count 2 state active The configuration of the host “Senegal”, the VDisk “,Senegal_bas0001”, and the mapping between the host and the VDisk are defined in the SAN Volume Controller, as described in Example 8-36 on page 241. In our example, the VDisk “Senegal_bas0002 and Senegal_bas003” have the same configuration as VDisk “Senegal_bas0001”.
240
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 8-36 VDisk mapping - Senegal
IBM_2145:ITSO-CLS2:admin>svcinfo lshostvdiskmap Senegal id name SCSI_id vdisk_id vdisk_name wwpn vdisk_UID 1 Senegal 0 7 Senegal_bas0001 210000E08B89B9C0 6005076801A180E9080000000000000F 1 Senegal 1 8 Senegal_bas0002 210000E08B89B9C0 6005076801A180E90800000000000010 1 Senegal 2 9 Senegal_bas0003 210000E08B89B9C0 6005076801A180E90800000000000011 IBM_2145:ITSO-CLS2:admin>svcinfo lsvdisk Senegal_bas0001 id 7 name Senegal_bas0001 IO_group_id 0 IO_group_name io_grp0 status online mdisk_grp_id 0 mdisk_grp_name MDG_0_DS45 capacity 10.0GB type striped formatted yes mdisk_id mdisk_name FC_id FC_name RC_id RC_name vdisk_UID 6005076801A180E9080000000000000F throttling 0 preferred_node_id 3 fast_write_state empty cache readwrite udid 0 fc_map_count 0 sync_rate 50 copy_count 1 copy_id 0 status online sync yes primary yes mdisk_grp_id 0 mdisk_grp_name MDG_0_DS45 type striped mdisk_id mdisk_name fast_write_state empty used_capacity 10.00GB real_capacity 10.00GB free_capacity 0.00MB overallocation 100 autoexpand warning grainsize
Chapter 8. Host configuration
241
We can also find the serial number of the VDisks by entering the following command (Example 8-37): svcinfo lsvdiskhostmap Senegal_bas0001 Example 8-37 VDisk serial number - Senegal_bas0001
IBM_2145:ITSO-CLS2:admin>svcinfo lsvdiskhostmap Senegal_bas0001 id name SCSI_id host_id host_name wwpn vdisk_UID 7 Senegal_bas0001 0 1 Senegal 210000E08B89B9C0 6005076801A180E9080000000000000F 7 Senegal_bas0001 0 1 Senegal 210000E08B89CCC2 6005076801A180E9080000000000000F After installing the necessary drivers and the rescan disks operation completes, the new disks are found in the Computer Management window, as shown in Figure 8-11.
Figure 8-11 Windows 2003 host system with three new volumes from SVC
In Windows Device Manager, the disks are shown as IBM 2145 SCSI Disk Device (Figure 8-12 on page 243). The number of IBM 2145 SCSI Disk Devices that you see is equal to: (# of VDisks) x (# of paths per IO group per HBA) x (# of HBAs) The IBM 2145 Multi-Path Disk Devices are the devices created by the multipath driver (Figure 8-12 on page 243). The number of these devices are equal to the VDisks presented to the host.
242
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 8-12 Windows 2003 Device Manager with assigned VDisks
When following the SAN zoning recommendation, this gives us, for one VDisk and a host with two HBAs: (# of VDisk) x (# of paths per IO group per HBA) x (# of HBAs) = 1 x 2 x 2 = 4 paths You can check if all paths are available if you select Start → All Programs → Subsystem Device Driver (DSM) → Subsystem Device Driver (DSM). The SDD (DSM) command-line interface will appear. Enter the following command to see which paths are available to your system (Example 8-38): Example 8-38 Datapath query device
Microsoft Windows [Version 5.2.3790] (C) Copyright 1985-2003 Microsoft Corp. C:\Program Files\IBM\SDDDSM>datapath query device Total Devices : 3
DEV#: 0 DEVICE NAME: Disk1 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 6005076801A180E9080000000000002A ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port2 Bus0/Disk1 Part0 OPEN NORMAL 47 0 1 Scsi Port2 Bus0/Disk1 Part0 OPEN NORMAL 0 0 2 Scsi Port3 Bus0/Disk1 Part0 OPEN NORMAL 0 0 3 Scsi Port3 Bus0/Disk1 Part0 OPEN NORMAL 28 0 DEV#: 1 DEVICE NAME: Disk2 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 6005076801A180E90800000000000010 ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port2 Bus0/Disk2 Part0 OPEN NORMAL 0 0 1 Scsi Port2 Bus0/Disk2 Part0 OPEN NORMAL 162 0 2 Scsi Port3 Bus0/Disk2 Part0 OPEN NORMAL 155 0 Chapter 8. Host configuration
243
3
Scsi Port3 Bus0/Disk2 Part0
OPEN
NORMAL
0
0
DEV#: 2 DEVICE NAME: Disk3 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 6005076801A180E90800000000000011 ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port2 Bus0/Disk3 Part0 OPEN NORMAL 51 0 1 Scsi Port2 Bus0/Disk3 Part0 OPEN NORMAL 0 0 2 Scsi Port3 Bus0/Disk3 Part0 OPEN NORMAL 0 0 3 Scsi Port3 Bus0/Disk3 Part0 OPEN NORMAL 25 0 C:\Program Files\IBM\SDDDSM> Note: All Path States have to be “OPEN”. The path state can be “OPEN” or “CLOSE”. If one path is “CLOSE”, it means that the system is missing a path that it saw during start up. If you restart your system, the “CLOSE” paths are removed from this view.
8.4.1 Extending a Windows 2000 or 2003 volume It is possible to expand a VDisk in the SVC cluster, even if it is mapped to a host. Some operating systems, such as Windows 2000 and Windows 2003, can handle the volumes being expanded even if the host has applications running. A VDisk that is defined to be in a FlashCopy, Metro Mirror, or Global Mirror mapping on the SVC cannot be expanded unless the mapping is removed, which means the FlashCopy, Metro Mirror, or Global Mirror on that VDisk has to be stopped before it is possible to expand the VDisk. Important: For VDisk expansion to work on Windows 2000, apply Windows 2000 Hotfix Q327020, which is available from the Microsoft Knowledge Base at: http://support.microsoft.com/kb/327020 If you want to expand a logical drive in a extended partition in Windows 2003, apply the Hotfix from KB 841650, which is available from the Microsoft Knowledge Base at: http://support.microsoft.com/kb/841650/en-us Use the updated Diskpart version for Windows 2003, which is available from the Microsoft Knowledge Base at: http://support.microsoft.com/kb/923076/en-us If the volume is part of a Microsoft Cluster (MSCS), Microsoft recommends that you shut down all nodes except one, and that applications in the resource that use the volume that is going to be expanded are stopped before expanding the volume. Applications running in other resources can continue. After expanding the volume, start the application and the resource, and then restart the other nodes in the MSCS.
244
Implementing the IBM System Storage SAN Volume Controller V4.3
To expand a volume in use on Windows 2000 and Windows 2003, we used Diskpart. The Diskpart tool is part of Windows 2003; for other Windows versions, you can download it free of charge from Microsoft. Diskpart is a tool developed by Microsoft to ease administration of storage. It is a command-line interface where you can manage disks, partitions, and volumes, by using scripts or direct input on the command line. You can list disks and volumes, select them, and after selecting get more detailed information, create partitions, extend volumes, and more. For more information, see the Microsoft Web site: http://www.microsoft.com or http://support.microsoft.com/default.aspx?scid=kb;en-us;304736&sd=tech An example of how to expand a volume on a Windows 2003 host, where the volume is a VDisk from the SVC, is shown in the following discussion. To list a VDisk size, use the command svcinfo lsvdisk . This gives this information for the Senegal_bas0001 before expanding the VDisk (Example 8-36 on page 241). Here we can see that the capacity is 10 GB, and also what the vdisk_UID is. To find what vpath this VDisk is on the Windows 2003 host, we use the SDD command, datapath query device, on the Windows host (Figure 8-13). Here we can see that the Serial 6005076801A180E9080000000000000F of Disk1 on the Windows host (Figure 8-13) matches the vdisk ID of “Senegal_bas0001” (Example 8-36 on page 241). To see the size of the volume on the Windows host, we use Disk Manager, as shown in Figure 8-13.
Figure 8-13 Windows 2003 - Disk Management
Chapter 8. Host configuration
245
This shows that the volume size is 10 GB. To expand the volume on the SVC, we use the command svctask expandvdisksize to increase the capacity on the VDisk. In this example, we expand the VDisk by 1 GB (Example 8-39). Example 8-39 svctask expandvdisksize command
IBM_2145:ITSO-CLS2:admin>svctask expandvdisksize -size 1 -unit gb Senegal_bas0001 IBM_2145:ITSO-CLS2:admin>svcinfo lsvdisk Senegal_bas0001 id 7 name Senegal_bas0001 IO_group_id 0 IO_group_name io_grp0 status online mdisk_grp_id 0 mdisk_grp_name MDG_0_DS45 capacity 11.0GB type striped formatted yes mdisk_id mdisk_name FC_id FC_name RC_id RC_name vdisk_UID 6005076801A180E9080000000000000F throttling 0 preferred_node_id 3 fast_write_state empty cache readwrite udid 0 fc_map_count 0 sync_rate 50 copy_count 1 copy_id 0 status online sync yes primary yes mdisk_grp_id 0 mdisk_grp_name MDG_0_DS45 type striped mdisk_id mdisk_name fast_write_state empty used_capacity 11.00GB real_capacity 11.00GB free_capacity 0.00MB overallocation 100 autoexpand warning grainsize To check that the VDisk has been expanded, we use the command svctask expandvdisksize. In Example 8-39, we can see that the VDisk Senegal_bas0001 has been expanded to 11 GB in capacity. 246
Implementing the IBM System Storage SAN Volume Controller V4.3
After performing a “Disk Rescan” in Windows, you will see the new unallocated space in Windows Disk Management, as shown in Figure 8-14.
Figure 8-14 Expanded volume in Disk Manager
This shows that Disk1 now has 1 GB unallocated new capacity. To make this capacity available for the file system, use the following commands, as shown in Example 8-40. diskpart
Starts DiskPart in a DOS prompt.
list volume
Shows you all available volumes.
select volume
Selects the volume to expand.
detail volume
Displays details for the selected volume, including the unallocated capacity.
extend
Extends the volume to the available unallocated space.
Example 8-40 Using Diskpart
C:\>diskpart Microsoft DiskPart version 5.2.3790.3959 Copyright (C) 1999-2001 Microsoft Corporation. On computer: SENEGAL DISKPART> list volume Volume ### ---------Volume 0 Volume 1 Volume 2
Ltr --C S D
Label ----------SVC_Senegal
Fs ----NTFS NTFS
Type ---------Partition Partition DVD-ROM
Size ------75 GB 10 GB 0 B
Status --------Healthy Healthy Healthy
Info -------System
DISKPART> select volume 1 Volume 1 is the selected volume. DISKPART> detail volume
Chapter 8. Host configuration
247
Disk ### -------* Disk 1
Status ---------Online
Size ------11 GB
Free ------1020 MB
Dyn ---
Gpt ---
Readonly : No Hidden : No No Default Drive Letter: No Shadow Copy : No DISKPART> extend DiskPart successfully extended the volume. DISKPART> detail volume Disk ### -------* Disk 1
Status ---------Online
Size ------11 GB
Free ------0 B
Dyn ---
Gpt ---
Readonly : No Hidden : No No Default Drive Letter: No Shadow Copy : No After extending the volume, the command detail volume shows that there is no free capacity on the volume anymore. The list volume command shows the file system size. The disk management window also shows the new disk size, as shown in Figure 8-15.
Figure 8-15 Disk Management after extending disk
The example here is referred to as a Windows Basic Disk. Dynamic disks can be expanded by expanding the underlying SVC VDisk. The new space will appear as unallocated space at the end of the disk.
248
Implementing the IBM System Storage SAN Volume Controller V4.3
In this case, you do not need to use the DiskPart Tool, just Windows Disk Management functions to allocate the new space. Expansion works irrespective of the volume type (simple, spanned, mirrored, and so on) on the disk. Dynamic disks can be expanded without stopping I/O in most cases. Important: Never try to upgrade your Basic Disk to Dynamic Disk or vice versa without backing up your data, because this operation is disruptive for the data, due to a different position of the LBA on the disks.
8.5 Example configuration of attaching an SVC to a Windows 2008 host This section describes an example configuration that shows the attachment of a Windows 2008 host system to the SVC. More details about Windows 2008 and SVC are covered in 8.3, “Windows-specific information” on page 233.
8.5.1 Installing SDDDSM on a Windows 2008 host Download the HBA driver and the SDDDSM Package and copy it to your host system. Information about the recommended SDDDSM Package is listed in 8.3.7, “SDDDSM driver installation on Windows” on page 238. HBA driver details are listed in 8.3.3, “Hardware lists, device driver, HBAs and firmware levels” on page 233. We will perform the steps described in 8.3.2, “Configuring Windows” on page 233 to achieve this task. As a prerequisite for this example, we have already performed steps 1 to 5 for the hardware installation, SAN configuration is done, and hotfixes are applied. The Disk timeout value is set to 60 seconds (see 8.3.5, “Changing the disk timeout on Microsoft Windows Server” on page 236) and we will start with the driver installation.
Installing the HBA driver 1. Extract the Qlogic driver package to your hard drive. 2. Select Start → Run. 3. Enter devmgmt.msc, click OK, and the Device Manager will appear. 4. Expand the Storage Controllers.
Chapter 8. Host configuration
249
5. Right-click the HBA and select Update driver Software. (Figure 8-16).
Figure 8-16 Windows 2008 driver update
6. Click Browse my computer for driver software (Figure 8-17).
Figure 8-17 Windows 2008 driver update
7. Enter the path to the extracted QLogic driver and click Next (Figure 8-18 on page 251).
250
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 8-18 Windows 2008 driver update
8. Windows installs the driver (Figure 8-19).
Figure 8-19 Windows 2008 driver installation
Chapter 8. Host configuration
251
9. When the driver update is complete, click Close to exit the wizard (Figure 8-20).
Figure 8-20 Windows 2008 driver installation
10.Repeat steps 1 to 8 for all HBAs installed in the system.
8.5.2 Installing SDDDSM To install the SDDDSM driver on your system, perform the following steps: 1. Extract the SDDDSM driver package to a folder on your hard drive. 2. Open the folder with the extracted files. 3. Run setup.exe and a DOS command prompt will appear. 4. Type Y and press Enter to install SDDDSM (Figure 8-21).
Figure 8-21 Installing SDDDSM
5. After the SDDDSM Setup is finished, type Y and press Enter to restart your system. After the reboot, the SDDDSM installation is complete. You can check this in Device Manager, as the SDDDSM device will appear (Figure 8-22 on page 253), and the SDDDSM tools will have been installed (Figure 8-23 on page 253).
252
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 8-22 SDDDSM installation
Figure 8-23 SDDDSM installation
Chapter 8. Host configuration
253
8.5.3 Attaching SVC VDisks to Windows 2008 Create the VDisks on the SVC and map them to the Windows 2008 host. In this example, we have mapped three SVC disks to the Windows 2008 host named Diomede, as shown in Example 8-41. Example 8-41 SVC host mapping to host Diomede
IBM_2145:ITSO-CLS2:admin>svcinfo lshostvdiskmap Diomede id name SCSI_id vdisk_id vdisk_name wwpn 0 Diomede 0 20 Diomede_0001 210000E08B0541BC 6005076801A180E9080000000000002B 0 Diomede 1 21 Diomede_0002 210000E08B0541BC 6005076801A180E9080000000000002C 0 Diomede 2 22 Diomede_0003 210000E08B0541BC 6005076801A180E9080000000000002D
vdisk_UID
Perform the following steps to use the devices on your Windows 2008 host: 1. Click Start and Run. 2. Enter diskmgmt.msc and click OK and the Disk Management window will appear. 3. Select Action and click Rescan Disks (Figure 8-24).
Figure 8-24 Windows 2008 - Rescan disks
4. The SVC disks will now appear in the Disk Management window (Figure 8-25 on page 255).
254
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 8-25 Windows 2008 Disk Management window
After you have assigned the SVC disks, they are also available in Device Manager. The three assigned drives are represented by SDDDSM/MPIO as IBM-2145 Multipath disk devices in the Device Manager (Figure 8-26).
Figure 8-26 Windows 2008 Device Manager
Chapter 8. Host configuration
255
5. To check that the disks are available, select Start → All Programs → Subsystem Device Driver DSM and click Subsystem Device Driver DSM. (Figure 8-27). The SDDDSM Command Line Utility will appear.
Figure 8-27 Windows 2008 Subsystem Device Driver DSM utility
6. Enter datapath query device and press Enter (Example 8-42). This command will display all disks and the available paths, including their state. Example 8-42 Windows 2008 SDDDSM command-line utility
Microsoft Windows [Version 6.0.6001] Copyright (c) 2006 Microsoft Corporation.
All rights reserved.
C:\Program Files\IBM\SDDDSM>datapath query device Total Devices : 3
DEV#: 0 DEVICE NAME: Disk1 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 6005076801A180E9080000000000002B ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port2 Bus0/Disk1 Part0 OPEN NORMAL 0 0 1 Scsi Port2 Bus0/Disk1 Part0 OPEN NORMAL 1429 0 2 Scsi Port3 Bus0/Disk1 Part0 OPEN NORMAL 1456 0 3 Scsi Port3 Bus0/Disk1 Part0 OPEN NORMAL 0 0 DEV#: 1 DEVICE NAME: Disk2 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 6005076801A180E9080000000000002C ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port2 Bus0/Disk2 Part0 OPEN NORMAL 1520 0 1 Scsi Port2 Bus0/Disk2 Part0 OPEN NORMAL 0 0
256
Implementing the IBM System Storage SAN Volume Controller V4.3
2 3
Scsi Port3 Bus0/Disk2 Part0 Scsi Port3 Bus0/Disk2 Part0
OPEN OPEN
NORMAL NORMAL
0 1517
0 0
DEV#: 2 DEVICE NAME: Disk3 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 6005076801A180E9080000000000002D ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port2 Bus0/Disk3 Part0 OPEN NORMAL 27 0 1 Scsi Port2 Bus0/Disk3 Part0 OPEN NORMAL 1396 0 2 Scsi Port3 Bus0/Disk3 Part0 OPEN NORMAL 1459 0 3 Scsi Port3 Bus0/Disk3 Part0 OPEN NORMAL 0 0 C:\Program Files\IBM\SDDDSM> Note: When following the SAN zoning recommendation, this gives us, using one VDisk and a host with two HBAs, (# of VDisk) x (# of paths per IO group per HBA) x (# of HBAs) = 1 x 2 x 2 = four paths. 7. Right-click the disk in Disk Management and select Online to place the disk online (Figure 8-28).
Figure 8-28 Windows 2008 - place disk online
8. Repeat step 7 for all of your attached SVC disks. 9. Right-click one disk again and select Initialize Disk (Figure 8-29).
Figure 8-29 Windows 2008 - Initialize Disk
Chapter 8. Host configuration
257
10.Mark all the disks you want to initialize and click OK (Figure 8-30).
Figure 8-30 Windows 2008 - Initialize Disk
11.Right-click the unallocated disk space and select New Simple Volume (Figure 8-31).
Figure 8-31 Windows 2008 - New Simple Volume
12.The New Simple Volume Wizard appears. Click Next. 13.Enter a disk size and click Next (Figure 8-32).
Figure 8-32 Windows 2008 - New Simple Volume
14.Assign a drive letter and click Next (Figure 8-33 on page 259). 258
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 8-33 Windows 2008 - New Simple Volume
15.Enter a volume label and click Next (Figure 8-34).
Figure 8-34 Windows 2008 - New Simple Volume
Chapter 8. Host configuration
259
16.Click Finish and repeat this step for every SVC disk on your host system (Figure 8-35).
Figure 8-35 Windows 2008 - Disk Management
8.5.4 Extending a Windows 2008 Volume Using SVC and Windows 2008 gives you the ability to extend volumes while they are in use. The steps to extend a volume are described in 8.4.1, “Extending a Windows 2000 or 2003 volume” on page 244. Windows 2008 also uses also the DiskPart utility to extend volumes. To start it, select Start → Run and enter DiskPart. The DiskPart utility will appear. The procedure is exactly the same as in Windows 2003. Follow the Windows 2003 description to extend your volume.
8.5.5 Removing a disk on Windows When we want to remove a disk from Windows, and the disk is an SVC VDisk, we need to follow the standard Windows procedure to make sure that there is no data we wish to preserve on the disk, that no applications are using the disk, and that no I/O is going to the disk. After completing this procedure, we remove the VDisk mapping on the SVC. Here we need to make sure we are removing the correct VDisk, and to check this we use SDD to find the serial number for the disk, and on the SVC we use lshostvdiskmap to find the VDisk name and number. We also check that the SDD Serial number on the host matches the UID on the SVC for the VDisk. When the VDisk mapping is removed, we will do a rescan for the disk, Disk Management on the server will remove the disk, and the vpath will go into the status of CLOSE on the server. We can check this by using the SDD command datapath query device, but the vpath that is closed will first be removed after a reboot of the server. In the following sequence of examples, we show how we can remove an SVC VDisk from a Windows server. We show it on a Windows 2003 operating system, but the steps also apply to Windows 2000 and 2008.
260
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 8-13 on page 245 shows the Disk Manager before removing the disk. We will remove Disk 1(S:). To find the correct VDisk information, we find the Serial/UID number using SDD (Example 8-43). Example 8-43 Removing SVC disk from Windows server
C:\Program Files\IBM\SDDDSM>datapath query device Total Devices : 3
DEV#: 0 DEVICE NAME: Disk1 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 6005076801A180E9080000000000000F ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port2 Bus0/Disk1 Part0 OPEN NORMAL 1471 0 1 Scsi Port2 Bus0/Disk1 Part0 OPEN NORMAL 0 0 2 Scsi Port3 Bus0/Disk1 Part0 OPEN NORMAL 0 0 3 Scsi Port3 Bus0/Disk1 Part0 OPEN NORMAL 1324 0 DEV#: 1 DEVICE NAME: Disk2 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 6005076801A180E90800000000000010 ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port2 Bus0/Disk2 Part0 OPEN NORMAL 20 0 1 Scsi Port2 Bus0/Disk2 Part0 OPEN NORMAL 94 0 2 Scsi Port3 Bus0/Disk2 Part0 OPEN NORMAL 55 0 3 Scsi Port3 Bus0/Disk2 Part0 OPEN NORMAL 0 0 DEV#: 2 DEVICE NAME: Disk3 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 6005076801A180E90800000000000011 ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port2 Bus0/Disk3 Part0 OPEN NORMAL 100 0 1 Scsi Port2 Bus0/Disk3 Part0 OPEN NORMAL 0 0 2 Scsi Port3 Bus0/Disk3 Part0 OPEN NORMAL 0 0 3 Scsi Port3 Bus0/Disk3 Part0 OPEN NORMAL 69 0
Chapter 8. Host configuration
261
Knowing the Serial/UID of the VDisk and the host name Senegal, we find the VDisk mapping to remove using the lshostvdiskmap command on the SVC, and after this we remove the actual VDisk mapping (Example 8-44). Example 8-44 Finding and removing the VDisk mapping
IBM_2145:ITSO-CLS2:admin>svcinfo lshostvdiskmap Senegal id name SCSI_id vdisk_id vdisk_name wwpn 1 Senegal 0 7 Senegal_bas0001 210000E08B89B9C0 6005076801A180E9080000000000000F 1 Senegal 1 8 Senegal_bas0002 210000E08B89B9C0 6005076801A180E90800000000000010 1 Senegal 2 9 Senegal_bas0003 210000E08B89B9C0 6005076801A180E90800000000000011
vdisk_UID
IBM_2145:ITSO-CLS2:admin>svctask rmvdiskhostmap -host Senegal Senegal_bas0001 IBM_2145:ITSO-CLS2:admin>svcinfo lshostvdiskmap Senegal id name SCSI_id vdisk_id vdisk_name wwpn 1 Senegal 1 8 Senegal_bas0002 210000E08B89B9C0 6005076801A180E90800000000000010 1 Senegal 2 9 Senegal_bas0003 210000E08B89B9C0 6005076801A180E90800000000000011
vdisk_UID
Here we can see that the VDisk is removed from the server. On the server, we then perform a disk rescan in Disk Management, and we now see that the correct disk (Disk1) has been removed, as shown in Figure 8-36.
Figure 8-36 Disk Management - Disk has been removed
SDD also shows us that the status for all paths to Disk1 has changed to CLOSE because the disk is not available (Example 8-45 on page 263).
262
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 8-45 SDD - closed path
C:\Program Files\IBM\SDDDSM>datapath query device Total Devices : 3
DEV#: 0 DEVICE NAME: Disk1 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 6005076801A180E9080000000000000F ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port2 Bus0/Disk1 Part0 CLOSE NORMAL 1471 0 1 Scsi Port2 Bus0/Disk1 Part0 CLOSE NORMAL 0 0 2 Scsi Port3 Bus0/Disk1 Part0 CLOSE NORMAL 0 0 3 Scsi Port3 Bus0/Disk1 Part0 CLOSE NORMAL 1324 0 DEV#: 1 DEVICE NAME: Disk2 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 6005076801A180E90800000000000010 ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port2 Bus0/Disk2 Part0 OPEN NORMAL 20 0 1 Scsi Port2 Bus0/Disk2 Part0 OPEN NORMAL 124 0 2 Scsi Port3 Bus0/Disk2 Part0 OPEN NORMAL 72 0 3 Scsi Port3 Bus0/Disk2 Part0 OPEN NORMAL 0 0 DEV#: 2 DEVICE NAME: Disk3 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 6005076801A180E90800000000000011 ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port2 Bus0/Disk3 Part0 OPEN NORMAL 134 0 1 Scsi Port2 Bus0/Disk3 Part0 OPEN NORMAL 0 0 2 Scsi Port3 Bus0/Disk3 Part0 OPEN NORMAL 0 0 3 Scsi Port3 Bus0/Disk3 Part0 OPEN NORMAL 82 0 The disk (Disk1) is now removed from the server. However, to remove the SDD information of the disk, we need to reboot the server, but this can wait until a more suitable time.
8.6 Using the SVC CLI from a Windows host To issue CLI commands, we must install and prepare the SSH client system on the Windows host system. We can install the PuTTY SSH client software on a Windows host using the PuTTY Installation program. This is in the SSHClient\PuTTY directory of the SAN Volume Controller Console CD-ROM, or you can download PuTTY from the following Web site: http://www.chiark.greenend.org.uk/~sgtatham/putty/ The following Web site offers SSH client alternatives for Windows: http://www.openssh.com/windows.html Cygwin software has an option to install an OpenSSH client. You can download Cygwin from the following Web site: http://www.cygwin.com/
Chapter 8. Host configuration
263
More information about the CLI is covered in Chapter 9, “SVC configuration and administration using the CLI” on page 303.
8.7 Microsoft Volume Shadow Copy The SAN Volume Controller provides support for the Microsoft Volume Shadow Copy Service. The Microsoft Volume Shadow Copy Service can provide a point-in-time (shadow) copy of a Windows host volume while the volume is mounted and files are in use. In this section, we discuss how to install the Microsoft Volume Copy Shadow Service. The following operating system versions are supported: Windows 2003 Standard Server Edition, 32-bit and 64-bit (x64) versions Windows 2003 Enterprise Edition, 32-bit and 64-bit (x64) versions Windows 2003 Standard Server R2 Edition, 32-bit and 64-bit (x64) versions Windows 2003 Enterprise R2 Edition, 32-bit and 64-bit (x64) versions Windows Server 2008 Standard Windows Server 2008 Enterprise The following components are used to provide support for the service: SAN Volume Controller SAN Volume Controller master console IBM System Storage hardware provider, known as the IBM System Storage Support for Microsoft Volume Shadow Copy Service Microsoft Volume Shadow Copy Service The IBM System Storage provider is installed on the Windows host. To provide the point-in-time shadow copy, the components complete the following process: 1. A backup application on the Windows host initiates a snapshot backup. 2. The Volume Shadow Copy Service notifies the IBM System Storage hardware provider that a copy is needed. 3. The SAN Volume Controller prepares the volume for a snapshot. 4. The Volume Shadow Copy Service quiesces the software applications that are writing data on the host and flushes file system buffers to prepare for a copy. 5. The SAN Volume Controller creates the shadow copy using the FlashCopy Service. 6. The Volume Shadow Copy Service notifies the writing applications that I/O operations can resume and notifies the backup application that the backup was successful. The Volume Shadow Copy Service maintains a free pool of virtual disks (VDisks) for use as a FlashCopy target and a reserved pool of VDisks. These pools are implemented as virtual host systems on the SAN Volume Controller.
264
Implementing the IBM System Storage SAN Volume Controller V4.3
8.7.1 Installation overview The steps for implementing the IBM System Storage Support for Microsoft Volume Shadow Copy Service must be completed in the correct sequence. Before you begin, you must have experience with, or knowledge of, administering a Windows operating system. And you must also have experience with, or knowledge of, administering a SAN Volume Controller. You will need to complete the following tasks: Verify that the system requirements are met. Install the SAN Volume Controller Console if it is not already installed. Install the IBM System Storage hardware provider. Verify the installation. Create a free pool of volumes and a reserved pool of volumes on the SAN Volume Controller.
8.7.2 System requirements for the IBM System Storage hardware provider Ensure that your system satisfies the following requirements before you install the IBM System Storage Support for Microsoft Volume Shadow Copy Service and Virtual Disk Service software on the Windows operating system: SAN Volume Controller and Master Console Version 2.1.0 or later with FlashCopy enabled. You must install the SAN Volume Controller Console before you install the IBM System Storage Hardware provider. IBM System Storage Support for Microsoft Volume Shadow Copy Service and Virtual Disk Service software Version 3.1 or later.
8.7.3 Installing the IBM System Storage hardware provider This section includes the steps to install the IBM System Storage hardware provider on a Windows server. You must satisfy all of the system requirements before starting the installation. During the installation, you will be prompted to enter information about the SAN Volume Controller master console, including the location of the truststore file. The truststore file is generated during the installation of the master console. You must copy this file to a location that is accessible to the IBM System Storage hardware provider on the Windows server. When the installation is complete, the installation program might prompt you to restart the system. Complete the following steps to install the IBM System Storage hardware provider on the Windows server: 1. Download the installation program files from the IBM Web site, and place a copy on the Windows server where you will install the IBM System Storage hardware provider: http://www-1.ibm.com/support/docview.wss?rs=591&context=STCCCXR&context=STCCCYH &dc=D400&uid=ssg1S4000663&loc=en_US&cs=utf-8&lang=en 2. Log on to the Windows server as an administrator and navigate to the directory where the installation program is located. 3. Run the installation program by double-clicking IBMVSS.exe.
Chapter 8. Host configuration
265
4. The Welcome window opens, as shown in Figure 8-37. Click Next to continue with the installation. You can click Cancel at any time to exit the installation. To move back to previous windows while using the wizard, click Back.
Figure 8-37 IBM System Storage Support for Microsoft Volume Shadow Copy installation
5. The License Agreement window opens (Figure 8-38). Read the license agreement information. Select whether you accept the terms of the license agreement, and click Next. If you do not accept, it means that you cannot continue with the installation.
Figure 8-38 IBM System Storage Support for Microsoft Volume Shadow Copy installation
266
Implementing the IBM System Storage SAN Volume Controller V4.3
6. The Choose Destination Location window opens (Figure 8-39). Click Next to accept the default directory where the setup program will install the files, or click Change to select a different directory. Click Next.
Figure 8-39 IBM System Storage Support for Microsoft Volume Shadow Copy installation
7. Click Install to begin the installation (Figure 8-40):
Figure 8-40 IBM System Storage Support for Microsoft Volume Shadow Copy installation
Chapter 8. Host configuration
267
8. From the next window, select the required CIM server, or select Enter the CIM Server address manually, and click Next (Figure 8-41).
Figure 8-41 IBM System Storage Support for Microsoft Volume Shadow Copy installation
9. The Enter CIM Server Details window appears. Enter the following information in the fields (Figure 8-42): a. In the CIM Server Address field, type the name of the server where the SAN Volume Controller Console is installed. b. In the CIM User field, type the user name that the IBM System Storage Support for Microsoft Volume Shadow Copy Service and Virtual Disk Service software will use to gain access to the server where the SAN Volume Controller Console is installed. c. In the CIM Password field, type the password for the user name that the IBM System Storage Support for Microsoft Volume Shadow Copy Service and Virtual Disk Service software will use to gain access to the SAN Volume Controller Console d. Click Next.
Figure 8-42 IBM System Storage Support for Microsoft Volume Shadow Copy installation
10.In the next window, click Finish. If necessary, the InstallShield Wizard prompts you to restart the system (Figure 8-43 on page 269).
268
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 8-43 IBM System Storage Support for Microsoft Volume Shadow Copy installation
Note: If these settings change after installation, you can use the ibmvcfg.exe tool to update Microsoft Volume Shadow Copy and Virtual Disk Services software with the new settings. If you do not have the CIM agent server, port, or user information, contact your CIM agent administrator.
8.7.4 Verifying the installation Perform the following steps to verify the installation. 1. Select Start → All Programs → Administrative Tools → Services from the Windows server task bar. 2. Ensure that the service named “IBM System Storage Support for Microsoft Volume Shadow Copy Service and Virtual Disk Service” software appears and that Status is set to Started and Startup Type is set to Automatic. 3. Open a command prompt window and issue the following command: vssadmin list providers
Chapter 8. Host configuration
269
This command ensures that the service named IBM System Storage Support for Microsoft Volume Shadow Copy Service and Virtual Disk Service software is listed as a provider (Example 8-46). Example 8-46 Microsoft Software Shadow copy provider
C:\Documents and Settings\Administrator>vssadmin list providers vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool (C) Copyright 2001 Microsoft Corp. Provider name: 'Microsoft Software Shadow Copy provider 1.0' Provider type: System Provider Id: {b5946137-7b9f-4925-af80-51abd60b20d5} Version: 1.0.0.7 Provider name: 'IBM System Storage Volume Shadow Copy Service Hardware Provider' Provider type: Hardware Provider Id: {d90dd826-87cf-42ce-a88d-b32caa82025b} Version: 3.1.0.1108 If you are able to successfully perform all of these verification tasks, the IBM System Storage Support for Microsoft Volume Shadow Copy Service and Virtual Disk Service software was successfully installed on the Windows server.
8.7.5 Creating the free and reserved pools of volumes The IBM System Storage hardware provider maintains a free and a reserved pool of volumes. Because these objects do not exist on the SAN Volume Controller, the free and reserved pool of volumes are implemented as virtual host systems. You must define these two virtual host systems on the SAN Volume Controller. When a shadow copy is created, the IBM System Storage hardware provider selects a volume in the free pool, assigns it to the reserved pool, and then removes it from the free pool. This protects the volume from being overwritten by other Volume Shadow Copy Service users. To successfully perform a Volume Shadow Copy Service operation, there must be enough virtual disks (VDisks) mapped to the free pool. The VDisks must be the same size as the source VDisks. Use the SAN Volume Controller Console or the SAN Volume Controller command-line interface (CLI) to perform the following steps: 1. Create a host for the free pool of VDisks. You can use the default name VSS_FREE or specify a different name. Associate the host with the worldwide port name (WWPN) 5000000000000000 (15 zeroes) (Example 8-47). Example 8-47 mkhost for free pool
IBM_2145:ITSO-CLS2:admin>svctask mkhost -name VSS_FREE -hbawwpn 5000000000000000 -force Host, id [2], successfully created 2. Create a virtual host for the reserved pool of volumes. You can use the default name VSS_RESERVED or specify a different name. Associate the host with the WWPN 5000000000000001 (14 zeroes) (Example 8-48 on page 271).
270
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 8-48 mkhost for reserved pool
IBM_2145:ITSO-CLS2:admin>svctask mkhost -name VSS_RESERVED -hbawwpn 5000000000000001 -force Host, id [3], successfully created 3. Map the logical units (VDisks) to the free pool of volumes. The VDisks cannot be mapped to any other hosts. If you already have VDisks created for the free pool of volumes, you must assign the VDisks to the free pool. 4. Create VDisk-to-host mappings between the VDisks selected in step 3 and the VSS_FREE host to add the VDisks to the free pool. Alternatively, you can use the ibmvcfg add command to add VDisks to the free pool (Example 8-49). Example 8-49 Host mappings
IBM_2145:ITSO-CLS2:admin>svctask mkvdiskhostmap -host VSS_FREE msvc0001 Virtual Disk to Host map, id [0], successfully created IBM_2145:ITSO-CLS2:admin>svctask mkvdiskhostmap -host VSS_FREE msvc0002 Virtual Disk to Host map, id [1], successfully created 5. Verify that the VDisks have been mapped. If you do not use the default WWPNs 5000000000000000 and 5000000000000001, you must configure the IBM System Storage hardware provider with the WWPNs (Example 8-50). Example 8-50 Verify hosts
IBM_2145:ITSO-CLS2:admin>svcinfo lshostvdiskmap VSS_FREE id name SCSI_id vdisk_id vdisk_name wwpn vdisk_UID 2 VSS_FREE 0 10 msvc0001 5000000000000000 6005076801A180E90800000000000012 2 VSS_FREE 1 11 msvc0002 5000000000000000 6005076801A180E90800000000000013
8.7.6 Changing the configuration parameters You can change the parameters that you defined when you installed the IBM System Storage Support for Microsoft Volume Shadow Copy Service and Virtual Disk Service software. Therefore, you must use the ibmvcfg.util. It is a command-line utility located in C:\Program Files\IBM\Hardware Provider for VSS-VDS (Example 8-51). Example 8-51 ibmvcfg.util help
C:\Program Files\IBM\Hardware Provider for VSS-VDS>ibmvcfg.exe IBM System Storage VSS Provider Configuration Tool Commands ---------------------------------------ibmvcfg.exe Commands: /h | /help | -? | /? showcfg listvols add (separated by spaces) rem (separated by spaces) Configuration: set user Chapter 8. Host configuration
271
set set set set set set set set set set set set set
password trace [0-7] trustpassword <trustpassword> truststore <truststore location> usingSSL vssFreeInitiator <WWPN> vssReservedInitiator <WWPN> FlashCopyVer <1 | 2> (only applies to ESS) cimomPort cimomHost namespace targetSVC <svc_cluster_ip> backgroundCopy <0-100>
The available commands are shown in Table 8-4. Table 8-4 ibmvcfg.util commands
272
Command
Description
Example
ibmvcfg showcfg
Lists the current settings.
ibmvcfg showcfg
ibmvcfg set username <username>
Sets the user name to access the SAN Volume Controller Console.
ibmvcfg set username Dan
ibmvcfg set password <password>
Sets the password of the user name that will access the SAN Volume Controller Console.
ibmvcfg set password mypassword
ibmvcfg set targetSVC
Specifies the IP address of the SAN Volume Controller on which the VDisks are located when VDisks are moved to and from the free pool with the ibmvcfg add and ibmvcfg rem commands. The IP address is overridden if you use the -s flag with the ibmvcfg add and ibmvcfg rem commands.
set targetSVC 9.43.86.120
set backgroundCopy
Sets the background copy rate for FlashCopy.
set backgroundCopy 80
ibmvcfg set usingSSL
Specifies whether to use Secure Sockets Layer protocol to connect to the SAN Volume Controller Console.
ibmvcfg set usingSSL yes
ibmvcfg set cimomPort <portnum>
Specifies the SAN Volume Controller Console port number. The default value is 5999.
ibmvcfg set cimomPort 5999
ibmvcfg set cimomHost <server name>
Sets the name of the server where the SAN Volume Controller Console is installed.
ibmvcfg set cimomHost cimomserver
ibmvcfg set namespace
Specifies the namespace value that master console is using. The default value is \root\ibm.
ibmvcfg set namespace \root\ibm
Implementing the IBM System Storage SAN Volume Controller V4.3
Command
Description
Example
ibmvcfg set vssFreeInitiator <WWPN>
Specifies the WWPN of the host. The default value is 5000000000000000. Modify this value only if there is a host already in your environment with a WWPN of 5000000000000000.
ibmvcfg set vssFreeInitiator 5000000000000000
ibmvcfg set vssReservedInitiator <WWPN>
Specifies the WWPN of the host. The default value is 5000000000000001. Modify this value only if there is a host already in your environment with a WWPN of 5000000000000001.
ibmvcfg set vssFreeInitiator 5000000000000001
ibmvcfg listvols
Lists all virtual disks (VDisks), including information about size, location, and VDisk to host mappings.
ibmvcfg listvols
ibmvcfg listvols all
Lists all VDisks, including information about size, location, and VDisk to host mappings.
ibmvcfg listvols all
ibmvcfg listvols free
Lists the volumes that are currently in the free pool.
ibmvcfg listvols free
ibmvcfg listvols unassigned
Lists the volumes that are currently not mapped to any hosts.
ibmvcfg listvols unassigned
ibmvcfg add -s ipaddress
Adds one or more volumes to the free pool of volumes. Use the -s parameter to specify the IP address of the SAN Volume Controller where the VDisks are located. The -s parameter overrides the default IP address that is set with the ibmvcfg set targetSVC command.
ibmvcfg add vdisk12 ibmvcfg add 600507 68018700035000000 0000000BA -s 66.150.210.141
ibmvcfg rem -s ipaddress
Removes one or more volumes from the free pool of volumes. Use the -s parameter to specify the IP address of the SAN Volume Controller where the VDisks are located. The -s parameter overrides the default IP address that is set with the ibmvcfg set targetSVC command.
ibmvcfg rem vdisk12 ibmvcfg rem 600507 68018700035000000 0000000BA -s 66.150.210.141
Chapter 8. Host configuration
273
8.8 Linux (on Intel) specific information The following sections details specific information pertaining to the connection of Linux on Intel-based hosts to the SVC environment.
8.8.1 Configuring the Linux host Follow these steps to configure the Linux host: 1. Use the latest firmware levels on your host system. 2. Install the HBA or HBAs on the Linux server, as described in 8.3.4, “Host adapter installation and configuration” on page 234. 3. Install the supported HBA driver / firmware and upgrade the kernel if required, as described in 8.8.2, “Configuration information” on page 274. 4. Connect the Linux server FC Host adapters to the switches. 5. Configure the switches (zoning) if needed. 6. Install SDD for Linux, as described in 8.8.5, “Multipathing in Linux” on page 275. 7. Configure the host, VDisks, and host mapping in the SAN Volume Controller. 8. Rescan for LUNs on the Linux server to discover the VDisks created on the SVC.
8.8.2 Configuration information The SAN Volume Controller supports hosts that run the following Linux distributions: Red Hat Enterprise Linux SUSE® Linux Enterprise Server For the latest information, always refer to this site: http://www.ibm.com/storage/support/2145 For SVC Version 4.3, the following support information was available at the time of writing: Software supported levels: http://www-1.ibm.com/support/docview.wss?rs=591&uid=ssg1S1003278 Hardware supported levels: http://www-1.ibm.com/support/docview.wss?rs=591&uid=ssg1S1003277 There you will find the hardware list for supported host bus adapters and device driver levels for Windows. Check the supported firmware and driver level for your host bus adapter and follow the manufacture’s instructions to upgrade the firmware and driver levels for each type of HBA.
8.8.3 Disabling automatic Linux system updates Many Linux distributions give you the ability to configure your systems for automatic system updates. Red Hat provides this ability in the form of a program called up2date, while Novell® SUSE provides the YaST Online Update utility. These features periodically query for updates available for each host and can be configured to automatically install any new updates that they find.
274
Implementing the IBM System Storage SAN Volume Controller V4.3
Often, the automatic update process also upgrades the system to the latest kernel level. If this is the case, hosts running SDD should consider turning off the automatic update of kernel levels. Some drivers supplied by IBM, such as SDD, are dependent on a specific kernel and will cease to function on a new kernel. Similarly, host bus adapter (HBA) drivers need to be compiled against specific kernels in order to function optimally. By allowing automatic updates of the kernel, you risk impacting your host systems unexpectedly.
8.8.4 Setting queue depth with QLogic HBAs The queue depth is the number of I/O operations that can be run in parallel on a device. Configure your host running the Linux operating system using the formula specified in 8.13, “Calculating the queue depth” on page 301. Perform the following steps to set the maximum queue depth: 1. Add the following line to the /etc/modules.conf file: a. For the 2.4 kernel (SUSE Linux Enterprise Server 8 or Red Hat Enterprise Linux): options qla2300 ql2xfailover=0 ql2xmaxqdepth=new_queue_depth b. For the 2.6 kernel (SUSE Linux Enterprise Server 9, or later, or Red Hat Enterprise Linux 4, or later): options qla2xxx ql2xfailover=0 ql2xmaxqdepth=new_queue_depth 2. Rebuild the RAM disk that is associated with the kernel being used by using one of the following commands: a. If you are running on an SUSE Linux Enterprise Server operating system, run the mk_initrd command. b. If you are running on a Red Hat Enterprise Linux operating system, run the mkinitrd command and then restart.
8.8.5 Multipathing in Linux Red Hat Enterprise Linux 5 and later, and SUSE Linux Enterprise Server 10 and later, provide their own multipath support by the operating system. On older systems, it is necessary to install the IBM SDD multipath driver.
Installing SDD This section describes how to install SDD for older distributions. Before performing these steps, always check for the currently supported levels, as described in 8.8.2, “Configuration information” on page 274.
Chapter 8. Host configuration
275
The cat /proc/scsi/scsi command in Example 8-52 shows the devices that the SCSI driver has probed. In our configuration, we have two HBAs installed in our server and we configured the zoning in order to access our VDisk from four paths. Example 8-52 cat /proc/scsi/scsi command example
[root@diomede sdd]# cat /proc/scsi/scsi Attached devices: Host: scsi4 Channel: 00 Id: 00 Lun: 00 Vendor: IBM Model: 2145 Type: Unknown Host: scsi5 Channel: 00 Id: 00 Lun: 00 Vendor: IBM Model: 2145 Type: Unknown [root@diomede sdd]#
Rev: 0000 ANSI SCSI revision: 04 Rev: 0000 ANSI SCSI revision: 04
The rpm -ivh IBMsdd-1.6.3.0-5.i686.rhel4.rpm command installs the package, as shown in Example 8-53. Example 8-53 rpm command example
[root@Palau sdd]# rpm -ivh IBMsdd-1.6.3.0-5.i686.rhel4.rpm Preparing... ########################################### [100%] 1:IBMsdd ########################################### [100%] Added following line to /etc/inittab: srv:345:respawn:/opt/IBMsdd/bin/sddsrv > /dev/null 2>&1 [root@Palau sdd]# To manually load and configure SDD on Linux, use the service sdd start command (SUSE Linux users can use the sdd start command). If you are not running a supported kernel, you will get an error message. If your kernel is supported, you should see an OK success message, as shown in Example 8-54. Example 8-54 Non-supported kernel for SDD
[root@Palau sdd]# sdd start Starting IBMsdd driver load: Issuing killall sddsrv to trigger respawn... Starting IBMsdd configuration:
[
OK
]
[
OK
]
Issue the cfgvpath query command to view the name and serial number of the VDisk configured in the SAN Volume Controller, as shown in Example 8-55. Example 8-55 cfgvpath query example
[root@Palau ~]# cfgvpath query RTPG command = a3 0a 00 00 00 00 00 00 08 20 00 00 total datalen=52 datalen_str=0x00 00 00 30 RTPG succeeded: sd_name=/dev/sda df_ctlr=0 /dev/sda ( 8, 0) host=0 ch=0 id=0 lun=0 vid=IBM pid=2145 serial=60050768018201bee000000000000035 lun_id=60050768018201bee000000000000035 ctlr_flag=1 ctlr_nbr=1 df_ctlr=0 RTPG command = a3 0a 00 00 00 00 00 00 08 20 00 00 total datalen=52 datalen_str=0x00 00 00 30
276
Implementing the IBM System Storage SAN Volume Controller V4.3
RTPG succeeded: sd_name=/dev/sdb df_ctlr=0 /dev/sdb ( 8, 16) host=0 ch=0 id=1 lun=0 vid=IBM pid=2145 serial=60050768018201bee000000000000035 lun_id=60050768018201bee000000000000035 ctlr_flag=1 ctlr_nbr=0 df_ctlr=0 RTPG command = a3 0a 00 00 00 00 00 00 08 20 00 00 total datalen=52 datalen_str=0x00 00 00 30 RTPG succeeded: sd_name=/dev/sdc df_ctlr=0 /dev/sdc ( 8, 32) host=1 ch=0 id=0 lun=0 vid=IBM pid=2145 serial=60050768018201bee000000000000035 lun_id=60050768018201bee000000000000035 ctlr_flag=1 ctlr_nbr=0 df_ctlr=0 RTPG command = a3 0a 00 00 00 00 00 00 08 20 00 00 total datalen=52 datalen_str=0x00 00 00 30 RTPG succeeded: sd_name=/dev/sdd df_ctlr=0 /dev/sdd ( 8, 48) host=1 ch=0 id=1 lun=0 vid=IBM pid=2145 serial=60050768018201bee000000000000035 lun_id=60050768018201bee000000000000035 ctlr_flag=1 ctlr_nbr=1 df_ctlr=0 [root@Palau ~]# The cfgvpath command configures the SDD vpath devices, as shown in Example 8-56. Example 8-56 cfgvpath command example
[root@Palau ~]# cfgvpath c--------- 1 root root 253, 0 Jun 5 WARNING: vpatha path sda has WARNING: vpatha path sdb has WARNING: vpatha path sdc has WARNING: vpatha path sdd has Writing out new configuration to file [root@Palau ~]#
09:04 /dev/IBMsdd already been configured. already been configured. already been configured. already been configured. /etc/vpath.conf
The configuration information is saved by default in the file /etc/vpath.conf. You can save the configuration information to a specified file name by entering the following command: cfgvpath -f file_name.cfg Issue the chkconfig command to enable SDD to run at system startup: chkconfig sdd on To verify the setting, enter the following command: chkconfig --list sdd This is shown in Example 8-57. Example 8-57 sdd run level example
[root@Palau sdd]# chkconfig --list sdd sdd 0:off 1:off 2:on [root@Palau sdd]#
3:on
4:on
5:on
6:off
If necessary, you can disable the startup option by entering: chkconfig sdd off
Chapter 8. Host configuration
277
Run the datapath query commands to display the online adapters and paths to the adapters. Notice that the preferred paths are used from one of the nodes, that is, path 0 and 2. Paths 1 and 3 connect to the other node and are used as alternate or backup paths for high availability, as shown in Example 8-58. Example 8-58 datapath query command example
[root@Palau ~]# datapath query adapter Active Adapters :2 Adpt# Name State Mode 0 Host0Channel0 NORMAL ACTIVE 1 Host1Channel0 NORMAL ACTIVE [root@Palau ~]# [root@Palau ~]# datapath query device
Select 1 0
Errors 0 0
Paths 2 2
Active 0 0
Total Devices : 1
DEV#: 0 DEVICE NAME: vpatha TYPE: 2145 POLICY: Optimized Sequential SERIAL: 60050768018201bee000000000000035 ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Host0Channel0/sda CLOSE NORMAL 1 0 1 Host0Channel0/sdb CLOSE NORMAL 0 0 2 Host1Channel0/sdc CLOSE NORMAL 0 0 3 Host1Channel0/sdd CLOSE NORMAL 0 0 [root@Palau ~]# SDD has three different path-selection policy algorithms. Failover only (fo): All I/O operations for the device are sent to the same (preferred) path unless the path fails because of I/O errors. Then an alternate path is chosen for subsequent I/O operations. Load balancing (lb): The path to use for an I/O operation is chosen by estimating the load on the adapter to which each path is attached. The load is a function of the number of I/O operations currently in process. If multiple paths have the same load, a path is chosen at random from those paths. Load-balancing mode also incorporates failover protection. The load-balancing policy is also known as the optimized policy. Round robin (rr): The path to use for each I/O operation is chosen at random from paths that were not used for the last I/O operation. If a device has only two paths, SDD alternates between the two. You can dynamically change the SDD path-selection policy algorithm by using the SDD command datapath set device policy. You can see the SDD path-selection policy algorithm that is active on the device when you use the datapath query device command. Example 8-58 shows that the active policy is optimized, which means that the SDD path-selection policy algorithm active is Optimized Sequential.
278
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 8-59 shows the VDisk information from the SVC command-line interface. Example 8-59 svcinfo redhat1
IBM_2145:ITSOSVC42A:admin>svcinfo lshost linux2 id 6 name linux2 port_count 2 type generic mask 1111 iogrp_count 4 WWPN 210000E08B89C1CD node_logged_in_count 2 state active WWPN 210000E08B054CAA node_logged_in_count 2 state active IBM_2145:ITSOSVC42A:admin> IBM_2145:ITSOSVC42A:admin>svcinfo lshostvdiskmap linux2 id name SCSI_id vdisk_id wwpn vdisk_UID 6 linux2 0 33 210000E08B89C1CD 60050768018201BEE000000000000035 IBM_2145:ITSOSVC42A:admin> IBM_2145:ITSOSVC42A:admin>svcinfo lsvdisk linux_vd1 id 33 name linux_vd1 IO_group_id 0 IO_group_name io_grp0 status online mdisk_grp_id 1 mdisk_grp_name MDG0 capacity 1.0GB type striped formatted no mdisk_id mdisk_name FC_id FC_name RC_id RC_name vdisk_UID 60050768018201BEE000000000000035 throttling 0 preferred_node_id 1 fast_write_state empty cache readwrite udid 0 fc_map_count 0 IBM_2145:ITSOSVC42A:admin>
vdisk_name linux_vd1
Chapter 8. Host configuration
279
8.8.6 Creating and preparing SDD volumes for use Follow these steps to create and prepare the volumes: 1. Create a partition on the vpath device, as shown in Example 8-60. Example 8-60 fdisk example
[root@Palau ~]# fdisk /dev/vpatha Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) Command (m for help): m Command action a toggle a bootable flag b edit bsd disklabel c toggle the dos compatibility flag d delete a partition l list known partition types m print this menu n add a new partition o create a new empty DOS partition table p print the partition table q quit without saving changes s create a new empty Sun disklabel t change a partition's system id u change display/entry units v verify the partition table w write table to disk and exit x extra functionality (experts only) Command (m for help): n Command action e extended p primary partition (1-4) e Partition number (1-4): 1 First cylinder (1-1011, default 1): Using default value 1 Last cylinder or +size or +sizeM or +sizeK (1-1011, default 1011): Using default value 1011 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [root@Palau ~]#
280
Implementing the IBM System Storage SAN Volume Controller V4.3
2. Create a file system on the vpath, as shown in Example 8-61. Example 8-61 mkfs command example
[root@Palau ~]# mkfs -t ext3 /dev/vpatha mke2fs 1.35 (28-Feb-2004) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 131072 inodes, 262144 blocks 13107 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=268435456 8 block groups 32768 blocks per group, 32768 fragments per group 16384 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376 Writing inode tables: done Creating journal (8192 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 27 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. [root@Palau ~]# 3. Create the mount point and mount the vpath drive, as shown in Example 8-62. Example 8-62 Mount point
[root@Palau ~]# mkdir /itsosvc [root@Palau ~]# mount -t ext3 /dev/vpatha /itsosvc 4. The drive is now ready for use. The df command shows us the mounted disk /itsosvc and the datapath query command shows that four paths are available (Example 8-63). Example 8-63 Display mounted drives
[root@Palau ~]# df Filesystem 1K-blocks /dev/mapper/VolGroup00-LogVol00 74699952 /dev/hda1 101086 none 1033136 /dev/vpatha 1032088 [root@Palau ~]#
Used Available Use% Mounted on 2564388 13472 0 34092
68341032 82395 1033136 945568
4% 15% 0% 4%
/ /boot /dev/shm /itsosvc
[root@Palau ~]# datapath query device Total Devices : 1
DEV#: 0 DEVICE NAME: vpatha TYPE: 2145 POLICY: Optimized Sequential SERIAL: 60050768018201bee000000000000035 ============================================================================
Chapter 8. Host configuration
281
Path# Adapter/Hard Disk 0 Host0Channel0/sda 1 Host0Channel0/sdb 2 Host1Channel0/sdc 3 Host1Channel0/sdd [root@Palau ~]#
State OPEN OPEN OPEN OPEN
Mode NORMAL NORMAL NORMAL NORMAL
Select 1 6296 6178 0
Errors 0 0 0 0
8.8.7 Using the operating system MPIO As mentioned before, Red Hat Enterprise Linux 5 and later and SUSE Linux Enterprise Server 10 and later provide their own multipath support for the operating system. This means you do not have to install an additional device driver. Always check if your operating system includes one of the supported multipath drivers. You will find this information in the links provided in 8.8.2, “Configuration information” on page 274. In SLES10, the multipath drivers and tools are installed by default, but for RHEL5, the user has to explicitly choose the multipath components during the OS installation to install them. Each of the attached SAN Volume Controller LUNs has a special device file in the Linux directory /dev. Hosts that use 2.6 kernel Linux operating systems can have as many Fibre Channel disks as r allowed by the SAN Volume Controller. The following Web site provides the most current information about the maximum configuration for the SAN Volume Controller: http://www.ibm.com/storage/support/2145
8.8.8 Creating and preparing MPIO volumes for use First, you have to start the MPIO daemon on your system. To do this, run the following commands on your host system: Enable MPIO for SLES10 by running the following commands: 1. /etc/init.d/boot.multipath {start|stop} 2. /etc/init.d/multipathd {start|stop|status|try-restart|restart|force-reload|reload|probe} Note: Run insserv boot.multipath multipathd to automatically load the multipath driver and multipathd daemon during boot up. Enable MPIO for RHEL5 by running the following commands: 1. modprobe dm-multipath 2. modprobe dm-round-robin 3. service multipathd start 4. chkconfig multipathd on Example 8-64 on page 283 shows the commands issued on a RHEL 5.1 operating system
282
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 8-64 Starting MPIO daemon on RHEL
[root@palau [root@palau [root@palau [root@palau
~]# modprobe dm-round-robin ~]# multipathd start ~]# chkconfig multipathd on ~]#
5. Open the multipath.conf file and follow the instructions to enable the multipathing for IBM devices. The file is located in the /etc directory. Example 8-65 shows the editing using vi. Example 8-65 Editing the multipath.conf file
[root@palau etc]# vi multipath.conf [root@palau etc]# service multipathd stop Stopping multipathd daemon: [root@palau etc]# service multipathd start Starting multipathd daemon: [root@palau etc]#
[
OK
]
[
OK
]
[
OK
]
[
OK
]
6. Add the following entry to the multipath.conf file: # SVC device { vendor "IBM" product "2105800" path_grouping_policy group_by_serial 7. Restart the multipath daemon (Example 8-66). Example 8-66 Restarting the multipath daemon
[root@palau ~]# service multipathd stop Stopping multipathd daemon: [root@palau ~]# service multipathd start Starting multipathd daemon:
8. Type the command multipath -dl to see the MPIO configuration. Example 8-67 shows two SVC VDisks, attached over four paths. Example 8-67 MPIO configuration
[root@palau scsi]# multipath -dl mpath1 (360050768018301bf280000000000001a) dm-3 IBM,2145 [size=4.0G][features=1 queue_if_no_path][hwhandler=0] \_ round-robin 0 [prio=0][active] \_ 0:0:2:1 sdb 8:16 [active][undef] \_ 0:0:3:1 sdd 8:48 [active][undef] \_ 1:0:1:1 sdf 8:80 [active][undef] \_ 1:0:3:1 sdh 8:112 [active][undef] mpath0 (360050768018301bf2800000000000019) dm-2 IBM,2145 [size=4.0G][features=1 queue_if_no_path][hwhandler=0] \_ round-robin 0 [prio=0][active] \_ 0:0:2:0 sda 8:0 [active][undef] \_ 0:0:3:0 sdc 8:32 [active][undef] \_ 1:0:1:0 sde 8:64 [active][undef] \_ 1:0:3:0 sdg 8:96 [active][undef]
Chapter 8. Host configuration
283
9. Use fdisk to create a partition on the SVC disk, as shown in Example 8-68. Example 8-68 fdisk
[root@palau scsi]# fdisk -l Disk /dev/hda: 80.0 GB, 80032038912 bytes 255 heads, 63 sectors/track, 9730 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot /dev/hda1 * /dev/hda2
Start 1 14
End 13 9730
Blocks 104391 78051802+
Id 83 8e
Disk /dev/sda: 4244 MB, 4244635648 bytes 131 heads, 62 sectors/track, 1020 cylinders Units = cylinders of 8122 * 512 = 4158464 bytes Disk /dev/sda doesn't contain a valid partition table Disk /dev/sdb: 4244 MB, 4244635648 bytes 131 heads, 62 sectors/track, 1020 cylinders Units = cylinders of 8122 * 512 = 4158464 bytes Disk /dev/sdb doesn't contain a valid partition table Disk /dev/sdc: 4244 MB, 4244635648 bytes 131 heads, 62 sectors/track, 1020 cylinders Units = cylinders of 8122 * 512 = 4158464 bytes Disk /dev/sdc doesn't contain a valid partition table Disk /dev/sdd: 4244 MB, 4244635648 bytes 131 heads, 62 sectors/track, 1020 cylinders Units = cylinders of 8122 * 512 = 4158464 bytes Disk /dev/sdd doesn't contain a valid partition table Disk /dev/sde: 4244 MB, 4244635648 bytes 131 heads, 62 sectors/track, 1020 cylinders Units = cylinders of 8122 * 512 = 4158464 bytes Disk /dev/sde doesn't contain a valid partition table Disk /dev/sdf: 4244 MB, 4244635648 bytes 131 heads, 62 sectors/track, 1020 cylinders Units = cylinders of 8122 * 512 = 4158464 bytes Disk /dev/sdf doesn't contain a valid partition table Disk /dev/sdg: 4244 MB, 4244635648 bytes 131 heads, 62 sectors/track, 1020 cylinders Units = cylinders of 8122 * 512 = 4158464 bytes Disk /dev/sdg doesn't contain a valid partition table
284
Implementing the IBM System Storage SAN Volume Controller V4.3
System Linux Linux LVM
Disk /dev/sdh: 4244 MB, 4244635648 bytes 131 heads, 62 sectors/track, 1020 cylinders Units = cylinders of 8122 * 512 = 4158464 bytes Disk /dev/sdh doesn't contain a valid partition table Disk /dev/dm-2: 4244 MB, 4244635648 bytes 255 heads, 63 sectors/track, 516 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk /dev/dm-2 doesn't contain a valid partition table Disk /dev/dm-3: 4244 MB, 4244635648 bytes 255 heads, 63 sectors/track, 516 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk /dev/dm-3 doesn't contain a valid partition table [root@palau scsi]# fdisk /dev/dm-2 Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) Command (m for help): n Command action e extended p primary partition (1-4) e Partition number (1-4): 1 First cylinder (1-516, default 1): Using default value 1 Last cylinder or +size or +sizeM or +sizeK (1-516, default 516): Using default value 516 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. WARNING: Re-reading the partition table failed with error 22: Invalid argument. The kernel still uses the old table. The new table will be used at the next reboot. [root@palau scsi]# shutdown -r now
Chapter 8. Host configuration
285
10.Create a file system using the mkfs command (Example 8-69). Example 8-69 mkfs command
[root@palau ~]# mkfs -t ext3 /dev/dm-2 mke2fs 1.39 (29-May-2006) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 518144 inodes, 1036288 blocks 51814 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=1061158912 32 block groups 32768 blocks per group, 32768 fragments per group 16192 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736 Writing inode tables: done Creating journal (16384 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 29 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. [root@palau ~]# 11.Create a mount point and mount the drive, as shown in Example 8-70. Example 8-70 Mount point
[root@palau ~]# mkdir /svcdisk_0 [root@palau ~]# cd /svcdisk_0/ [root@palau svcdisk_0]# mount -t ext3 /dev/dm-2 /svcdisk_0 [root@palau svcdisk_0]# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/VolGroup00-LogVol00 73608360 1970000 67838912 3% / /dev/hda1 101086 15082 80785 16% /boot tmpfs 967984 0 967984 0% /dev/shm /dev/dm-2 4080064 73696 3799112 2% /svcdisk_0
286
Implementing the IBM System Storage SAN Volume Controller V4.3
8.9 VMware configuration information This section explains the requirements and additional information for attaching the SAN Volume Controller to a variety of guest host operating systems running on the VMware operating system.
8.9.1 Configuring VMware hosts To configure the VMware hosts, follow these steps: 1. Install the HBAs into your host system, as described in 8.9.4, “HBAs for hosts running VMware” on page 287 2. Connect the server FC Host adapters to the switches. 3. Configure the switches (zoning), as described in 8.9.6, “VMware storage and zoning recommendations” on page 289. 4. Install the VMware operating system (if not already done) and check the HBA timeouts, as described in 8.9.7, “Setting the HBA timeout for failover in VMware” on page 290. 5. Configure the host, VDisks, and host mapping in the SVC, as described in 8.9.9, “Attaching VMware to VDisks” on page 291.
8.9.2 Operating system versions and maintenance levels For the latest information about VMware support, refer to: http://www-03.ibm.com/systems/storage/software/virtualization/svc/interop.html At the time of writing, the following versions are supported: ESX V3.5 ESX V3.51 ESX V3.02 ESX V2.5.3 ESX V2.5.2 ESX V2.1 with VMFS disks Note: Customers who are running the VMware V3.01 build are required to move to a minimum VMware level of V3.02 for continued support.
8.9.3 Guest operating systems Also make sure that you are using supported guest operating systems. The latest information is available at: http://www-1.ibm.com/support/docview.wss?rs=591&uid=ssg1S1003278#_VMWare
8.9.4 HBAs for hosts running VMware Ensure that your hosts running on VMware operating systems use the correct host bus adapters (HBAs) and firmware levels. Install the host adapter(s) into your system. Refer to the manufacturer’s instructions for installation and configuration of the HBAs. Chapter 8. Host configuration
287
In IBM System x servers, the HBA should always be installed in the first slots. This means that if you install, for example, two HBAs and two network cards, the HBAs should be installed in slot 1 and slot 2 and the network cards can be installed in the remaining slots. For older ESX versions, you will find the supported HBAs at the IBM Web Site: http://www.ibm.com/storage/support/2145 The interoperability matrix for ESX V3.02, V3.5, and V3.51 are available at the VMware Web Site (clicking this link opens or downloads the PDF): V3.02 http://www.vmware.com/pdf/vi3_io_guide.pdf V3.5 http://www.vmware.com/pdf/vi35_io_guide.pdf The supported HBA device drivers are already included in the ESX server build. After installing, load the default configuration of your FC HBAs. We recommend using the same model of HBA with the same firmware in one server. It is not supported to have Emulex and QLogic HBAs that access the same target in one server.
8.9.5 Multipath solutions supported Only single path is supported in ESX V2.1 and multipathing is supported in ESX V2.5.x. The VMware operating system provides multipathing support, so installing multipathing software is not required.
VMware multipathing software dynamic pathing VMware multipathing software does not support dynamic pathing. Preferred paths set in the SAN Volume Controller are ignored. The VMware multipathing software performs static load balancing for I/O, based upon a host setting that defines the preferred path for a given volume.
Multipathing configuration maximums When you configure, keep in mind the maximum configuration for the VMware multipathing software: 256 is the maximum number of SCSI devices supported by the VMware software, and the maximum number of paths to each VDisk is four, giving you a total number of paths, on a server, of 1024. Note: Each path to a VDisk equates to a single SCSI device.
Clustering support for hosts running VMware The SVC provides cluster support on VMware guest operating systems. The following Web Site provides the current interoperability information: http://www-1.ibm.com/support/docview.wss?rs=591&uid=ssg1S1003277#_VMware
SAN boot support SAN boot of any guest OS is supported under VMware. The very nature of VMware means that this is a requirement on any guest OS. The guest OS itself must reside on a SAN disk.
288
Implementing the IBM System Storage SAN Volume Controller V4.3
If you are not familiar with the VMware environments and the advantages of storing virtual machines and application data on a SAN, we recommend that you get an overview about the VMware products before continuing with the section below. VMware documentation is available at: http://www.vmware.com/support/pubs/
8.9.6 VMware storage and zoning recommendations The VMware ESX server is able to use a Virtual Machine File System (VMFS). This is a file system that is optimized to run multiple virtual machines as one workload to minimize disk I/O. It is also able to handle concurrent access from multiple physical machines because it enforces the appropriate access controls. This means multiple ESX hosts can share the same set of LUNs (Figure 8-44).
Figure 8-44 VMware - SVC zoning example
This means that theoretically you are able to run all your virtual machines on one LUN, but for performance reasons, in more complex scenarios, it can be better to load balance virtual machines over separate HBAs, storages, or arrays. For example, if you run an ESX host, with several virtual machines, it would make sense to use one “slow” array, for example, for Print and Active Directory® Services guest operating systems without high I/O, and another fast array for database guest operating systems.
Chapter 8. Host configuration
289
Using fewer VDisks does have the following advantages: More flexibility to create virtual machines without creating new space on the SVC More possibilities for taking VMware snapshots Fewer VDisks to manage Using more and smaller VDisks can have the following advantages: Different I/O characteristics of the guest operating systems More flexibility (the multipathing policy and disk shares are set per VDisk) Microsoft Cluster Service requires a own VDisk for each cluster disk resource More documentation about designing your VMware infrastructure is provided at: http://www.vmware.com/vmtn/resources/ or: http://www.vmware.com/resources/techresources/1059 Note: ESX Server hosts that use shared storage for virtual machine failover or load balancing must be in the same zone. You can have only one VMFS volume per VDisk.
8.9.7 Setting the HBA timeout for failover in VMware The timeout for failover for ESX hosts should be set to 30 seconds. For QLogic HBAs, the timeout depends on the PortDownRetryCount parameter. The timeout value is 2 * PortDownRetryCount + 5 sec. It is recommended to set the qlport_down_retry parameter to 14. For Emulex HBAs, the lpfc_linkdown_tmo and the lpcf_nodev_tmo parameters should be set to 30 seconds. To make these changes on your system, perform the following steps (Example 8-71): 1. Back up the file /etc/vmware/esx.cof. 2. Open /etc/vmware/esx.cof for editing. 3. The file includes a section for every installed SCSI device. 4. Locate your SCSI adapters and edit the parameters described above. 5. Repeat this for every installed HBA. Example 8-71 Setting HBA Timeout
[root@nile svc]# cp /etc/vmware/esx.conf /etc/vmware/esx.confbackup [root@nile svc]# vi /etc/vmware/esx.conf
290
Implementing the IBM System Storage SAN Volume Controller V4.3
8.9.8 Multipathing in ESX The ESX Server performs multipathing itself, and you do not need to install a multipathing driver such as SDD, either on the ESX server or on the guest operating systems.
8.9.9 Attaching VMware to VDisks First we make sure that the VMware host is logged into the SAN Volume Controller. In our examples, VMware ESX server V3.5 and the host name “Nile” is used. Enter the following command to check the status of the host: svcinfo lshost Example 8-72 shows that the host Nile is logged into the SVC with two HBAs. Example 8-72 lshost Nile
IBM_2145:ITSO-CLS1:admin>svcinfo lshost Nile id 1 name Nile port_count 2 type generic mask 1111 iogrp_count 2 WWPN 210000E08B892BCD node_logged_in_count 4 state active WWPN 210000E08B89B8C0 node_logged_in_count 4 state active Then we have to set the SCSI Controller Type in VMware. By default, ESX Server disables the SCSI bus sharing, and does not allow multiple virtual machines to access the same VMFS file at the same time (Figure 8-45 on page 292). But in many configurations, such as those for high availability, the virtual machines have to share the same VMFS file to share a disk. Log on to your Infrastructure Client, shut down the virtual machine, right-click it, and select Edit settings. Highlight the SCSI Controller, and select one of the three available settings, depending on your configuration: None: Disks cannot be shared by other virtual machines. Virtual: Disks can be shared by virtual machines on the same server. Physical: Disks can be shared by virtual machines on any server. Click OK to apply the setting.
Chapter 8. Host configuration
291
Figure 8-45 Changing SCSI Bus settings
Create your VDisks on the SVC and map them to the ESX hosts, as described in 7.8, “Assigning a VDisk to a host” on page 207. Note: If you want to use features such as VMotion®, the VDisks that own the VMFS file have to be visible to every ESX host that should be able to host the virtual machine. In SVC, this can be achieved by selecting the Allow the virtual disks to be mapped even if they are already mapped to a host check box. The VDisk has to have the same SCSI ID on each ESX host. For this example configuration, we have created one VDisk and have mapped it to our ESX host, as shown in Example 8-73. Example 8-73 Mapped VDisk to ESX host “Nile”
IBM_2145:ITSO-CLS1:admin>svcinfo lshostvdiskmap Nile id name SCSI_id vdisk_id vdisk_name wwpn 1 Nile 0 12 VMW_pool 210000E08B892BCD 60050768018301BF2800000000000010
vdisk_UID
ESX does not automatically scan for SAN changes (except when rebooting the whole ESX server). If you have made any changes to your SVC or SAN configuration, perform the following steps: 1. Open your VMware Infrastructure Client. 2. Select the host. 3. In the Hardware window, choose Storage Adapters. 4. Click Rescan.
292
Implementing the IBM System Storage SAN Volume Controller V4.3
To configure a storage device to use it in VMware, perform the following steps: 1. Open your VMware Infrastructure Client. 2. Select the host for which you want to see the assigned VDisks and open the Configuration tab. 3. In the Hardware window on the left side, click Storage. 4. To create a new storage pool, select Click here to create a datastore or Add storage if the yellow field does not appear (Figure 8-46).
Figure 8-46 VMWare Add Datastore
5. The Add storage wizard will appear. 6. Select create Disk/Lun and click Next. 7. Select the SVC VDisk you want to use for the datastore and click Next. 8. Review the Disk Layout and click Next. 9. Enter a datastore name and click Next. 10.Select a Block Size and enter the size of the new partition, then click Next. 11.Review your selections and click Finish. Now the created VMFS datastore appears in the Storage window (Figure 8-47). You will see the details for the highlighted datastore. Check if all the paths are available, and that the Path Selection is set to Most Recently Used.
Figure 8-47 VMWare Storage Configuration
If not all paths are available, check your SAN and storage configuration. After fixing the problem, select Refresh to perform a path rescan. The view will be updated to the new configuration.
Chapter 8. Host configuration
293
The recommended Multipath Policy for SVC is Most Recently Used. If you have to edit this policy, perform the following steps: 1. Highlight the datastore. 2. Click Properties. 3. Click Managed Paths. 4. Click Change (see Figure 8-48). 5. Select Most Recently Used. 6. Click OK. 7. Click Close. Now your VMFS datastore has been created and you can start using it for your guest operating systems.
8.9.10 VDisk naming in VMware In the Virtual Infrastructure Client, a VDisk is displayed as a sequence of three or four numbers, separated by colons (Figure 8-48): <SCSI HBA>:<SCSI target>:<SCSi VDIsk>: Where SCSI HBA The number of the SCSI HBA (may change). SCSI target The number of the SCSI target (may change). SCSI VDisk The number of the VDisk (never changes). disk partition The number of the disk partition (never changes). If the last number is not displayed, the name stands for the entire VDisk.
Figure 8-48 VDisk naming in VMware
294
Implementing the IBM System Storage SAN Volume Controller V4.3
8.9.11 Setting the Microsoft guest operating system timeout For a Microsoft Windows 2000 or 2003 Server installed as a VMware guest operating system, the disk timeout value should be set to 60 seconds. The instructions to perform this task are provided in 8.3.5, “Changing the disk timeout on Microsoft Windows Server” on page 236.
8.9.12 Extending a VMFS volume It is possible to extend VMFS volumes while virtual machines are running. First, you have to extend the VDisk on the SVC, and then you are able to extend the VMFS volume. Before performing these steps, we recommend having a backup of your data. Perform the following steps to extend a volume: 1. The VDisk can be expanded with the svctask expandvdisksize -size 1 -unit gb command (Example 8-74). Example 8-74 Expanding a VDisk in SVC
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk VMW_pool id 12 name VMW_pool IO_group_id 0 IO_group_name io_grp0 status online mdisk_grp_id 0 mdisk_grp_name MDG_DS45 capacity 60.0GB type striped formatted yes mdisk_id mdisk_name FC_id FC_name RC_id RC_name vdisk_UID 60050768018301BF2800000000000010 throttling 0 preferred_node_id 2 fast_write_state empty cache readwrite udid 0 fc_map_count 0 sync_rate 50 copy_count 1 copy_id 0 status online sync yes primary yes mdisk_grp_id 0 mdisk_grp_name MDG_DS45 type striped mdisk_id mdisk_name Chapter 8. Host configuration
295
fast_write_state empty used_capacity 60.00GB real_capacity 60.00GB free_capacity 0.00MB overallocation 100 autoexpand warning grainsize IBM_2145:ITSO-CLS1:admin>svctask expandvdisksize -size 5 -unit gb VMW_pool IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk VMW_pool id 12 name VMW_pool IO_group_id 0 IO_group_name io_grp0 status online mdisk_grp_id 0 mdisk_grp_name MDG_DS45 capacity 65.0GB type striped formatted yes mdisk_id mdisk_name FC_id FC_name RC_id RC_name vdisk_UID 60050768018301BF2800000000000010 throttling 0 preferred_node_id 2 fast_write_state empty cache readwrite udid 0 fc_map_count 0 sync_rate 50 copy_count 1 copy_id 0 status online sync yes primary yes mdisk_grp_id 0 mdisk_grp_name MDG_DS45 type striped mdisk_id mdisk_name fast_write_state empty used_capacity 65.00GB real_capacity 65.00GB free_capacity 0.00MB overallocation 100 autoexpand warning grainsize IBM_2145:ITSO-CLS1:admin>
296
Implementing the IBM System Storage SAN Volume Controller V4.3
2. Open the Virtual Infrastructure Client. 3. Select the host. 4. Select Configuration. 5. Select Storage Adapters. 6. Click Rescan. 7. Make sure that the Scan for new Storage Devices check box is marked and click OK. After the scan has completed, the new capacity is displayed in the Details section. 8. Click Storage. 9. Right-click the VMFS volume and click Properties. 10.Click Add Extend. 11.Select the new free space and click Next. 12.Click Next. 13.Click Finish. The VMFS volume has now been extended and the new space is ready for use.
8.9.13 Removing a datastore from an ESX host Before you remove a datastore from an ESX host, you have to migrate or delete all virtual machines that reside on this datastore. To remove it, perform the following steps: 1. Back up the data. 2. Open the Virtual Infrastructure Client. 3. Select the host. 4. Select Configuration. 5. Select Storage. 6. Highlight the datastore you want to remove. 7. Click Remove. 8. Read the warning, and if you are sure that you want to remove the datastore and delete all data on it, click Yes. 9. Remove the host mapping on the SVC or delete the VDisk (as shown in Example 8-75). 10.In the VI Client, select Storage Adapters. 11.Click Rescan. 12.Make sure that the Scan for new Storage Devices check box is marked and click OK. 13.After the scan completes, the disk disappears from the view. Your datastore has now been successfully removed from the system. Example 8-75 Remove VDisk host mapping - Delete VDisk
IBM_2145:ITSO-CLS1:admin>svctask rmvdiskhostmap -host Nile VMW_pool IBM_2145:ITSO-CLS1:admin>svctask rmvdisk VMW_pool
Chapter 8. Host configuration
297
8.10 SUN Solaris support information For the latest information about supported software and driver levels, always refer to this site: http://www-03.ibm.com/systems/storage/software/virtualization/svc/interop.html
8.10.1 Operating system versions and maintenance levels At the time of writing, Sun Solaris 8, Sun Solaris 9, and Sun Solaris 10 are supported in 64-bit only.
8.10.2 SDD dynamic pathing Solaris supports dynamic pathing when you either add more paths to an existing VDisk, or if you present a new VDisk to a host. No user intervention is required. SDD is aware of the preferred paths that SVC sets per VDisk. SDD will use a round robin algorithm when failing over paths, that is, it will try the next known preferred path. If this fails and all preferred paths have been tried, it will use a round robin algorithm on the non-preferred paths until it finds a path that is available. If all paths are unavailable, the VDisk will go offline. Therefore, it can take some time to perform path failover when multiple paths go offline. SDD under Solaris performs load balancing across the preferred paths where appropriate.
Veritas Volume Manager with DMP Dynamic Pathing Veritas VM with DMP automatically selects the next available I/O path for I/O requests dynamically without action from the administrator. VM with DMP is also informed when you repair or restore a connection, and when you add or remove devices after the system has been fully booted (provided that the operating system recognizes the devices correctly). The new JNI™ drivers support the mapping of new VDisks without rebooting the Solaris host. Note the following support characteristics: Veritas VM with DMP does not support preferred pathing with SVC. Veritas VM with DMP does support load balancing across multiple paths with SVC.
Co-existence with SDD and Veritas VM with DMP Veritas Volume Manager with DMP will coexist in “pass-thru” mode with SDD. This means that DMP will use the vpath devices provided by SDD.
OS Cluster Support Solaris with Symantec Cluster V4.1, Symantec SFHA and SFRAC V4.1/ 5.0, and Solaris with Sun Cluster V3.1/3.2 are supported at the time of writing.
SAN Boot support Note the following support characteristics: Boot from SAN is supported under Solaris 9 running Symantec Volume Manager. Boot from SAN is not supported when SDD is used as the multi-pathing software.
298
Implementing the IBM System Storage SAN Volume Controller V4.3
8.11 HP-UX configuration information For the latest information about HP-UX support, refer to: http://www-03.ibm.com/systems/storage/software/virtualization/svc/interop.html
8.11.1 Operating system versions and maintenance levels At the time of writing, HP-UX V11.0 and V11i v1/ v2 / v3 is supported (64-bit only).
8.11.2 Multipath solutions supported At the time of writing, SDD V1.6.3.0 for HP-UX is supported. Multipathing Software PV Link and Cluster Software Service Guard V11.14 / 11.16 / 11.17 / 11.18 are also supported, but in a cluster environment SDD is recommended.
SDD dynamic pathing HP-UX supports dynamic pathing when you either add more paths to an existing VDisk or if you present a new VDisk to a host. SDD is aware of the preferred paths that SVC sets per VDisk. SDD will use a round robin algorithm when failing over paths, that is, it will try the next known preferred path. If this fails and all preferred paths have been tried, it will use a round robin algorithm on the non-preferred paths until it finds a path that is available. If all paths are unavailable, the VDisk will go offline. It can take some time, therefore, to perform path failover when multiple paths go offline. SDD under HP-UX performs load balancing across the preferred paths where appropriate.
Physical Volume Links (PVLinks) Dynamic Pathing Unlike SDD, PVLinks does not load balance and is unaware of the preferred paths that SVC sets per VDisk. Therefore, SDD is strongly recommended, except when in a clustering environment or when using an SVC VDisk as your boot disk. When creating a Volume Group, specify the primary path you want HP-UX to use when accessing the Physical Volume presented by SVC. This path, and only this path, will be used to access the PV as long as it is available, no matter what the SVC's preferred path to that VDisk is. Therefore, care needs to be taken when creating Volume Groups so that the primary links to the PVs (and load) are balanced over both HBAs, FC switches, SVC nodes, and so on. When extending a Volume Group to add alternate paths to the PVs, the order in which you add these paths is HP-UX's order of preference should the primary path become unavailable. Therefore, when extending a Volume Group, the first alternate path you add should be from the same SVC node as the primary path, to avoid unnecessary node failover due to an HBA, FC link, or FC switch failure.
8.11.3 Co-existence of SDD and PV Links If you want to multipath a VDisk with PVLinks while SDD is installed, you need to make sure SDD does not configure a vpath for that VDisk. To do this, you need to put the serial number of any VDisks you want SDD to ignore in /etc/vpathmanualexcl.cfg. In the case of SAN Boot, if you are booting from an SVC VDisk, when you install SDD (from Version 1.6 onwards), SDD will automatically ignore the boot VDisk.
Chapter 8. Host configuration
299
SAN Boot support SAN Boot is supported on HP-UX by using PVLinks as the multi-pathing software on the boot device. PVLinks or SDD can be used to provide the multi-pathing support for the other devices attached to the system.
8.11.4 Using an SVC VDisk as a cluster lock disk ServiceGuard does not provide a way to specify alternate links to a cluster lock disk. When using an SVC VDisk as your lock disk, should the path to FIRST_CLUSTER_LOCK_PV become unavailable, the HP node will not be able to access the lock disk should a 50-50 split in quorum occur. To ensure redundancy, when editing you Cluster Configuration ASCII file, make sure that the variable FIRST_CLUSTER_LOCK_PV is a different path to the lock disk for each HP node in your cluster. For example, when configuring a two node HP cluster, make sure that FIRST_CLUSTER_LOCK_PV on HP server A is on a different SVC node and through a different FC switch to the FIRST_CLUSTER_LOCK_PV on HP server B.
8.11.5 Support for HP-UX greater than eight LUNs HPUX will not recognize more than eight LUNS per port using the generic SCSI behavior. In order to accommodate this behavior, SVC supports a “type” associated with a host. This can be set using the command svctask mkhost and modified using the command svctask chhost. The type can be set to generic, which is the default or HPUX. When an initiator port that is a member of a host that is of type HPUX accesses an SVC, the SVC will behave in the following way: Flat Space Addressing mode is used rather than the Peripheral Device Addressing Mode. When an Inquiry command for any page is sent to LUN 0 using Peripheral Device Addressing, it is reported as Peripheral Device Type 0Ch (controller). When any command other than an inquiry is sent to LUN 0 using Peripheral Device Addressing, SVC will respond as an unmapped LUN 0 would normally respond. When an inquiry is sent to LUN 0 using Flat Space Addressing, it is reported as Peripheral Device Type 00h (Direct Access Device) if a LUN is mapped at LUN 0 or 1Fh Unknown Device Type. When an inquiry is sent to an unmapped LUN that is not LUN 0 using Peripheral Device Addressing, the Peripheral qualifier returned is 001b and the Peripheral Device type is 1Fh (unknown or no device type). This is in contrast to the behavior for generic hosts, where peripheral Device Type 00h is returned.
8.12 Using SDDDSM, SDDPCM, and SDD Web interface After installing the SDDDSM or SDD driver there are specific commands available. To open a command window for SDDDSM or SDD, from the desktop, select Start → Programs → Subsystem Device Driver → Subsystem Device Driver Management. The command documentation for the different operating systems is available in the Multipath Subsystem Device Driver User Guides: http://www-1.ibm.com/support/docview.wss?rs=540&context=ST52G7&dc=DA400&uid=ssg1S7 000303&loc=en_US&cs=utf-8&lang=en 300
Implementing the IBM System Storage SAN Volume Controller V4.3
It is also possible to configure the multipath driver so that it offers a Web interface to run the commands. Before this can work, we need to configure the Web interface. Sddsrv does not bind to any TCP/IP port by default, but allows port binding to be dynamically enabled or disabled. For all platforms except Linux, the multipath driver package ships a template file of sddsrv.conf that is named sample_sddsrv.conf. On all UNIX platforms except Linux, the sample_sddsrv.conf file is located in the /etc directory. On Windows platforms, the sample_sddsrv.conf file is in the directory where SDD is installed. You must use the sample_sddsrv.conf file to create the sddsrv.conf file in the same directory as sample_sddsrv.conf by simply copying it and naming the copied file sddsrv.conf. You can then dynamically change port binding by modifying the parameters in sddsrv.conf. and changing the values of Enableport,Loopbackbind to True. Figure 8-49 shows the start window of the multipath driver Web interface.
Figure 8-49 SDD Web interface
8.13 Calculating the queue depth The queue depth is the number of I/O operations that can be run in parallel on a device. It is usually possible to set a limit on the queue depth on the subsystem device driver (SDD) paths (or equivalent) or the host bus adapter (HBA). Ensure that you configure the servers to limit the queue depth on all of the paths to the SAN Volume Controller disks in configurations that contain a large number of servers or virtual disks (VDisks). You might have a number of servers in the configuration that are idle, or do not initiate the calculated quantity of I/O operations. If so, you might not need to limit the queue depth.
Chapter 8. Host configuration
301
More information about the queue length is covered in “I/O queue depth handling in large SANs” on page 83.
8.14 Further sources of information For more information about host attachment and configuration to the SVC, refer to the IBM System Storage SAN Volume Controller: Host Attachment Guide, SC26-7563. For more information about SDDDSM or SDD configuration, refer to the IBM TotalStorage® Multipath Subsystem Device Driver User's Guide, SC30-4096.
302
Implementing the IBM System Storage SAN Volume Controller V4.3
9
Chapter 9.
SVC configuration and administration using the CLI In this chapter, we describe how to use the command-line interface (CLI) to perform additional and advanced configuration and administration tasks that were not covered in Chapter 6, “Quickstart configuration using the command-line interface” on page 157. We also discuss the backup and recovery function.
© Copyright IBM Corp. 2003-2008. All rights reserved.
303
9.1 Managing users using the CLI The access to the cluster and the right to perform different commands on the CLI is dependent on two functions: Adding an SSH key to the cluster. The user must be defined as an admin or service user. Changing the role of this admin user has an impact on the commands this user can perform on the CLI. Note: Every user that is designated as an admin user also has a default role of Monitor. Commands that can be performed by the admin or service user are: Admin user All command-line activities are permitted. Service user Can only run the following service commands: – svcservicetask – svcinfo
9.1.1 Maintaining SSH keys using the CLI You can use the command-line interface (CLI) to maintain SSH keys. (Details about how to create an SSH key pair are shown in 5.4.1, “Generating public and private SSH key pairs using PuTTY” on page 98). Perform the following steps to maintain the SSH keys: 1. Issue the svcinfo lssshkeys command to list the SSH keys that are available on the cluster (Example 9-1). Example 9-1 svctask lssshkeys IBM_2145:ITSO-CLS1:admin>svcinfo lsauth id ssh_label Role 0 admin Administrator
2. Open a command window on your server and upload the SSH key, using the pscp command, to the cluster, as shown in Example 9-2. Example 9-2 Upload the SSH key to the cluster using the pscp command pscp -load SVC_CL1 C:\sshkey\copyoperator1.pub [email protected] :/tmp/
3. Issue the svctask addsshkey command to install a new SSH key on the cluster. Each key is associated with an ID string that you define that can consist of up to 30 characters. Up to 100 keys can be stored on a cluster. You can add keys to provide either administrator access or service access. As shown in Example 9-3 on page 305, /tmp/copyoper1.pub is the name of the file that the SSH key will be saved in and copyoper1 is the label to associate with this key.
304
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 9-3 svctask addsshkey IBM_2145:ITSO-CLS1:admin>svctask addsshkey -label copyoper1 -file /tmp/copyoper1.pub -user admin IBM_2145:ITSO-CLS1:admin>svcinfo lssshkeys -user all id userid key_identifier 1 admin admin 2 admin copyoper1
4. You can issue the svctask rmsshkey command to remove an SSH key from the cluster. 5. You can issue the svctask rmallsshkeys command to remove all of the SSH keys from the cluster.
9.1.2 Managing user roles Authorization is based on roles that map to the administrator and service roles in an installation. Role-based security commands are used to restrict the administrative abilities of a user. These commands consist of the mkauth command (to assign a specific role of CopyOperator or Administrator), the rmauth command (to revert the assigned role to the default Monitor role), and the lsauth command (to view entries in the role-based security authorization table). The roles that are assigned by the mkauth command relate only to SSH sessions established with the SAN Volume Controller cluster using the admin user. The commands that you can initiate in an assigned role are determined by the role that will also be associated with the SSH key that established the session.
Viewing user roles Use the svcinfo lsauth command, as shown in Example 9-4, to list all users. Example 9-4 svcinfo lsauth
IBM_2145:ITSO-CLS1:admin>svcinfo lsauth id ssh_label Role 0 admin Administrator 1 copyoper1 Monitor
Changing an admin user role The mkauth command allows you to change the default role of Monitor to either CopyOperator or Administrator. The roles that are assigned by the mkauth command apply only to SSH sessions that have been established within the SAN Volume Controller cluster by an Administrator. The commands that you can initiate in an assigned role are determined by the role that is associated with the SSH key that established the session. The full syntax of this command is: >>- svctask -- -- mkauth -- -- -label -- ssh_key_label -- -role -role_name--------->< Note the following explanation: ssh_key_label: Specifies the identifier associated with the SSH key. role: The name of the role assigned to the user.
Chapter 9. SVC configuration and administration using the CLI
305
Commands allowed per role The Monitor role allows a user to initiate the following SVC CLI commands: – svcinfo commands All svcinfo commands – svctask commands Only the following commands: dumpinternallog, dumperrlog, finderr – svcservicetask commands Only the following commands: finderr and dumperrlog – Other commands: svqueryclock The CopyOperator role allows a user to initiate the following SAN Volume Controller CLI commands and functions: – svcinfo commands All svcinfo commands – svctask commands Only the following commands: finderr, dumperrlog, dumpinternallog, prestartfcconsistgrp, startfcconsistgrp, stopfcconsistgrp, chfcconsistgrp, prestartfcmap, startfcmap, stopfcmap, chfcmap, startrcconsistgrp, stoprcconsistgrp, switchrcconsistgrp, chrcconsistgrp, startrcrelationship, stoprcrelationship, switchrcrelationship, chrcrelationship, chpartnership – svcservicetask commands Only the following commands: finderr and dumperrlog – svcconfig backup and restore tool A configuration backup can be performed by an Administrator. A configuration restore can only be performed by an Administrator. – Other commands: svcqueryclock As an Administrator, a user can initiate all commands, and perform configurations and backups using the svcconfig backup and restore tool. The command to create an admin user role is svctask mkauth, as shown in Example 9-5. Example 9-5 svctask mkauth
IBM_2145:ITSO-CLS1:admin>svctask mkauth -label copyoper1 -role CopyOperator This command creates a user authorization role called copyoper1 as CopyOperator, and we can display it using the svcinfo lsauth command, as shown in Example 9-6. Example 9-6 svcinfo lsauth command
IBM_2145:ITSO-CLS1:admin>svcinfo lsauth id ssh_label Role 0 admin Administrator 1 copyoper1 CopyOperator Use the svctask rmauth command to change a user´s assigned authorization role from that of CopyOperator or Administrator back to the default authorization of Monitor. To verify the change, run the svcinfo lsauth command. Both of these commands are shown in Example 9-7 on page 307. 306
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 9-7 svctask rmauth command
IBM_2145:ITSO-CLS1:admin>svctask rmauth -label copyoper1 IBM_2145:ITSO-CLS1:admin>svcinfo lsauth id ssh_label Role 0 admin Administrator 1 copyoper1 CopyOperator
Removing an admin user role The rmauth command removes the authorization that is assigned to CopyOperator and Administrator users. In Example 9-8, the role of admincl1 has been reset to the default role of Monitor, and the new role can be seen by issuing the svcinfo lsauth command, as shown in the same example. Example 9-8 New role
IBM_2145:ITSO-CLS1:admin>svctask rmauth -label copyoper1 IBM_2145:ITSO-CLS1:admin>svcinfo lsauth id ssh_label Role 0 admin Administrator 1 copyoper1 Monitor You have now completed the tasks required to manage admin user roles using the CLI.
9.2 Managing the cluster Here we discuss the procedures used to manage the cluster.
Command syntax Two major command sets are available: The svcinfo command set allows us to query the various components within the IBM System Storage SAN Volume Controller (SVC) environment. The svctask command set allows us to make changes to the various components within the SVC. When the command syntax is shown, you see some parameters in square brackets, for example, [parameter]. This indicates that the parameter is optional in most if not all instances. Anything that is not in square brackets is required information. You can view the syntax of a command by entering one of the following commands:
svcinfo -?: Shows a complete list of information commands. svctask -?: Shows a complete list of task commands. svcinfo commandname -?: Shows the syntax of information commands. svctask commandname -?: Shows the syntax of task commands. svcinfo commandname -filtervalue?: Shows what filters you can use to reduce output of the information commands. Note: You can also use -h instead of -?, for example, svcinfo -h or svctask commandname -h.
Chapter 9. SVC configuration and administration using the CLI
307
If you look at the syntax of the command by typing svcinfo command name -?, you often see -filter listed as a parameter. Be aware that the correct parameter is -filtervalue, as stated above. Tip: You can use the up and down keys on your keyboard to recall commands recently issued. Then, you can use the left and right, backspace, and delete keys to edit commands before you resubmit them.
9.2.1 Organizing on screen content Sometimes the output of a command can be long and difficult to read on screen. In cases where you need information about a subset of the total number of available items, you can use filtering to reduce the output to a more manageable size.
Filtering To reduce the output that is displayed by an svcinfo command, you can specify a number of filters, depending on which svcinfo command you are running. To see which filters are available, type the command followed by the -filtervalue? flag, as shown in Example 9-9. Example 9-9 svcinfo lsvdisk -filtervalue? command
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk -filtervalue? Filters for this view are : name id IO_group_id IO_group_name status mdisk_grp_name mdisk_grp_id capacity type FC_id FC_name RC_id RC_name vdisk_name vdisk_id vdisk_UID fc_map_count copy_count When you know the filters, you can be more selective in generating output: Multiple filters can be combined to create specific searches. You can use an * as a wildcard when using names. When capacity is used, the units must also be specified using -u b | kb | mb | gb | tb | pb. For example, if we issue the svcinfo lsvdisk command with no filters, we see the output shown in Example 9-10 on page 309.
308
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 9-10 svcinfo lsvdisk command: no filters
id,name,IO_group_id,IO_group_name,status,mdisk_grp_id,mdisk_grp_name,capacity,typ e,FC_id,FC_name,RC_id,RC_name,vdisk_UID,fc_map_count,copy_count 0,vdisk0,0,io_grp0,online,1,MDG_DS47,10.0GB,striped,,,,,60050768018301BF2800000000 000000,0,1 1,vdisk1,1,io_grp1,online,1,MDG_DS47,100.0GB,striped,,,,,60050768018301BF280000000 0000001,0,1 2,vdisk2,1,io_grp1,online,0,MDG_DS45,40.0GB,striped,,,,,60050768018301BF2800000000 000002,0,1 3,vdisk3,1,io_grp1,online,0,MDG_DS45,80.0GB,striped,,,,,60050768018301BF2800000000 000003,0,1 Tip: The -delim : parameter truncates the on screen content and separates data fields with colons as opposed to wrapping text over multiple lines. That is normally used in case you need to grab some reports during script execution. If we now add a filter to our svcinfo command (such as FC_name), we can reduce the output, as shown in Example 9-11. Example 9-11 svcinfo lsvdisk command: with filter
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk -filtervalue IO_group_id=0 -delim , id,name,IO_group_id,IO_group_name,status,mdisk_grp_id,mdisk_grp_name,capacity,type ,FC_id,FC_name,RC_id,RC_name,vdisk_UID,fc_map_count,copy_count 0,vdisk0,0,io_grp0,online,1,MDG_DS47,10.0GB,striped,,,,,60050768018301BF2800000000 000000,0,1 IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk -filtervalue mdisk_grp_name=*7 -delim , id,name,IO_group_id,IO_group_name,status,mdisk_grp_id,mdisk_grp_name,capacity,type ,FC_id,FC_name,RC_id,RC_name,vdisk_UID,fc_map_count,copy_count 0,vdisk0,0,io_grp0,online,1,MDG_DS47,10.0GB,striped,,,,,60050768018301BF2800000000 000000,0,1 1,vdisk1,1,io_grp1,online,1,MDG_DS47,100.0GB,striped,,,,,60050768018301BF280000000 0000001,0,1 The first command shows all Virtual Disks (VDisks) with the IO_group_id=0. The second command shows us all VDisks where the mdisk_grp_name ending is 7. The wildcard * can be used when names are used.
9.2.2 Viewing cluster properties Use the svcinfo lscluster command to display summary information about all clusters visible to the SVC. To display more detailed information about a specific cluster, run the command again and append the cluster name parameter (for example, SVC1). Both of these commands are shown in Example 9-12. Example 9-12 svcinfo lscluster command
IBM_2145:ITSO-CLS1:admin>svcinfo lscluster -delim , id,name,location,partnership,bandwidth,cluster_IP_address,cluster_service_IP_addre ss,cluster_IP_address_6,cluster_service_IP_address_6,id_alias 0000020060C06FCA,ITSO-CLS1,local,,,9.43.86.117,9.43.86.118,,,0000020060C06FCA IBM_2145:ITSO-CLS1:admin>svcinfo lscluster ITSO-CLS1 id 0000020060C06FCA Chapter 9. SVC configuration and administration using the CLI
309
name ITSO-CLS1 location local partnership bandwidth cluster_IP_address 9.43.86.117 cluster_service_IP_address 9.43.86.118 total_mdisk_capacity 756.0GB space_in_mdisk_grps 756.0GB space_allocated_to_vdisks 152.00GB total_free_space 604.0GB statistics_status off statistics_frequency 15 required_memory 8192 cluster_locale en_US SNMP_setting none SNMP_community SNMP_server_IP_address subnet_mask 255.255.252.0 default_gateway 9.43.85.1 time_zone 520 US/Pacific email_setting email_id code_level 4.3.0.0 (build 8.15.0806110000) FC_port_speed 2Gb console_IP 9.43.86.115:9080 id_alias 0000020060C06FCA gm_link_tolerance 300 gm_inter_cluster_delay_simulation 0 gm_intra_cluster_delay_simulation 0 email_server email_server_port email_reply email_contact email_contact_primary email_contact_alternate email_contact_location email_state invalid email_user_count 0 inventory_mail_interval 0 cluster_IP_address_6 cluster_service_IP_address_6 prefix_6 default_gateway_6 total_vdiskcopy_capacity 230.00GB total_used_capacity 90.00GB total_overallocation 30 total_vdisk_capacity 230.00GB
310
Implementing the IBM System Storage SAN Volume Controller V4.3
9.2.3 Changing cluster settings Use the svctask chcluster command to change the settings of the cluster. This command modifies specific features of a cluster. Multiple features can be changed by issuing a single command. If the cluster IP address is changed, the open command-line shell closes during the processing of the command. You must reconnect to the new IP address. The service IP address is not used until a node is expelled from the cluster. If this node cannot rejoin the cluster, you can bring the node up in service mode. In this mode, the node can be accessed as a stand-alone node using the service IP address. All command parameters are optional; however, you must specify at least one parameter. The full syntax of the svctask chcluster command is: >>- svctask -- -- chcluster -- ---------------------------------> >--+------------------------------------+-- --------------------> '- -clusterip -- cluster_ip_address -' >--+----------------------------------------+-- ----------------> '- -serviceip --+- service_ip_address -+-' '- DHCP ---------------' >--+-------------------------+-- --+-----------------------+----> '- -name -- cluster_name -' '- -admpwd -- password -' >-- --+---------------------------+-- --------------------------> '- -servicepwd -- password -' >--+--------------------------+-- ------------------------------> '- -gw -- default_gateway -' >--+------------------------+-- --------------------------------> '- -mask -- subnet_mask -' >--+--------------------------+-- --+----------------------+----> '- -speed -- fabric_speed -' '- -alias -- id_alias -' >-- --+--------------------------------------+-- ---------------> '- -icatip -- icat_console_ip_address -' >--+---------------------------------+-- -----------------------> '- -invemailinterval -- interval -' >--+--------------------------------------+-- ------------------> '- -gmlinktolerance -- link_tolerance -' >--+-------------------------------------------------------------+--> '- -gminterdelaysimulation -- inter_cluster_delay_simulation -' >--+-------------------------------------------------------------+--> '- -gmintradelaysimulation -- intra_cluster_delay_simulation -' >--+-------------------------------------------+----------------> '- -clusterip_6 -- ipv6_cluster_ip_address -' Chapter 9. SVC configuration and administration using the CLI
311
>--+-------------------------------------------+-- -------------> '- -serviceip_6 -- ipv6_service_ip_address -' >--+----------+-- --+------------+-- ---------------------------> '- -rm_ip -' '- -rm_ip_6 -' >--+---------------------------------+-- -----------------------> '- -gw_6 -- ipv6_default_gateway -' >--+------------------------------------+-- --------------------> '- -prefix_6 -- ipv6_network_prefix -' >--+-------------------------------------+-- ------------------>< '- -icatip_6 -- ipv6_icat_ip_address -' The parameters of this syntax are: -clusterip cluster_ip_address (Optional) Specifies the new cluster IP address. Note: After the cluster IP address is changed, you lose the open shell connection to the cluster. You must reconnect with the newly specified IP address. -serviceip service_ip_address (Optional) Specifies the new service IPv4 address. This address is the address to use if the node must be started after it has been released from the cluster. Specify either a fixed IPv4 address or, to use a dynamic IP address, specify DHCP. -name cluster_name (Optional) Specifies a new name for the cluster. -admpwd password (Optional) Specifies a new administrator password. You can specify this parameter with or without the password. If this parameter is not followed by a password, you are prompted for the password. Note: Only a user with administrator authority can change the password. -servicepwd password (Optional) Specifies a new service user password. You can specify this parameter with or without the password. If the parameter is not followed by a password, you are prompted for the password. When you type the password in response to the prompt, the password is not displayed. Note: Only a user with administrator authority can change the password. -gw default_gateway (Optional) Specifies the new default gateway IPv4 address of the cluster. -mask subnet_mask (Optional) Specifies the new IPv4 subnet mask of the cluster.
312
Implementing the IBM System Storage SAN Volume Controller V4.3
-speed fabric_speed (Optional) Specifies the speed of the fabric to which this cluster is attached. Valid values are 1 or 2 (GB). For 4/8 Gbps fabrics, leave his option empty; the fabric speed will be automatically negotiated. Attention: Changing the speed on a running cluster breaks I/O service to the attached hosts. Before changing the fabric speed, stop I/O from active hosts and force these hosts to flush any cached data by unmounting volumes (for UNIX host types) or by removing drive letters (for Windows host types). Some hosts might need to be rebooted to detect the new fabric speed. The fabric speed setting applies only to the 4F2 and 8F2 model nodes in a cluster. The 8F4 nodes automatically negotiate the fabric speed on a per-port basis. -alias id_alias (Optional) Specifies an alternate name that does not change the basic ID for the cluster, but does influence the VDisk_UID of every vdiskhostmap, both existing and new. These objects appear to have been created for a cluster whose ID matches the alias. -icatip icat_console_ip_address (Optional) Specifies the new IP address that is used by the cluster. The format of this IP address must be a dotted decimal notation with the port, for example, 255.255.255.255:8080. If you specify this parameter, it overwrites any existing -icatip_6 address. -icatip_6 icat_console_ipv6_address (Optional) Specifies the new IPv6 address that is used by the cluster. If you specify this parameter, it overwrites any existing -icatip address. The format of the IPv6 address must be one of the following: – Eight colon-separated groups of four hexadecimal digits, for example, [1234:1234:abcd:0123:0000:0000:7689:6576]:23 – Eight colon-separated groups of hexadecimal digits with leading zeros omitted, for example, [1234:1234:abcd:123:0000:0000:7689:6576]:23 – With suppression of one or more consecutive all 0 groups, for example, [1234:1234:abcd:123:7689:6576]:23 -invemailinterval interval (Optional) Specifies the interval at which inventory e-mails are sent to the designated e-mail recipients. The interval range is 0 to 15. The interval is measured in days. Setting the value to 0 turns the inventory e-mail notification function off. -gmlinktolerance link_tolerance (Optional) Specifies the length of time, in seconds, for which an inadequate intercluster link is tolerated for a Global Mirror operation. The parameter accepts values from 60 to 86 400 seconds in steps of 10 seconds. The default is 300 seconds. You can disable the link tolerance by entering a value of zero (0) for this parameter. -gminterdelaysimulation inter_cluster_delay_simulation (Optional) Specifies the intercluster delay simulation, which simulates the Global Mirror round trip delay between two clusters, in milliseconds. The default is 0; the valid range is 0 to 100 milliseconds.
Chapter 9. SVC configuration and administration using the CLI
313
-gmintradelaysimulation intra_cluster_delay_simulation (Optional) Specifies the intracluster delay simulation, which simulates the Global Mirror round trip delay in milliseconds. The default is 0; the valid range is 0 to 100 milliseconds. -rm_ip (Optional) Deletes all IPv4 addresses in the cluster. -rm_ip_6 (Optional) Deletes all IPv6 addresses in the cluster. -clusterip_6 ipv6_cluster_ip_address (Optional) Specifies the new cluster IPv6 address. Note: After the cluster IP address is changed, you lose the open shell connection to the cluster. You must reconnect with the newly specified IP address. -serviceip_6 ipv6_service_ip_address (Optional) Specifies the service IPv6 address for the cluster. Use this address if the node must be started after it has been released from the cluster. Specify either a fixed IPv6 address, or, to use a dynamic IPv6 address, specify DHCP. -gw_6 ipv6_default_gateway (Optional) Specifies the IPv6 default gateway for the cluster. -prefix_6 ipv6_network_prefix (Optional) Specifies the IPv6 network prefix for the cluster. The ipv6_network_prefix value is 0 - 127.
9.2.4 Maintaining cluster passwords To change the administrator user password, issue the svctask chcluster -admpwd command. To change the service user password, issue the svctask chcluster -admpwd command. Note: If you do not want the password to display as you enter it on the command line, omit the new password. The command-line tool then prompts you to enter and confirm the password without the password being displayed. The command to change the admin and the service password is shown in Example 9-13. Example 9-13 svctask chcluster -admpwd
IBM_2145:ITSO-CLS1:admin>svctask chcluster -admpwd admin -servicepwd service This command changes the current admin password to admin and the current service password to service.
9.2.5 Modifying IP addresses List the IP address of the cluster by issuing the svcinfo lscluster command. Modify the IP address by issuing the svctask chcluster command. You can either specify a static IP address or have the system assign a dynamic IP address, as shown in Example 9-14 on page 315.
314
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 9-14 svctask chcluster -clusterip / - serviceip
IBM_2145:ITSO-CLS1:admin>svctask chcluster -clusterip 9.43.86.130 -serviceip 9.43.86.131 This command changes the current IP address of the cluster to 9.43.86.130 and the current service IP address to 9.43.86.131. Important: If you specify a new cluster IP address, the existing communication with the cluster through the CLI is broken and the PuTTY application automatically closes. You must relaunch the PuTTY application and point to the new IP address. Modifying the IP address of the cluster, although quite simple, means some reconfiguration for other items within the SVC environment (such as reconfiguring our PuTTY application and the central administration GUI).
Supported IP address formats Table 9-1 shows the IP address formats. Table 9-1 ip_address_list formats IP type
ip_address_list format
IPv4 (no port set, SVC uses default)
1.2.3.4
IPv4 with specific port
1.2.3.4:22
Full IPv6, default port
1234:1234:0001:0123:1234:1234:1234:1234
Full IPv6, default port,leading zeros suppressed
1234:1234:1:123:1234:1234:1234:1234
Full IPv6 with port
[2002:914:fc12:848:209:6bff:fe8c:4ff6]:23
Zero-compressed IPv6, default port
2002::4ff6
Zero-compressed IPv6 with port
[2002::4ff6]:23
We have now completed the tasks required to change the IP addresses (cluster and service) of the SVC environment.
9.2.6 Setting the cluster time zone and time Use the -timezone parameter to specify the numeric ID of the time zone that you want to set. Issue the svcinfo lstimezones command to list the timezones that are available on the cluster. A list of valid time zone settings are displayed in a list. Note: If you have changed the time zone, you must clear the error log dump directory before you can view the error log through the Web application. Refer to 6.2, “Setting the cluster time zone and time” on page 161 to see more information about the cluster time zone.
Chapter 9. SVC configuration and administration using the CLI
315
9.2.7 Start statistics collection Statistics are collected at the end of each sampling period (as specified by the -interval parameter). These statistics are written to a file. A new file is created at the end of each sampling period. Separate files are created for MDisks, VDisks, and node statistics. Use the svctask startstats command to start the collection of statistics, as shown in Example 9-15. Example 9-15 svctask startstats command
IBM_2145:ITSO-CLS1:admin>svctask startstats -interval 15 The interval we specify (minimum 1, maximum 60) is in minutes. This command starts statistics collection and gathers data at 15 minute intervals. Note: To verify that statistics collection is set, display the cluster properties again, as shown in Example 9-16. Example 9-16 Statistics collection status and frequency
IBM_2145:ITSO-CLS1:admin>svcinfo lscluster ITSO-CLS1 id 0000020060C06FCA name ITSO-CLS1 location local partnership bandwidth cluster_IP_address 9.43.86.117 cluster_service_IP_address 9.43.86.118 total_mdisk_capacity 756.0GB space_in_mdisk_grps 756.0GB space_allocated_to_vdisks 182.00GB total_free_space 574.0GB statistics_status on statistics_frequency 15 required_memory 8192 cluster_locale en_US SNMP_setting none SNMP_community SNMP_server_IP_address subnet_mask 255.255.252.0 default_gateway 9.43.85.1 time_zone 520 US/Pacific email_setting email_id code_level 4.3.0.0 (build 8.15.0806110000) FC_port_speed 2Gb console_IP 127.0.0.1:9080 id_alias 0000020060C06FCA gm_link_tolerance 300 gm_inter_cluster_delay_simulation 0 gm_intra_cluster_delay_simulation 0 email_server email_server_port email_reply email_contact 316
Implementing the IBM System Storage SAN Volume Controller V4.3
email_contact_primary email_contact_alternate email_contact_location email_state invalid email_user_count 0 inventory_mail_interval 0 cluster_IP_address_6 cluster_service_IP_address_6 prefix_6 default_gateway_6 total_vdiskcopy_capacity 260.00GB total_used_capacity 120.00GB total_overallocation 34 total_vdisk_capacity 260.00GB We have now completed the tasks required to start statistics collection on the cluster.
9.2.8 Stopping a statistics collection Use the svctask stopstats command to start the collection of statistics within the cluster (Example 9-17). Example 9-17 svctask stopstats
IBM_2145:ITSO-CLS1:admin>svctask stopstats This command stops the statistics collection. Do not expect any prompt message from this command. To verify that the statistics collection is stopped, display the cluster properties again, as shown in Example 9-18. Example 9-18 Statistics collection status and frequency
IBM_2145:ITSO-CLS1:admin>svcinfo lscluster ITSO-CLS1 id 0000020060C06FCA name ITSO-CLS1 location local partnership bandwidth cluster_IP_address 9.43.86.117 cluster_service_IP_address 9.43.86.118 total_mdisk_capacity 756.0GB space_in_mdisk_grps 756.0GB space_allocated_to_vdisks 182.00GB total_free_space 574.0GB statistics_status off statistics_frequency 15 required_memory 8192 cluster_locale en_US SNMP_setting none SNMP_community SNMP_server_IP_address subnet_mask 255.255.252.0 default_gateway 9.43.85.1 time_zone 520 US/Pacific Chapter 9. SVC configuration and administration using the CLI
317
email_setting email_id code_level 4.3.0.0 (build 8.15.0806110000) FC_port_speed 2Gb console_IP 9.43.86.115:9080 id_alias 0000020060C06FCA gm_link_tolerance 300 gm_inter_cluster_delay_simulation 0 gm_intra_cluster_delay_simulation 0 email_server email_server_port email_reply email_contact email_contact_primary email_contact_alternate email_contact_location email_state invalid email_user_count 0 inventory_mail_interval 0 cluster_IP_address_6 cluster_service_IP_address_6 prefix_6 default_gateway_6 total_vdiskcopy_capacity 260.00GB total_used_capacity 120.00GB total_overallocation 34 total_vdisk_capacity 260.00GB Notice that the interval parameter is not changed but the status is off. We have now completed the tasks required to stop statistics collection on our cluster. For further information about statistics, refer to 4.3.1, “Collecting performance statistics” on page 88.
9.2.9 Audit Log commands Starting from software release 4.1.0, all action commands issued as a result of actions in the CLI, ICAT GUI, and native GUI are logged to the audit log. View commands and commands in service mode are not logged. The audit log cannot be disabled in any way. The audit log entries provide the following information: Identity of the user who issued the action command. – From the command-line interface, the user name (administrator or service), and the label that is associated with the user’s public SSH key in the authorized keys file. – From the native Web pages, the user’s identity (admin[web] or service[web]) according to which user name the user authenticated with. – From the SAN Volume Controller Console, the user’s identity (administrator), the label that is associated with the CIMOM key in the authorized keys file, and the user name that has been recorded by the CIMOM when the SAN Volume Controller Console user authenticated with the CIMOM. The name of the actionable command. The time stamp of when the actionable command was issued on the configuration node.
318
Implementing the IBM System Storage SAN Volume Controller V4.3
The parameters that were issued with the actionable command. Use the svcinfo catauditlog -first 5 command to return a list of five in-memory Audit Log entries, as shown in Example 9-19. Example 9-19 svcinfo catauditlog command
IBM_2145:ITSO-CLS1:admin>svcinfo catauditlog -delim , -first 5 audit_seq_no,timestamp,cluster_user,ssh_label,ssh_ip_address,icat_user,result,res_ obj_id,action_cmd 76,080619104712,admin,admincl1,9.43.86.111,superuser,0,2,svctask mkhost -name Kanaga -hbawwpn 10000000C932A800:10000000C932A7FB -force -iogrp io_grp0:io_grp1:io_grp2:io_grp3 -mask 15 77,080619110346,admin,admincl1,9.43.86.111,superuser,0,3,svctask mkhost -name Siam -hbawwpn 210000E08B18D48F:210000E08B18FF8A -force -iogrp io_grp0:io_grp1:io_grp2:io_grp3 -mask 15 78,080619111924,admin,admin,9.145.130.56,,0,,svctask startstats -interval 15 79,080619134016,admin,admin,9.43.86.115,superuser,0,,svctask chcluster -icatip 9.43.86.115:9080 80,080619134139,admin,admin,9.145.130.56,,0,,svctask stopstats If you need to dump the contents of the in-memory audit log to a file on the current configuration node, use the command svctask dumpauditlog. This command does not provide any feedback, just the prompt. To obtain a list of the audit log dumps, use svcinfo lsauditlogdumps, as described in Example 9-20. Example 9-20 svctask dumpauditlog / svcinfo lsauditlogdumps command
IBM_2145:ITSO-CLS1:admin>svctask dumpauditlog IBM_2145:ITSO-CLS1:admin>svcinfo lsauditlogdumps id auditlog_filename 0 auditlog_0_80_20080619134139_0000020060c06fca
9.2.10 Status of discovery Use the svcinfo lsdiscoverystatus command, as shown in Example 9-21, to determine if a discovery operation is in progress or not. The output of this command is the status of active or inactive. Example 9-21 lsdiscoverystatus command
IBM_2145:ITSO-CLS1:admin>svcinfo lsdiscoverystatus status inactive
9.2.11 Status of copy operation Use the svcinfo lscopystatus command, as shown in Example 9-22, to determine if a file copy operation is in progress or not. Only one file copy operation can be performed at a time. The output of this command is the status of active or inactive. Example 9-22 lscopystatus command
IBM_2145:ITSO-CLS1:admin>svcinfo lsdiscoverystatus status inactive
Chapter 9. SVC configuration and administration using the CLI
319
9.2.12 Shutting down a cluster If all input power to an SVC cluster is to be removed for more than a few minutes (for example, if the machine room power is to be shut down for maintenance), it is important to shut down the cluster before removing the power. The reason for this is that if the input power is removed from the uninterruptible power supply units without first shutting down the cluster and the uninterruptible power supplies themselves, the uninterruptible power supply units remain operational and eventually become drained of power. When input power is restored to the uninterruptible power supplies, they start to recharge. However, the SVC does not permit any input/output (I/O) activity to be performed to the VDisks until the uninterruptible power supplies are charged enough to enable all the data on the SVC nodes to be destaged in the event of a subsequent unexpected power loss. Recharging the uninterruptible power supply can take as long as two hours. Shutting down the cluster prior to removing input power to the uninterruptible power supply units prevents the battery power from being drained. It also makes it possible for I/O activity to be resumed as soon as input power is restored. You can use the following procedure to shut down the cluster: 1. Use the svctask stopcluster command to shut down your SVC cluster (Example 9-23). Example 9-23 svctask stopcluster IBM_2145:ITSO-CLS1:admin>svctask stopcluster Are you sure that you want to continue with the shut down?
This command shuts down the SVC cluster. All data is flushed to disk before the power is removed. At this point you lose administrative contact with your cluster, and the PuTTY application automatically closes. 2. You will be presented with the following message: Warning: Are you sure that you want to continue with the shut down? Ensure that you have stopped all FlashCopy mappings, Metro Mirror (Remote Copy) relationships, data migration operations, and forced deletions before continuing. Entering y to this will execute the command. No feedback is then displayed. Entering anything other than y(es) or Y(ES) will result in the command not executing. No feedback is displayed. Important: Before shutting down a cluster, quiesce all I/O operations that are destined for this cluster because you will lose access to all VDisks being provided by this cluster. Failure to do so can result in failed I/O operations being reported to the host operating systems. There is no need to do this when you shut down a node. Begin the process of quiescing all I/O to the cluster by stopping the applications on the hosts that are using the VDisks provided by the cluster. 3. We have now completed the tasks required to shut down the cluster. To shut down the uninterruptible power supplies, press the power button on their front panels.
320
Implementing the IBM System Storage SAN Volume Controller V4.3
Note: To restart the cluster, you must first restart the uninterruptible power supply units by pressing the power button on their front panels. Then you go to the service panel of one of the nodes within the cluster and press the power on button. After it is fully booted up (for example, displaying Cluster: on line 1 and the cluster name on line 2 of the panel), you can start the other nodes in the same way. As soon as all nodes are fully booted, you can re-establish administrative contact using PuTTY, and your cluster is fully operational again.
9.3 Working with nodes This section explains the various configuration and administration tasks that you can perform on the nodes within an SVC cluster.
9.3.1 I/O groups This section explains the tasks that you can perform at an I/O group level.
9.3.2 Viewing I/O group details Use the svcinfo lsiogrp command, as shown in Example 9-24, to view information about I/O groups defined within the SVC environment. Example 9-24 I/O group details
IBM_2145:ITSO-CLS1:admin>svcinfo lsiogrp id name node_count 0 io_grp0 2 1 io_grp1 2 2 io_grp2 0 3 io_grp3 0 4 recovery_io_grp 0
vdisk_count 3 4 0 0 0
host_count 3 3 2 2 0
As we can see, the SVC predefines five I/O groups. In a four node cluster (like ours), only two I/O groups are actually in use. The other I/O groups (io_grp2 and io_grp3) are for a six or eight node cluster. The recovery I/O group is a temporary home for VDisks when all nodes in the I/O group that normally owns them have suffered multiple failures. This allows us to move the VDisks to the recovery I/O group and then into a working I/O group. Of course, while temporarily assigned to the recovery I/O group, I/O access is not possible.
9.3.3 Renaming an I/O group Use the svctask chiogrp command to rename an I/O group (Example 9-25). Example 9-25 svctask chiogrp
IBM_2145:ITSO-CLS1:admin>svctask chiogrp -name io_grpA io_grp1 This command renames the I/O group io_grp1 to io_grpA.
Chapter 9. SVC configuration and administration using the CLI
321
Note: The chiogrp command specifies the new name first. If you want to provide a name, you can use letters A to Z, a to z, numbers 0 to 9, the dash ’-’, and the underscore ’_’. It can be between one and 15 characters in length. However, it cannot start with a number, dash, or the word iogrp, since this prefix is reserved for SVC assignment only. To see whether the renaming was successful, issue the svcinfo lsiogrp command again and you should see the change reflected. We have now completed the tasks required to rename an I/O group.
9.3.4 Adding and removing hostiogrp To map or unmap a specific host object to a specific I/O group in order to reach the maximum hosts supported by an SVC cluster, use the svctask addhostiogrp command to map a specific host to a specific I/O group, as shown in Example 9-26. Example 9-26 svctask addhostiogrp
IBM_2145:ITSO-CLS1:admin>svctask addhostiogrp -iogrp 1 Kanaga Parameters: -iogrp iogrp_list -iogrpall Specifies a list of one or more I/O groups that must be mapped to the host. This parameter is mutually exclusive with -iogrpall. The -iogrpall specifies that all the I/O groups must be mapped to the specified host. This parameter is mutually exclusive with -iogrp. -host host_id_or_name Identify the host either by ID or name to which the I/O groups must be mapped. Use the svctask rmhostiogrp command to unmap a specific host to a specific I/O group, as shown in Example 9-27. Example 9-27 svctask rmhostiogrp command
IBM_2145:ITSO-CLS1:admin>svctask rmhostiogrp -iogrp 0 Kanaga Parameters: -iogrp iogrp_list -iogrpall Specifies a list of one or more I/O groups that must be unmapped to the host. This parameter is mutually exclusive with -iogrpall. The -iogrpall specifies that all the I/O groups must be unmapped to the specified host. This parameter is mutually exclusive with -iogrp. -force If the removal of a host to I/O group mapping will result in the loss of VDisk to host mappings, then the command must fail if the -force flag has not been used. The -force flag will, however, override such behavior and force the host to I/O group mapping to be deleted. host_id_or_name Identify the host either by ID or name to which the I/O groups must be mapped.
322
Implementing the IBM System Storage SAN Volume Controller V4.3
9.3.5 Listing I/O groups To list all the I/O groups mapped to the specified host and vice versa, use the svcinfo lshostiogrp command, as shown in Example 9-28. Example 9-28 svcinfo lshostiogrp
IBM_2145:ITSO-CLS1:admin>svcinfo lshostiogrp Kanaga id name 1 io_grp1 Where Kanaga is, for example, the host name. To list all the host objects mapped to the specified I/O group, use the svcinfo lsiogrphost command, as shown in Example 9-29. Example 9-29 svcinfo lsiogrphost
IBM_2145:ITSO-CLS1:admin>svcinfo lsiogrphost io_grp1 id name 1 Nile 2 Kanaga 3 Siam Where iogrp_1 is the I/0 group name.
9.4 Nodes This section details the tasks that can be performed at an individual node level.
9.4.1 Viewing node details Use the svcinfo lsnode command to view summary information about nodes defined within the SVC environment. To view more details about a specific node, append the node name (for example, SVCNode_1) to the command. Both of these commands are shown in Example 9-30. Tip: The -delim, parameter truncates the on screen content and separates data fields with colons as opposed to wrapping text over multiple lines. Example 9-30 svcinfo lsnode command
IBM_2145:ITSO-CLS1:admin>svcinfo lsnode -delim , id,name,UPS_serial_number,WWNN,status,IO_group_id,IO_group_name,config_node,UPS_un ique_id,hardware 1,node1,1000739007,50050768010037E5,online,0,io_grp0,yes,20400001C3240007,8G4 2,node2,1000739004,50050768010037DC,online,0,io_grp0,no,20400001C3240004,8G4 3,node3,100066C107,5005076801001D1C,online,1,io_grp1,no,20400001864C1007,8G4 4,node4,100066C108,50050768010027E2,online,1,io_grp1,no,20400001864C1008,8G4 IBM_2145:ITSO-CLS1:admin>svcinfo lsnode node1 id 1 name node1 UPS_serial_number 1000739007 WWNN 50050768010037E5 Chapter 9. SVC configuration and administration using the CLI
323
status online IO_group_id 0 IO_group_name io_grp0 partner_node_id 2 partner_node_name node2 config_node yes UPS_unique_id 20400001C3240007 port_id 50050768014037E5 port_status active port_speed 4Gb port_id 50050768013037E5 port_status active port_speed 4Gb port_id 50050768011037E5 port_status active port_speed 4Gb port_id 50050768012037E5 port_status active port_speed 4Gb hardware 8G4
9.4.2 Adding a node Before you can add a node, you must know which unconfigured nodes you have as “candidates”. You can find this out by issuing the svcinfo lsnodecandidate command (Example 9-31). Example 9-31 svctask lsnodecandiidate
IBM_2145:ITSO-CLS1:admin>svcinfo lsnodecandidate id panel_name UPS_serial_number 50050768010027E2 108283 100066C108
UPS_unique_id hardware 20400001864C1008 8G4
Note: The node you want to add must be on a different UPS serial number than the UPS on the first node. Now that we know the available nodes, we can use the svctask addnode command to add the node to the SVC cluster configuration. The complete syntax of the addnode command is: >>- svctask -- -- addnode -- -----------------------------------> >--+- -panelname -- -- panel_name -+-- -------------------------> '- -wwnodename -- -- wwnn_arg --' >--+----------------------------+-- ----------------------------> '- -name -- -- new_name_arg -' >-- -iogrp -- --+- iogroup_name -+----------------------------->< '- iogroup_id ---' In the following explanation, note that panelname and wwnodename are mutually exclusive:
324
Implementing the IBM System Storage SAN Volume Controller V4.3
-panelname panel_name (Required if you do not specify the -wwnodename parameter.) Specifies the node that you want to add to a cluster by the name that is displayed on the panel. You cannot use this parameter with the -wwnodename parameter. -wwnodename wwnn_arg (Required if you do not specify the -panelname parameter.) Specifies the node that you want to add to the cluster by the worldwide node name (WWNN). You cannot use this parameter with the -panelname parameter. -name new_name_arg (Optional) Specifies a name for the node that you want to add to the cluster. -iogrp iogroup_name | iogroup_id (Required) Specifies the I/O group to which you want to add this node. The command to add a node to the SVC cluster is shown in Example 9-35. Example 9-32 svctask addnode (wwnodename)
IBM_2145:ITSO-CLS1:admin>svctask addnode -wwnodename 50050768010027E2 -name Node4 -iogrp io_grp1 Node, id [5], successfully added This command adds the candidate node with the wwnodename 50050768010027E2 to the I/O group io_grp1. We used the -wwnodename parameter (50050768010027E2), but we could have used the -panelname parameter (108283) instead (Example 9-33). Example 9-33 svctask addnode (panelname)
IBM_2145:ITSO-CLS1:admin>svctask addnode -panelname 108283 -name Node4 -iogrp io_grp1 We also used the optional -name parameter (Node4). If you do not provide the -name parameter, the SVC automatically generates the name nodeX (where X is the ID sequence number assigned by the SVC internally). In our case, it would be node6. Note: If you want to provide a name, you can use letters A to Z, a to z, numbers 0 to 9, the dash ’-’, and the underscore ’_’. It can be between one and 15 characters in length. However, it cannot start with a number, dash, or the word node, since this prefix is reserved for SVC assignment only.
9.4.3 Renaming a node Use the svctask chnode command to rename a node within the SVC cluster configuration. From now on, we are following our naming convention (Example 9-34). Example 9-34 svctask chnode -name
IBM_2145:ITSO-CLS1:admin>svctask chnode -name n4 Node4 This command renames node Node4 to n4.
Chapter 9. SVC configuration and administration using the CLI
325
Note: The chnode command specifies the new name first. You can use letters A to Z, a to z, numbers 0 to 9, the dash ’-’, and the underscore ’_’. It can be between one and 15 characters in length. However, it cannot start with a number, dash, or the word node, since this prefix is reserved for SVC assignment only.
9.4.4 Deleting a node Use the svctask rmnode command to remove a node from the SVC cluster configuration (Example 9-35). Example 9-35 svctask rmnode
IBM_2145:ITSO-CLS1:admin>svctask rmnode node4 This command removes the node node4 from the SVC cluster. Since node4 was also the configuration node, the SVC transfers the configuration node responsibilities to a surviving node (in our case, node1). Unfortunately the PuTTY session cannot be dynamically passed to the surviving node. Therefore, the PuTTY application loses communication and closes automatically. We must restart the PuTTY application to establish a secure session with the new configuration node. Important: If this is the last node in an I/O Group, and there are Virtual Disks still assigned to the I/O Group, the node will not be deleted from the cluster. If this is the last node in the cluster, and the I/O Group has no Virtual Disks remaining, the cluster will be destroyed and all virtualization information will be lost. Any data that is still required should be backed up or migrated prior to destroying the cluster.
9.4.5 Shutting down a node Earlier we showed how to shut down the complete SVC cluster in a controlled manner. On occasion, it can be necessary to shut down a single node within the cluster, to perform such tasks as scheduled maintenance, while leaving the SVC environment up and running. Use the svctask stopcluster -node command, as shown in Example 9-36, to shut down a node. Example 9-36 svctask stopcluster -node command
IBM_2145:ITSO-CLS1:admin>svctask stopcluster -node n4 Are you sure that you want to continue with the shut down? This command shuts down node n4 in a graceful manner. When this is done, the other node in the I/O Group will destage the contents of its cache and will go into write through mode until the node is powered up and rejoins the cluster. Note: There is no need to stop FlashCopy mappings, Remote Copy relationships, and data migration operations. The other cluster will handle this, but be aware that this cluster is a single point of failure now.
326
Implementing the IBM System Storage SAN Volume Controller V4.3
If this is the last node in an I/O Group, all access to the Virtual Disks in the I/O Group will be lost. Ensure that this is what you want to do before executing this command and we will need to specify the -force flag. By re-issuing the svcinfo lsnode command (Example 9-37), we can see that the node is now offline. Example 9-37 svcinfo lsnode
IBM_2145:ITSO-CLS1:admin>svcinfo lsnode -delim , id,name,UPS_serial_number,WWNN,status,IO_group_id,IO_group_name,config_node,UPS_un ique_id,hardware 1,n1,1000739007,50050768010037E5,online,0,io_grp0,yes,20400001C3240007,8G4 2,n2,1000739004,50050768010037DC,online,0,io_grp0,no,20400001C3240004,8G4 3,n3,100066C107,5005076801001D1C,online,1,io_grp1,no,20400001864C1007,8G4 6,n4,100066C108,0000000000000000,offline,1,io_grp1,no,20400001864C1008,unknown IBM_2145:ITSO-CLS1:admin>svcinfo lsnode n4 CMMVC5782E The object specified is offline. To restart the node, simply go to the service panel of the node and push the power on button. We have now completed the tasks required to view, add, delete, rename, and shut down a node within an SVC environment.
9.5 Working with managed disks This section details the various configuration and administration tasks that you can perform on the managed disks (MDisks) within the SVC environment.
9.5.1 Disk controller systems This section details the tasks that you can perform on a disk controller level.
Viewing disk controller details Use the svcinfo lscontroller command to display summary information about all available back-end storage systems. To display more detailed information about a specific controller, run the command again and append the controller name parameter (for example, controller0). Both of these commands are shown in Example 9-38. Tip: The -delim, parameter truncates the on screen content and separates data fields with colons, as opposed to wrapping text over multiple lines. Example 9-38 svcinfo lscontroller command
IBM_2145:ITSO-CLS1:admin>svctask chcontroller -name controller0 DS4500 IBM_2145:ITSO-CLS1:admin>svcinfo lscontroller -delim , id,controller_name,ctrl_s/n,vendor_id,product_id_low,product_id_high 0,controller0,,IBM ,1742-900, 1,DS4700,,IBM ,1814 , FAStT IBM_2145:ITSO-CLS1:admin>svcinfo lscontroller controller0 id 0 controller_name controller0 WWNN 200400A0B8174431 Chapter 9. SVC configuration and administration using the CLI
327
mdisk_link_count 13 max_mdisk_link_count 13 degraded no vendor_id IBM product_id_low 1742-900 product_id_high product_revision 0520 ctrl_s/n WWPN 200500A0B8174433 path_count 0 max_path_count 13 WWPN 200400A0B8174433 path_count 52 max_path_count 52
Renaming a controller Use the svctask chcontroller command to change the name of a storage controller. To verify the change, run the svcinfo lscontroller command. Both of these commands are shown in Example 9-39. Example 9-39 svctask chcontroller command
IBM_2145:ITSO-CLS1:admin>svctask chcontroller -name DS4500 controller0 IBM_2145:ITSO-CLS1:admin>svcinfo lscontroller -delim , id,controller_name,ctrl_s/n,vendor_id,product_id_low,product_id_high 0,DS4500,,IBM ,1742-900, 1,DS4700,,IBM ,1814 , FAStT This command renames the controller named controller0 to DS4500. Note: The chcontroller command specifies the new name first. You can use letters A to Z, a to z, numbers 0 to 9, the dash ’-’, and the underscore ’_’. It can be between one and 15 characters in length. However, it cannot start with a number, dash, or the word controller, since this prefix is reserved for SVC assignment only.
9.5.2 MDisk information Use the svcinfo lsmdisk command to display summary information about all available managed disks. To display more detailed information about a specific MDisk, run the command again and append the MDisk name parameter (for example, mdisk0). Both of these commands are shown in Example 9-40. Example 9-40 svcinfo lsmdisk command
IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk -delim , id,name,status,mode,mdisk_grp_id,mdisk_grp_name,capacity,ctrl_LUN_#,controller_nam e,UID 0,mdisk0,online,managed,0,MDG_DS45,36.0GB,0000000000000000,DS4500,600a0b8000174431 000000eb47139cca00000000000000000000000000000000 1,mdisk1,online,managed,0,MDG_DS45,36.0GB,0000000000000001,DS4500,600a0b8000174431 000000ef47139e1c00000000000000000000000000000000 2,mdisk2,online,managed,0,MDG_DS45,36.0GB,0000000000000002,DS4500,600a0b8000174431 000000f147139e7200000000000000000000000000000000
328
Implementing the IBM System Storage SAN Volume Controller V4.3
3,mdisk3,online,managed,0,MDG_DS45,36.0GB,0000000000000003,DS4500,600a0b8000174431 000000e44713575400000000000000000000000000000000 4,mdisk4,online,managed,0,MDG_DS45,36.0GB,0000000000000004,DS4500,600a0b8000174431 000000e64713576000000000000000000000000000000000 5,mdisk5,online,managed,1,MDG_DS47,36.0GB,0000000000000000,DS4700,600a0b800026b282 00003ea34851577c00000000000000000000000000000000 6,mdisk6,online,managed,0,MDG_DS45,36.0GB,0000000000000005,DS4500,600a0b8000174431 000000e747139cb600000000000000000000000000000000 7,mdisk7,online,managed,1,MDG_DS47,36.0GB,0000000000000001,DS4700,600a0b80002904de 00004188485157a400000000000000000000000000000000 8,mdisk8,online,managed,0,MDG_DS45,36.0GB,0000000000000006,DS4500,600a0b8000174431 000000ea47139cc400000000000000000000000000000000 9,mdisk9,online,managed,1,MDG_DS47,36.0GB,0000000000000002,DS4700,600a0b800026b282 00003ed6485157b600000000000000000000000000000000 . . IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk mdisk0 id 0 name mdisk0 status online mode managed mdisk_grp_id 0 mdisk_grp_name MDG_DS45 capacity 36.0GB quorum_index 0 block_size 512 controller_name DS4500 ctrl_type 4 ctrl_WWNN 200400A0B8174431 controller_id 0 path_count 4 max_path_count 4 ctrl_LUN_# 0000000000000000 UID 600a0b8000174431000000eb47139cca00000000000000000000000000000000 preferred_WWPN 200400A0B8174433 active_WWPN 200400A0B8174433
9.5.3 Renaming an MDisk Use the svctask chmdisk command to change the name of an MDisk. To verify the change, run the svcinfo lsmdisk command. Both of these commands are shown in Example 9-41. Example 9-41 svctask chmdisk command
IBM_2145:ITSO-CLS1:admin>svctask chmdisk -name mdisk_6 mdisk6 IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk -delim , id,name,status,mode,mdisk_grp_id,mdisk_grp_name,capacity,ctrl_LUN_#,controller_nam e,UID 0,mdisk0,online,managed,0,MDG_DS45,36.0GB,0000000000000000,DS4500,600a0b8000174431 000000eb47139cca00000000000000000000000000000000 1,mdisk1,online,managed,0,MDG_DS45,36.0GB,0000000000000001,DS4500,600a0b8000174431 000000ef47139e1c00000000000000000000000000000000 2,mdisk2,online,managed,0,MDG_DS45,36.0GB,0000000000000002,DS4500,600a0b8000174431 000000f147139e7200000000000000000000000000000000
Chapter 9. SVC configuration and administration using the CLI
329
3,mdisk3,online,managed,0,MDG_DS45,36.0GB,0000000000000003,DS4500,600a0b8000174431 000000e44713575400000000000000000000000000000000 4,mdisk4,online,managed,0,MDG_DS45,36.0GB,0000000000000004,DS4500,600a0b8000174431 000000e64713576000000000000000000000000000000000 5,mdisk5,online,managed,1,MDG_DS47,36.0GB,0000000000000000,DS4700,600a0b800026b282 00003ea34851577c00000000000000000000000000000000 6,mdisk_6,online,managed,0,MDG_DS45,36.0GB,0000000000000005,DS4500,600a0b800017443 1000000e747139cb600000000000000000000000000000000 . . This command renamed the MDisk named mdisk6 to mdisk_6. Note: The chmdisk command specifies the new name first. You can use letters A to Z, a to z, numbers 0 to 9, the dash ’-’, and the underscore ’_’. It can be between one and 15 characters in length. However, it cannot start with a number, dash, or the word mdisk, since this prefix is reserved for SVC assignment only.
9.5.4 Discovering MDisks In general, the cluster detects the MDisks automatically when they appear in the network. However, some Fibre Channel controllers do not send the required SCSI primitives that are necessary to automatically discover the new MDisks. If new storage has been attached and the cluster has not detected it, it might be necessary to run this command before the cluster will detect the new MDisks. Use the svctask detectmdisk command to scan for newly added MDisks (Example 9-42). Example 9-42 svctask detectmdisk
IBM_2145:ITSO-CLS1:admin>svctask detectmdisk To check whether any newly added MDisks were successfully detected, run the svcinfo lsmdisk command as before. If the disks do not appear, check that the disk is appropriately assigned to the SVC in the disk subsystem, and that the zones are properly set up, as explained in Chapter 3, “Planning and configuration” on page 25. Note: If you have assigned a large number of LUNs to your SVC, the discovery process could take a while. Check, several times, using the svcinfo lsmdisk command if all the MDisks you were expecting are present.
9.5.5 Setting up a quorum disk The SVC cluster, after the process of node discovery, automatically chooses three MDisks as quorum disks. Each disk is assigned an index number of 0, 1, or 2. The quorum disks are only created once when at least one managed MDisk with an available extent is placed in managed mode. In the event that half the nodes in a cluster are missing for any reason, the other half cannot simply assume that the nodes are “dead”. It can simply mean that the cluster state information is not being successfully passed between nodes for some reason (network failure
330
Implementing the IBM System Storage SAN Volume Controller V4.3
for example). For this reason, if half of the cluster disappears from the view of the other half, each surviving half attempts to lock the active quorum disk. Note: There can be only one active quorum disk. When SVC first discovers LUNs as MDisks, it chooses three MDisks as quorum disk candidates. One is then chosen as active, and the others are not considered quorum disks in any way. Only if the active quorum disk becomes unavailable will the cluster go out and choose any of the other two candidates to take its place. Since the other quorum disk candidates are nothing but candidates, they are not relevant and not even considered in any cluster quorum event. So, in the event of quorum disk index 0 not being available, the next disk (index 1) becomes the quorum, and so on. The half of the cluster that is successful in locking the quorum disk becomes the exclusive processor of I/O activity. It attempts to reform the cluster with any nodes it can still see. The other half will stop processing IO. This provides a tie-breaker solution and ensures that both halves of the cluster do not continue to operate. In the case where all clusters can see the quorum disk, they will use this quorum disk to communicate with each other and will decide which half will become the exclusive processor of I/O activity. If, for any reason (for example, additional back-end storage has been installed and you want to move one or two quorum disks on this newly installed back-end storage subsystem) you want to set your own quorum disks, you can use the svctask setquorum command, as shown in Example 9-43, to reassign the quorum indexes. The managed disk that is currently assigned the quorum index number is set to a non-quorum disk. Example 9-43 svctask setquorum command
IBM_2145:ITSO-CLS1:admin>svctask setquorum -quorum 0 mdisk5 IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk mdisk5 id 5 name mdisk5 status online mode managed mdisk_grp_id 1 mdisk_grp_name MDG_DS47 capacity 36.0GB quorum_index 0 block_size 512 controller_name DS4700 ctrl_type 4 ctrl_WWNN 200400A0B82904DE controller_id 1 path_count 4 max_path_count 4 ctrl_LUN_# 0000000000000000 UID 600a0b800026b28200003ea34851577c00000000000000000000000000000000 preferred_WWPN 202500A0B82904DE active_WWPN 202500A0B82904DE As you can see, this command has set mdisk5 as a quorum disk using quorum index 0. You can also do this for quorum index 1 and 2.
Chapter 9. SVC configuration and administration using the CLI
331
9.5.6 Including an MDisk If a significant number of errors occur on an MDisk, the SVC automatically excludes it. These errors can be from a hardware problem, a storage area network (SAN) zoning problem, or the result of poorly planned maintenance. If it was a hardware fault, you should receive Simple Network Management Protocol (SNMP) alerts about the state of the disk subsystem (before the disk was excluded) and undertake preventative maintenance. If not, the hosts that were using VDisks, which used the excluded MDisk, now have I/O errors. By running the svcinfo lsmdisk command, you can see that mdisk10 is excluded in Example 9-44. Example 9-44 svcinfo lsmdisk command: Excluded MDisk
IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk -delim , id,name,status,mode,mdisk_grp_id,mdisk_grp_name,capacity,ctrl_LUN_#,controller_nam e,UID 0,mdisk0,online,managed,0,MDG_DS45,36.0GB,0000000000000000,DS4500,600a0b8000174431 000000eb47139cca00000000000000000000000000000000 1,mdisk1,online,managed,0,MDG_DS45,36.0GB,0000000000000001,DS4500,600a0b8000174431 000000ef47139e1c00000000000000000000000000000000 2,mdisk2,online,managed,0,MDG_DS45,36.0GB,0000000000000002,DS4500,600a0b8000174431 000000f147139e7200000000000000000000000000000000 3,mdisk3,online,managed,0,MDG_DS45,36.0GB,0000000000000003,DS4500,600a0b8000174431 000000e44713575400000000000000000000000000000000 4,mdisk4,online,managed,0,MDG_DS45,36.0GB,0000000000000004,DS4500,600a0b8000174431 000000e64713576000000000000000000000000000000000 5,mdisk5,online,managed,1,MDG_DS47,36.0GB,0000000000000000,DS4700,600a0b800026b282 00003ea34851577c00000000000000000000000000000000 6,mdisk_6,online,managed,0,MDG_DS45,36.0GB,0000000000000005,DS4500,600a0b800017443 1000000e747139cb600000000000000000000000000000000 7,mdisk7,online,managed,1,MDG_DS47,36.0GB,0000000000000001,DS4700,600a0b80002904de 00004188485157a400000000000000000000000000000000 8,mdisk8,online,managed,0,MDG_DS45,36.0GB,0000000000000006,DS4500,600a0b8000174431 000000ea47139cc400000000000000000000000000000000 9,mdisk9,excluded,managed,1,MDG_DS47,36.0GB,0000000000000002,DS4700,600a0b800026b2 8200003ed6485157b600000000000000000000000000000000 . . After taking the necessary corrective action to repair the MDisk (for example, replace the failed disk, repair the SAN zones, and so on), we must tell the SVC to include the MDisk again by issuing the svctask includemdisk command (Example 9-45). Example 9-45 svctask includemdisk
IBM_2145:ITSO-CLS1:admin>svctask includemdisk mdisk9 Running the svcinfo lsmdisk command again should show mdisk10 online again, as shown in Example 9-46. Example 9-46 svcinfo lsmdisk command: Verifying that MDisk is included
IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk -delim , id,name,status,mode,mdisk_grp_id,mdisk_grp_name,capacity,ctrl_LUN_#,controller_nam e,UID 0,mdisk0,online,managed,0,MDG_DS45,36.0GB,0000000000000000,DS4500,600a0b8000174431 000000eb47139cca00000000000000000000000000000000 332
Implementing the IBM System Storage SAN Volume Controller V4.3
1,mdisk1,online,managed,0,MDG_DS45,36.0GB,0000000000000001,DS4500,600a0b8000174431 000000ef47139e1c00000000000000000000000000000000 2,mdisk2,online,managed,0,MDG_DS45,36.0GB,0000000000000002,DS4500,600a0b8000174431 000000f147139e7200000000000000000000000000000000 3,mdisk3,online,managed,0,MDG_DS45,36.0GB,0000000000000003,DS4500,600a0b8000174431 000000e44713575400000000000000000000000000000000 4,mdisk4,online,managed,0,MDG_DS45,36.0GB,0000000000000004,DS4500,600a0b8000174431 000000e64713576000000000000000000000000000000000 5,mdisk5,online,managed,1,MDG_DS47,36.0GB,0000000000000000,DS4700,600a0b800026b282 00003ea34851577c00000000000000000000000000000000 6,mdisk_6,online,managed,0,MDG_DS45,36.0GB,0000000000000005,DS4500,600a0b800017443 1000000e747139cb600000000000000000000000000000000 7,mdisk7,online,managed,1,MDG_DS47,36.0GB,0000000000000001,DS4700,600a0b80002904de 00004188485157a400000000000000000000000000000000 8,mdisk8,online,managed,0,MDG_DS45,36.0GB,0000000000000006,DS4500,600a0b8000174431 000000ea47139cc400000000000000000000000000000000 9,mdisk9,online,managed,1,MDG_DS47,36.0GB,0000000000000002,DS4700,600a0b800026b282 00003ed6485157b600000000000000000000000000000000 . .
9.5.7 Showing the MDisk group Use the svcinfo lsmdisk command as before to display information about the managed disk group (MDG) to which an MDisk belongs, as shown in Example 9-47. Example 9-47 svcinfo lsmdisk command
id,name,status,mdisk_count,vdisk_count,capacity,extent_size,free_capacity,virtual_ capacity,used_capacity,real_capacity,overallocation,warning 0,MDG_DS45,online,13,4,468.0GB,512,355.0GB,140.00GB,100.00GB,112.00GB,29,0 1,MDG_DS47,online,8,3,288.0GB,512,217.5GB,120.00GB,20.00GB,70.00GB,41,0 See 9.6, “Managed Disk Groups” on page 334 for more details about MDGs.
9.5.8 Showing a VDisk on an MDisk Use the svcinfo lsmdiskmember command to display information about the VDisks that use space on a specific MDisk, as shown in Example 9-48. Example 9-48 svcinfo lsmdiskmember command
IBM_2145:ITSO-CLS1:admin>svcinfo lsmdiskmember mdisk1 id copy_id 0 0 2 0 3 0 4 0 5 0 This command displays the list of all Vdisk IDs that correspond to the VDisk copies that are using mdisk1. To correlate the IDs displayed in this output to VDisk names, we can run the svcinfo lsvdisk command, which we discuss in more detail in 9.9, “Working with virtual disks” on page 346. Chapter 9. SVC configuration and administration using the CLI
333
9.6 Managed Disk Groups This section explains the tasks that we can perform at an MDG level.
Viewing MDisk group information Use the svcinfo lsmdiskgrp command, as shown in Example 9-49, to display information about the MDGs defined in the SVC. Example 9-49 svcinfo lsmdiskgrp command
IBM_2145:ITSO-CLS1:admin>svcinfo lsmdiskgrp -delim , id,name,status,mdisk_count,vdisk_count,capacity,extent_size,free_capacity,virtual_ capacity,used_capacity,real_capacity,overallocation,warning 0,MDG_DS45,online,13,5,468.0GB,512,345.0GB,150.00GB,110.00GB,122.00GB,32,0 1,MDG_DS47,online,8,2,288.0GB,512,227.5GB,110.00GB,10.00GB,60.00GB,38,0
9.6.1 Creating an MDisk group Use the svctask mkmdiskgrp command to create an MDG. The full syntax of this command is: >>- svctask -- -- mkmdiskgrp -- --+-------------------------+---> '- -name -- new_name_arg -' >-- --+---------------------------------+-- --------------------> '- -mdisk --+- mdisk_id_list ---+-' '- mdisk_name_list -' >-- -ext -- extent_size ----------------------------------------> >--+--------------------------------------------------+---------> '- -warning --disk_size |--disk_size_percentage--%-' >--+-------------------+--------------------------------------->< '- -unit --+- b --+-' +- kb -+ +- mb -+ +- gb -+ +- tb -+ '- pb -' Note the following explanation: The parameters are: -name new_name_arg (Optional) Specifies a name to assign to the new group. -mdisk mdisk_id_list | mdisk_name_list (Optional) Specifies a colon-separated list of managed disk IDs or names to add to the group. You can create an empty MDisk group by not specifying the -mdisk parameter. -ext extent_size (Required) Specifies the size of the extents for this group in MB. The extent_size parameter must be one of the following values: 16, 32, 64, 128, 256, 512, 1024, or 2048 (MB).
334
Implementing the IBM System Storage SAN Volume Controller V4.3
-warning disk_size | disk_size_percentage% (Optional) Generates a warning when the used disk capacity in the MDisk group first exceeds the specified threshold. You can specify a disk_size integer, which defaults to megabytes (MB) unless the -unit parameter is specified; or you can specify a disk_size, which is a percentage of the MDisk group size. To disable warnings, specify 0 or 0%. The default value is 0. -unit b | kb | mb | gb | tb | pb (Optional) Specifies the data units for the -warning parameter. The command to create an MDG is shown in Example 9-50. Example 9-50 svctask mkmdiskgrp command
IBM_2145:ITSO-CLS1:admin>svctask mkmdiskgrp -name MDG_DS83 -ext 512 -warning 85% MDisk Group, id [2], successfully created This command creates an MDG called MDG_DS83 with an extent size of 512 MB. Since we did not specify any MDisks to add to the group with the -mdisk parameter, this is an empty MDG. A warning will be generated when the used disk capacity of the MDisk first exceeds the threshold of 85%. If we run the svcinfo lsmdiskgrp command, we should see the MDG created, as shown in Example 9-51. Example 9-51 svcinfo lsmdiskgrp command
IBM_2145:ITSO-CLS1:admin>svcinfo lsmdiskgrp -delim , id,name,status,mdisk_count,vdisk_count,capacity,extent_size,free_capacity,virtual_ capacity,used_capacity,real_capacity,overallocation,warning 0,MDG_DS45,online,13,5,468.0GB,512,345.0GB,150.00GB,110.00GB,122.00GB,32,0 1,MDG_DS47,online,8,2,288.0GB,512,227.5GB,110.00GB,10.00GB,60.00GB,38,0 2,MDG_DS83,online,0,0,0,512,0,0.00MB,0.00MB,0.00MB,0,85
9.6.2 Renaming an MDisk group Use the svctask chmdiskgrp command to change the name of an MDG. To verify the change, run the svcinfo lsmdiskgrp command. Both of these commands are shown in Example 9-52. Example 9-52 svctask chmdiskgrp command
IBM_2145:ITSO-CLS1:admin>svctask chmdiskgrp -name MDG_DS81 MDG_DS83 IBM_2145:ITSO-CLS1:admin>svcinfo lsmdiskgrp -delim , id,name,status,mdisk_count,vdisk_count,capacity,extent_size,free_capacity,virtual_ capacity,used_capacity,real_capacity,overallocation,warning 0,MDG_DS45,online,13,5,468.0GB,512,345.0GB,150.00GB,110.00GB,122.00GB,32,0 1,MDG_DS47,online,8,2,288.0GB,512,227.5GB,110.00GB,10.00GB,60.00GB,38,0 2,MDG_DS81,online,0,0,0,512,0,0.00MB,0.00MB,0.00MB,0,85 This command renamed the MDG from MDG_DS83 to MDG_DS81.
Chapter 9. SVC configuration and administration using the CLI
335
Note: The chmdiskgrp command specifies the new name first. You can use letters A to Z, a to z, numbers 0 to 9, the dash ’-’, and the underscore ’_’. It can be between one and 15 characters in length. However, it cannot start with a number, dash, or the word mdiskgrp, since this prefix is reserved for SVC assignment only.
9.6.3 Deleting an MDisk group Use the svctask rmmdiskgrp command to remove an MDG from the SVC cluster configuration (Example 9-53). Example 9-53 svctask rmmdiskgrp
IBM_2145:ITSO-CLS1:admin>svctask rmmdiskgrp MDG_DS81 IBM_2145:ITSO-CLS1:admin>svcinfo lsmdiskgrp -delim , id,name,status,mdisk_count,vdisk_count,capacity,extent_size,free_capacity,virtual_ capacity,used_capacity,real_capacity,overallocation,warning 0,MDG_DS45,online,13,5,468.0GB,512,345.0GB,150.00GB,110.00GB,122.00GB,32,0 1,MDG_DS47,online,8,2,288.0GB,512,227.5GB,110.00GB,10.00GB,60.00GB,38,0 This command removes MDG_DS81 from the SVC configuration. Note: If there are MDisks within the MDG, you must use the -force flag, for example: svctask rmmdiskgrp MDG_DS81 -force Ensure that you really want to use this flag, as it destroys all mapping information and data held on the VDisks, which cannot be recovered.
9.6.4 Adding MDisks to an MDisk group If you created an empty MDG as we did, or you simply assign additional MDisks to your SVC environment later, you can use the svctask addmdisk command to populate the MDG (Example 9-54). Example 9-54 svctask addmdisk command
IBM_2145:ITSO-CLS1:admin>svctask addmdisk -mdisk mdisk6 MDG_DS45 You can only add unmanaged MDisks to an MDG. This command adds MDisk mdisk6 to the MDG named MDG_DS45. Important: Do not do this if you want to create an image mode VDisk from the MDisk you are adding. As soon as you add an MDisk to an MDG, it becomes managed, and extent mapping is not necessarily one to one anymore.
9.6.5 Removing MDisks from MDisk group Use the svctask rmmdisk command to remove an MDisk from an MDG (Example 9-55). Example 9-55 svctask rmmdisk command
IBM_2145:ITSO-CLS1:admin>svctask rmmdisk -mdisk 6 -force MDG_DS45
336
Implementing the IBM System Storage SAN Volume Controller V4.3
This command removes the MDisk called mdisk6 from the MDG named MDG_DS45.The -force flag is set because there are VDisks using this MDG. Note: The removal only takes place if there is sufficient space to migrate the VDisk data to other extents on other MDisks that remain in the MDG. After you remove the MDisk group, it takes some time to change the mode from managed to unmanaged.
9.6.6 Showing MDisks in a MDisk group Use the svcinfo lsmdisk -filtervalue command, as shown in Example 9-56, to see which MDisks are part of a specific MDG. This command shows all MDisks that are part of the MDG MDG2. Example 9-56 svcinfo lsmdisk -filtervalue: mdisks in MDG
IBM_2145:ITSOSVC42A:admin>svcinfo lsmdisk -filtervalue mdisk_grp_name=MDG2 -delim : id:name:status:mode:mdisk_grp_id:mdisk_grp_name:capacity:ctrl_LUN_#:controller_nam e:UID 6:mdisk6:online:managed:2:MDG2:3.0GB:0000000000000006:DS4000:600a0b800017423300000 044465c0a2700000000000000000000000000000000 7:mdisk7:online:managed:2:MDG2:6.0GB:0000000000000007:DS4000:600a0b800017443100000 06f465bf93200000000000000000000000000000000 21:mdisk21:online:image:2:MDG2:2.0GB:0000000000000015:DS4000:600a0b800017443100000 0874664018600000000000000000000000000000000
9.6.7 Showing VDisks using a MDisk group Use the svcinfo lsvdisk -filtervalue command, as shown in Example 9-57, to see which VDisks are part of a specific MDG. This command shows all VDisks that are part of the MDG MDG2. Example 9-57 svcinfo lsvdisk -filtervalue: vdisks in MDG
IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk -filtervalue mdisk_grp_name=MDG_DS47 -delim , id,name,status,mode,mdisk_grp_id,mdisk_grp_name,capacity,ctrl_LUN_#,controller_nam e,UID 5,mdisk5,online,managed,1,MDG_DS47,36.0GB,0000000000000000,DS4700,600a0b800026b282 00003ea34851577c00000000000000000000000000000000 7,mdisk7,online,managed,1,MDG_DS47,36.0GB,0000000000000001,DS4700,600a0b80002904de 00004188485157a400000000000000000000000000000000 9,mdisk9,online,managed,1,MDG_DS47,36.0GB,0000000000000002,DS4700,600a0b800026b282 00003ed6485157b600000000000000000000000000000000 12,mdisk12,online,managed,1,MDG_DS47,36.0GB,0000000000000003,DS4700,600a0b80002904 de000041ba485157d000000000000000000000000000000000 14,mdisk14,online,managed,1,MDG_DS47,36.0GB,0000000000000004,DS4700,600a0b800026b2 8200003f6c4851585200000000000000000000000000000000 18,mdisk18,online,managed,1,MDG_DS47,36.0GB,0000000000000005,DS4700,600a0b80002904 de000042504851586800000000000000000000000000000000 19,mdisk19,online,managed,1,MDG_DS47,36.0GB,0000000000000006,DS4700,600a0b800026b2 8200003f9f4851588700000000000000000000000000000000 20,mdisk20,online,managed,1,MDG_DS47,36.0GB,0000000000000007,DS4700,600a0b80002904 de00004282485158aa00000000000000000000000000000000
Chapter 9. SVC configuration and administration using the CLI
337
We have now completed the tasks required to manage the disk controller systems, managed disks, and MDGs within an SVC environment.
9.7 Hosts This section explains the tasks that can be performed at a host level.
9.7.1 Host information Use the svcinfo lshost command to display summary information about all hosts defined within the SVC environment. To display more detailed information about a specific host, run the command again and append the host name parameter (for example, win2k). Both of these commands are shown in Example 9-58. Tip: The -delim, parameter truncates the on screen content and separates data fields with colons as opposed to wrapping text over multiple lines. Example 9-58 svcinfo lshost command
IBM_2145:ITSO-CLS1:admin>svcinfo lshost id name port_count 0 Palau 2 1 Nile 2 2 Kanaga 2 3 Siam 2
iogrp_count 1 1 1 2
IBM_2145:ITSO-CLS1:admin>svcinfo lshost Nile id 1 name Nile port_count 2 type generic mask 1111 iogrp_count 1 WWPN 210000E08B892BCD node_logged_in_count 2 state active WWPN 210000E08B89B8C0 node_logged_in_count 2 state active
9.7.2 Creating a host Before creating a host, you need to know that its host bus adapter (HBA) worldwide port names (WWPNs) are visible to the SVC. To do this, issue the svcinfo lshbaportcandidate command, as shown in Example 9-59. Example 9-59 svcinfo lshbaportcandidate command
IBM_2145:ITSO-CLS1:admin>svcinfo lshbaportcandidate id 210000E08B89C1CD 210000E08B054CAA
338
Implementing the IBM System Storage SAN Volume Controller V4.3
After you know that the WWPNs that are displayed match our host (use host or SAN switch utilities to verify), use the svctask mkhost command to create an image mode VDisk. The full syntax of this command is: >>- svctask -- -- mkhost -- --+---------------------+-- --------> '- -name -- new_name -' >-- -hbawwpn -- wwpn_list -- --+-----------------------+-- -----> '- -iogrp--iogrp_list -' >-- --+-------------------------+-- --+----------+--------------> '- -mask--port_login_mask-' '- -force -' >--+------------------------+---------------------------------->< '- -type --+- hpux ----+-' +- tpgs ----+ '- generic -' Note the following explanation of the parameters: -name new_name (Optional) Specifies a name or label for the new host object. -hbawwpn wwpn_list (Required) Specifies a list of host bus adapter (HBA) worldwide port names (WWPNs) to add to the specified host object. -iogrp iogrp_list (Optional) Specifies a set of one or more I/O groups that the host can access the VDisks from. I/O groups are specified using their names or IDs, separated by a colon. Names and IDs can be mixed in the list. If this parameter is not specified, the host is associated with all I/O groups. -mask port_login_mask (Optional) Specifies which node target ports that a host can access. The port mask is four binary bits and is made up of a combination of 0s and 1s, where 0 indicates that the corresponding target port cannot be used and 1 indicates that it can be used. The right-most bit in the mask corresponds to the lowest numbered target port (1 not 4) on a node. Valid mask values range from 0000 (no ports enabled) to 1111 (all ports enabled). For example, a mask of 0011 enables port 1 and port 2. The default value is 1111 (all ports enabled). -force (Optional) Specifies that a logical host object be created without validation of the WWPNs. -type hpux | tpgs | generic (Optional) Specifies the type of host: hpux, tpgs, or generic. The default is generic. The tpgs parameter enables extra target port unit attentions. See the IBM System Storage Open Software Family SAN Volume Controller: Host Attachment Guide, SC26-7563 for more information about the hosts that require the -type parameter.
Chapter 9. SVC configuration and administration using the CLI
339
Note: If you do not provide the -name parameter, the SVC automatically generates the name hostX (where X is the ID sequence number assigned by the SVC internally). You can use letters A to Z, a to z, numbers 0 to 9, the dash ’-’, and the underscore ’_’. It can be between one and 15 characters in length. However, it cannot start with a number, dash, or the word host, since this prefix is reserved for SVC assignment only. The command to create a host is shown here in Example 9-60. Example 9-60 svctask mkhost
IBM_2145:ITSO-CLS1:admin>svctask mkhost -name Palau -hbawwpn 210000E08B89C1CD:210000E08B054CAA Host, id [0], successfully created This command creates a host called Palau using WWPN 21:00:00:E0:8B:89:C1:CD and 21:00:00:E0:8B:05:4C:AA Note: You can define from one up to eight ports per host or you can use the addport command, which we show in 9.7.5, “Adding ports” on page 341. Perhaps your WWPN or WWPNs did not display when you issued the svcinfo lshbaportcandidate command, but you are sure your adapter is functioning (for example, you can see the WWPN in the switch name server) and your zones are correctly setup. In this case, you can type the WWPN of your HBA or HBAs and use the -force flag to create the host regardless, as shown in Example 9-61. Example 9-61 mkhost -force
IBM_2145:ITSO-CLS1:admin>svctask mkhost -name Guinea -hbawwpn 210000E08B89C1DC -force Host, id [4], successfully created This command forces the creation of a host called Guinea using WWPN 210000E08B89C1DC. Note: WWPNs are not case sensitive in the CLI. If you run the svcinfo lshost command again, you should now see your host.
9.7.3 Modify a host Use the svctask chhost command to change the name of a host. To verify the change, run the svcinfo lshost command. Both of these commands are shown in Example 9-62 on page 341.
340
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 9-62 svctask chhost command
IBM_2145:ITSO-CLS1:admin>svctask chhost -name Angola Guinea IBM_2145:ITSO-CLS1:admin>svcinfo lshost id name port_count 0 Palau 2 1 Nile 2 2 Kanaga 2 3 Siam 2 4 Angola 1
iogrp_count 4 1 1 2 4
This command renamed the host from Guinea to Angola. Note: The chhost command specifies the new name first. You can use letters A to Z, a to z, numbers 0 to 9, the dash ’-’, and the underscore ’_’. It can be between one and 15 characters in length. However, it cannot start with a number, dash, or the word host, since this prefix is reserved for SVC assignment only.
Note: To get more than eight LUNs support for HP-UX, there is a -type flag for this command. Valid options are -type hpux (for HP-UX only) or -type generic (default)
9.7.4 Deleting a host Use the svctask rmhost command to delete a host from the SVC configuration. This command deletes the host called sanfs1 from the SVC configuration (Example 9-63). Example 9-63 rmhost Angola
IBM_2145:ITSO-CLS1:admin>svctask rmhost Angola Note: If there are any VDisks assigned to the host, you must use the -force flag, for example: svctask rmhost -force Angola
9.7.5 Adding ports If you add an HBA to a server that is already defined within the SVC, you can use the svctask addhostport command to add WWPN definitions to it. Before you add the new WWPN, you need to know that it is visible to the SVC. To do this, you issue the svcinfo lshbaportcandidate command, as shown in Example 9-64. Example 9-64 svcinfo lshbaportcandidate
IBM_2145:ITSO-CLS1:admin>svcinfo lshbaportcandidate id 210000E08B054CAA
Chapter 9. SVC configuration and administration using the CLI
341
After you know the WWPNs that are displayed match our host (use host or SAN switch utilities to verify), use the svctask addhostport command to add the port or ports to the host. The full syntax of this command is: >>- svctask -- -- addhostport -- -- -hbawwpn -- wwpn_list -- ---> >--+----------+-- --+- host_name -+---------------------------->< '- -force -' '- host_id ---' Note the following explanation: -hbawwpn wwpn_list (Required) Specifies the list of ports to add to the host. -force (Optional) Specifies that the list of ports be added to the host without the validation of any WWPNs. host_id | host_name (Required) Specifies the host object to add ports to, either by ID or by name. The command to add a host port is shown in Example 9-65. Example 9-65 svctask addhostport
IBM_2145:ITSO-CLS1:admin>svctask addhostport -hbawwpn 210000E08B054CAA Palau This command adds the WWPN of 210000E08B054CAA to the host Palau. Note: You can add multiple ports all at once by using the separator (:) between WWPNs, for example: svctask addhostport -hbawwpn 210000E08B054CAA:210000E08B89C1CD Palau Perhaps your WWPN or WWPNs did not display when you issued the svcinfo lshbaportcandidate command, but you are sure your adapter is functioning (for example, you see WWN in the switch name server) and your zones are correctly setup. In this case, you can manually type the WWPN of your HBA or HBAs and use the -force flag to create the host regardless, as shown in Example 9-66. Example 9-66 svctask addhostport
IBM_2145:ITSO-CLS1:admin>svctask addhostport -hbawwpn 210000E08B054CAA -force Palau This command forces the addition of the WWPN 210000E08B054CAA to the host called Palau. Note: WWPNs are one of the few things within the CLI that are not case sensitive. If you run the svcinfo lshost command again, you should see your host with an updated port count (2 in Example 9-67 on page 343).
342
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 9-67 svcinfo lshost command: port count
IBM_2145:ITSO-CLS1:admin>svcinfo lshost id name port_count 0 Palau 2 1 Nile 2 2 Kanaga 2 3 Siam 2 4 Angola 1
iogrp_count 4 1 1 2 4
9.7.6 Deleting ports If you make a mistake when adding, or if you remove an HBA from a server that is already defined within the SVC, you can use the svctask rmhostport command to remove WWPN definitions from an existing host. Before you remove the WWPN, be sure that it is the right one. To find this out, issue the svcinfo lshost command (our host is win2k), as shown in Example 9-68. Example 9-68 svcinfo lshost command
IBM_2145:ITSO-CLS1:admin>svcinfo lshost Palau id 0 name Palau port_count 2 type generic mask 1111 iogrp_count 4 WWPN 210000E08B054CAA node_logged_in_count 2 state inactive WWPN 210000E08B89C1CD node_logged_in_count 2 state inactive When you know the WWPN, use the svctask rmhostport command to delete a host port. The full syntax of this command is: >>- svctask -- -- rmhostport -- -- -hbawwpn -- wwpn_list -- ----> >--+----------+-- --+- host_name -+---------------------------->< '- -force -' '- host_id ---' Note the following explanation: -hbawwpn wwpn_list (Required) Specifies the list of ports that you can delete from the host. -force (Optional) Specifies that you want the system to delete the ports that you have specified without performing the validation check. The validation check ensures that the list of ports that you want to delete are actually mapped to the specified host. When the ports are deleted, they become unconfigured WWPNs. host_name | host_id (Required) Specifies the host name or the host ID.
Chapter 9. SVC configuration and administration using the CLI
343
The command to remove a host port is shown in Example 9-69. Example 9-69 svctask rmhostport
IBM_2145:ITSO-CLS1:admin>svctask rmhostport -hbawwpn 210000E08B89C1CD Palau This command removes the WWPN of 210000E08B89C1CD from host Palau. Note: You can remove multiple ports at a time by using the separator (:) between WWPNs, for example: svctask rmhostport -hbawwpn 210000E08B054CAA:210000E08B892BCD Angola
9.8 SAN debugging There are SVC commands to help to debug and to display connectivity between SAN Volume Controller nodes, storage subsystems, and hosts. The lsfabric command generates a report that displays the connectivity between nodes and other controllers and hosts: >>- svcinfo -- -- lsfabric -- ----------------------------------> >--+-------------------------------------------------------+--->< +- -node -- node_id_or_name -- --+--------------------+-+ | '- -port -- port_id -' | +- -wwpn -- wwpn ---------------------------------------+ +- -host -- host_id_or_name ----------------------------+ +- -controller -- controller_id_or_name ----------------+ '- -cluster -- cluster_id_or_name ----------------------' These are the parameters: -node node_id_or_name (Optional) Displays the output for all ports for the specified node. The only parameter that you can specify with the -node parameter is the -port parameter. -port port_id (Optional) Displays a concise view of all WWPNs that are logged into the specified port ID and node. The -port parameter must be specified with the -node parameter only. A valid port_id value is a number from 1 - 4 that specifies the port number in the vital product data (VPD), or the hexadecimal WWPN of the local port. -wwpn wwpn (Optional) Displays a list of all ports that have a login to the specified WWPN. You cannot use the -wwpn parameter with any other parameter. -host host_id_or_name (Optional) Specifies a host name or ID. Issuing the lsfabric command with the -host parameter is equivalent to issuing the svcinfo lsfabric -wwpn WWPN command for every configured WWPN of the specified host. For example, a host with two ports that are zoned to one port of every node in a eight-node cluster produces 16 lines of output. You cannot use the -host parameter with any other parameter.
344
Implementing the IBM System Storage SAN Volume Controller V4.3
-controller controller_id_or_name (Optional) Specifies a controller ID or name. You cannot use the -controller parameter with any other parameter in this command. Issuing the lsfabric command with the -controller parameter is equivalent to issuing the svcinfo lsfabric -wwpn WWPN command for every configured WWPN of the specified controller. For example, a controller with four ports connected to a eight node cluster with two counterpart SANs produces 64 lines of output. -cluster cluster_id_or_name (Optional) Specifies a cluster ID or name. You cannot use the -cluster parameter with any other parameter. Issuing the lsfabric command with the -cluster parameter is equivalent to issuing the svcinfo lsfabric -wwpn WWPN command for every known WWPN in the specified cluster. Output is sorted by remote WWPNs and then cluster WWPNs. This parameter can be used to check the state of connections within the local cluster or between the local and remote cluster. When the local cluster ID or name is specified, each node-to-node connection is listed twice: once from each end. For example, an 8 node cluster with two counterpart SANs produces eight nodes, multiplied by seven other nodes, multiplied by two SANs, multiplied by four point-to-point logins, equals 448 lines of output. Use the command shown in Example 9-70. Example 9-70 svcinfo lsfabric
IBM_2145:ITSO-CLS1:admin>svcinfo lsfabric -delim , remote_wwpn,remote_nportid,id,node_name,local_wwpn,local_port,local_nportid,state, name,cluster_name,type 10000000C932A800,011F00,2,n2,50050768011037DC,3,010300,inactive,Kanaga,,host 50050768012027E2,010700,1,n1,50050768013037E5,2,010000,active,n4,ITSO-CLS1,node 50050768012027E2,010700,1,n1,50050768011037E5,3,010100,active,n4,ITSO-CLS1,node 50050768012027E2,010700,2,n2,50050768014037DC,1,010200,active,n4,ITSO-CLS1,node 50050768012027E2,010700,2,n2,50050768011037DC,3,010300,active,n4,ITSO-CLS1,node 50050768012027E2,010700,3,n3,5005076801301D1C,2,010400,active,n4,ITSO-CLS1,node 50050768012027E2,010700,3,n3,5005076801201D1C,4,010500,active,n4,ITSO-CLS1,node 5005076801301D1C,010400,1,n1,50050768013037E5,2,010000,active,n3,ITSO-CLS1,node 5005076801301D1C,010400,1,n1,50050768011037E5,3,010100,active,n3,ITSO-CLS1,node 5005076801301D1C,010400,2,n2,50050768014037DC,1,010200,active,n3,ITSO-CLS1,node 5005076801301D1C,010400,2,n2,50050768011037DC,3,010300,active,n3,ITSO-CLS1,node . . 202400A0B82904DE,011000,1,n1,50050768013037E5,2,010000,inactive,DS4700,,controller 202400A0B82904DE,011000,1,n1,50050768011037E5,3,010100,inactive,DS4700,,controller 202400A0B82904DE,011000,2,n2,50050768014037DC,1,010200,inactive,DS4700,,controller 202400A0B82904DE,011000,2,n2,50050768011037DC,3,010300,inactive,DS4700,,controller 202400A0B82904DE,011000,3,n3,5005076801301D1C,2,010400,inactive,DS4700,,controller 202400A0B82904DE,011000,3,n3,5005076801201D1C,4,010500,inactive,DS4700,,controller 202400A0B82904DE,011000,6,n4,50050768013027E2,2,010600,inactive,DS4700,,controller 202400A0B82904DE,011000,6,n4,50050768012027E2,4,010700,inactive,DS4700,,controller . . 200400A0B8174433,011100,1,n1,50050768014037E5,1,010000,inactive,DS4500,,controller 200400A0B8174433,011100,1,n1,50050768012037E5,4,010100,inactive,DS4500,,controller 200400A0B8174433,011100,2,n2,50050768013037DC,2,010200,inactive,DS4500,,controller 200400A0B8174433,011100,2,n2,50050768012037DC,4,010300,inactive,DS4500,,controller 200400A0B8174433,011100,3,n3,5005076801401D1C,1,010400,inactive,DS4500,,controller 200400A0B8174433,011100,3,n3,5005076801101D1C,3,010500,inactive,DS4500,,controller 200400A0B8174433,011100,6,n4,50050768014027E2,1,010600,inactive,DS4500,,controller Chapter 9. SVC configuration and administration using the CLI
345
200400A0B8174433,011100,6,n4,50050768011027E2,3,010700,inactive,DS4500,,controller 5005076801201D1C,010500,1,n1,50050768013037E5,2,010000,active,n3,ITSO-CLS1,node . . 210000E08B89B8C0,011900,1,n1,50050768014037E5,1,010000,inactive,Nile,,host 210000E08B89B8C0,011900,3,n3,5005076801401D1C,1,010400,active,Nile,,host . . 210000E08B89C1CD,011B00,1,n1,50050768014037E5,1,010000,inactive,Palau,,host 210000E08B89C1CD,011B00,3,n3,5005076801401D1C,1,010400,inactive,Palau,,host . . 10000000C932A7FB,011F00,1,n1,50050768014037E5,1,010000,inactive,Kanaga,,host 10000000C932A7FB,011F00,3,n3,5005076801401D1C,1,010400,inactive,Kanaga,,host . . 210000E08B054CAA,011B00,2,n2,50050768011037DC,3,010300,inactive,Palau,,host 210000E08B054CAA,011B00,6,n4,50050768012027E2,4,010700,inactive,Palau,,host . . 210000E08B18D48F,011800,1,n1,50050768014037E5,1,010000,active,Siam,,host 210000E08B18D48F,011800,3,n3,5005076801401D1C,1,010400,active,Siam,,host 210000E08B18FF8A,011800,2,n2,50050768011037DC,3,010300,active,Siam,,host 210000E08B18FF8A,011800,6,n4,50050768012027E2,4,010700,active,Siam,,host Tip: The -delim, parameter truncates the on screen content and separates data fields with colons as opposed to wrapping text over multiple lines.
9.9 Working with virtual disks This section details the various configuration and administration tasks that can be performed on the VDisks within the SVC environment.
9.9.1 VDisk information Use the svcinfo lsvdisk command to display summary information about all VDisks defined within the SVC environment. To display more detailed information about a specific VDisk, run the command again and append the VDisk name parameter (for example, Vdisk_D). Both of these commands are shown in Example 9-71. Example 9-71 svcinfo lsvdisk command
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk -delim , id,name,IO_group_id,IO_group_name,status,mdisk_grp_id,mdisk_grp_name,capacity,type ,FC_id,FC_name,RC_id,RC_name,vdisk_UID,fc_map_count,copy_count 0,vdisk_A,0,io_grp0,online,0,MDG_DS45,10.0GB,striped,,,,,60050768018301BF280000000 0000008,0,1 1,vdisk_B,1,io_grp1,online,1,MDG_DS47,100.0GB,striped,,,,,60050768018301BF28000000 00000001,0,1 2,vdisk_C,1,io_grp1,online,0,MDG_DS45,40.0GB,striped,,,,,60050768018301BF280000000 0000002,0,1 3,vdisk_D,1,io_grp1,online,0,MDG_DS45,80.0GB,striped,,,,,60050768018301BF280000000 0000003,0,1
346
Implementing the IBM System Storage SAN Volume Controller V4.3
4,MM_DBLog_Pri,0,io_grp0,online,0,MDG_DS45,10.0GB,striped,,,4,MMREL2,6005076801830 1BF2800000000000004,0,1 5,MM_DB_Pri,0,io_grp0,online,0,MDG_DS45,10.0GB,striped,,,5,MMREL1,60050768018301BF 2800000000000005,0,1 6,MM_App_Pri,1,io_grp1,online,1,MDG_DS47,10.0GB,striped,,,,,60050768018301BF280000 0000000006,0,1 IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk vdisk_D id 3 name vdisk_D IO_group_id 1 IO_group_name io_grp1 status online mdisk_grp_id 0 mdisk_grp_name MDG_DS45 capacity 80.0GB type striped formatted no mdisk_id mdisk_name FC_id FC_name RC_id RC_name vdisk_UID 60050768018301BF2800000000000003 throttling 0 preferred_node_id 6 fast_write_state empty cache readwrite udid fc_map_count 0 sync_rate 50 copy_count 1 copy_id 0 status online sync yes primary yes mdisk_grp_id 0 mdisk_grp_name MDG_DS45 type striped mdisk_id mdisk_name fast_write_state empty used_capacity 80.00GB real_capacity 80.00GB free_capacity 0.00MB overallocation 100 autoexpand warning grainsize
Chapter 9. SVC configuration and administration using the CLI
347
9.9.2 Creating a VDisk Use the svctask mkvdisk command to create an image mode VDisk. Please see the full syntax and its parameters in 6.7, “Creating a virtual disk” on page 167. In Example 9-72, we show an example of how to create a space-efficient VDisk (SEV). Example 9-72 mkvdisk
IBM_2145:ITSO-CLS1:admin>svctask mkvdisk -mdiskgrp MDG_DS45 -iogrp 1 -vtype striped -size 10 -unit gb -rsize 50% -autoexpand -grainsize 32 Virtual Disk, id [7], successfully created IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk 7 id 7 name vdisk7 IO_group_id 1 IO_group_name io_grp1 status online mdisk_grp_id 0 mdisk_grp_name MDG_DS45 capacity 10.0GB type striped formatted no mdisk_id mdisk_name FC_id FC_name RC_id RC_name vdisk_UID 60050768018301BF280000000000000A throttling 0 preferred_node_id 6 fast_write_state empty cache readwrite udid fc_map_count 0 sync_rate 50 copy_count 1 copy_id 0 status online sync yes primary yes mdisk_grp_id 0 mdisk_grp_name MDG_DS45 type striped mdisk_id mdisk_name fast_write_state empty used_capacity 0.41MB real_capacity 5.02GB free_capacity 5.02GB overallocation 199 autoexpand on warning 80
348
Implementing the IBM System Storage SAN Volume Controller V4.3
grainsize 32 This command creates a space-efficient VDisk with a VDisk ID 7 of 10 GB. The VDisk belongs to the mdiskgrp with the name MDG_DS45 and is owned by the I/O group io_grp1. The real_capacity will automatically expand until the VDisk size of 10 GB is reached. The grainsize is set to 32 KB. Note: The auto option creates a VDisk copy that uses the entire size of the MDisk; if you specify the -rsize auto option, you must also specify the -vtype image option. If you are using the space-efficient VDisk directly with a host system, use a small grain size
Note: An entry of 1 GB uses 1024 MB.
9.9.3 Creating a VDisk in image mode This virtualization type allows image mode virtual disks to be created when a managed disk already has data on it, perhaps from a pre-virtualized subsystem. When an image mode virtual disk is created, it directly corresponds to the previously unmanaged, managed disk that it was created from. Therefore, with the exception of space-efficient image mode VDisks, virtual disk logical block address (LBA) x equals managed disk LBA x. You can use this command to bring a non-virtualized disk under the control of the cluster. After it is under the control of the cluster, you can migrate the virtual disk from the single managed disk. When it is migrated, the virtual disk is no longer an image mode virtual disk. You can add image mode VDisks to an already populated MDisk group with other types of VDisks, such as a striped or sequential. Note: An image mode VDisk must be at least 512 bytes (the capacity cannot be 0). That is, the minimum size that can be specified for an image mode VDisk must be the same as the MDisk group extent size that it is added to, with a minimum of 16 MB. You must use the -mdisk parameter to specify an MDisk that has a mode of unmanaged. The -fmtdisk parameter cannot be used to create an image mode VDisk. Note: If you create a mirrored VDisk from two image mode MDisks without specifying a -capacity value, the capacity of the resulting VDisk is the smaller of the two MDisks, and the remaining space on the larger MDisk is not accessible.
Note: If you do not specify the -size parameter when you create an image mode disk, the entire MDisk capacity is used. Use the svctask mkvdisk command to create an image mode VDisk. The full syntax of this command is: >>- svctask -- -- mkvdisk -- -----------------------------------> >-- -mdiskgrp --+- mdisk_group_id_list ---+-- ------------------> '- mdisk_group_name_list -' >-- -iogrp --+- io_group_id ---+-- --+------------+-- ---------->
Chapter 9. SVC configuration and administration using the CLI
349
'- io_group_name -'
'- -fmtdisk -'
>--+------------------------------------------------... '- -rsize -- disk_size | disk_size_percentage -- ... ...-------------------------------------------------------------------... ...| auto--+-------------------------------------------------------+--... '- -warning disk_size | disk_size_percentage% | off-' ...-------------------------------------------------------+--> ...+---------------+--+-----------------------------------+-' '- -autoexpand -' '- -grainsize 32 | 64 | 128 | 256 -' >--+-----------------+--+-------------------------+-------------> '- -import -- -- -' '- -copies -- num_copies -' >--+---------------------------+--+---------------+-------------> '- -syncrate -- percentage -' '- -createsync -' >--+-------------------------+-- --+-----------------------+----> '- -udid -- vdisk_udid -' '- -vtype --+- seq ---+-' '- image -' >-- --+--------------------------+-- --+-------------------+----> '- -node --+- node_name -+-' '- -unit --+- b --+-' '- node_id ---' +- kb -+ +- mb -+ +- gb -+ +- tb -+ '- pb -' >-- --+---------------------------------+-- --------------------> '- -mdisk --+- mdisk_id_list ---+-' '- mdisk_name_list -' >--+-------------------------+-- -------------------------------> '- -name -- new_name_arg -' >--+------------------------------+---------------------------->< '- -cache -- readwrite | none -' For the detailed description of the parameters, please refer to 6.7, “Creating a virtual disk” on page 167. The command to create an image mode VDisk and the system response are shown in Example 9-73. Example 9-73 svctask mkvdisk (image mode)
IBM_2145:ITSO-CLS1:admin>svctask mkvdisk -mdiskgrp MDG_Image -iogrp 0 -mdisk mdisk20 -vtype image -name Image_Vdisk_A Virtual Disk, id [8], successfully created This command creates an image mode VDisk called Image_VdiskA using MDisk mdisk20. The VDisk belongs to the MDG MDG_Image and is owned by the I/O group io_grp0. If we run the svcinfo lsmdisk command again, notice that mdisk20 now has a status of image, as shown in Example 9-74 on page 351. 350
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 9-74 svcinfo lsmdisk
IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk -delim , id,name,status,mode,mdisk_grp_id,mdisk_grp_name,capacity,ctrl_LUN_#,controller_nam e,UID 0,mdisk0,online,managed,0,MDG_DS45,36.0GB,0000000000000000,DS4500,600a0b8000174431 000000eb47139cca00000000000000000000000000000000 1,mdisk1,online,managed,0,MDG_DS45,36.0GB,0000000000000001,DS4500,600a0b8000174431 000000ef47139e1c00000000000000000000000000000000 2,mdisk2,online,managed,0,MDG_DS45,36.0GB,0000000000000002,DS4500,600a0b8000174431 000000f147139e7200000000000000000000000000000000 3,mdisk3,online,managed,0,MDG_DS45,36.0GB,0000000000000003,DS4500,600a0b8000174431 000000e44713575400000000000000000000000000000000 4,mdisk4,online,managed,0,MDG_DS45,36.0GB,0000000000000004,DS4500,600a0b8000174431 000000e64713576000000000000000000000000000000000 5,mdisk5,online,managed,1,MDG_DS47,36.0GB,0000000000000000,DS4700,600a0b800026b282 00003ea34851577c00000000000000000000000000000000 6,mdisk_6,online,managed,0,MDG_DS45,36.0GB,0000000000000005,DS4500,600a0b800017443 1000000e747139cb600000000000000000000000000000000 7,mdisk7,online,managed,1,MDG_DS47,36.0GB,0000000000000001,DS4700,600a0b80002904de 00004188485157a400000000000000000000000000000000 8,mdisk8,online,managed,0,MDG_DS45,36.0GB,0000000000000006,DS4500,600a0b8000174431 000000ea47139cc400000000000000000000000000000000 9,mdisk9,online,managed,1,MDG_DS47,36.0GB,0000000000000002,DS4700,600a0b800026b282 00003ed6485157b600000000000000000000000000000000 10,mdisk10,online,managed,0,MDG_DS45,36.0GB,0000000000000007,DS4500,600a0b80001744 31000000e34713574e00000000000000000000000000000000 11,mdisk11,online,managed,0,MDG_DS45,36.0GB,0000000000000008,DS4500,600a0b80001744 31000000de4713567600000000000000000000000000000000 12,mdisk12,online,managed,1,MDG_DS47,36.0GB,0000000000000003,DS4700,600a0b80002904 de000041ba485157d000000000000000000000000000000000 13,mdisk13,online,managed,0,MDG_DS45,36.0GB,0000000000000009,DS4500,600a0b80001744 31000000e14713574200000000000000000000000000000000 14,mdisk14,online,managed,1,MDG_DS47,36.0GB,0000000000000004,DS4700,600a0b800026b2 8200003f6c4851585200000000000000000000000000000000 15,mdisk15,online,managed,0,MDG_DS45,36.0GB,000000000000000A,DS4500,600a0b80001744 31000000e24713574800000000000000000000000000000000 16,mdisk16,online,managed,0,MDG_DS45,36.0GB,000000000000000B,DS4500,600a0b80001744 31000000e54713575a00000000000000000000000000000000 17,mdisk17,online,managed,0,MDG_DS45,36.0GB,000000000000000C,DS4500,600a0b80001744 31000000e947139cbe00000000000000000000000000000000 18,mdisk18,online,managed,1,MDG_DS47,36.0GB,0000000000000005,DS4700,600a0b80002904 de000042504851586800000000000000000000000000000000 19,mdisk19,online,managed,1,MDG_DS47,36.0GB,0000000000000006,DS4700,600a0b800026b2 8200003f9f4851588700000000000000000000000000000000 20,mdisk20,online,image,2,MDG_Image,36.0GB,0000000000000007,DS4700,600a0b80002904d e00004282485158aa00000000000000000000000000000000
9.9.4 Adding a mirrored VDisk copy You can create a mirrored copy of a VDisk. This keeps a VDisk accessible even when the MDisk on which it depends has become unavailable. You can create a copy of a VDisk either on different MDisk groups or by creating an image mode copy of the VDisk. Copies increase the availability of data; however, they are not separate objects. You can only create or change mirrored copies from the VDisk. Chapter 9. SVC configuration and administration using the CLI
351
In addition, you can use VDisk mirroring as an alternative method of migrating VDisks between MDisk groups. For example, if you have a non-mirrored VDisk in one MDisk group and want to migrate that VDisk to another MDisk group, you can add a new copy of the VDisk and specify the second MDisk group. After the copies are synchronized, you can delete the copy on the first MDisk group. The VDisk is migrated to the second MDisk group while remaining online during the migration. To create a mirror copy from a VDisk copy, use the addvdiskcopy command. This adds a copy to an existing VDisk, which changes a non-mirrored VDisk into a mirrored VDisk. The following syntax specifies the addition of a sequential or striped mode VDisk: >>- svctask -- --addvdiskcopy-- --------------------------------> >-- -mdiskgrp --+- mdisk_group_id_list ---+-- ------------------> '- mdisk_group_name_list -' >--+-------------------------+-- -- ----------------------------> '- -vtype --+- seq -----+-' '- striped -' >--+---------------------------------+-- -- --------------------> '- -mdisk --+- mdisk_id_list ---+-' '- mdisk_name_list -' >--+---------------+-- -----------------------------------------> '- -createsync -' >--+-------------------------------------------------------------... '- -rsize -- disk_size | disk_size_percentage--%-- | ...------------------------------------------------------------+--> ...auto-----+--------------------------------------------------+-' ............+- -warning disk_size |disk_size_percentage% | off-+ ............+- -autoexpand ------------------------------------+ ............'- -grainsize 32 | 64 | 128 | 256 ----------------' >-- --+-----------+-- --+-------------------------+-- -- -------> '- -fmtdisk-' '- -syncrate --percentage-' >--+-------------------+-- --+- vdisk_name -+------------------>< '- -unit --+- b --+-' '- vdisk_id ---' +- kb -+ +- mb -+ +- gb -+ +- tb -+ '- pb -' -mdiskgrp mdisk_group_id_list | mdisk_group_name_list (Required) Specifies the managed disk groups to use to create copies for the virtual disk. You must specify a group for each copy that is being added.
352
Implementing the IBM System Storage SAN Volume Controller V4.3
-vtype seq | striped | image (Optional) Specifies the virtualization type for the copy: sequential, striped, or image. The type can be different than the virtualization types for other copies on the VDisk. The default virtualization type is striped. -mdisk mdisk_id_list | mdisk_name_list (Optional) Specifies one or more managed disks (MDisks). For sequential and image mode copies, you must specify a single MDisk that has sufficient free extents. For image mode copies, the MDisk must be in unmanaged mode. For sequential mode copies, the MDisk must be in managed mode. -syncrate percentage (Optional) Specifies the copy synchronization rate, as a percentage of the peak synchronization rate. A value of zero (0) prevents synchronization. The default value is 50. -createsync (Optional) Suppresses the synchronization of the new VDisk copy with the primary copy. Using this parameter can cause data corruption if the primary copy fails and leaves an unsynchronized secondary copy to provide data. Using this parameter can cause loss of read stability in unwritten are if the primary copy fails, data is read from the primary copy, and then different data is read from the secondary copy. To avoid data loss or read stability loss, use this parameter only for a primary copy that has been formatted and not written to, and with the -fmtdisk parameter. -fmtdisk (Optional) Formats a sequential or striped mode copy. You must also specify the -createsync parameter, which labels the formatted copy as identical to the primary copy. The -fmtdisk parameter causes the VDisk to go offline until new VDisk copy formatting completes. To query the formatting progress, use the lsvdiskprogress command. -rsize disk_size | disk_size_percentage% | auto (Optional) Makes the copy space-efficient and specifies the real size of the copy. Specify the disk_size | disk_size_percentage value using an integer, or an integer immediately followed by the percent character (%). The default units for disk_size are megabytes (MB); to specify different units, use the -unit parameter. The auto option creates a VDisk copy that uses the entire size of the MDisk; if you specify the -rsize auto option, you must also specify the -vtype image option. -warning disk_size | disk_size_percentage% (Optional) Requires that the -rsize parameter also be specified. Generates a warning when the used disk capacity on the space-efficient copy first exceeds the specified threshold. You can specify a disk_size integer, which defaults to megabytes (MB) unless the -unit parameter is specified; or you can specify a disk_size%, which is a percentage of the virtual disk size. If -autoexpand is enabled, the default value for -warning is 80% of the virtual disk capacity. If -autoexpand is not enabled, the default value for warning is 80% of the real capacity. To disable warnings, specify 0 or 0%. -autoexpand (Optional) Specifies that space-efficient copies automatically expand their real capacities by allocating new extents from their managed disk group. Requires that the -rsize parameter also be specified. If the -autoexpand parameter is specified, the -rsize parameter specifies a capacity that is reserved by the copy. This protects the copy from going offline when its managed disk group runs out of space by allowing it to consume this reserved space first.
Chapter 9. SVC configuration and administration using the CLI
353
-grainsize 32 | 64 | 128 | 256 (Optional) Sets the grain size (KB) for a space-efficient VDisk. Requires that the -rsize parameter also be specified. The default is 32 KB. -import name (Optional) Imports an image mode disk that contains a space-efficient volume into the cluster. Requires that the -rsize and -vtype image parameters also be specified. -unit b | kb | mb | gb | tb | pb (Optional) Specifies the data units for the -rsize and -warning parameters. In the following scenario, we show how to add a VDisk copy mirror to an existing VDisk copy. As you can see in Example 9-75, the VDisk has a copy with copy_id 0. Example 9-75 lsvdisk
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk vdisk_C id 2 name vdisk_C IO_group_id 1 IO_group_name io_grp1 status online mdisk_grp_id 1 mdisk_grp_name MDG_DS47 capacity 45.0GB type striped formatted no mdisk_id mdisk_name FC_id FC_name RC_id RC_name vdisk_UID 60050768018301BF2800000000000002 virtual_disk_throttling (MB) 20 preferred_node_id 3 fast_write_state empty cache readwrite udid fc_map_count 0 sync_rate 50 copy_count 1 copy_id 0 status online sync yes primary yes mdisk_grp_id 1 mdisk_grp_name MDG_DS47 type striped mdisk_id mdisk_name fast_write_state empty used_capacity 0.41MB real_capacity 12.00GB
354
Implementing the IBM System Storage SAN Volume Controller V4.3
free_capacity 12.00GB overallocation 375 autoexpand off warning 23 grainsize 32 In Example 9-76, the VDisk copy mirror is being added using the svctask addvdiskcopy command. Example 9-76 svctask addvdiskcopy
IBM_2145:ITSO-CLS1:admin>svctask addvdiskcopy -mdiskgrp MDG_DS45 -vtype striped -rsize 20 -autoexpand -grainsize 64 -unit gb vdisk_C Vdisk [2] copy [1] successfully created During the synchronization process, the status can be seen using the command svcinfo lsvdisksyncprogress. As shown in Example 9-77, the first time the status has been checked the synchronization progress was at 86%, and the estimated completion time was 19:16:54. The second time the command is performed, the progress status is at 100%, and the synchronization has completed. Example 9-77 Synchronization
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisksyncprogress -copy 1 vdisk_C vdisk_id vdisk_name copy_id progress estimated_completion_time 2 vdisk_C 1 86 080710191654 IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisksyncprogress -copy 1 vdisk_C vdisk_id vdisk_name copy_id progress estimated_completion_time 2 vdisk_C 1 100 As you can see in Example 9-78, the new VDisk copy mirror (copy_id 1) has been added and can be seen using the svcinfo lsvdisk command. Example 9-78 svcinfo lsvidsk
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk vdisk_C id 2 name vdisk_C IO_group_id 1 IO_group_name io_grp1 status online mdisk_grp_id many mdisk_grp_name many capacity 45.0GB type many formatted no mdisk_id many mdisk_name many FC_id FC_name RC_id RC_name vdisk_UID 60050768018301BF2800000000000002
Chapter 9. SVC configuration and administration using the CLI
355
virtual_disk_throttling (MB) 20 preferred_node_id 3 fast_write_state empty cache readwrite udid fc_map_count 0 sync_rate 50 copy_count 2 copy_id 0 status online sync yes primary yes mdisk_grp_id 1 mdisk_grp_name MDG_DS47 type striped mdisk_id mdisk_name fast_write_state empty used_capacity 0.41MB real_capacity 12.00GB free_capacity 12.00GB overallocation 375 autoexpand off warning 23 grainsize 32 copy_id 1 status online sync yes primary no mdisk_grp_id 0 mdisk_grp_name MDG_DS45 type striped mdisk_id mdisk_name fast_write_state empty used_capacity 0.44MB real_capacity 20.02GB free_capacity 20.02GB overallocation 224 autoexpand on warning 80 grainsize 64 Notice that the VDisk copy mirror (copy_id 1) does not have the same values as the VDisk copy. While adding a VDisk copy mirror, you are able to define a mirror with different parameters than the VDisk copy. This means that you can define a space-efficient VDisk copy mirror for a non-space-efficient VDisk copy and vice-versa. This is one of the ways to migrate a non-space-efficient VDisk to a space-efficient VDisk Note: To change the parameters of a VDisk copy mirror, it must be deleted and redefined with the new values.
356
Implementing the IBM System Storage SAN Volume Controller V4.3
9.9.5 Splitting a VDisk Copy The splitvdiskcopy command creates a new VDisk in the specified I/OGroup from a copy of the specified VDisk. If the copy that you are splitting is not synchronized, you must use the -force parameter. The command fails if you are attempting to remove the only synchronized copy. To avoid this, wait for the copy to synchronize or split the unsynchronized copy from the VDisk by using the -force parameter. You can run the command when either VDisk copy is offline. Example 9-79 shows the command svctask splitvdiskcopy, which is used to split a VDisk copy. It creates a new vdisk_N from the copy of vdisk_B. Example 9-79 Split VDisk
IBM_2145:ITSO-CLS1:admin>svctask splitvdiskcopy -copy 1 -iogrp 0 -name vdisk_N vdisk_B Virtual Disk, id [2], successfully created As you can see in Example 9-80, the new VDisk, vdisk_N, has been created as an independent VDisk. Example 9-80 svcinfo lsvdisk
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk vdisk_N id 2 name vdisk_N IO_group_id 0 IO_group_name io_grp0 status online mdisk_grp_id 0 mdisk_grp_name MDG_DS45 capacity 100.0GB type striped formatted no mdisk_id mdisk_name FC_id FC_name RC_id RC_name vdisk_UID 60050768018301BF280000000000002F throttling 0 preferred_node_id 2 fast_write_state empty cache readwrite udid fc_map_count 0 sync_rate 50 copy_count 1 copy_id 0 status online sync yes primary yes mdisk_grp_id 0 mdisk_grp_name MDG_DS45 type striped Chapter 9. SVC configuration and administration using the CLI
357
mdisk_id mdisk_name fast_write_state empty used_capacity 84.75MB real_capacity 20.10GB free_capacity 20.01GB overallocation 497 autoexpand on warning 80 grainsize 64 The VDisk copy of VDisk vdisk_B has now lost its mirror. Therefore, a new VDisk has been created.
9.9.6 Deleting a VDisk When executing this command on an existing managed mode VDisk, any data that remained on it will be lost. The extents that made up this VDisk will be returned to the pool of free extents available in the Managed Disk Group. If any Remote Copy, FlashCopy, or Host mappings still exist for this virtual disk, then the delete will fail unless the -force flag is specified. Now, any mapping that remains will be deleted and then the VDisk will be deleted. If the VDisk is currently the subject of a migrate to image mode, then the delete will fail unless the -force flag is specified. Now the migration will be halted and the VDisk deleted. If the command succeeds (without the -force flag) for an image mode disk, then the underlying back-end controller logical unit will be consistent with the data that a host could previously have read from the Image Mode Virtual Disk, that is, all fast write data will have been flushed to the underlying LUN. If the -force flag is used, then this guarantee does not hold. If any Remote FlashCopy or Host mappings still exist for this VDisk, then the delete will fail unless the -force flag is specified. Now, any mapping that remains will be deleted and then the VDisk will be deleted. If there is any un-destaged data in the fast write cache for this VDisk, then the deletion of the VDisk will fail unless the -force flag is specified. Now any un-destaged data in the fast write cache will be deleted. Use the svctask rmvdisk command to delete a VDisk from your SVC configuration, as shown in Example 9-81. Example 9-81 svctask rmvdisk
IBM_2145:ITSO-CLS1:admin>svctask rmvdisk vdisk_A This command deletes VDisk vdisk_A from the SVC configuration. If the VDisk is assigned to a host, you need to use the -force flag to delete the VDisk (Example 9-82). Example 9-82 svctask rmvdisk (force)
IBM_2145:ITSO-CLS1:admin>svctask rmvdisk -force vdisk_A
358
Implementing the IBM System Storage SAN Volume Controller V4.3
9.9.7 Expanding a VDisk Expanding a VDisk presents a larger capacity disk to your operating system. Although this can be easily done using the SVC, you must ensure your operating system(s) supports expansion before using this function. Assuming your operating system(s) supports it, you can use the svctask expandvdisksize command to increase the capacity of a given VDisk. The full syntax of the expandvdisksize command is: >>- svctask -- -- expandvdisksize -- --+- -size disk_size -+---> '- -rsize disk_size-' >-- --+---------------------------------+-- --+------------+----> '- -mdisk --+- mdisk_id_list ---+-' '- -fmtdisk -' '- mdisk_name_list -' >-- --+-------------------+--+--------------+-- ----------------> '- -unit --+- b --+-' '- -copy-- id -' +- kb -+ +- mb -+ +- gb -+ +- tb -+ '- pb -' >--+- vdisk_name -+-------------------------------------------->< '- vdisk_id ---' Note the following explanation of the parameters: -size disk_size (Optional) Specifies the capacity by which the virtual disk is expanded. Disk size is used with the value of the unit. All capacities, including changes, must be in multiples of 512 bytes. An error occurs if you specify a capacity that is not a multiple of 512, which can only occur when byte units (-unit b) are used. However, an entire extent is reserved even if it is only partially used. The default disk_size unit is megabytes (MB). The -size parameter is mutually exclusive with the -rsize parameter. You must specify either -size or -rsize. If the VDisk is space-efficient, MDisks cannot be specified. -rsize disk_size (Optional) Specifies the capacity by which to increase the real size of a space-efficient VDisk. Specify the disk_size value using an integer. Specify the unit for a disk_size integer using the -unit parameter; the default unit is megabytes (MB). The -rsize value can be greater than, equal to, or less than the size of the VDisk. The -size parameter is mutually exclusive with the -rsize parameter. You must specify either -size or -rsize. -copy id (Optional) Specifies the copy to change the real capacity for. You must also specify the -rsize parameter; you can only modify the real capacity of a VDisk copy. The -copy parameter is required if the specified VDisk is mirrored and only one copy is space-efficient. If the VDisk is mirrored, both copies are space-efficient and -copy is not specified, both copies are modified by the same amount.
Chapter 9. SVC configuration and administration using the CLI
359
-mdisk mdisk_id_list | mdisk_name_list (Optional) Specifies the list of one or more MDisks to be used as the stripe set. The extents that expand the VDisk come from the specified list of MDisks. All MDisks in the list must be part of the same MDisk group. The -mdisk parameter cannot be used if the specified VDisk is mirrored. -fmtdisk (Optional) Specifies that the VDisk be formatted before use. This parameter formats the new extents that have been added to the VDisk as a result of the expandvdisksize command. The expandvdisksize command completes asynchronously if you use this parameter. -unit b | kb | mb | gb | tb | pb (Optional) Specifies the disk_size unit for the -size or -rsize parameter. The default value is megabytes (MB). vdisk_name | vdisk_id (Required) Specifies the virtual disk to modify, either by ID or by name. A sample of this command is shown in Example 9-83. Example 9-83 svctask expandvdisksize
IBM_2145:ITSO-CLS1:admin>svctask expandvdisksize -size 5 -unit gb vdisk_C This command expands the vdisk_C by another 5 GB to give a total of 40 GB. To expand a space-efficient VDisk, you can use the -rsize option, as shown in Example 9-84. This command changes the real size of VDisk vdisk_B to real capacity of 55 GB. The capacity of the Vdisk remains unchanged. Example 9-84 svcinfo lsvdisk
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk vdisk_B id 1 name vdisk_B . . capacity 100.0GB . . mdisk_name fast_write_state empty used_capacity 0.41MB real_capacity 50.00GB free_capacity 50.00GB overallocation 200 autoexpand off warning 40 grainsize 32 IBM_2145:ITSO-CLS1:admin>svctask expandvdisksize -rsize 5 -unit gb vdisk_B IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk vdisk_B id 1 name vdisk_B 360
Implementing the IBM System Storage SAN Volume Controller V4.3
. . capacity 100.0GB . . mdisk_grp_name MDG_DS47 type striped mdisk_id mdisk_name fast_write_state empty used_capacity 0.41MB real_capacity 55.00GB free_capacity 55.00GB overallocation 181 autoexpand off warning 40 grainsize 32 Important: If a VDisk is expanded, its type will become striped even if it was previously sequential or in image mode. If there are not enough extents to expand your VDisk to the specified size, you will receive the following error message: CMMVC5860E Ic_failed_vg_insufficient_virtual_extents
9.9.8 Mapping a VDisk to a host Use the svctask mkvdiskhostmap to map a VDisk to a host. The full syntax is: >>- svctask -- -- mkvdiskhostmap -- --+----------+-- -----------> '- -force -' >-- -host --+- host_id ---+-- --+-------------------------+-----> '- host_name -' '- -scsi -- scsi_num_arg -' >-- --+- vdisk_name -+----------------------------------------->< '- vdisk_id ---' When executed, this command will create a new mapping between the virtual disk and the host specified. This will essentially present this virtual disk to the host, as though the disk was directly attached to the Host. It is only after this command is executed that the host can perform I/O to the virtual disk. Optionally, a SCSI LUN ID can be assigned to the mapping. When the HBA on the host scans for devices attached to it, it will discover all Virtual Disks that are mapped to its Fibre Channel ports. When the devices are found, each one is allocated an identifier (SCSI LUN ID). For example, the first disk found will generally be SCSI LUN 1, and so on. You can control the order in which the HBA discovers Virtual Disks by assigning the SCSI LUN ID as required. If you do not specify a SCSI LUN ID, then the cluster will automatically assign the next available SCSI LUN ID, given any mappings that already exist with that Host.
Chapter 9. SVC configuration and administration using the CLI
361
It is worth noting that some HBA device drivers will stop when they find a gap in the SCSI LUN IDs. For example: Virtual Disk 1 is mapped to Host 1 with SCSI LUN ID 1. Virtual Disk 2 is mapped to Host 1 with SCSI LUN ID 2. Virtual Disk 3 is mapped to Host 1 with SCSI LUN ID 4. When the device driver scans the HBA, it might stop after discovering Virtual Disks 1 and 2, because there is no SCSI LUN mapped with ID 3. Care should therefore be taken to ensure that the SCSI LUN ID allocation is contiguous. It is not possible to map a virtual disk to a host more than once at different LUN numbers (Example 9-85). Example 9-85 svctask mkvdiskhostmap
IBM_2145:ITSO-CLS1:admin>svctask mkvdiskhostmap -host Siam vdisk_A Virtual Disk to Host map, id [0], successfully created This command maps the VDisk called vdisk_A to the host called Siam.
9.9.9 Deleting a VDisk-to-host mapping If you mapped a VDisk to a host by mistake, or you simply want to reassign the VDisk to another host, use the svctask rmvdiskhostmap command to unmap a VDisk from a host (Example 9-86). Example 9-86 svctask rmvdiskhostmap
IBM_2145:ITSO-CLS1:admin>svctask rmvdiskhostmap -host Nile vdisk_D This command unmaps the VDisk called vdisk_D from the host Nile.
9.9.10 Showing the VDisks mapped to a host Use the svcinfo lshostvdiskmap command to show which VDisks are assigned to a specific host (Example 9-87). Example 9-87 svcinfo lshostvdiskmap
IBM_2145:ITSO-CLS1:admin>svcinfo lshostvdiskmap -delim , Siam id,name,SCSI_id,vdisk_id,vdisk_name,wwpn,vdisk_UID 3,Siam,0,0,vdisk_A,210000E08B18FF8A,60050768018301BF280000000000000C From this command, you can see that the host Siam has only one VDisk called vdisk_A assigned. The SCSI LUN ID is also shown. This is the ID by which the Virtual Disk is being presented to the host. If no host is specified, all defined host to VDisk mappings will be returned. Note: Although the -delim, flag normally comes at the end of the command string, in this case you must specify this flag before the host name. Otherwise, it returns the following message: CMMVC6070E An invalid or duplicated parameter, unaccompanied argument, or incorrect argument sequence has been detected. Ensure that the input is as per the help.
362
Implementing the IBM System Storage SAN Volume Controller V4.3
9.9.11 Modifying a VDisk Executing the svctask chvdisk command will modify a single property of a Virtual Disk. Only one property can be modified at a time. So, to change the name and modify the I/O Group would require two invocations of the command. A new name, or label, can be specified. The new name can be used subsequently to reference the Virtual Disk. The I/O Group with which this Virtual Disk is associated can be changed. Note that this requires a flush of the cache within the nodes in the current I/O Group to ensure that all data is written to disk. I/O must be suspended at the host level before performing this operation. The full syntax of the svctask chvdisk command is: >>- svctask -- -- chvdisk -- --+-------------------------+------> '- -name -- new_name_arg -' >--+-------------------------------+--+----------+--------------> '- -iogrp --+- io_group_id ---+-' '- -force -' '- io_group_name -' >--+--------------------------+--+--------------------------+---> '- -node --+- node_id ---+-' '- -rate -- throttle_rate -' '- node_name -' >--+-----------+--+-------------------------+-- ----------------> '- -unitmb -' '- -udid -- vdisk_udid -' >--+--------------------------------------------------+---------> '- -warning --disk_size |--disk_size_percentage--%-' >--+-------------------------+--+--------------+----------------> '- -autoexpand --on | off-' '- -copy-- id -' >--+----------------------+--+------------------------------+---> '- -primary-- copy_id -' '- -syncrate-- percentage_arg -' >--+-------------------+--+- vdisk_name -+--------------------->< '- -unit --+- b --+-' '- vdisk_id ---' +- kb -+ +- mb -+ +- gb -+ +- tb -+ '- pb -' -name new_name_arg (Optional) Specifies a new name to assign to the virtual disk. You cannot use this parameter with the -iogrp, -rate, -node, or -udid parameters. This parameter is required if you do not use the -iogrp, -rate, or -udid parameters. -iogrp io_group_id | io_group_name (Optional) Specifies a new I/O group to move the virtual disk to, by IO group ID or IO group name. You can use the -node parameter with the -iogrp parameter to specify a preferred node for the specified VDisk.
Chapter 9. SVC configuration and administration using the CLI
363
Note: If the VDisk has a mapping to any hosts, it is not possible to move the VDisk to an I/O group that does not include any of those hosts. This parameter can fail if there is not enough space to allocate bitmaps for a mirrored VDisk in the target IO group. This parameter can fail if any copy is not synchronized. The -force parameter can be used to force the move, but this synchronizes the VDisk again. -force (Optional) Forces the VDisk to be removed from an I/O group. This parameter can only be used with the -iogrp parameter. Attention: 1. If the -force parameter is used and the cluster is unable to destage all write data from the cache, the contents of the VDisk are corrupted by the loss of the cached data. 2. If the -force parameter is used to move a VDisk that has out-of-sync copies, a full resynchronization is required. -rate throttle_rate [-unitmb] (Optional) Specifies the I/O governing rate for the VDisk, which caps the amount of I/O that is accepted. The default throttle_rate units are I/Os. To change the throttle_rate units to megabytes per second (MBps), specify the -unitmb parameter. The governing rate for a virtual disk can be specified by I/Os or by MBps, but not both. However, you can set the rate to I/Os for some virtual disks and to MBps for others.You cannot use this parameter with the -name, -iogrp, -node, or -udid parameters. -udid vdisk_udid (Optional) Specifies the unit number (udid) for the disk. The vdisk_udid is an identifier that is required to support OpenVMS hosts; no other systems use this parameter. Valid options are a decimal number from 0 to 32 767 or a hexadecimal number from 0 to 0x7FFF. A hexadecimal number must be preceded by 0x (for example, 0x1234). If you do not use the -udid parameter, the default udid is 0.You cannot use this parameter with the -name, -iogrp, -node, or -rate parameters. -warning disk_size | disk_size_percentage% (Optional) Generates a warning when the used disk capacity on the space-efficient copy first exceeds the specified threshold. You can specify a disk_size integer, which defaults to MBs unless the -unit parameter is specified; or you can specify a disk_size_percentage, which is a percentage of the virtual disk size. To disable warnings, specify 0 or 0%. -unit b | kb | mb | gb | tb | pb (Optional) Specifies the data units to use for the -warning disk_size parameter. -autoexpand on | off (Optional) Specifies whether space-efficient VDisk copies automatically expand their real capacities by allocating new extents from their managed disk group. To use this parameter, the VDisk must be space-efficient.
364
Implementing the IBM System Storage SAN Volume Controller V4.3
-copy id (Optional) Specifies the copy to apply the changes to. You must specify this parameter with the -autoexpand or -warning parameter. The -copy parameter is required if the specified VDisk is mirrored and only one VDisk copy is space-efficient. If both copies are space-efficient and the -copy parameter is not specified, the specified -autoexpand or -warning parameter is set on both copies. -primary copy_id (Optional) Specifies the primary copy. Changing the primary copy only takes effect when the new primary copy is online and synchronized. If the new primary is online and synchronized when the command is issued, the change takes effect immediately. -syncrate percentage (Optional) Specifies the copy synchronization rate, as a percentage of the peak synchronization rate. A value of zero (0) prevents synchronization. -node node_id | node_name (Optional) Specifies a preferred node for the specified VDisk. When using this parameter, you must also specify the -iogrp parameter. You cannot use this parameter with the -name, -rate, or -udid parameters. vdisk_name | vdisk_id (Required) Specifies the virtual disk to modify, either by ID or by name. Changing the name of a VDisk is quite an obvious task. However, the I/O governing parameter is a new concept.
I/O governing You can set a limit on the amount of I/O transactions that is accepted for this virtual disk. It is set in terms of I/Os per second or MB per second. By default, no I/O governing rate is set when a virtual disk is created. The choice between I/O and MB as the I/O governing throttle should be based on the disk access profile of the application. Database applications generally issue large amounts of I/O, but only transfer a relatively small amount of data. In this case, setting an I/O governing throttle based on MBs per second does not achieve much. It is better to use an I/O as a second throttle. On the other extreme, a streaming video application generally issues a small amount of I/O, but transfers large amounts of data. In contrast to the database example, setting an I/O governing throttle based on I/Os per second does not achieve much, so it is better to use an MB per second throttle. Note: An I/O governing rate of 0 (displayed as throttling in CLI output of svcinfo lsvdisk command) does not mean that zero I/O per second (or MBs per second) can be achieved. It means that no throttle is set. An example of the chvdisk command is shown in Example 9-88. Example 9-88 svctask chvdisk (rate/warning SEV)
IBM_2145:ITSO-CLS1:admin>svctask chvdisk -rate 20 -unitmb vdisk_C IBM_2145:ITSO-CLS1:admin>svctask chvdisk -warning 85% vdisk7
Chapter 9. SVC configuration and administration using the CLI
365
Note: The chvdisk command specifies the new name first. The name can consist of letters A to Z, a to z, numbers 0 to 9, the dash ’-’, and the underscore ’_’. It can be between one and 15 characters in length. However, it cannot start with a number, the dash, or the word vdisk, because this prefix is reserved for SVC assignment only. The first command changes the VDisk throttling of vdisk7 to 20 MBps, while the second command changes the SEV warning to 85%. If you want to verify the changes, issue the svcinfo lsvdisk command, as shown in Example 9-89. Example 9-89 svcinfo lsvdisk command: verifying throttling
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk vdisk7 id 7 name vdisk7 IO_group_id 1 IO_group_name io_grp1 status online mdisk_grp_id 0 mdisk_grp_name MDG_DS45 capacity 10.0GB type striped formatted no mdisk_id mdisk_name FC_id FC_name RC_id RC_name vdisk_UID 60050768018301BF280000000000000A virtual_disk_throttling (MB) 20 preferred_node_id 6 fast_write_state empty cache readwrite udid fc_map_count 0 sync_rate 50 copy_count 1 copy_id 0 status online sync yes primary yes mdisk_grp_id 0 mdisk_grp_name MDG_DS45 type striped mdisk_id mdisk_name fast_write_state empty used_capacity 0.41MB real_capacity 5.02GB free_capacity 5.02GB overallocation 199 autoexpand on 366
Implementing the IBM System Storage SAN Volume Controller V4.3
warning 85 grainsize 32
9.9.12 Migrating a VDisk From time to time, you might want to migrate VDisks from one set of MDisks to another to retire an old disk subsystem, to have better balance performance across your virtualized environment, or simply to migrate data into the SVC environment transparently using image mode. To do so, use the svctask migratevdisk command. The full syntax of the command is: >>- svctask -- -- migratevdisk -- ------------------------------> >-- -mdiskgrp --+- mdisk_group_id ---+-- -----------------------> '- mdisk_group_name -' >--+---------------------------------+-- --+--------------+-----> '- -threads -- number_of_threads -' '- -copy-- id -' >-- -- -vdisk --+- vdisk_id ---+------------------------------->< '- vdisk_name -' The parameters are: -mdiskgrp mdisk_group_id | mdisk_group_name (Required) Specifies the new managed disk group ID or name. -threads number_of_threads (Optional) Specifies the number of threads to use during the migration of these extents. You can specify one to four threads. The default number of threads is four. -copy id (Required if the specified VDisk has more than one copy) Specifies the VDisk copy to migrate. -vdisk vdisk_id | vdisk_name (Required) Specifies the virtual disk ID or name to migrate in to a new managed disk group. Important: After migration is started, it continues to completion unless it is stopped or suspended by an error condition or the VDisk being migrated is deleted. As you can see from the above parameters, before you can migrate your VDisk, you must know the name of the VDisk you want to migrate and the name of the MDG to which you want to migrate. To find the name, simply run the svcinfo lsvdisk and svcinfo lsmdiskgrp commands. When you know these details, you can issue the migratevdisk command, as shown in Example 9-90. Example 9-90 svctask migratevdisk
IBM_2145:ITSO-CLS1:admin>svctask migratevdisk -mdiskgrp MDG_DS47 -vdisk vdisk_C This command moves the VDisk vdisk7 to the MDG MDG_DS47.
Chapter 9. SVC configuration and administration using the CLI
367
Note: If insufficient extents are available within your target MDG, you receive an error message. Make sure the source and target MDisk group have the same extent size. The optional threads parameter allows you to assign a priority to the migration process. The default is 4, which is the highest priority setting. However, if you want the process to take a lower priority over other types of I/O, you can specify 3, 2, or 1. You can run the svcinfo lsmigrate command at any time to see the status of the migration process. This is shown in Example 9-91. Example 9-91 svcinfo lsmigrate command
IBM_2145:ITSO-CLS1:admin>svcinfo lsmigrate migrate_type MDisk_Group_Migration progress 12 migrate_source_vdisk_index 2 migrate_target_mdisk_grp 1 max_thread_count 4 migrate_source_vdisk_copy_id 0 IBM_2145:ITSO-CLS1:admin>svcinfo lsmigrate migrate_type MDisk_Group_Migration progress 16 migrate_source_vdisk_index 2 migrate_target_mdisk_grp 1 max_thread_count 4 migrate_source_vdisk_copy_id 0 Note: The progress is given as percent complete. If you get no more replies, then the process has finished.
9.9.13 Migrate a VDisk to an image mode VDisk Migrating a VDisk to an image mode VDisk allows SVC to be removed from the data path.This might be useful where SVC is used as a data mover appliance. You can use the svctask migratetoimage command to do this. To migrate a VDisk to an image mode VDisk, the following rules apply: The destination MDisk must be greater than or equal to the size of the VDisk. The MDisk specified as the target must be in an unmanaged state. Regardless of the mode that the VDisk starts in, it is reported as managed mode during the migration. Both of the MDisks involved are reported as being image mode during the migration. If the migration is interrupted by a cluster recovery or by a cache problem, then the migration will resume after the recovery completes. The full syntax of the svctask migratetoimage command is: >>- svctask -- -- migratetoimage -- --+--------------+-- -------> '- -copy-- id -' >-- -vdisk --+- source_vdisk_id ---+-- -------------------------> '- source_vdisk_name -' 368
Implementing the IBM System Storage SAN Volume Controller V4.3
>--+---------------------------------+-- -----------------------> '- -threads -- number_of_threads -' >-- -mdisk --+- unmanaged_target_mdisk_id ---+-- ---------------> '- unmanaged_target_mdisk_name -' >-- -mdiskgrp --+- managed_disk_group_id ---+------------------>< '- managed_disk_group_name -' The parameters are: -vdisk source_vdisk_id | name (Required) Specifies the name or ID of the source VDisk to be migrated. -copy id (Required if the specified VDisk has more than one copy.) Specifies the VDisk copy to migrate from. -threads number_of_threads (Optional) Specifies the number of threads to use during the migration of extents. You can specify one to four threads. The default number of threads is four, which is the highest priority. -mdisk unmanaged_target_mdisk_id | name (Required) Specifies the name of the MDisk to which the data must be migrated. This disk must be unmanaged and large enough to contain the data of the disk that is being migrated. -mdiskgrp managed_disk_group_id | name (Required) Specifies the MDisk group into which the MDisk must be placed, after the migration has completed. An example of the command is shown in Example 9-92. Example 9-92 svctask migratetoimage
IBM_2145:ITSO-CLS1:admin>svctask migratetoimage -vdisk vdisk_A -mdisk mdisk8 -mdiskgrp MDG_Image In this example, you migrate the data from vdisk_A onto mdisk8 and the MDisk must be put into the MDisk group MDG_Image.
9.9.14 Shrinking a VDisk The shrinkvdisksize command reduces the capacity that is allocated to the particular virtual disk by the amount that you specify. You cannot shrink the real size of a space-efficient volume below its used size. All capacities, including changes, must be in multiples of 512 bytes. An entire extent is reserved even if it is only partially used. The default capacity units are MB. The command can be used to shrink the physical capacity that is allocated to a particular VDisk by the specified amount. The command can also be used to shrink the virtual capacity of a space-efficient VDisk without altering the physical capacity assigned to the VDisk. To change the capacity of a non-space-efficient disk, use the -size parameter. To change the real capacity of a space-efficient disk, use the -rsize parameter. To change the virtual capacity of a space-efficient disk, use the -size parameter. Chapter 9. SVC configuration and administration using the CLI
369
When the virtual size of a space-efficient VDisk is changed, the warning threshold is automatically scaled to match. The new threshold is stored as a percentage. The cluster arbitrarily reduces the capacity of the VDisk by removing a partial, one or more extents from those allocated to the VDisk. You cannot control which extents are removed and so you cannot assume that it is unused space that is removed. Note: Image Mode Virtual Disks cannot be reduced in size. They must first be migrated to Managed Mode. To run the shrinkvdisksize command on a mirrored VDisk, all copies of the VDisk must be synchronized.
Attention: 1. If the virtual disk contains data, do not shrink the disk. 2. Some operating systems or file systems use what they consider to be the outer edge of the disk for performance reasons. This command can shrink FlashCopy target virtual disks to the same capacity as the source. 3. Before you shrink a VDisk, validate that the VDisk is not mapped to any host objects. If the VDisk is mapped, data is displayed. You can determine the exact capacity of the source or master VDisk by issuing the svcinfo lsvdisk -bytes vdiskname command. Shrink the VDisk by the required amount by issuing the svctask shrinkvdisksize -size disk_size -unit b | kb | mb | gb | tb | pb vdisk_name | vdisk_id command. Assuming your operating system supports it, you can use the svctask shrinkvdisksize command to decrease the capacity of a given VDisk. The full syntax of the svctask shrinkvdisksize command is: >>- svctask -- -- shrinkvdisksize -- --+- -size disk_size -+---> '- -rsize disk_size-' >-- --+--------------+--+-------------------+-- ----------------> '- -copy-- id -' '- -unit --+- b --+-' +- kb -+ +- mb -+ +- gb -+ +- tb -+ '- pb -' >--+- vdisk_name -+-------------------------------------------->< '- vdisk_id ---' The parameters are: -size disk_size (Required) Specifies the size reduction for the designated virtual disk. The -size parameter cannot be used with the -rsize parameter. You must specify either -size or -rsize. -rsize disk_size (Optional) Reduces the real size of a space-efficient VDisk by the specified amount. Specify the disk_size value using an integer. Specify the units for a disk_size integer using the -unit parameter; the default is MB. The -rsize value can be greater than, equal to, or less than the size of the VDisk. You must specify either the -size parameter or the -rsize parameter.
370
Implementing the IBM System Storage SAN Volume Controller V4.3
-copy id (Optional) Specifies the copy to change the real capacity for. You must also specify the -rsize parameter. If the -copy parameter is not specified, all copies of the VDisk are reduced. This parameter is required if the VDisk is mirrored and only one copy is space-efficient. -unit b | kb | mb | gb | tb | pb (Optional) Specifies the data units to be used in conjunction with the value that is specified by the -size parameter. vdisk_name | vdisk_id (Required) Specifies the virtual disk that you want to modify, either by ID or by name. An example of this command is shown in Example 9-93. Example 9-93 svctask shrinkvdisksize
IBM_2145:ITSO-CLS1:admin>svctask shrinkvdisksize -size 44 -unit gb vdisk_A This command shrinks the VDisk Vdisk_A by 44 GB to a new total size of 36 GB.
9.9.15 Showing the MDisks Use the svcinfo lsvdiskmember command, as shown in Example 9-94, to show which MDisks are used by a specific VDisk. Example 9-94 svcinfo lsvdiskmember command
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdiskmember vdisk_D id 0 1 2 3 4 6 10 11 13 15 16 17 If you want to know more about these MDisks, you can run the svcinfo lsmdisk command, as explained in 9.5.2, “MDisk information” on page 328 (using the ID displayed above rather than the name).
Chapter 9. SVC configuration and administration using the CLI
371
9.9.16 Showing the MDisk group Use the svcinfo lsvdisk command, as shown in Example 9-95, to show to which MDG a specific VDisk belongs. Example 9-95 svcinfo lsvdisk command: MDG name
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk vdisk_D id 3 name vdisk_D IO_group_id 1 IO_group_name io_grp1 status online mdisk_grp_id 0 mdisk_grp_name MDG_DS45 capacity 80.0GB type striped formatted no mdisk_id mdisk_name FC_id FC_name RC_id RC_name vdisk_UID 60050768018301BF2800000000000003 throttling 0 preferred_node_id 6 fast_write_state empty cache readwrite udid fc_map_count 0 sync_rate 50 copy_count 1 copy_id 0 status online sync yes primary yes mdisk_grp_id 0 mdisk_grp_name MDG_DS45 type striped mdisk_id mdisk_name fast_write_state empty used_capacity 80.00GB real_capacity 80.00GB free_capacity 0.00MB overallocation 100 autoexpand warning grainsize If you want to know more about these MDGs, you can run the svcinfo lsmdiskgrp command, as explained in 9.6, “Managed Disk Groups” on page 334.
372
Implementing the IBM System Storage SAN Volume Controller V4.3
9.9.17 Showing the host to which the VDisk is mapped To show the hosts to which a specific VDisk has been assigned, run the svcinfo lsvdiskhostmap command, as shown in Example 9-96. Example 9-96 svcinfo lsvdiskhostmap command
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdiskhostmap -delim , vdisk_B id,name,SCSI_id,host_id,host_name,wwpn,vdisk_UID 1,vdisk_B,2,1,Nile,210000E08B892BCD,60050768018301BF2800000000000001 1,vdisk_B,2,1,Nile,210000E08B89B8C0,60050768018301BF2800000000000001 This command shows the host or hosts to which the VDisk vdisk_B was mapped. It is normal for you to see duplicated entries, as there are more paths between the cluster and the host. To be sure that the operating system on the host sees the disk only one time, you must install and configure a multipath software application like SDD. For more information, see Chapter 6, “Quickstart configuration using the command-line interface” on page 157, where SDD is explained. Note: Although the optional -delim, flag normally comes at the end of the command string, in this case you must specify this flag before the VDisk name. Otherwise, the command does not return any data. You have now completed the tasks required to manage the hosts and VDisks within an SVC environment.
9.9.18 Showing the VDisk to which the host is mapped To show the VDisk to which a specific host has been assigned, run the svcinfo lshostvdiskmap command, as shown in Example 9-97. Example 9-97 lshostvdiskmap command example
id,name,SCSI_id,vdisk_id,vdisk_name,wwpn,vdisk_UID 3,Siam,0,5,MM_DB_Pri,210000E08B18FF8A,60050768018301BF2800000000000005 3,Siam,1,4,MM_DBLog_Pri,210000E08B18FF8A,60050768018301BF2800000000000004 3,Siam,2,6,MM_App_Pri,210000E08B18FF8A,60050768018301BF2800000000000006 This command shows which VDisks are mapped to the host called Siam. Note: Although the optional -delim, flag normally comes at the end of the command string, in this case you must specify this flag before the VDisk name. Otherwise, the command does not return any data.
Chapter 9. SVC configuration and administration using the CLI
373
9.9.19 Tracing a host disk back to its source physical disk Follow this procedure: 1. On your host, run the datapath query device command from your host. You see a long disk serial number for each vpath device, as shown in Example 9-98. Example 9-98 datapath query device DEV#: 0 DEVICE NAME: Disk1 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 60050768018301BF2800000000000005 ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port2 Bus0/Disk1 Part0 OPEN NORMAL 20 0 1 Scsi Port3 Bus0/Disk1 Part0 OPEN NORMAL 2343 0 DEV#: 1 DEVICE NAME: Disk2 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 60050768018301BF2800000000000004 ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port2 Bus0/Disk2 Part0 OPEN NORMAL 2335 0 1 Scsi Port3 Bus0/Disk2 Part0 OPEN NORMAL 0 0 DEV#: 2 DEVICE NAME: Disk3 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 60050768018301BF2800000000000006 ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port2 Bus0/Disk3 Part0 OPEN NORMAL 2331 0 1 Scsi Port3 Bus0/Disk3 Part0 OPEN NORMAL 0 0
Note: In Example 9-98, the state of each path is OPEN. Sometimes you will find the state CLOSED, and this does not necessarily indicate any kind of problem, as it might be due to the stage of processing that the path is in. 2. Run the svcinfo lshostvdiskmap command to return a list of all assigned VDisks (Example 9-99). Example 9-99 svcinfo lshostvdiskmap IBM_2145:ITSO-CLS1:admin>svcinfo lshostvdiskmap -delim , Siam id,name,SCSI_id,vdisk_id,vdisk_name,wwpn,vdisk_UID 3,Siam,0,5,MM_DB_Pri,210000E08B18FF8A,60050768018301BF2800000000000005 3,Siam,1,4,MM_DBLog_Pri,210000E08B18FF8A,60050768018301BF2800000000000004 3,Siam,2,6,MM_App_Pri,210000E08B18FF8A,60050768018301BF2800000000000006
Look for the disk serial number that matches your datapath query device output. This host was defined in our SVC as Siam. 3. Run the svcinfo lsvdiskmember vdiskname command for a list of the MDisk or MDisks that make up the specified VDisk (Example 9-100). Example 9-100 svcinfo lsvdiskmember IBM_2145:ITSO-CLS1:admin>svcinfo lsvdiskmember MM_DBLog_Pri id 0 1 2 3 4 10
374
Implementing the IBM System Storage SAN Volume Controller V4.3
11 13 15 16 17
4. Query the MDisks with the svcinfo lsmdisk mdiskID to find their controller and LUN number information, as shown in Example 9-101. The output displays the controller name and the controller LUN ID, which should be enough (provided you named your controller something unique, such as a serial number) to track back to a LUN within the disk subsystem (Example 9-101). Example 9-101 svcinfo lsmdisk command IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk 3 id 3 name mdisk3 status online mode managed mdisk_grp_id 0 mdisk_grp_name MDG_DS45 capacity 36.0GB quorum_index block_size 512 controller_name DS4500 ctrl_type 4 ctrl_WWNN 200400A0B8174431 controller_id 0 path_count 4 max_path_count 4 ctrl_LUN_# 0000000000000003 UID 600a0b8000174431000000e44713575400000000000000000000000000000000 preferred_WWPN 200400A0B8174433 active_WWPN 200400A0B8174433
9.10 Service and maintenance This section details the various service and maintenance tasks that you can execute within the SVC environment.
9.10.1 Upgrading software This section explains how to upgrade the SVC software.
Package numbering and version The format for software upgrade packages is four positive integers separated by dots. For example, a software upgrade package contains something similar to 4.2.0.0 Each software package is given a unique number. You can upgrade from any post-3.1.0.5 level to 4.2.0. All pre-3.1.0.5 levels should be upgraded first to 3.1.0.5. Check the recommended software levels on the Web at: http://www.ibm.com/storage/support/2145
Chapter 9. SVC configuration and administration using the CLI
375
Software utility This utility, which resides on the master console, will check software levels in the system against recommended levels, which will be documented on the support Web site. You will be informed if the software levels are up-to-date, or if you need to download and install newer levels. After the software file has been uploaded to the cluster (to the /home/admin/upgrade directory), it can be selected and applied to the cluster. This is performed by the Web script by using the svcservicetask applysoftware command. When a new code level is applied, it is automatically installed on all the nodes within the cluster. The underlying command-line tool runs the script sw_preinstall, which checks the validity of the upgrade file, and whether it can be applied over the current level. If the upgrade file is unsuitable, the preinstall script deletes the files. This prevents the build up of invalid files on the cluster.
Precaution before upgrade Software installation is normally considered to be a customer task. The SVC supports concurrent software upgrade. That is to say, that software upgrade can be performed concurrently with I/O user operations and some management activities, but only limited CLI commands will be operational from the time the install command is started until the upgrade operation has either terminated successfully or been backed-out. Some commands will fail with a message indicating that a software upgrade is in progress: Before you upgrade the SVC software, ensure that all I/O paths between all hosts and SANs are working. Otherwise, the applications might have I/O failures during the software upgrade. You can do that using the SDD command, as shown in Example 9-102. Example 9-102 query adapter
#datapath query adapter Active Adapters :2 Adpt# 0 1
Name State fscsi0 NORMAL fscsi1 NORMAL
Mode ACTIVE ACTIVE
Select 1445 1888
Errors 0 0
Paths 4 4
Active 4 4
#datapath query device Total Devices : 2 DEV#: 0 DEVICE NAME: vpath0 TYPE: 2145 POLICY: Optimized SERIAL: 60050768018201BF2800000000000000 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk3 OPEN NORMAL 0 0 1 fscsi1/hdisk7 OPEN NORMAL 972 0 DEV#: 1 DEVICE NAME: vpath1 TYPE: 2145 POLICY: Optimized SERIAL: 60050768018201BF2800000000000002 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk4 OPEN NORMAL 784 0 1 fscsi1/hdisk8 OPEN NORMAL 0 0
376
Implementing the IBM System Storage SAN Volume Controller V4.3
Note: During a software upgrade, there are periods where not all of the nodes in the cluster are operational, and as a result the cache operates in write through mode. This will have an impact upon throughput, latency, and bandwidth aspects of performance. It is also worth double checking that your UPS power configuration is also set up correctly (even if your cluster is running without problems). Specifically, make sure: That your UPSs are all getting their power from an external source, and that they are not daisy chained. In other words, make sure that each UPS is not supplying power to another node’s UPS. That the power cable, and the serial cable coming from each node, go back to the same UPS. If the cables are crossed, and are going back to a different UPS, then during the upgrade, as one node is shut down, another node might also be mistakenly shut down. Important: Do not share the SVC UPS with any other devices. You must also ensure that all I/O paths are working for each host that is running I/O operations to the SAN during the software upgrade. You can check the I/O paths by using datapath query commands. Refer to Chapter 8, “Host configuration” on page 209 for more information about datapath query commands. You do not need to check for hosts that have no active I/O operations to the SAN during the software upgrade.
Procedure To upgrade the SVC cluster software, perform the following steps: 1. Before starting the upgrade, you must back up the configuration (see 9.11.1, “Backing up the SVC cluster configuration” on page 390) and save the backup config file in a safe place. 2. Also, save the data collection for support diagnosis just in case of problems, as shown in Example 9-103. Example 9-103 svc_snap
IBM_2145:ITSO-CLS1:admin>svc_snap Collecting system information... Copying files, please wait... Copying files, please wait... Listing files, please wait... Copying files, please wait... Listing files, please wait... Copying files, please wait... Listing files, please wait... Dumping error log... Creating snap package... Snap data collected in /dumps/snap.104643.080617.002427.tgz
Chapter 9. SVC configuration and administration using the CLI
377
3. List the dump generated by the previous command, as shown in Example 9-104. Example 9-104 svcinfo ls2145dumps
IBM_2145:ITSO-CLS1:admin>svcinfo ls2145dumps id 2145_filename 0 svc.config.cron.bak_node3 1 svc.config.cron.bak_SVCNode_2 2 svc.config.cron.bak_node1 3 dump.104643.070803.015424 4 dump.104643.071010.232740 5 svc.config.backup.bak_ITSOCL1_N1 6 svc.config.backup.xml_ITSOCL1_N1 7 svc.config.backup.tmp.xml 8 svc.config.cron.bak_ITSOCL1_N1 9 dump.104643.080609.202741 10 104643.080610.154323.ups_log.tar.gz 11 104643.trc.old 12 dump.104643.080609.212626 13 104643.080612.221933.ups_log.tar.gz 14 svc.config.cron.bak_Node1 15 svc.config.cron.log_Node1 16 svc.config.cron.sh_Node1 17 svc.config.cron.xml_Node1 18 dump.104643.080616.203659 19 104643.trc 20 ups_log.a 21 snap.104643.080617.002427.tgz 22 ups_log.b 4. Save the generated dump in a safe place using the pscp command, as shown in Example 9-105. Example 9-105 pscp -load
C:\>pscp -load ITSOCL1 [email protected] :/dumps/snap.104643.080617.002427.tgz c:\ snap.104643.080617.002427 | 597 kB | 597.7 kB/s | ETA: 00:00:00 | 100% 5. Upload the new software package using PuTTY Secure Copy. Enter the command as shown in Example 9-106. Example 9-106 pscp -load
C:\>pscp -load ITSOCL1 IBM2145_INSTALL_4.3.0.0 [email protected] :/home/admin/upgrade IBM2145_INSTALL_4.3.0.0-0 | 103079 kB | 9370.8 kB/s | ETA: 00:00:00 | 100% 6. Upload the SAN Volume Controller Software Upgrade Test Utility using PuTTY Secure Copy. Enter the command as shown in Example 9-107. Example 9-107 Upload utility
C:\>pscp -load ITSOCL1 IBM2145_INSTALL_svcupgradetest_1.11 [email protected] :/home/admin/upgrade IBM2145_INSTALL_svcupgrad | 11 kB | 12.0 kB/s | ETA: 00:00:00 | 100%
378
Implementing the IBM System Storage SAN Volume Controller V4.3
7. Check that the packages were successfully delivered through the PuTTY command-line application by entering the svcinfo lssoftwaredumps command, as shown in Example 9-108. Example 9-108 svcinfo lssoftwaredumps
IBM_2145:ITSO-CLS1:admin>svcinfo lssoftwaredumps id software_filename 0 IBM2145_INSTALL_4.3.0.0 1 IBM2145_INSTALL_svcupgradetest_1.11 8. Now that the packages are uploaded, first install the SAN Volume Controller Software Upgrade Test Utility, as shown in Example 9-109. Example 9-109 svcservicetask applysoftware
IBM_2145:ITSO-CLS1:admin>svcservicetask applysoftware -file IBM2145_INSTALL_svcupgradetest_1.11 CMMVC6227I The package installed successfully. 9. Using the following command, test the upgrade for known issues that may prevent a software upgrade from completing successfully, as shown in Example 9-110. Example 9-110 svcupgradetest
IBM_2145:ITSO-CLS1:admin>svcupgradetest svcupgradetest version 1.11. Please wait while the tool tests for issues that may prevent a software upgrade from completing successfully. The test will take approximately one minute to complete. The test has not found any problems with the 2145 cluster. Please proceed with the software upgrade. Important: If the above command produces any errors, troubleshoot the error using the maintenance procedures before continuing further. 10.Now use the hidden svcservicetask command set to apply the software upgrade, as shown in Example 9-111. Example 9-111 Apply upgrade command example
IBM_2145:ITSOSVC42A:admin>svcservicetask applysoftware -file IBM2145_INSTALL_4.3.0.0 While the upgrade is running, you can check the status, as shown in Example 9-112 Example 9-112 Check update status
IBM_2145:ITSO-CLS1:admin>svcinfo lssoftwareupgradestatus status upgrading 11.The new code is distributed and applied to each node in the SVC cluster. After installation, each node is automatically restarted in turn. If a node does not restart automatically during the upgrade, it will have to be repaired manually.
Chapter 9. SVC configuration and administration using the CLI
379
12.Eventually both nodes should display Cluster: on line one on the SVC front panel and the name of your cluster on line 2. Be prepared for a long wait (in our case, we waited approximately 40 minutes). Note: During this process, both your CLI and GUI vary from sluggish (very slow) to unresponsive. The important thing is that I/O to the hosts can continue. 13.To verify that the upgrade was successful, you can perform either of the following options: – Run the svcinfo lscluster and svcinfo lsnodevpd commands, as shown in Example 9-113. We have truncated the lscluster and lsnodevpd information for this example. Example 9-113 svcinfo lscluster and lsnodevpd commands
IBM_2145:ITSO-CLS1:admin>svcinfo lscluster ITSO-CLS1 id 0000020060806FCA name ITSO-CLS1 location local partnership bandwidth cluster_IP_address 9.43.86.117 cluster_service_IP_address 9.43.86.118 total_mdisk_capacity 756.0GB space_in_mdisk_grps 756.0GB space_allocated_to_vdisks 156.00GB total_free_space 600.0GB statistics_status off statistics_frequency 15 required_memory 8192 cluster_locale en_US SNMP_setting none SNMP_community SNMP_server_IP_address 0.0.0.0 subnet_mask 255.255.252.0 default_gateway 9.43.85.1 time_zone 522 UTC email_setting email_id code_level 4.3.0.0 (build 8.15.0806110000) FC_port_speed 2Gb console_IP 9.43.86.115:9080 id_alias 0000020060806FCA gm_link_tolerance 300 gm_inter_cluster_delay_simulation 0 gm_intra_cluster_delay_simulation 0 email_server 127.0.0.1 email_server_port 25 email_reply [email protected] email_contact ITSO User email_contact_primary 555-1234 email_contact_alternate email_contact_location ITSO email_state running email_user_count 1 inventory_mail_interval 0 380
Implementing the IBM System Storage SAN Volume Controller V4.3
cluster_IP_address_6 cluster_service_IP_address_6 prefix_6 default_gateway_6 total_vdiskcopy_capacity 156.00GB total_used_capacity 156.00GB total_overallocation 20 total_vdisk_capacity 156.00GB IBM_2145:ITSO-CLS1:admin>
IBM_2145:ITSO-CLS1:admin>svcinfo lsnodevpd 1 id 1 system board: 24 fields part_number 31P0906 system_serial_number 13DVT31 number_of_processors 4 number_of_memory_slots 8 number_of_fans 6 number_of_FC_cards 1 number_of_scsi/ide_devices 2 BIOS_manufacturer IBM BIOS_version -[GFE136BUS-1.09]BIOS_release_date 02/08/2008 system_manufacturer IBM system_product IBM System x3550 -[21458G4]. . software: 6 fields code_level 4.3.0.0 (build 8.15.0806110000) node_name Node1 ethernet_status 1 WWNN 0x50050768010037e5 id 1 – Copy the error log to your management workstation, as explained in 9.10.2, “Running maintenance procedures” on page 381. Open it in WordPad and search for Software Install completed. You have now completed the tasks required to upgrade the SVC software.
9.10.2 Running maintenance procedures Use the svctask finderr command to generate a list of any unfixed errors in the system. This command analyzes the last generated log that resides in the /dumps/elogs/ directory on the cluster. If you want to generate a new log before analyzing unfixed errors, run the svctask dumperrlog command (Example 9-114). Example 9-114 svctask dumperrlog
IBM_2145:ITSO-CLS2:admin>svctask dumperrlog
Chapter 9. SVC configuration and administration using the CLI
381
This generates a file called errlog_timestamp, such as errlog_100048_080618_042419, where:
errlog is part of the default prefix for all error log files. 100048 is the panel name of the current configuration node. 080618 is the date (YYMMDD). 042419 is the time (HHMMSS).
You can add the -prefix parameter to your command to change the default prefix of errlog to something else (Example 9-115). Example 9-115 svctask dumperrlog -prefix
IBM_2145:ITSO-CLS2:admin>svctask dumperrlog -prefix svcerrlog This command creates a file called svcerrlog_timestamp. To see what the file name is, you must enter the following command (Example 9-116). Example 9-116 svcinfo lserrlogdumps
IBM_2145:ITSO-CLS2:admin>svcinfo lserrlogdumps id filename 0 errlog_100048_080618_042049 1 errlog_100048_080618_042128 2 errlog_100048_080618_042355 3 errlog_100048_080618_042419 4 errlog_100048_080618_175652 5 errlog_100048_080618_175702 6 errlog_100048_080618_175724 7 errlog_100048_080619_205900 8 errlog_100048_080624_170214 9 svcerrlog_100048_080624_170257 Note: A maximum of ten error log dump files per node will be kept on the cluster. When the eleventh dump is made, the oldest existing dump file for that node will be overwritten. Note that the directory might also hold log files retrieved from other nodes. These files are not counted. The SVC will delete the oldest file (when necessary) for this node in order to maintain the maximum number of files. The SVC will not delete files from other nodes unless you issue the cleandumps command. After you generate your error log, you can issue the svctask finderr command to scan it for any unfixed errors, as shown in Example 9-117. Example 9-117 svctask finderr
IBM_2145:ITSO-CLS2:admin>svctask finderr Highest priority unfixed error code is [1230] As you can see, we have one unfixed error on our system. To get it analyzed, you need to download it onto your own PC. To know more about this unfixed error, you need to look at the error log in more detail. Use the PuTTY Secure Copy process to copy the file from the cluster to your local management workstation, as shown in Example 9-118 on page 383.
382
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 9-118 pscp command: Copy error logs off SVC
In W2K3 → Start → Run → cmd C:\Program Files\PuTTY>pscp -load SVC_CL2 [email protected] :/dumps/elogs/svcerrlog_100048_080624_170257 c:\temp\svcerrlog.txt svcerrlog.txt | 6390 kB | 3195.1 kB/s | ETA: 00:00:00 | 100% In order to use the Run option, you must know where your pscp.exe is located. In this case, it is in C:\Program Files\PuTTY\. This command copies the file called svcerrlog_100048_080624_170257 to the C:\temp directory on our local workstation and calls the file svcerrlog.txt. Open the file in WordPad (Notepad does not format the screen as well). You should see information similar to what is shown in Example 9-119. The list was truncated for the purposes of this example. Example 9-119 errlog in WordPad
Error Log Entry 400 Node Identifier Object Type Object ID Copy ID Sequence Number Root Sequence Number First Error Timestamp Last Error Timestamp Error Count Error ID Error Code Status Flag Type Flag 03 33 33 04 00 00 00 00
00 44 00 00 00 00 00 00
00 17 33 04 00 00 00 00
00 B8 00 00 00 00 00 00
03 A0 05 00 00 00 00 00
00 00 00 00 00 00 00 00
00 05 0B 01 00 00 00 00
00 20 00 00 00 00 00 00
: : : : : : : : : : : : : : :
Node2 device 0
31 00 00 00 00 00 00 00
37404 37404 Sat Jun 21 00:08:21 2008 Epoch + 1214006901 Sat Jun 21 00:11:36 2008 Epoch + 1214007096 2 10013 : Login Excluded 1230 : Login excluded UNFIXED TRANSIENT ERROR 44 11 00 00 00 00 00 00
17 01 01 00 00 00 00 00
B8 00 00 00 00 00 00 00
A0 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00
04 01 01 00 00 00 00 00
20 00 00 00 00 00 00 00
Scrolling through, or searching for the term unfixed, you should find more detail about the problem. There can be more entries in the errorlog that have the status of unfixed.
Chapter 9. SVC configuration and administration using the CLI
383
After you take the necessary steps to rectify the problem, you can mark the error as fixed in the log by issuing the svctask cherrstate command against its sequence numbers (Example 9-120). Example 9-120 svctask cherrstate
IBM_2145:ITSO-CLS2:admin>svctask cherrstate -sequencenumber 37404 If you accidentally mark the wrong error as fixed, you can mark it as unfixed again by entering the same command and appending the -unfix flag to the end, as shown in Example 9-121. Example 9-121 unfix
IBM_2145:ITSO-CLS2:admin>svctask cherrstate -sequencenumber 37406 -unfix
9.10.3 Setting up error notification To set up error notification, use the svctask setevent command. The full syntax of the setevent command is: >--+----------------------------------+-- ----------------------> '- -snmptrap --+- all -----------+-' +- hardware_only -+ '- none ----------' >--+------------------------------+-- -- -----------------------> '- -snmpip -- ip_address_list -' >--+---------------------------+-- ---------------------------->< '- -community -- community -' Note the following explanation of the parameters: -snmptrap all | hardware_only | none (Optional) Specifies the SNMP trap setting, which specifies when to receive a message that reports a problem or significant event. You can set the following values for this parameter: – all Sends an SNMP trap for all errors and state changes that are logged. – hardware_only Sends an SNMP trap for all errors, but not for object state changes. – none Does not send any SNMP traps or errors. This is the default setting for a new cluster. -snmpip ip_address_list (Optional) Specifies the IP addresses of host systems running the SNMP manager software. You can specify up to six IP addresses, using one of following formats: – A colon-separated list of IPv4 addresses – A comma-separated list of IPv6 addresses – A comma-separated list of IP – V4 and IPv6 addresses, including an optional port number for each address. For example:
384
Implementing the IBM System Storage SAN Volume Controller V4.3
•
For IPv4: ##.##.##.##:8080
•
For IPv6: [####:####:####:####:####:####:####:####]:8080 Entries in excess of the number specified using the -community parameter are ignored.
-community community (Optional) Specifies the SNMP community string. This is a colon-separated list of values with up to six items per list. The maximum length of the community string that is used in SNMP trap generation cannot be more than 60 characters. An example of the setevent command is shown in Example 9-122. Example 9-122 svctask setevent
IBM_2145:ITSO-CLS1:admin>svctask setevent -snmptrap all -snmpip 9.43.86.160 -community SVC This command sends all events (errors and changes in state) to the SVC community on the SNMP manager with the IP address 9.43.86.160.
9.10.4 Analyzing the error log The following types of events and errors are logged in the error log: Events: State changes that are detected by the cluster software and that are logged for informational purposes. Events are recorded in the cluster error log. Errors: Hardware or software problems that are detected by the cluster software and that require some repair. Errors are recorded in the cluster error log. Unfixed errors: Errors that were detected and recorded in the cluster error log and that have not yet been corrected or repaired. Fixed errors: Errors that were detected and recorded in the cluster error log and that have subsequently been corrected or repaired. To display the error log, use the svcinfo lserrlog or svcinfo caterrlog commands, as shown in Example 9-123 (the output is the same). Example 9-123 svcinfo caterrlog command
IBM_2145:ITSOSVC42A:admin>svcinfo caterrlog -delim : id:type:fixed:SNMP_trap_raised:error_type:node_name:sequence_number:root_sequence_ number:first_timestamp:last_timestamp:number_of_errors:error_code 0:cluster:no:no:5:SVCNode_1:0:0:070606094909:070606094909:1:00990101 0:cluster:no:no:5:SVCNode_1:0:0:070606094909:070606094909:1:00990101 12:mdisk_grp:no:no:5:SVCNode_1:0:0:070606094858:070606094858:1:00990145 12:mdisk_grp:no:no:5:SVCNode_1:0:0:070606094539:070606094539:1:00990173 0:internal:no:no:5:SVCNode_1:0:0:070606094507:070606094507:1:00990219 12:mdisk_grp:no:no:5:SVCNode_1:0:0:070606094208:070606094208:1:00990148 12:mdisk_grp:no:no:5:SVCNode_1:0:0:070606094139:070606094139:1:00990145 12:mdisk_grp:no:no:5:SVCNode_1:0:0:070606094110:070606094110:1:00990148 6:host:no:no:5:SVCNode_1:0:0:070605234048:070605234048:1:00990178 30:vdisk:no:no:5:SVCNode_1:0:0:070605173704:070605173704:1:00990182 0:flash:no:yes:6:n/a:3546:3546:070605163708:070605163708:1:00983002 0:flash:no:no:5:SVCNode_1:0:0:070605162833:070605162833:1:00990189 0:flash:no:yes:6:n/a:3545:3545:070605162833:070605162833:1:00983001 0:flash:no:no:5:SVCNode_1:0:0:070605162832:070605162832:1:00990187
Chapter 9. SVC configuration and administration using the CLI
385
0:flash:no:no:5:SVCNode_1:0:0:070605162809:070605162809:1:00990184 37:vdisk:no:no:5:SVCNode_1:0:0:070605162630:070605162630:1:00990169 0:migrate:no:yes:6:n/a:3544:3544:070605162536:070605162536:1:00982009 32:vdisk:no:no:5:SVCNode_1:0:0:070605162527:070605162527:1:00990236 32:vdisk:no:no:5:SVCNode_1:0:0:070605162507:070605162507:1:00990236 11:mdisk_grp:no:no:5:SVCNode_1:0:0:070605162402:070605162402:1:00990173 11:mdisk_grp:no:no:5:SVCNode_1:0:0:070605162402:070605162402:1:00990148 35:vdisk:no:no:5:SVCNode_1:0:0:070605161054:070605161054:1:00990236 35:vdisk:no:no:5:SVCNode_1:0:0:070605161007:070605161007:1:00990236 35:vdisk:no:no:5:SVCNode_1:0:0:070605160903:070605160903:1:00990182 35:vdisk:no:no:5:SVCNode_1:0:0:070605160723:070605160723:1:00990168 ......... IBM_2145:ITSO-CLS1:admin>svcinfo caterrlog -delim , id,type,fixed,SNMP_trap_raised,error_type,node_name,sequence_number,root_sequence_ number,first_timestamp,last_timestamp,number_of_errors,error_code,copy_id 0,cluster,no,yes,6,n4,171,170,080624115947,080624115947,1,00981001, 0,cluster,no,yes,6,n4,170,170,080624115932,080624115932,1,00981001, 0,cluster,no,no,5,n1,0,0,080624105428,080624105428,1,00990101, 0,internal,no,no,5,n1,0,0,080624095359,080624095359,1,00990219, 0,internal,no,no,5,n1,0,0,080624094301,080624094301,1,00990220, 0,internal,no,no,5,n1,0,0,080624093355,080624093355,1,00990220, 11,vdisk,no,no,5,n1,0,0,080623150020,080623150020,1,00990183, 4,vdisk,no,no,5,n1,0,0,080623145958,080623145958,1,00990183, 5,vdisk,no,no,5,n1,0,0,080623145934,080623145934,1,00990183, 11,vdisk,no,no,5,n1,0,0,080623145017,080623145017,1,00990182, 6,vdisk,no,no,5,n1,0,0,080623144153,080623144153,1,00990183, 6,remote,no,no,5,n1,0,0,080623134641,080623134641,1,00990230, 6,remote,no,yes,6,n/a,169,169,080623133647,080623133647,1,00985001, 6,remote,no,no,5,n1,0,0,080623132932,080623132932,1,00990229, 6,remote,no,no,5,n1,0,0,080623132857,080623132857,1,00990225, 6,remote,no,yes,6,n/a,168,168,080623132857,080623132857,1,00985002, 6,remote,no,no,5,n1,0,0,080623132741,080623132741,1,00990227, 6,remote,no,no,5,n1,0,0,080623132713,080623132713,1,00990230, 6,remote,no,no,5,n1,0,0,080623132525,080623132525,1,00990229, 6,remote,no,no,5,n1,0,0,080623132315,080623132315,1,00990230, 6,remote,no,yes,6,n/a,167,167,080623132130,080623132130,1,00985001, 6,remote,no,no,5,n1,0,0,080623132120,080623132120,1,00990229, . . This command views the error log that was last generated. It shows that six events are logged. Use the method described in 9.10.2, “Running maintenance procedures” on page 381 to upload and analyze the error log in more detail. To clear the error log, you can issue the svctask clearerrlog command, as shown in Example 9-124. Example 9-124 svctask clearerrlog
IBM_2145:ITSO-CLS1:admin>svctask clearerrlog Do you really want to clear the log? y Using the -force flag will stop any confirmation requests from appearing.
386
Implementing the IBM System Storage SAN Volume Controller V4.3
When executed, this command will clear all entries from the error log. This will proceed even if there are unfixed errors in the log. It also clears any status events that are in the log. This is a destructive command for the error log and should only be used when you have either rebuilt the cluster, or when you have fixed a major problem that has caused many entries in the error log that you do not wish to manually fix.
9.10.5 License settings To change the licensing feature settings, use the svctask chlicense command. The full syntax of the svctask chlicense command is: >>- svctask -- -- chlicense -- ---------------------------------> >--+- -flash capacity_TB ----------+-------------------------->< +- -remote capacity_TB ---------+ '- -virtualization capacity_TB -' Note the following explanation of the parameters: -flash capacity_TB (Optional) Changes cluster licensing for the FlashCopy feature. To change the licensed capacity for the FlashCopy feature, specify a capacity in terabytes (TB). -remote capacity_TB (Optional) Changes cluster licensing for the Metro Mirror and Global Mirror feature. To change the licensed capacity for the Metro Mirror and Global Mirror feature, specify a capacity in terabytes (TB). -virtualization capacity_TB (Optional) Changes cluster licensing for the Virtualization feature. To change the licensed capacity for the Virtualization feature, specify a capacity in terabytes (TB). All three arguments are mutually exclusive. Before you change the licensing, you can display the licenses you already have by issuing the svcinfo lslicense command, as shown in Example 9-125. Example 9-125 svcinfo lslicense command
IBM_2145:ITSO-CLS1:admin>svcinfo lslicense used_flash 0.00 used_remote 0.00 used_virtualization 0.74 license_flash 50 license_remote 20 license_virtualization 80 The current license settings for the cluster are displayed in the viewing license settings log panel. These settings show whether you are licensed to use the FlashCopy, Metro Mirror, Global Mirror, or Virtualization feature. They also show the storage capacity that is licensed for virtualization. Typically, the license settings log contains entries because feature options must be set as part of the Web-based cluster creation process.
Chapter 9. SVC configuration and administration using the CLI
387
Consider, for example, that you have purchased an additional 5 TB of licensing for the Metro Mirror and Global Mirror feature. The command you need to enter is shown in Example 9-126. Example 9-126 -svctask chlicense
IBM_2145:ITSO-CLS1:admin>svctask chlicense -remote 25 To turn a feature off, just add 0 TB as capacity for the feature you want to disable. To verify that the changes you have made are reflected in your SVC configuration, you can issue the svcinfo lslicense command as before (see Example 9-127). Example 9-127 svcinfo lslicense command: Verifying changes
IBM_2145:ITSO-CLS1:admin>svcinfo lslicense used_flash 0.00 used_remote 0.00 used_virtualization 0.74 license_flash 50 license_remote 25 license_virtualization 80
9.10.6 Viewing the feature log To view the feature log using the CLI, you must first create a feature log dump. Then copy the feature log to your management workstation using PuTTY Secure Copy. Finally, open the file in WordPad. To create the feature log dump, enter the svctask dumpinternallog command, as shown in Example 9-128. Example 9-128 svctask dumpinternallog
IBM_2145:ITSO-CLS1:admin>svctask dumpinternallog This creates a file called feature.txt in the /dumps/feature directory on the cluster. To see whether creation was successful, you can enter the svcinfo lsfeaturedumps command, as shown in Example 9-129, to check that the file was created. Example 9-129 svcinfo lsfeaturedumps
IBM_2145:ITSO-CLS1:admin>svcinfo lsfeaturedumps id feature_filename 0 feature.txt Note: Only one of these files exists at a time. Therefore, each time you run the dumpinternallog command, it overwrites any existing feature.txt file. Now that you have created the file, you must copy it to your management workstation using PuTTY Secure Copy, as shown in Example 9-130. Example 9-130 Copying the file
C:\Program Files\PuTTY>pscp -load SVC_CL1 [email protected] :/dumps/feature/feature.txt c:\temp\feature.txt feature.txt | 18 kB | 18.5 kB/s | ETA: 00:00:00 | 100%
388
Implementing the IBM System Storage SAN Volume Controller V4.3
Now open the file in WordPad (Notepad does a poor job of formatting it) to view the output. It should look similar to that shown in Example 9-131. The output list was truncated for purposes of this example. Example 9-131 Feature dump in WordPad
//--------------------//--------------------// Feature Log Entries //--------------------time type value0 48583bd9 00000011 00000000 48583bd9 00000016 00000000 48583bd9 0000000c 00000000 48614e3d 00000016 00000000 48614f00 00000016 00000000 48614f88 00000016 00000000 48614f97 00000011 00000000 48614f97 00000011 00000000 48614fa6 0000000c 00000000 48614fc4 00000011 00000000 48614fd3 00000016 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 . . . 00000000 00000000 00000000 00000000 00000000 00000000 60c06fca 00000200 085a6dc5 eaf1a05f 8a450a08
value1 00000000 00000000 00000000 061a7c00 00000000 00001400 061a7c00 0000c800 061a7c00 00005000 00005000 00000000 00000000 00000000 00000000 00000000 00000000
value2 061a7c00 061a7c00 061a7c00 00000000 00001400 00005000 0000c800 00005000 00014000 0000c800 00006400 00000000 00000000 00000000 00000000 00000000 00000000
value3 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000002f4 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
value4 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
value5 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 e75aeac4
9.11 SVC cluster configuration backup and recovery The SVC configuration data is stored on all the nodes in the cluster. In normal circumstances, the SVC should never lose its configuration settings. However, in exceptional circumstances, such as a rogue fire sprinkler soaking the SVC cluster, or a multiple hardware failure, this data might become corrupted or lost. This section details the tasks that you can perform to save the configuration data from an SVC configuration node and restore it. The following configuration information is backed up:
SVC cluster Storage controllers Hosts I/O groups Software licenses Managed disks MDGs SVC nodes SSH keys Chapter 9. SVC configuration and administration using the CLI
389
Virtual disks VDisk-to-host mappings Important: Before you begin the restore process, you must consult IBM Support to determine why you cannot access your original configuration data. After the restore process starts, the original data on the VDisks is destroyed. Therefore, you must ensure that you have a backup of all user data on the VDisks. IBM has a procedure guided by L3 support to help you recover your data that is still on the back-end storage. The svcconfig command-line tool is a script, used under the CLI, to save and restore configuration data. It uses secure communications to communicate with a configuration node. The tool is designed to work if the hardware configuration for restoration is identical to that during saving. The prerequisites for having a successful backup are as follows: All nodes in the cluster must be online. No object name can begin with an underscore (_). Do not run any independent operations that could change the cluster configuration while the backup command runs. Do not make any changes to the fabric or the cluster between backup and restore. If changes are made, back up your configuration again or you might not be able to restore it later. Note: We recommend that you make a backup of the SVC configuration data after each major change in the environment, such as defining or changing a VDisks, VDisk-to-host mappings, and so on. In addition, you can make a backup after each change. Be aware that only two versions of the backup file are maintained for each cluster (the previous one has .bak appended), unless you copy the XML or XML BAK files to another folder.
9.11.1 Backing up the SVC cluster configuration You can back up your cluster configuration by using the Backing Up a Cluster Configuration screen or the CLI svcconfig command. This section describes the overall procedure for backing up your cluster configuration and the conditions that must be satisfied to perform a successful backup. Important: We recommend that you make a backup of the SVC configuration data after each major change in the environment, such as defining or changing VDisks, VDisk-to-host mappings, and so on. The backup command extracts configuration data from the cluster and saves it to svc.config.backup.xml in /tmp. A file svc.config.backup.sh is also produced. You can study this file to see what other commands were issued to extract information. A log svc.config.backup.log is also produced. You can study this log for details in regard to what was done and when. This log also includes information about the other commands issued. Any pre-existing svc.config.backup.xml file is archived as svc.config.backup.bak. Only one such archive is kept. We recommend that you immediately move the .XML file and related KEY files (see limitations below) off the cluster for archiving. Then erase the files from /tmp using the svcconfig clear -all command. We also recommend that you change all objects 390
Implementing the IBM System Storage SAN Volume Controller V4.3
having default names to non-default names. Otherwise, a warning is produced for objects with default names. Also the object with the default name is restored with its original name with an “_r” appended. The prefix _(underscore) is reserved for backup and restore command usage, and should not be used in any object names. Important: The tool backs up logical configuration data only, not client data. It does not replace a traditional data backup and restore tool, but supplements such a tool with a way to back up and restore the client's configuration. To provide a complete backup and disaster recovery solution, you must back up both user (non-configuration) data and configuration (non-user) data. After restoration of the SVC configuration, you are expected to fully restore user (non-configuration) data to the cluster's disks.
Prerequisites You must have the following prerequisites in place: All nodes must be online. No object name can begin with an underscore. All objects should have non-default names, that is, names that are not assigned by the SAN Volume Controller. Although we recommend that objects have non-default names at the time the backup is taken, this is not mandatory. Objects with default names are renamed when they are restored. Example 9-132 shows an example of the svcconfig backup command. Example 9-132 svcconfig backup command
IBM_2145:ITSO-CLS1:admin>svcconfig backup ...... CMMVC6130W Inter-cluster partnership fully_configured will not be restored ................... CMMVC6112W io_grp io_grp0 has a default name . CMMVC6112W io_grp io_grp1 has a default name . CMMVC6112W io_grp io_grp2 has a default name . CMMVC6112W io_grp io_grp3 has a default name ...... CMMVC6112W mdisk mdisk0 has a default name . CMMVC6112W mdisk mdisk1 has a default name . CMMVC6112W mdisk mdisk2 has a default name . CMMVC6112W mdisk mdisk3 has a default name . CMMVC6112W mdisk mdisk4 has a default name . CMMVC6112W mdisk mdisk5 has a default name .. CMMVC6112W mdisk mdisk7 has a default name . CMMVC6112W mdisk mdisk8 has a default name .
Chapter 9. SVC configuration and administration using the CLI
391
CMMVC6112W mdisk mdisk9 has a default name . CMMVC6112W mdisk mdisk10 has a default name . CMMVC6112W mdisk mdisk11 has a default name . CMMVC6112W mdisk mdisk12 has a default name . CMMVC6112W mdisk mdisk13 has a default name . CMMVC6112W mdisk mdisk14 has a default name . CMMVC6112W mdisk mdisk15 has a default name . CMMVC6112W mdisk mdisk16 has a default name . CMMVC6112W mdisk mdisk17 has a default name . CMMVC6112W mdisk mdisk18 has a default name . CMMVC6112W mdisk mdisk19 has a default name . CMMVC6112W mdisk mdisk20 has a default name ................ CMMVC6136W No SSH key file svc.config.admin.admin.key CMMVC6136W No SSH key file svc.config.admincl1.admin.key CMMVC6136W No SSH key file svc.config.ITSOSVCUser1.admin.key ....................... CMMVC6112W vdisk vdisk7 has a default name ................... CMMVC6155I SVCCONFIG processing completed successfully Example 9-133 shows the pscp command. Example 9-133 pscp command
C:\Program Files\PuTTY>pscp -load SVC_CL1 [email protected] :/tmp/svc.config.backup.xml c:\temp\clibackup.xml clibackup.xml | 97 kB | 97.2 kB/s | ETA: 00:00:00 | 100%
Context The following scenario illustrates the value of configuration backup: 1. Use the svcconfig command to create a backup file on the cluster that contains details about the current cluster configuration. 2. Store the backup configuration on some form of tertiary storage. You must copy the backup file from the cluster or it becomes lost if the cluster crashes. 3. If a severe enough failure occurs, the cluster might be lost. Both configuration data (for example, the cluster definitions of hosts, I/O groups, MDGs, and MDisks) and the application data on the virtualized disks are lost. In this scenario, it is assumed that the application data can be restored from normal client backup procedures. However, before you can carry this out, you must reinstate the cluster, as configured at the time of the failure. This means you restore the same MDGs, I/O groups, host definitions, and the
392
Implementing the IBM System Storage SAN Volume Controller V4.3
VDisks that existed prior to the failure. Then you can copy the application data back onto these VDisks and resume operations. 4. Recover the hardware. This includes hosts, SVCs, disk controller systems, disks, and SAN fabric. The hardware and SAN fabric must physically be the same as those used before the failure. 5. Re-initialize the cluster, just with the config node; the other nodes will be recovered restoring the configuration. 6. Restore your cluster configuration using the backup configuration file generated prior to the failure. 7. Restore the data on your virtual disks (VDisks) using your preferred restore solution or with help from IBM Service. 8. Resume normal operations.
9.11.2 Restoring the SVC cluster configuration In this section, we discuss restoration of the SVC cluster configuration. Important: Always consult IBM Support to restore the SVC cluster configuration from backup to determine the cause of the loss of your cluster configuration. After the svcconfig restore -execute command is started, any prior user data on the VDisks should be considered destroyed and must be recovered through your usual application data backup process. See IBM TotalStorage Open Software Family SAN Volume Controller: Command-Line Interface User's Guide, SC26-7544 for more information about this topic. For a detailed description of the SVC configuration backup and restore functions, see IBM TotalStorage Open Software Family SAN Volume Controller: Configuration Guide, SC26-7543.
9.11.3 Deleting configuration backup This section details the tasks that you can perform to delete the configuration backup files from the default folder in the SVC master console. You can do this if you already copied them to an external and secure place. Delete the SVC Configuration backup files using the svcconfig clear -all command.
Chapter 9. SVC configuration and administration using the CLI
393
9.12 Listing dumps Several commands are available for you to list the dumps that were generated over a period of time. You can use the lsxxxxdumps command, where xxxx is the object dumps, to return a list of dumps in the appropriate directory. The syntax of this command is: >>- svcinfo -- --- lserrlogdumps -- --+----------+-- ------------> lsfeaturedumps '- -nohdr -' lsiotracedumps lsiostatsdumps lssoftwaredumps ls2145dumps >--+-----------------------+-- --+-------------+--------------->< '- -delim -- delimiter -' +- node_id ---+ '- node_name -' If no node is specified, the dumps that are available on the configuration node are listed.
9.12.1 Error or event dump Dumps contained in the /dumps/elogs directory are dumps of the contents of the error and event log at the time that the dump was taken. An error or event log dump is created by using the svctask dumperrlog command. This dumps the contents of the error or event log to the /dumps/elogs directory. If no file name prefix is supplied, the default errlog_ is used. The full, default file name is errlog_NNNNNN_YYMMDD_HHMMSS. Here NNNNNN is the node front panel name. If the command is used with the -prefix option, then the value entered for the -prefix is used instead of errlog. The command to list all dumps in the /dumps/elogs directory is svcinfo lserrlogdumps (Example 9-134). Example 9-134 svcinfo lserrlogdumps
IBM_2145:ITSO-CLS1:admin>svcinfo lserrlogdumps id filename 0 errlog_104643_080617_172859 1 errlog_104643_080618_163527 2 errlog_104643_080619_164929 3 errlog_104643_080619_165117 4 errlog_104643_080624_093355 5 svcerrlog_104643_080624_094301 6 errlog_104643_080624_120807 7 errlog_104643_080624_121102 8 errlog_104643_080624_122204 9 errlog_104643_080624_160522
394
Implementing the IBM System Storage SAN Volume Controller V4.3
9.12.2 Featurization log dump Dumps contained in the /dumps/feature directory are dumps of the featurization log. A featurization log dump is created by using the svctask dumpinternallog command. This dumps the contents of the featurization log to the /dumps/feature directory to a file called feature.txt. Only one of these files exists, so every time the svctask dumpinternallog command is run, this file is overwritten. The command to list all dumps in the /dumps/feature directory is svcinfo lsfeaturedumps (Example 9-135). Example 9-135 svctask lsfeaturedumps
IBM_2145:ITSO-CLS1:admin>svcinfo lsfeaturedumps id feature_filename 0 feature.txt
9.12.3 I/O trace dump Dumps contained in the /dumps/iotrace directory are dumps of I/O trace data. The type of data that is traced depends on the options specified by the svctask settrace command. The collection of the I/O trace data is started by using the svctask starttrace command. The I/O trace data collection is stopped when the svctask stoptrace command is used. When the trace is stopped, the data is written to the file. The file name is prefix_NNNNNN_YYMMDD_HHMMSS, where NNNNNN is the node front panel name, and prefix is the value entered by the user for the -filename parameter in the svctask settrace command. The command to list all dumps in the /dumps/iotrace directory is svcinfo lsiotracedumps (Example 9-136). Example 9-136 svcinfo lsiotracedumps
IBM_2145:ITSO-CLS1:admin>svcinfo lsiotracedumps id iotrace_filename 0 tracedump_104643_080624_172208 1 iotrace_104643_080624_172451
9.12.4 I/O statistics dump Dumps contained in the /dumps/iostats directory are dumps of the I/O statistics for disks on the cluster. An I/O statistics dump is created by using the svctask startstats command. As part of this command, you can specify a time interval at which you want the statistics to be written to the file (the default is 15 minutes). Every time the time interval is encountered, the I/O statistics that are collected up to this point are written to a file in the /dumps/iostats directory. The file names used for storing I/O statistics dumps are m_stats_NNNNNN_YYMMDD_HHMMSS, or v_stats_NNNNNN_YYMMDD_HHMMSS, depending on whether the statistics are for MDisks or VDisks. Here NNNNNN is the node front panel name.
Chapter 9. SVC configuration and administration using the CLI
395
The command to list all dumps in the /dumps/iostats directory is svcinfo lsiostatsdumps (Example 9-137). Example 9-137 svcinfo lsiostatsdumps
IBM_2145:ITSO-CLS1:admin>svcinfo lsiostatsdumps id iostat_filename 0 Nm_stats_104603_071115_020054 1 Nn_stats_104603_071115_020054 2 Nv_stats_104603_071115_020054 3 Nv_stats_104603_071115_022057 4 Nm_stats_104603_071115_022057 5 Nn_stats_104603_071115_022057 6 Nv_stats_104603_071115_024100 7 Nm_stats_104603_071115_024100 8 Nn_stats_104603_071115_024100 9 Nm_stats_104603_071115_030103 10 Nv_stats_104603_071115_030103 11 Nn_stats_104603_071115_030103 12 Nn_stats_104603_071115_032106 13 Nm_stats_104603_071115_032106 14 Nv_stats_104603_071115_032106 15 Nn_stats_104603_071115_034108 16 Nm_stats_104603_071115_034108 17 Nv_stats_104603_071115_034108 18 Nn_stats_104603_071115_040111 19 Nm_stats_104603_071115_040111 20 Nv_stats_104603_071115_040111 21 Nv_stats_104603_071115_042114 22 Nn_stats_104603_071115_042114 23 Nm_stats_104603_071115_042114 24 Nv_stats_104603_071115_044117 25 Nm_stats_104603_071115_044117 . . .
9.12.5 Software dump The svcinfo lssoftwaredump command lists the contents of the /home/admin/upgrade directory. Any files in this directory are copied there at the time that you want to perform a software upgrade. Example 9-138 shows the command. Example 9-138 svcinfo lssoftwaredumps
IBM_2145:ITSO-CLS1:admin>svcinfo lssoftwaredumps id software_filename 0 IBM2145_INSTALL_4.3.0.0
9.12.6 Application abends dump Dumps contained in the /dumps directory are dumps resulting from application (abnormal ends) abends. Such dumps are written to the /dumps directory. The default file names are dump.NNNNNN.YYMMDD.HHMMSS. Here NNNNNN is the node front panel name. In
396
Implementing the IBM System Storage SAN Volume Controller V4.3
addition to the dump file, it is possible that there might be some trace files written to this directory. These are named NNNNNN.trc. The command to list all dumps in the /dumps directory is svcinfo ls2145dumps (Example 9-139). Example 9-139 svcinfo ls2145dumps
IBM_2145:ITSO-CLS1:admin>svcinfo ls2145dumps id 2145_filename 0 svc.config.cron.bak_node3 1 svc.config.cron.bak_SVCNode_2 2 dump.104643.070803.015424 3 dump.104643.071010.232740 4 svc.config.backup.bak_ITSOCL1_N1 5 svc.config.backup.tmp.xml 6 svc.config.cron.bak_ITSOCL1_N1 7 dump.104643.080609.202741 8 104643.080610.154323.ups_log.tar.gz 9 104643.trc.old 10 dump.104643.080609.212626 11 104643.080612.221933.ups_log.tar.gz 12 svc.config.cron.bak_Node1 13 dump.104643.080616.203659 14 snap.104643.080617.002427.tgz 15 104643.080617.030706.ups_log.tar.gz 16 104643.080617.145015.ups_log.tar.gz 17 svc.config.cron.bak_node1 18 svc.config.cron.bak_n1 19 svc.config.cron.log_n1 20 svc.config.cron.xml_n1 21 svc.config.cron.sh_n1 22 svc.config.backup.xml_n1 23 ups_log.a 24 ups_log.b 25 104643.trc
9.12.7 Other node dumps All of the svcinfo lsxxxxdumps commands can accept a node identifier as input (for example, append the node name to the end of any of the above commands). If this identifier is not specified, then the list of files on the current configuration node (in our case ITSO_node2) is displayed. If the node identifier is specified, then the list of files on that node is displayed. However, files can only be copied from the current configuration node (using PuTTY Secure Copy). Therefore, you must issue the svctask cpdumps command to copy the files from a non-configuration node to the current configuration node. Subsequenty, you can copy them to the management workstation using PuTTY Secure Copy. For example, you discover a dump file and want to copy it to your management workstation for further analysis. In this case, you must first copy the file to your current configuration node.
Chapter 9. SVC configuration and administration using the CLI
397
To copy dumps from other nodes to the configuration node, use the svctask cpdumps command. The full syntax of the svctask cpdumps command is: >>- svctask -- -- cpdumps -- -- -prefix --+- directory ---+-----> '- file_filter -' >-- --+- node_name -+------------------------------------------>< '- node_id ---' The parameters are: -prefix directory | file_filter (Required) Specifies the directory, or files, or both to be retrieved. If a directory is specified with no file filter, all relevant dump or log files in that directory are retrieved. You can use the following directory arguments (filters): – – – – – – – –
* /dumps (retrieves all files in all subdirectories) * /dumps/audit * /dumps/configs * /dumps/elogs * /dumps/feature * /dumps/iostats * /dumps/iotrace * /home/admin/upgrade
In addition to the directory, you can specify a file filter. For example, if you specified /dumps/elogs/*.txt, all files in the /dumps/elogs directory that end in .txt are copied. Note: The following rules apply to the use of wildcards with the SAN Volume Controller CLI: The wildcard character is an asterisk (*). The command can contain a maximum of one wildcard. When you use a wildcard, you must surround the filter entry with double quotation marks (""), as follows: >svctask cleardumps -prefix "/dumps/elogs/*.txt"
node_id | node_name (Required) Specifies the node from which to retrieve the dumps. The variable that follows the parameter can be one of the following: – The node name or label that you assigned when you added the node to the cluster. – The node ID that is assigned to the node (not the worldwide node name). If the node specified is the current configuration node, no file is copied. An example of the command is shown in Example 9-140. Example 9-140 svctask cpdumps
IBM_2145:ITSO-CLS1:admin>svctask cpdumps -prefix /dumps/configs n4 Now that you have copied the configuration dump file from Node n4 to your configuration node, you can use PuTTY Secure Copy to copy the file to your management workstation for further analysis, as described earlier.
398
Implementing the IBM System Storage SAN Volume Controller V4.3
To clear the dumps, you can run the svctask cleardumps command. Again, you can append the node name if you want to clear dumps off a node other than the current configuration node (the default for the svctask cleardumps command). The commands in Example 9-141 clear all logs or dumps from the SVC Node n1. Example 9-141 svctask cleardumps command
IBM_2145:ITSO-CLS1:admin>svctask IBM_2145:ITSO-CLS1:admin>svctask IBM_2145:ITSO-CLS1:admin>svctask IBM_2145:ITSO-CLS1:admin>svctask IBM_2145:ITSO-CLS1:admin>svctask IBM_2145:ITSO-CLS1:admin>svctask IBM_2145:ITSO-CLS1:admin>svctask
cleardumps cleardumps cleardumps cleardumps cleardumps cleardumps cleardumps
-prefix -prefix -prefix -prefix -prefix -prefix -prefix
/dumps n1 /dumps/iostats n1 /dumps/iotrace n1 /dumps/feature n1 /dumps/config n1 /dumps/elog n1 /home/admin/upgrade
n1
9.13 T3 recovery process A procedure called “T3 recovery” has been tested and used in select cases where the cluster has been completely destroyed. (An example would be simultaneously pulling power cords from all nodes to their UPSs; in this case, all nodes would boot up to node error 578 when power was restored.) This procedure, in certain circumstances, is able to recover most user data. However, this procedure is not to be used by the customer or IBM CE without direct involvement from IBM level 3 support. It is not published, but we refer to it here only to indicate that loss of a cluster can be recoverable without total data loss, but requires a restore of application data from backup. It is a very sensitive procedure and only to be used as a last resort, and cannot recover any data unstaged from cache at the time of total cluster failure.
9.14 Scripting and its usage under CLI for SVC task automation Usage of scripting constructs works better for automation of regular operational jobs. You can use available shells to develop it. To run an SVC console where the operating system is Windows 2000 and higher, you can either purchase licensed shell emulation software or download Cygwin from: http://www.cygwin.com Scripting enhances the productivity of SVC administrators and integration of their storage virtualization environment. We show an example of scripting in Appendix C, “Scripting” on page 901. You can create your own customized scripts to automate a large number of tasks for completion at a variety of times and run them through the CLI.
Chapter 9. SVC configuration and administration using the CLI
399
400
Implementing the IBM System Storage SAN Volume Controller V4.3
10
Chapter 10.
SVC configuration and administration using the GUI In this chapter, we describe how to use the IBM System Storage SAN Volume Controller graphical user interface (GUI). This allows you to perform additional and advanced configuration and administration tasks, which are not covered in Chapter 7, “Quickstart configuration using the GUI” on page 173.
© Copyright IBM Corp. 2003-2008. All rights reserved.
401
10.1 Managing users Users are managed from within the Users window in the SAN Volume Controller console GUI (see Figure 10-1 on page 403). The user accounts are specific to each SVC Console, not to the cluster, so if there is more than one SVC Console managing a cluster, then the user account should be set up on each SVC Console. Each user account has a name, a role, and password assigned to it. This differs from the ssh-key based role approach used by the CLI. The role-based security feature organizes the SVC administrative functions into groups, known as roles, so that permissions to execute the various functions can be granted differently to the different administrative users. There are three main roles and one special one. The user roles are as follows: Administrator This role can perform all actions for a cluster. Monitor This role allows the user to execute all view commands, plus a limited set of action commands that do not materially change the status of the SVC or its managed resources. A monitor may not execute any other action commands. This is the minimum level of permission. Operator This role allows operational control over the pre-existing copy relationships. This role has the ability to prepare/start/stop FlashCopy mappings, and start/stop/switch remote copy relationships. They may not create/delete mappings or relationships, and may not execute any other action commands. The Operator is also able to execute view commands. This is the equivalent to the CLI CopyOperator role. Service A special service role that has effectively the same access as the monitor role. The superuser user is a built-in account that has the Administrator user role permissions. You cannot change permissions on this account, only the password. Note: SVC Console user management is done exclusively by the special account superuser. To create users over the GUI, refer to 10.1.1, “Creating a user using the GUI” on page 402and for the CLI overview, refer to 9.1, “Managing users using the CLI” on page 304.
10.1.1 Creating a user using the GUI Perform the following steps to view and create a user: 1. On the SVC Welcome window, select the Users option in the My Work pane, as shown in Figure 10-1 on page 403.
402
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-1 Viewing users
2. Select Add a User in the drop-down menu, then press Go, as shown in Figure 10-2.
Figure 10-2 Add a User in the Users window
Chapter 10. SVC configuration and administration using the GUI
403
3. To define a new user role, enter the user name and password and select a role. Then click OK (Figure 10-3).
Figure 10-3 Add Users window
4. The newly created user is shown in the Viewing Users window, as shown in Figure 10-4.
Figure 10-4 Showing all users
10.1.2 Modifying a user role Perform the following steps to modify a role: 1. Select the radio button to the left of the user, as shown in Figure 10-5 on page 405, to change the assigned role. Select Modify a User from the drop-down menu and click Go.
404
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-5 Modify a User
2. You have the option of changing the password or assigning a new role for the given user name. Click OK (Figure 10-6).
Figure 10-6 Modifying a user window
Chapter 10. SVC configuration and administration using the GUI
405
3. The modified user is now shown in the Viewing Users list (Figure 10-7).
Figure 10-7 Viewing the users list
10.1.3 Deleting a user role Perform the following steps to delete a user role. 1. Select the radio button to the left of the user(s) that you want to delete. Select Delete a User from the drop-down list (Figure 10-8) and click Go.
Figure 10-8 Delete a user
2. Click OK to confirm that you want to delete the user, as shown in Figure 10-9.
Figure 10-9 Confirming deleting a user
You have now completed the tasks required to create, modify, and delete a user within the SVC Console. 406
Implementing the IBM System Storage SAN Volume Controller V4.3
10.2 Managing the cluster using the GUI This section explains the various configuration and administration tasks that you can perform on the cluster. Installing certificates: Perhaps you have already accepted certificates, as suggested in 7.1.1, “Installing certificates” on page 178. If you did not, you might notice many instances where you are prompted with security warnings regarding unrecognized certificates. Return to 7.1.1, “Installing certificates” on page 178, and complete those steps to avoid getting these messages. Lack of correct certificates could cause the browser to exit.
10.2.1 Organizing on screen content In the following sections, there are several windows within the SVC GUI where you can perform filtering (to minimize the amount of data shown on window) and sorting (to organize the content on the window). As we have not covered these functions elsewhere, this section provides a brief overview of these functions. To show how the filtering features work in the GUI, go to the SVC Welcome window, click the Work with Virtual Disks option, and click the Virtual Disks link.
Table filtering When you are in the Viewing Virtual Disks list, you can use the table filter option to filter the visible list, which is useful if the list of entries is too large to work with. You can change the filtering here as many times as you like, to further reduce the lists or for different views. Use the Filter Row Icon, as shown in Figure 10-10, or use the Show Filter Row option in the drop-down menu and click Go.
Figure 10-10 Filter Row icon
Chapter 10. SVC configuration and administration using the GUI
407
This enables you to filter based on the column names, as shown in Figure 10-11. The Filter under each column name shows that no filter is in effect for that column.
Figure 10-11 Show filter row
If you want to filter on a column, click the word Filter, which opens up a filter dialog, as shown in Figure 10-12 on page 409. Our example shows us filtering on the Name field, to only show entries that have App.
408
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-12 Filter option on Name
A list with VDisks is displayed that only contains App somewhere in the name, as shown in Figure 10-13. (Notice the filter line under each column heading showing that our filter is in place.) If you want, you can perform some additional filtering on the other columns to further narrow your view.
Figure 10-13 Filtered on Name containing the word App
Chapter 10. SVC configuration and administration using the GUI
409
The option to reset the filters is shown in Figure 10-14. Use the Clear All Filters icon or use the Clear All Filters option in the drop-down menu and click Go.
Figure 10-14 Clear All Filter options
Sorting Regardless of whether you use the pre-filter or additional filter options, when you are in the Viewing Virtual Disks window, you can sort the displayed data by selecting Edit Sort from the list and clicking Go, or you can click the small icon highlighted by the mouse pointer in Figure 10-15.
Figure 10-15 Selecting Edit Sort
As shown in Figure 10-16 on page 411, you can sort based on up to three criteria, including Name, State, I/O Group, MDisk Group, Capacity (MB), space-efficient, Type, Hosts, FC Pair, FC Map Count, Relationship Name, UID, and Copies. Note: The actual sort criteria differs based on the information that you are sorting.
410
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-16 Sorting criteria
When you finish making your choices, click OK to regenerate the display based on your sorting criteria. Look at the icons next to each column name to see the sort criteria currently in use, as shown in Figure 10-17. If you want to clear the sort, simply select Clear All Sorts from the list and click Go, or click the icon highlighted by the mouse pointer in Figure 10-17.
Figure 10-17 Selecting to clear all sorts
Chapter 10. SVC configuration and administration using the GUI
411
Documentation If you need to access the online documentation, in the upper right corner of the window, click the icon. This opens the Help Assistant pane on the left side of the window, as shown in Figure 10-18.
Figure 10-18 Online help using the i icon
Help If you need to access the online help, in the upper right corner of the window, click the icon. This opens a new window called Information Center. Here you can search on any item you want help for (see Figure 10-19 on page 413).
412
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-19 Online help using the ? icon
General housekeeping If at any time the content in the right side of the frame is “cut off”, you can collapse the My Work column by clicking the icon at the top of the My Work column. When collapsed, the small arrow changes from pointing to the left to pointing to the right ( ). Clicking the small arrow that points right expands the My Work column back to its original size. In addition, each time you open a configuration or administration window using the GUI in the following sections, it creates a link for that window along the top of your Web browser beneath the main banner graphic. As a general housekeeping task, we recommend that you close each window when you finish using it by clicking the icon to the right of the window name, but below the icon. Be careful not to close the entire browser.
10.2.2 Viewing cluster properties Perform the following steps to display the cluster properties: 1. From the SVC Welcome window, select the Manage Cluster option and then the View Cluster Properties link.
Chapter 10. SVC configuration and administration using the GUI
413
2. The Viewing General Properties window (Figure 10-20) opens. Click the IP Addresses, Space, SNMP, Statistics, or Metro & Global Mirror links and you see additional information that pertains to your cluster.
Figure 10-20 View Cluster Properties: General Properties
10.2.3 Maintain cluster passwords Perform the following steps to maintain the cluster passwords: 1. From the SVC Welcome window, select the Manage Cluster option and then the Maintain Cluster Passwords link. 2. The Maintain Passwords window (Figure 10-21 on page 415) opens. Enter the new passwords for the cluster administrator account, the cluster service account, or both. Click Modify Password. Note: Passwords are a maximum of 15 alphanumeric case-sensitive characters. Valid characters are uppercase letters [A through Z], lowercase letters [a through z], digits [0 through 9], dash [ - ], and underscore [ _ ]. The first character cannot be a dash [ - ]. 3. Before the next window is displayed, enter the new user ID and password combination when prompted (Figure 10-21 on page 415).
414
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-21 Maintain Passwords window
When complete, you see the successful update messages, as shown in Figure 10-22.
Figure 10-22 Modifying passwords successful update messages
You have now completed the tasks required to change the admin and service passwords for your SVC cluster.
10.2.4 Modifying IP Addresses In this section, we discuss the modification of IP addresses. IPv6 addressing was introduced in SVC V4.3. You can use both IPv4 and IPv6 addresses in a cluster at the same time, that is, the SVC can run in a dual stack mode. The windows have both IPv4 and IPv6 settings.
Chapter 10. SVC configuration and administration using the GUI
415
Important: If you specify a new cluster IP address, the existing communication with the cluster through the GUI is broken. You need to relaunch the SAN Volume Controller Application from the GUI Welcome window. Modifying the IP address of the cluster, although quite simple, requires some reconfiguration for other items within the SVC environments. This includes reconfiguring the central administration GUI by re-adding the cluster with its new IP address. Perform the following steps to modify the cluster and service IP addresses of our SVC configuration: 1. From the SVC Welcome window, select the Manage Cluster option and the Modify IP Addresses link. 2. The Modify IP Addresses window (Figure 10-23 on page 417) opens. Make any necessary changes. Then click Modify Settings.
416
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-23 Modify IP Addresses
Chapter 10. SVC configuration and administration using the GUI
417
3. You advance to the next window, which shows a message indicating that the IP addresses were updated. You have now completed the tasks required to change the IP addresses (cluster, service, gateway, and master console) for your SVC environment.
10.2.5 Setting the cluster time zone and time Perform the following steps to set the cluster time zone and time: 1. From the SVC Welcome window, select the Manage Cluster options and the Set Cluster Time link. 2. The Cluster Date and Time Settings window (Figure 10-24) opens. At the top of the window, you see the current settings. If necessary, make adjustments and ensure that the Update cluster data and time and Update cluster time zone check boxes are selected. Click Update. Note: You might be prompted for the cluster user ID and password. If you are, type admin and the password you set earlier.
Figure 10-24 Cluster Date and Time Settings window
418
Implementing the IBM System Storage SAN Volume Controller V4.3
3. You return to the Cluster Date and Time Settings window (Figure 10-25), which shows the new settings.
Figure 10-25 Cluster Date and Time Settings update confirmation
You have now completed the tasks necessary to set the cluster time zone and time.
10.2.6 Starting the statistics collection Perform the following steps to start statistics collection on your cluster: 1. From the SVC Welcome window, select the Manage Cluster option and the Start Statistics Collection link. 2. The Starting the Collection of Statistics window (Figure 10-26) opens. Make an interval change, if desired. The interval you specify (minimum 1, maximum 60) is in minutes. Click OK.
Figure 10-26 Starting the Collection of Statistics
Chapter 10. SVC configuration and administration using the GUI
419
3. Although it does not state the current status, clicking OK turns on the statistics collection. To verify, click the Cluster Properties link, as you did in 10.2.2, “Viewing cluster properties” on page 413. Then click the Statistics link. You see the interval as specified in Step 2 and the status of On, as shown in Figure 10-27.
Figure 10-27 Verifying that statistics collection is on
You have now completed the tasks required to start statistics collection on your cluster.
10.2.7 Stopping the statistics collection Perform the following steps to stop statistics collection on your cluster: 1. From the SVC Welcome window, select the Manage Cluster option and the Stop Statistics Collection link. 2. The Stopping the Collection of Statistics window (Figure 10-28) opens, and you see a message asking whether you are sure that you want to stop the statistics collection. Click Yes to stop the ongoing task.
Figure 10-28 Stopping the collection of statistics
3. The window closes. To verify that the collection has stopped, click the Cluster Properties link, as you did in 10.2.2, “Viewing cluster properties” on page 413. Then click the Statistics link. Now you see the status has changed to Off, as shown in Figure 10-29 on page 421.
420
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-29 Verifying that statistics collection is off
You have now completed the tasks required to stop statistics collection on your cluster.
10.2.8 Shutting down a cluster If all input power to a SAN Volume Controller cluster is removed for more than a few minutes (for example, if the machine room power is shut down for maintenance), it is important that you shut down the cluster before you remove the power. Shutting down the cluster while still connected to the main power will ensure that the UPS batteries are still fully charged (when power is restored). If you remove the main power while the cluster is still running, the UPS will detect the loss of power and instruct the nodes to shut down. This can take several minutes to complete, and although the UPS has sufficient power to do this, you will be unnecessarily draining the UPS batteries. When power is restored, the SVC nodes will start, however, one of the first checks they make is to ensure that the UPS’s batteries have sufficient power to survive another power failure, enabling the node to perform a clean shutdown. (We do not want the UPS to run out of power while the node’s shutdown activities have not yet completed!) If the UPS’s batteries are not fully charged enough, the node will not start. It can take up to three hours to charge the batteries sufficiently for a node to start. Note: When a node shuts down due to loss of power, it will dump the cache to an internal hard drive so the cache data can be retrieved when the cluster starts. With the 8F2/8G4 nodes, the cache is 8 GB, and as such, it can take several minutes to dump to the internal drive. SVC UPSs are designed to survive at least two power failures in a short time, before nodes will refuse to start until the batteries have sufficient power (to survive another immediate power failure). If, during your maintenance activities, the UPS detected power and power-loss more than once (and thus the nodes start and shut down more than once in a short time frame), you might find that you have unknowingly drained the UPS batteries, and have to wait until they are charged sufficiently before the nodes will start.
Chapter 10. SVC configuration and administration using the GUI
421
Perform the following steps to shut down your cluster: Important: Before shutting down a cluster, you should quiesce all I/O operations that are destined for this cluster because you will lose access to all VDisks being provided by this cluster. Failure to do so might result in failed I/O operations being reported to your host operating systems. There is no need to do this if you will only shut down one SVC node. Begin the process of quiescing all I/O to the cluster by stopping the applications on your hosts that are using the VDisks provided by the cluster. If you are unsure which hosts are using the VDisks provided by the cluster, follow the procedure in 10.8.16, “Showing the host to which the VDisk is mapped” on page 507. Repeat the previous step for all VDisks. 1. From the SVC Welcome window, select the Manage Cluster option and the Shut Down Cluster link. 2. The Shutting Down cluster window (Figure 10-30) opens. You see a message asking you to confirm whether you want to shut down the cluster. Ensure that you have stopped all FlashCopy mappings, Remote Copy relationships, data migration operations, and forced deletions before continuing. Click Yes to begin the shutdown process. Note: At this point, you lose administrative contact with your cluster.
Figure 10-30 Shutting down the cluster
You have now completed the tasks required to shut down the cluster. Now you can shut down the uninterruptible power supplies by pressing the power buttons on their front panels. Tip: When you shut down the cluster, it will not automatically start, and will have to be manually started. If the cluster shuts down because the UPS has detected a loss of power, it will automatically restart when the UPS has detected the power has been restored (and the batteries have sufficient power to survive another immediate power failure).
422
Implementing the IBM System Storage SAN Volume Controller V4.3
Note: To restart the SVC cluster, you must first restart the uninterruptible power supply units by pressing the power buttons on their front panels. After they are on, go to the service panel of one of the nodes within your SVC cluster and press the power on button, releasing it quickly. After it is fully booted (for example, displaying Cluster: on line 1 and the cluster name on line 2 of the SVC front panel), you can start the other nodes in the same way. As soon as all nodes are fully booted and you have re-established administrative contact using the GUI, your cluster is fully operational again.
10.3 Working with nodes using the GUI This section discusses the various configuration and administration tasks that you can perform on the nodes within an SVC cluster.
10.3.1 I/O groups This section details the tasks that can be performed at an I/O group level.
Renaming an I/O group Perform the following steps to rename an I/O group: 1. From the SVC Welcome window, select the Work with Nodes option and the I/O Groups link. 2. The Viewing Input/Output Groups window (Figure 10-31) opens. Select the radio button to the left of the I/O group you want to rename. In this case, we select io_grp1. Ensure that Rename an I/O Group is selected from the drop-down list. Click Go.
Figure 10-31 Viewing Input/Output Groups
Chapter 10. SVC configuration and administration using the GUI
423
3. On the Renaming I/O Group window (I/O Group Name is the I/O group you selected in the previous step), type the New Name you want to assign to the I/O group. Click OK, as shown in Figure 10-32. Our new name is IO_grp_SVC02.
Figure 10-32 Renaming the I/O group
Note: The name can consist of the letters A to Z, a to z, the numbers 0 to 9, the dash, and the underscore. It can be between one and 15 characters in length, but cannot start with a number, the dash, or the word iogrp, because this prefix is reserved for SVC assignment only. SVC also uses io_grp as a reserve word prefix. A node name cannot therefore be changed to io_grpN where N is a numeric; however, io_grpNy or io_grpyN, where y is any non-numeric character used in conjunction with N, is acceptable. You have now completed the tasks required to rename an I/O group.
10.3.2 Nodes This section discusses the tasks that you can perform at a node level. You perform each task from the Viewing Nodes window (Figure 10-33). To access this window, from the SVC Welcome window, select the Work with Nodes options and the Nodes link.
Figure 10-33 Viewing Nodes
The drop-down shows the options available at a node level. In this example, we will work with Node1.
424
Implementing the IBM System Storage SAN Volume Controller V4.3
Viewing the node details Perform the following steps to view information about a node within the SVC cluster: 1. From the Viewing Nodes window, (Figure 10-33 on page 424), click the underlined name of the node, in the Name column (in this case, Node1). 2. The Viewing General Details nodename window (where nodename is the node you chose) opens, as shown in Figure 10-34. Click the Ports and Vital Product Data links to view additional information about your selected node.
Figure 10-34 General node details
Adding a node Perform the following steps to add a node to the SVC cluster: 1. From the Viewing Nodes window (Figure 10-33 on page 424), select Add a Node and click Go.
Chapter 10. SVC configuration and administration using the GUI
425
2. In the Adding a Node to a Cluster window (Figure 10-35), select a node from the list of available nodes. Select the I/O group to which you want to assign the new node. Enter a suitable name for the new node. Click OK. Note: If you do not provide the name, the SVC automatically generates the name nodeX (where X is the ID sequence number assigned by the SVC internally). The name can consist of the letters A to Z, a to z, the numbers 0 to 9, the dash, and the underscore. It can be between one and 15 characters in length, but cannot start with a number, the dash, or just the word node, because this prefix is reserved for SVC assignment only.
Figure 10-35 Adding a node
3. Use the Refresh button in Figure 10-36 until the new_node has the status Online.
Figure 10-36 Add node Refresh button
426
Implementing the IBM System Storage SAN Volume Controller V4.3
Renaming a node Perform the following steps to rename a node in the SVC cluster: 1. From the Viewing Nodes window (Figure 10-33 on page 424), select the radio button to the left of the node you want to rename. Select Rename a Node from the drop-down list, and click Go. 2. In the Renaming Node nodename window (where nodename is the node you selected previously), type the new name you want to assign to the node. Click OK (Figure 10-37). Note: The name can consist of the letters A to Z, a to z, the numbers 0 to 9, the dash, and the underscore. It can be between one and 15 characters in length, but cannot start with a number, the dash, or the word node, because this prefix is reserved for SVC assignment only.
Figure 10-37 Renaming a node
Deleting a node Perform the following steps to delete a node from the SVC cluster: 1. From the Viewing Nodes window (Figure 10-33 on page 424), select the radio button to the left of the node you want to delete. Select Delete a Node from the drop-down list; click Go.
Chapter 10. SVC configuration and administration using the GUI
427
2. In the Deleting a Node from Cluster nodename window (where nodename is the name of the node you selected in the previous step), confirm your decision by selecting Yes. See Figure 10-38.
Figure 10-38 Deleting node from a cluster
3. A confirmation window will appear. Select Delete (Figure 10-39).
Figure 10-39 Delete node confirmation
Note: If the node you are deleting is the Configuration Node, then that responsibility will automatically be passed to another node in the cluster before it is deleted. 4. Use the Refresh button in Figure 10-40 on page 429 until Node2 is no longer in the list.
428
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-40 Delete node refresh button
Shutting down a node Earlier we showed how to shut down the complete SVC cluster in a controlled manner (see 10.2.8, “Shutting down a cluster” on page 421). On occasion, it might be necessary to shut down a single node within the cluster to perform such tasks as scheduled maintenance, while leaving the SVC environment up and running. This function shuts down one node in a graceful manner. When this is done, the other node in the I/O Group will destage the contents of its cache and will go into write-through mode until the node is powered up again and rejoins the cluster. To shut down a single node in an SVC cluster, perform the following steps: 1. From the Viewing Nodes window (Figure 10-33 on page 424), select the radio button to the left of the node you want to shut down. Select Shut Down a Node from the list. Click Go. 2. In the confirmation window (Figure 10-41), select Yes to continue with the shutdown process.
Figure 10-41 Shutting down a node
To restart the SVC node, simply go to the front panel of that node and push the power on button.
Chapter 10. SVC configuration and administration using the GUI
429
Note: The 2145 UPS-1U does not power off when the SAN Volume Controller is shut down. However, the previous models 2145 UPS-2U will go into standby mode after five minutes if the last node attached to this UPS is powered down. To be able to turn on the SVC node running on the 2145 UPS-2U, you will first need to press the power button on the UPS front panel. You have now completed the tasks that are required to view, add, delete, rename, and shut down a node within the SVC environment.
10.4 Viewing progress With this view you can see the status of activities like VDisk Migration, MDisk Removal, Image Mode Migration, Extend Migration, FlashCopy, Metro Mirror, and VDisk Formatting. Figure 10-42 shows the status of a MDisk Removal that we performed in 10.6.6, “Removing MDisks” on page 457. You can see detailed information of the item by pressing the underlined (progress) number in the Progress column.
Figure 10-42 Showing MDisk Removal Status
10.5 Working with managed disks This section describes the various configuration and administration tasks that you can perform on the managed disks (MDisks) within the SVC environment.
430
Implementing the IBM System Storage SAN Volume Controller V4.3
10.5.1 Disk controller systems This section details the tasks that you can perform at a disk controller level.
Viewing disk controller details Perform the following steps to view information about a back-end disk controller in use by the SVC environment: 1. Select the Work with Managed Disks option and then the Disk Controller Systems link. 2. The Viewing Disk Controller Systems window (Figure 10-43) opens. For more detailed information about a specific controller, click its ID (highlighted by the mouse cursor in Figure 10-43).
Figure 10-43 Disk controller systems
3. When you click the controller Name (Figure 10-43), the Viewing General Details window (Figure 10-44) opens for the controller (where Name is the Controller you selected). Review the details and click Close to return to the previous window.
Figure 10-44 Viewing general details about a disk controller
Chapter 10. SVC configuration and administration using the GUI
431
Renaming a disk controller Perform the following steps to rename a disk controller used by the SVC cluster: 1. Select the radio button to the left of the controller you want to rename. Then select Rename a Disk Controller System from the list and click Go. 2. In the Renaming Disk Controller System controllername window (where controllername is the controller you selected in the previous step), type the new name you want to assign to the controller and click OK. See Figure 10-45.
Figure 10-45 Renaming a controller
3. You return to the Disk Controller Systems window. You should now see the new name of your controller displayed. Note: The name can consist of the letters A to Z, a to z, the numbers 0 to 9, the dash, and the underscore. It can be between one and 15 characters in length. However, it cannot start with a number, the dash, or the word controller, because this prefix is reserved for SVC assignment only.
432
Implementing the IBM System Storage SAN Volume Controller V4.3
10.5.2 Discovery status You can view the status of a managed disk (MDisk) discovery from the Viewing Discovery Status window. This tells you if there is an ongoing MDisk discovery. A running MDisk discovery will be displayed with a status of Active. Perform the following steps to view the status of an MDisk discovery: 1. Select Work with Managed Disks → Discovery Status. The Viewing Discovery Status window is displayed, as shown in Figure 10-46.
Figure 10-46 Discovery status view
2. Click Close to close this window.
10.5.3 Managed disks This section details the tasks that can be performed at an MDisk level. You perform each of the following tasks from the Managed Disks window (Figure 10-47). To access this window, from the SVC Welcome window, click the Work with Managed Disks option and then the Managed Disks link.
Figure 10-47 Viewing Managed Disks window
Chapter 10. SVC configuration and administration using the GUI
433
10.5.4 MDisk information To retrieve information about a specific MDisk, perform the following steps: 1. In the Viewing Managed Disks window (Figure 10-48), click the underlined name of any MDisk in the list to reveal more detailed information about the specified MDisk.
Figure 10-48 Managed disk details
Tip: If at any time the content in the right side of frame is “cut off”, you can minimize the My Work column by clicking the arrow to the right of the My Work heading at the top right of the column (highlighted with the mouse pointer in Figure 10-47 on page 433). After you minimize the column, you see an arrow in the far left position in the same location where the My Work column formerly appeared. 2. Review the details and then click Close to return to the previous window.
10.5.5 Renaming an MDisk Perform the following steps to rename an MDisk controlled by the SVC cluster: 1. Select the radio button to the left of the MDisk that you want to rename in the window shown in Figure 10-47 on page 433. Select Rename an MDisk from the list and click Go. 2. On the Renaming Managed Disk MDiskname window (where MDiskname is the MDisk you selected in the previous step), type the new name you want to assign to the MDisk and click OK. See Figure 10-49 on page 435.
434
Implementing the IBM System Storage SAN Volume Controller V4.3
Note: The name can consist of the letters A to Z, a to z, the numbers 0 to 9, the dash, and the underscore. It can be between one and 15 characters in length. However, it cannot start with a number, the dash, or the word MDisk, because this prefix is reserved for SVC assignment only.
Figure 10-49 Renaming an MDisk
10.5.6 Discovering MDisks Perform the following steps to discover newly assigned MDisks: 1. Select Discover MDisks from the drop-down list shown in Figure 10-47 on page 433 and click Go. 2. Any newly assigned MDisks are displayed in the window shown in Figure 10-50.
Figure 10-50 Newly discovered managed disks
Chapter 10. SVC configuration and administration using the GUI
435
10.5.7 Setting up a quorum disk The SVC cluster, after the process of node discovery, automatically chooses three MDisks as quorum disks. Each disk is assigned an index number of either 0, 1, or 2. In the event that half the nodes in a cluster are missing for any reason, the other half cannot simply assume that the nodes are “dead”. It might just mean that the cluster state information is not being successfully passed between nodes for some reason (network failure, for example). For this reason, if half the cluster disappears from the view of the other, each surviving half attempts to lock the first quorum disk (index 0). In the event of quorum disk index 0 not available on any node, the next disk (index 1) becomes the quorum, and so on. The half of the cluster that is successful in locking the quorum disk becomes the exclusive processor of I/O activity. It attempts to reform the cluster with any nodes it can still see. The other half will stop processing I/O. This provides a tie-breaker solution and ensures that both halves of the cluster do not continue to operate. In the case that all clusters can see the quorum disk, then they will use this quorum disk to communicate with each other, and will decide which half will become the exclusive processor of I/O activity. If, for any reason, you want to set your own quorum disks (for example, if you have installed additional back-end storage and you want to move one or two quorum disks onto this newly installed back-end storage subsystem), complete the following tasks: 1. Select the radio button to the left of the MDisk that you want to designate as a quorum. Then select Set a quorum disk from the list and click Go. 2. In the Setting a Quorum Disk window shown in Figure 10-51, assign a quorum index of 0, 1, or 2 and click OK.
Figure 10-51 Setting a quorum disk
Quorum disks are only created if at least one MDisk is in managed mode (that is, it was formatted by SVC with extents in it). Otherwise, a 1330 cluster error message is displayed in the SVC front window. You can correct it only when you place MDisks in managed mode.
10.5.8 Including an MDisk If a significant number of errors occurs on an MDisk, the SVC automatically excludes it. These errors can be from a hardware problem, a storage area network (SAN) zoning problem, or the result of poorly planned maintenance. If it is a hardware fault, you should receive SNMP alerts in regard to the state of the hardware (before the disk were excluded)
436
Implementing the IBM System Storage SAN Volume Controller V4.3
and undertaken preventative maintenance. If not, the hosts that were using VDisks, which used the excluded MDisk, now have I/O errors. After you take the necessary corrective action to repair the MDisk (for example, replace the failed disk and repair SAN zones), you can tell the SVC to include the MDisk again.
10.5.9 Showing an MDisk group To display information about the managed disk group (MDG) to which an MDisk belongs, perform the following steps: 1. Select the radio button to the left of the MDisk you want to obtain MDG information about. Select Show MDisk Group from the list and click Go, as shown in Figure 10-52.
Figure 10-52 Show MDisk Group select
2. Click the name of the Managed Disk Group, as shown in Figure 10-53.
Figure 10-53 Show MDisk Group
Chapter 10. SVC configuration and administration using the GUI
437
3. You now see a subset (specific to the MDisk you chose in the previous step), as shown in Figure 10-54.
Figure 10-54 View MDG details
10.5.10 Showing a VDisk for an MDisk To display information about VDisks that reside on an MDisk, perform the following steps: 1. Select the radio button, as shown in Figure 10-55 on page 439, to the left of the MDisk you want to obtain VDisk information about. Select Show VDisks using this MDisk from the list and click Go.
438
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-55 Show VDisk using an MDisk
2. You now see a subset (specific to the MDisk you chose in the previous step) of the View Virtual Disks window in Figure 10-56. We cover the View Virtual Disks window in more detail in 10.7, “Working with hosts” on page 460.
Figure 10-56 VDisk list from a selected MDisk
Chapter 10. SVC configuration and administration using the GUI
439
10.5.11 Creating a VDisk in image mode An image mode disk is a VDisk that has an exact one-to-one (1:1) mapping of VDisk extents with the underlying MDisk. For example, extent 0 on the VDisk contains the same data as extent 1 on the MDisk and so on. Without this 1:1 mapping (for example, if extent 0 on the VDisk mapped to extent 3 on the MDisk), there is little chance that the data on a newly introduced MDisk is still readable. Image mode is intended for the purpose of migrating data from an environment without the SVC to an environment with the SVC. A LUN that was previously directly assigned to a SAN-attached host can now be reassigned to the SVC (during a short outage) and returned to the same host as an image mode VDisk, with the user’s data intact. During the same outage, the host, cables, and zones can be reconfigured to access the disk, now through the SVC. After access is re-established, the host workload can resume while the SVC manages the transparent migration of the data to other SVC managed MDisks on the same or another disk subsystem. We recommend that, during the migration phase of the SVC implementation, you add one image mode VDisk at a time to the SVC environment. This reduces the possibility of error. It also means that the short outages required to reassign the LUNs from the subsystem or subsystems and reconfigure the SAN and host can be staggered over a period of time to minimize the business impact. SVC V4.3 introduces the ability to create a VDisk mirror or a space-efficient VDisk while you are creating an image mode VDisk. Using the mirroring option while making the image mode VDisk could be used as a storage array migration tool, as the Copy1 mdisk will also be in image mode. To create a space-efficient image mode VDisk, you need the same amount of real disk space as the original MDisk. This is because the SVC is unable to detect how much physical space a host is utilizing on a LUN. Important: You can create an image mode VDisk only by using an unmanaged disk, that is, you must do this before you add the MDisk that corresponds to your original logical volume to a Managed Disk Group. To create an image mode VDisk, perform the following steps: 1. Select the radio button to the left of the unmanaged MDisk, as shown in Figure 10-57 on page 441, on which you want to create an image mode VDisk. Select Create VDisk in image mode from the list and click Go.
440
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-57 Create VDisk in image mode
2. The first thing you see is the MDisk creation wizard; after reading the steps, click Next. 3. The Set attributes window should appear (Figure 10-58 on page 442), where you enter the name of the VDisk you want to create. You can also select to have read and write operations stored in cache by specifying a cache mode. Additionally, you can specify a unit device identifier. You can optionally choose to have it as a mirrored or space-efficient vdisk. Click Next to continue. Attention: You must specify the cache mode when you create the VDisk. After the VDisk is created, you cannot change the cache mode. a. We describe the VDisk cache modes in Table 10-1. Table 10-1 VDisk cache modes Read/Write
All read and write I/O operations that are performed by the VDisk are stored in cache. This is the default cache mode for all VDisks.
None
All read and write I/O operations that are performed by the VDisk are not stored in cache.
Chapter 10. SVC configuration and administration using the GUI
441
b. Figure 10-58 shows how to set the attributes.
Figure 10-58 Set attributes
Note: If you do not provide a name, the SVC automatically generates the name VDiskX (where X is the ID sequence number assigned by the SVC internally). If you want to provide a name, you can use the letters A to Z, a to z, the numbers 0 to 9, a dash, and the underscore. It can be between one and 15 characters in length, but cannot start with a number, a dash, or the word VDisk, because this prefix is reserved for SVC assignment only. 4. Figure 10-59 shows you Copy 0 (primary copy) of the MDisk. Click Next to proceed.
Figure 10-59 Copy 0 MDisk
442
Implementing the IBM System Storage SAN Volume Controller V4.3
5. In Figure 10-60, you can optionally select an I/O group and preferred node and you can select another MDG if the one entered before does not have enough space available. In our case, we selected MDG_0_DS45. Click Next to proceed.
Figure 10-60 Choose an I/O group and an MDG
Chapter 10. SVC configuration and administration using the GUI
443
6. Figure 10-61 shows you the characteristics of the new image VDisk. Click Finish to complete this task.
Figure 10-61 Verify imaged VDisk attributes
You can now map the newly created VDisk to your host.
10.5.12 Creating an image mode mirrored VDisk This procedure defines a mirror copy to the image mode VDisk creation process. The second copy (Copy1) will also be an image mode MDisk. This could be used as a storage array migration tool, using the SVC as the data mover. 1. Select the radio button to the left of the unmanaged MDisk, as shown in Figure 10-62 on page 445, on which you want to create an image mode VDisk. Select Create VDisk in image mode from the list and click Go.
444
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-62 Create mirrored VDisk in image mode
2. The first thing you see is the MDisk creation wizard; after reading the steps, click Next.
Chapter 10. SVC configuration and administration using the GUI
445
3. Then the Set attributes window should appear (Figure 10-63): a. Enter the name of the VDisk you want to create. b. Select the Mirrored Disk check box and a sub-section expands. The mirror synchronization rate is a percentage of the peak rate. The synchronized option is only when the original disk is unused (or going to be otherwise formatted by the host).
Figure 10-63 Set attributes
446
Implementing the IBM System Storage SAN Volume Controller V4.3
4. Figure 10-64 shows you Copy 0 (primary copy) of the MDisk. Select Copy 1 (secondary copy). Notice we selected a second MDisk that is larger than the original. Click Next to proceed.
Figure 10-64 Copy 0 MDisk
Chapter 10. SVC configuration and administration using the GUI
447
5. Now you can optionally select an I/O group and preferred node, and you can select an MDG for each of the MDisk copies, as shown in Figure 10-65. In our case, we selected MDG_0_DS45 for the Copy 0 MDisk and MDG_SE_0 for the Copy 1 MDisk. Click Next to proceed.
Figure 10-65 Choose an I/O group and an MDG for each of the MDisk copies
6. Figure 10-66 on page 449 shows you the characteristics of the new image mode VDisk. Click Finish to complete this task.
448
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-66 Verify imaged VDisk attributes
You can monitor the MDisk copy synchronization progress by selecting the Manage Progress option and then the View Progress link, as shown in Figure 10-67.
Figure 10-67 VDisk copy synchronization status
You have the option of assigning the VDisk to the host or waiting until it is synchronized and, after deleting the MDisk mirror Copy 1, map the MDisk copy to the host.
Chapter 10. SVC configuration and administration using the GUI
449
10.6 Managed disk groups This section details the tasks that can be performed at an MDG level.
10.6.1 Viewing MDisk group information Each of the following tasks are performed from the View Managed Disk Groups window (Figure 10-68). To access this window, from the SVC Welcome window, click the Work with Managed Disks option and then the Managed Disk Groups link.
Figure 10-68 Viewing MDGs
To retrieve information about a specific MDG, perform the following steps: 1. In the Viewing Managed Disk Groups window (Figure 10-68), click the underlined name of any MDG in the list. 2. In the View MDisk Group Details window (Figure 10-69 on page 451), you see more detailed information about the specified MDisk. Here you see information pertaining to the number of MDisks and VDisks as well as the capacity (both total and free space) within the MDG. When you finish viewing the details, click Close to return to the previous window.
450
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-69 MDG details
10.6.2 Creating an MDisk group To create an MDG, perform the following steps: 1. Select Create an MDisk group from the list in Figure 10-68 on page 450 and click Go. 2. In the Create Managed Disk Group wizard window, click Next. 3. In the Name the group and select the managed disks window (Figure 10-70 on page 452), name the MDG. Optionally, select the MDisk Candidates and add them to the Selected MDisks list (one at a time) in the desired order. Optionally, you can specify a threshold to send a warning to the error log when the capacity is first exceeded. It can either be a percentage or a specific amount. Selecting No MDisk Candidates creates an “empty” MDG. You can add MDisks to an “empty” MDG at a later time. Click Next. Note: If you do not provide a name, the SVC automatically generates the name mdiskgrpX (where X is the ID sequence number assigned by the SVC internally). If you want to provide a name, you can use the letters A to Z, a to z, the numbers 0 to 9, a dash, and the underscore. It can be between one and 15 characters in length. However, it cannot start with a number, the dash, or the word mdiskgrp, because this prefix is reserved for SVC assignment only.
Chapter 10. SVC configuration and administration using the GUI
451
Figure 10-70 Name the group and select managed disks
4. In the Select Extent Size window (Figure 10-71), select the extent size in MB that you want to use to format your MDG. When you select a specific extent size, it will display the total cluster size in TB. Click Next.
Figure 10-71 Select the extent size
452
Implementing the IBM System Storage SAN Volume Controller V4.3
5. Verify the information that you specified in the previous windows (Figure 10-72) and if it is correct, click Finish. If you need to correct something, click Back.
Figure 10-72 Verifying the information about the MDG
10.6.3 Renaming an MDisk group To rename an MDG, perform the following steps: 1. Select the radio button in the Viewing Managed Disk Groups window (Figure 10-73) to the left of the MDG you want to rename. Select Modify an MDisk Group from the list and click Go.
Figure 10-73 Renaming an MDG
Chapter 10. SVC configuration and administration using the GUI
453
2. From the Renaming Managed Disk Group MDGname window (where MDGname is the MDG you selected in the previous step), type the new name you want to assign and click OK (see Figure 10-74). You can also set/change the usage threshold from this window. Note: The name can consist of letters A to Z, a to z, numbers 0 to 9, a dash, and the underscore. It can be between one and 15 characters in length, but cannot start with a number, a dash, or the word mdiskgrp, because this prefix is reserved for SVC assignment only.
Figure 10-74 Renaming an MDG
10.6.4 Deleting an MDisk group To delete an MDG, perform the following steps: 1. Select the radio button to the left of the MDG you want to delete. Select Delete an MDisk Group from the list and click Go. 2. In the Deleting a Managed Disk Group MDGname window (where MDGname is the MDG you selected in the previous step), click OK to confirm that you want to delete the MDG (see Figure 10-75).
Figure 10-75 Deleting an MDG
3. If there are MDisks and VDisks within the MDG you are deleting, you are required to click Forced delete for the MDG (Figure 10-76 on page 455).
454
Implementing the IBM System Storage SAN Volume Controller V4.3
Important: If you delete an MDG with the Forced Delete option, and VDisks were associated with that MDisk group, you lose the data on your VDisks, since they is deleted before the MDisk Group. If you want to save your data, migrate or mirror the VDisks to another MDisk group before you delete the MDisk group previously assigned to it.
Figure 10-76 Confirming forced deletion of an MDG
Chapter 10. SVC configuration and administration using the GUI
455
10.6.5 Adding MDisks If you created an empty MDG as we did, or you simply assign additional MDisks to your SVC environment later, you can add MDisks to existing MDGs by performing the following steps: Note: You can only add unmanaged MDisks to an MDG. 1. Select the radio button (Figure 10-77) to the left of the MDG to which you want to add MDisks. Select Add MDisks from the list and click Go.
Figure 10-77 Adding an MDisk to an existing MDG
2. From the Adding Managed Disks to Managed Disk Group MDiskname window (where MDiskname is the MDG you selected in the previous step), select the desired MDisk or MDisks from the MDisk Candidates list (Figure 10-78). After you select all the desired MDisks, click OK.
Figure 10-78 Adding MDisks to an MDG
456
Implementing the IBM System Storage SAN Volume Controller V4.3
10.6.6 Removing MDisks To remove an MDisk from an MDG, perform the following steps: 1. Select the radio button to the left (Figure 10-79) of the MDG from which you want to remove an MDisk. Select Remove MDisks from the list and click Go.
Figure 10-79 Viewing MDGs
2. From the Deleting Managed Disks from Managed Disk Group MDGname window (where MDGname is the MDG you selected in the previous step), select the desired MDisk or MDisks from the list (Figure 10-80). After you select all the desired MDisks, click OK.
Figure 10-80 Removing MDisks from an MDG
Chapter 10. SVC configuration and administration using the GUI
457
3. If VDisks are using the MDisks that you are removing from the MDG, you are required to click the Forced Delete button to confirm the removal of the MDisk, as shown in Figure 10-81. Even then, the removal only takes place if there is sufficient space to migrate the VDisk data to other extents on other MDisks that remain in the MDG.
Figure 10-81 Confirming forced deletion of MDisk from MDG
10.6.7 Showing MDisks in this group To show a list of MDisks within an MDG, perform the following steps: 1. Select the radio button to the left (Figure 10-82) of the MDG from which you want to retrieve MDisk information. Select Show MDisks in this group from the list and click Go.
Figure 10-82 View MDGs
458
Implementing the IBM System Storage SAN Volume Controller V4.3
2. You now see a subset (specific to the MDG you chose in the previous step) of the Viewing Managed Disk window (Figure 10-83) shown in 10.5.3, “Managed disks” on page 433.
Figure 10-83 Viewing MDisks in an MDG
Note: Remember, you can collapse the column entitled My Work at any time by clicking the arrow to the right of the My Work column heading.
10.6.8 Showing VDisks using this group To show a list of VDisks associated with MDisks within an MDG, perform the following steps: 1. Select the radio button to the left (Figure 10-84) of the MDG from which you want to retrieve VDisk information. Select Show VDisks using this group from the list and click Go.
Figure 10-84 View MDisks
Chapter 10. SVC configuration and administration using the GUI
459
2. You see a subset (specific to the MDG you chose in the previous step) of the Viewing Virtual Disks window in Figure 10-85. We cover the Viewing Virtual Disks window in more detail in 10.8.2, “VDisk information” on page 475.
Figure 10-85 VDisks belonging to selected MDG
You have now completed the tasks required to manage the disk controller systems, managed disks, and MDGs within the SVC environment.
10.7 Working with hosts In this section, we describe the various configuration and administration tasks that you can perform on the VDisks within the SVC environment.
10.7.1 Hosts This section details the tasks that you can perform at a host level. To access the Viewing Hosts window at the SVC Welcome window, click the Work with Hosts option and then the Hosts link. The Viewing Hosts window will appear, as shown in Figure 10-86 on page 461. Each of the tasks shown in the following sections are performed from the Viewing Hosts window.
460
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-86 Viewing hosts
10.7.2 Host information To retrieve information about a specific host, perform the following steps: 1. In the Viewing Hosts window (see Figure 10-86), click the underlined name of any host in the list displayed. 2. Next, you can obtain details for the host you requested: a. In the Viewing General Details window (Figure 10-87), you can see more detailed information about the specified host.
Figure 10-87 Host details
Chapter 10. SVC configuration and administration using the GUI
461
b. You can click the Port Details (Figure 10-88) link to see information about the Fibre Channel Host Bus Adapters (HBAs) that were defined within the host.
Figure 10-88 Host port details
c. You can click Mapped I/O Groups (Figure 10-89) to see which I/O groups this host can access.
Figure 10-89 Host mapped I/O groups
When you are finished viewing the details, click Close to return to the previous window.
462
Implementing the IBM System Storage SAN Volume Controller V4.3
10.7.3 Creating a host To create a new host, perform the following steps: 1. As shown in Figure 10-90, select the option Create a Host from the list and click Go.
Figure 10-90 Create a host
2. In the Creating Hosts window (Figure 10-91 on page 465), type a name for your host (Host Name). Note: If you do not provide a name, the SVC automatically generates the name hostX (where X is the ID sequence number assigned by the SVC internally). If you want to provide a name, you can use the letters A to Z, a to z, the numbers 0 to 9, and the underscore. It can be between one and 15 characters in length. However, it cannot start with a number or the word host, because this prefix is reserved for SVC assignment only. Although using an underscore might work in some circumstances, it violates the RFC 2396 definition of Uniform Resource Identifiers (URIs) and can cause problems. So we recommend that you do not use the underscore in host names. 3. Select the mode (Type) for the host. You must choose HP_UX to have more than eight LUNs supported for HP_UX machines and TPGS for Sun hosts using MPxIO. For all other hosts, select Generic mode (default). You can use a Port Mask to control the node target ports that a host can access. The port mask applies to logins from the host initiator port that are associated with the host object.
Chapter 10. SVC configuration and administration using the GUI
463
Note: For each login between a host HBA port and node port, the node examines the port mask that is associated with the host object for which the host HBA is a member and determines if access is allowed or denied. If access is denied, the node responds to SCSI commands as though the HBA port is unknown. The port mask is four binary bits. Valid mask values range from 0000 (no ports enabled) to 1111 (all ports enabled). The right-most bit in the mask corresponds to the lowest numbered SVC port (1 not 4) on a node. As shown in Figure 10-91 on page 465, our port mask is 1111; this means that the host HBA port can access all node ports. If, for example, a port mask is set to 0011, only port 1 and port 2 are enabled for this host access. 4. Select and add the worldwide port names (WWPNs) that correspond to your HBA or HBAs. Click OK. In some cases, your WWPN or WWPNs might not display, although you are sure that your adapter is functioning (for example, you see the WWPN in the switch name server) and your zones are correctly set up. In this case, you can manually type the WWPN of your HBA or HBAs into the Additional Ports field (type in WWPNs, one per line) at the bottom of the window before you click OK.
464
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-91 Creating a new host
Chapter 10. SVC configuration and administration using the GUI
465
5. This brings you back to the viewing host window (Figure 10-92) where you can see the added host.
Figure 10-92 Create host results
466
Implementing the IBM System Storage SAN Volume Controller V4.3
10.7.4 Modifying a host To modify a host, perform the following steps: 1. Select the radio button to the left of the host you want to rename (Figure 10-93). Select Rename a host from the list and click Go.
Figure 10-93 Modifying a host
Chapter 10. SVC configuration and administration using the GUI
467
2. From the Modifying Host window (Figure 10-94), type the new name you want to assign or change the Type parameter and click OK. Note: The name can consist of the letters A to Z, a to z, the numbers 0 to 9, and the underscore. It can be between one and 15 characters in length. If you want to provide a name, you can use the letters A to Z, a to z, the numbers 0 to 9, and the underscore. It can be between one and 15 characters in length. However, it cannot start with a number or the word host, because this prefix is reserved for SVC assignment only. While using an underscore might work in some circumstances, it violates the RFC 2396 definition of Uniform Resource Identifiers (URIs) and thus can cause problems. So we recommend that you do not use the underscore in host names.
Figure 10-94 Modifying a host (choosing a new name)
10.7.5 Deleting a host To delete a Host, perform the following steps: 1. Select the radio button to the left of the host you want to delete (Figure 10-95 on page 469). Select Delete a Host from the list and click Go.
468
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-95 Deleting a host
2. In the Deleting Host hostname window (where hostname is the host you selected in the previous step), click OK if you are sure you want to delete the host. See Figure 10-96.
Figure 10-96 Deleting a host
Chapter 10. SVC configuration and administration using the GUI
469
3. If you still have VDisks associated with the host, you see a window (Figure 10-97) requesting confirmation for the forced deletion of the host. Click OK and all the mappings between this host and its VDisks are deleted before the host is deleted.
Figure 10-97 Forcing a deletion
10.7.6 Adding ports If you add an HBA to a server that is already defined within the SVC, you can simply add additional ports to your host definition by performing the following steps: 1. Select the radio button to the left of the host to which you want to add WWPNs (Figure 10-98). Select Add Ports from the list and click Go.
Figure 10-98 Add ports to a host
2. From the Adding ports to hostname window (where hostname is the host you selected in the previous step), select the desired WWPN from the Available Ports list (one at a time) and click Add. After you select all the desired WWPNs, click OK. See Figure 10-99 on page 471.
470
Implementing the IBM System Storage SAN Volume Controller V4.3
If your WWPNs are not in the list of the Available Ports and you are sure your adapter is functioning (for example, you see WWPN in the switch name server) and your zones are correctly set up, then you can manually type the WWPN of your HBAs into the Add Additional Ports field at the bottom of the window before you click OK.
Figure 10-99 Adding ports to a host
Chapter 10. SVC configuration and administration using the GUI
471
10.7.7 Deleting ports To delete a port from a host, perform the following steps: 1. Select the radio button to the left of the host from which you want to delete a port (Figure 10-100). Select Delete Ports from the list and click Go.
Figure 10-100 Delete ports from a host
2. On the Deleting Ports From hostname window (where hostname is the host you selected in the previous step), select the ports you want to delete from the Available Ports list and click Add. When you have selected all the ports you want to delete from your host to the column to the right, click OK. See Figure 10-101 on page 473.
472
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-101 Deleting ports from a host
3. If you have VDisks that are associated with the host, you receive a warning about deleting a host port. You need to confirm your action when prompted, as shown in Figure 10-102.
Figure 10-102 Port deletion confirmation
10.7.8 Fabrics This view was added to the SVC management interface in Version 4.1. With it you can easily collect information about the attached hosts and controller subsystems, their local and remote WWPN, local and remote N_Port ID, the type of connection (host, node, and controller), and the current state (active or inactive). 1. Click Work with Hosts and then Fabrics.
Chapter 10. SVC configuration and administration using the GUI
473
2. The Viewing Fabrics window should open, as shown in Figure 10-103. In this view, you can search/filter as described 10.2.1, “Organizing on screen content” on page 407.
Figure 10-103 Viewing Fabrics
You have now completed the tasks required to manage the hosts within an SVC environment.
10.8 Working with virtual disks In this section, we describe the tasks that you can perform at a VDisk level.
10.8.1 Using the Virtual Disks window for VDisks Each of the following tasks are performed from the Viewing Virtual Disks window (Figure 10-104 on page 475). To access this window, from the SVC Welcome window, click the Work with Virtual Disks option and then the Virtual Disks link. The drop-down menu contains all the actions you can perform in the Virtual Disk window.
474
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-104 Viewing Virtual Disks
10.8.2 VDisk information To retrieve information about a specific VDisk, perform the following steps: 1. In the Viewing Virtual Disks window, click the underlined name of the desired VDisk in the list.
Chapter 10. SVC configuration and administration using the GUI
475
2. The next window (Figure 10-105) that opens shows detailed information. Review the information. When you are done, click Close to return to the Viewing Virtual Disks window.
Figure 10-105 VDisk details
476
Implementing the IBM System Storage SAN Volume Controller V4.3
10.8.3 Creating a VDisk To create a new VDisk, perform the following steps: 1. Select Create a VDisk from the list (Figure 10-104 on page 475) and click Go. 2. The Create Virtual Disks wizard launches. Click Next. 3. The Select groups window opens. Choose an I/O group and then a preferred node (see Figure 10-106). In our case, we let the system choose. Click Next.
Figure 10-106 Creating a VDisk wizard: Select Groups
Chapter 10. SVC configuration and administration using the GUI
477
4. The Set attributes window opens (Figure 10-107). a. Choose what type of VDisk you want to create, striped or sequential. b. Select the cache mode, Read/Write or None. c. If you want, enter a unit device identifier. d. Enter the number of VDisks you want to create e. You can select the Space-efficient or Mirrored Disk check box, which will expand their respective sections with extra options. f. Optionally, format the new VDisk by selecting the Format VDisk before use check box (write zeros to its managed disk extents). g. Click Next.
Figure 10-107 Creating a VDisk wizard: Set Attributes
478
Implementing the IBM System Storage SAN Volume Controller V4.3
5. Select the MDG from which you want the VDisk to be a member of. a. If you selected Striped, you will see the window shown in Figure 10-108. You must select the MDisk group, and then the Managed Disk Candidates window will appear. You can optionally add some MDisks to be striped.
Figure 10-108 Creating a VDisk wizard: Select attributes for striped mode VDisks
Chapter 10. SVC configuration and administration using the GUI
479
b. If you selected Sequential mode, you will see the window shown in Figure 10-109. You must select the MDisk group, and then Managed Disks will appear. You need to choose at least one MDisk as a managed disk.
Figure 10-109 Creating a VDisk wizard: Select attributes for sequential mode VDisks
c. Enter the size of the VDisk you want to create and select the capacity measurement (MB or GB) from the list. Note: An entry of 1 GB uses 1024 MB. d. Click Next. 6. You can enter the VDisk name if you want to create just one VDisk, or the naming prefix if you want to create multiple VDisks. Click Next. Tip: When you create more than one VDisk, the wizard will not ask you for a name for each VDisk to be created. Instead, the name you use here will be a prefix and have a number, starting at zero, appended to it as each one is created.
480
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-110 Creating a VDisk wizard: Name the VDisk(s)
Note: If you do not provide a name, the SVC automatically generates the name VDiskX (where X is the ID sequence number assigned by the SVC internally). If you want to provide a name, you can use the letters A to Z, a to z, the numbers 0 to 9, and the underscore. It can be between one and 15 characters in length, but cannot start with a number or the word VDisk, because this prefix is reserved for SVC assignment only.
Chapter 10. SVC configuration and administration using the GUI
481
7. In the Verify VDisk window (see Figure 10-111 for striped and Figure 10-112 on page 483 for sequential), check if you are satisfied with the information shown, then click Finish to complete the task. Otherwise, click Back to return and make any corrections.
Figure 10-111 Creating a VDisk wizard: Verify VDisk Striped type
482
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-112 Creating a VDisk wizard: Verify VDisk sequential type
8. Figure 10-113 on page 484 shows the progress of the creation of your VDisks on storage and the final results.
Chapter 10. SVC configuration and administration using the GUI
483
Figure 10-113 Creating a VDisk wizard: final result
10.8.4 Creating a space-efficient VDisk with auto-expand Using space-efficient VDisks allows you to commit the minimal amount of space while promising an allocation that may be larger than the available free storage. As the host using this VDisk starts utilizing up to the level of the real allocation, the SVC can dynamically grow (when you enable autoextend) until it reaches the virtual capacity limit or the Managed Disk Group physically runs out of free space. For the latter scenario, this will cause the growing VDisk to go offline, affecting the host using that VDisk. Therefore, enabling threshold warnings is very important. Do the following steps to create a space-efficient VDisk with autoextend: 1. Select Create a VDisk from the list (Figure 10-104 on page 475) and click Go. 2. The Create Virtual Disks wizard launches. Click Next. 3. The Select groups window opens. Choose an I/O group and then a preferred node (see Figure 10-106 on page 477). In our case, we let the system choose. Click Next.
484
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-114 Creating a VDisk wizard: Select Groups
4. The Set attributes window opens (Figure 10-107 on page 478). a. Choose what type of VDisk you want to create, striped or sequential. b. Select the cache mode, Read/Write or None. c. If you want, enter a unit device identifier. d. Enter the number of VDisks you want to create e. Select the Space-efficient check box, which will expand this section with the following options: i. Type the size of the VDisk Capacity (remember, this is the virtual size). ii. Type in a percentage or select a specific size for the usage threshold warning. iii. Select the Autoexpand check box. This will allow the real disk size to grow as required. iv. Select the Grain size (choose 32 KB normally, but match the FlashCopy grain size, which is 256 KB, if the VDisk is being used for FlashCopy). f. Optionally, format the new VDisk by selecting the Format VDisk before use check box (write zeros to its managed disk extents) g. Click Next.
Chapter 10. SVC configuration and administration using the GUI
485
Figure 10-115 Creating a VDisk wizard: Set Attributes
5. In the window, Select MDisk(s) and Size for a <modetype>-Mode VDisk, as shown in Figure 10-116 on page 487, and follow these steps: a. Select the Managed Disk Group from the list. b. Optionally, choose the Managed Disk Candidates upon which to create the VDisk. Click Add to move them to the Managed Disks Striped in this Order box. c. Type in the Real size you wish to allocate. This is how much disk space will actually be allocated. It can either be a percentage of the virtual size or a specific number.
486
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-116 Creating a VDisk wizard: Selecting MDisk(s) and sizes
6. In the window Name the VDisk(s) (Figure 10-117), type a name for the VDIsk you are creating. In our case, we used vdisk_sev2. Click Next.
Figure 10-117 Name the VDisk(s) window
Chapter 10. SVC configuration and administration using the GUI
487
7. In the Verify Attributes window (Figure 10-118), verify the selections. We can select the Back button at any time to make changes.
Figure 10-118 Verifying space-efficient VDisk Attributes window
8. After selecting the Finish option, we are presented with a window (Figure 10-119 on page 489) that tells us the result of the action.
488
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-119 Space-efficient VDisk creation success
10.8.5 Deleting a VDisk To delete a VDisk, perform the following steps: 1. Select the radio button to the left of the VDisk you want to delete (Figure 10-104 on page 475). Select Delete a VDisk from the list and click Go. 2. In the Deleting Virtual Disk VDiskname window (where VDiskname is the VDisk you just selected), click OK to confirm your desire to delete the VDisk. See Figure 10-120.
Figure 10-120 Deleting a VDisk
Chapter 10. SVC configuration and administration using the GUI
489
If the VDisk is currently assigned to a host, you receive a secondary message where you must click Forced Delete to confirm your decision. See Figure 10-121. This deletes the VDisk-to-host mapping before deleting the VDisk. Important: Deleting a VDisk is a destructive action for user data residing in that VDisk.
Figure 10-121 Deleting a VDisk: Forcing a deletion
10.8.6 Deleting a VDisk-to-host mapping To unmap (unassign) a VDisk from a host, perform the following steps: 1. Select the radio button to the left of the VDisk you want to unmap. Select Delete a VDisk-to-host mapping from the list and click Go. 2. In the Deleting a VDisk-to-host mapping window (Figure 10-122), from the Host Name list, select the host from which to unassign the VDisk. Click OK. Tip: Make sure that the host is no longer using that disk. Unmapping a disk from a host will not destroy its contents. Unmapping a disk has the same effect as powering off the computer without first performing a clean shutdown, and thus might leave the data in an inconsistent state. Also, any running application that was using the disk will start to receive I/O errors.
Figure 10-122 Deleting a VDisk-to-host mapping
490
Implementing the IBM System Storage SAN Volume Controller V4.3
10.8.7 Expanding a VDisk Expanding a VDisk presents a larger capacity disk to your operating system. Although you can do this easily using the SVC, you must ensure that your operating system is prepared for it and supports the volume expansion before you use this function. Dynamic expansion of a VDisk is only supported when the VDisk is in use by: AIX 5L V5.2 and above W2K and W2K3 for basic disks W2K and W2K3 with a hot fix from Microsoft (Q327020) for dynamic disks Assuming that your operating system supports it, to expand a VDisk, perform the following steps: 1. Select the radio button to the left of the VDisk you want to expand, as shown in Figure 10-106 on page 477. Select Expand a VDisk from the list and click Go. 2. The Expanding Virtual Disks VDiskname window (where VDiskname is the VDisk you selected in the previous step) opens. See Figure 10-123 on page 492. Follow these steps: a. Select the new size of the VDisk. This is the increment to add. For example, if you have a 5 GB disk and you want it to become 10 GB, you specify 5 GB in this field. b. Optionally, select the managed disk candidates from which to obtain the additional capacity. The default for a striped VDisk is to use equal capacity from each MDisk in the MDG. Notes: With sequential VDisks, you must specify the MDisk from which you want to obtain space. There is no support for the expansion of image mode VDisks. If there are not enough extents to expand your VDisk to the specified size, you receive an error message. If you are using VDisk mirroring, all copies must be synchronized before expanding. c. Optionally, you can format the extra space with zeros by selecting the Format Additional Managed Disk Extents check box. This does not format the entire VDisk, just the newly expanded space. When you are done, click OK.
Chapter 10. SVC configuration and administration using the GUI
491
Figure 10-123 Expanding a VDisk
3. Go to your host and perform the necessary operations to discover the additional space and expand your volumes into it. This procedure differs depending on the operating system.
10.8.8 Mapping a VDisk to a host To map (assign) a virtual disk to a host, perform the following steps: 1. Select the radio button to the left of the VDisk you want to assign to a host (Figure 10-104 on page 475). Select Map VDisks to Host from the list and click Go. 2. In the Creating a Virtual Disk-to-Host mapping VDiskname window (where VDiskname is the VDisk you selected in the previous step), from the Target Host list, select the desired host. The SCSI LUN ID increments based on what is already assigned to the host. Click OK. See Figure 10-124 on page 493. Tip: The option “Allow the virtual disks to be mapped even if they are already mapped to a host” allows you to map a VDisk to more than one host. This would normally be used in clustered environments, where the responsibility of access to the disks is negotiated between the hosts (and not enforced by the SVC), or when using global file systems, such as the IBM System Storage SAN File System.
492
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-124 Mapping a VDisk to a host
3. Figure 10-125 shows you the progress of the VDisk to host mapping.
Figure 10-125 Progress of VDisk to host mapping
10.8.9 Modifying a VDisk The Modifying Virtual Disk menu item allows you to rename the VDisk, reassign the VDisk to another I/O group, and set throttling parameters. To modify a VDisk, perform the following steps: 1. Select the radio button to the left of the VDisk you want to modify (Figure 10-104 on page 475). Select Modify a VDisk from the list and click Go.
Chapter 10. SVC configuration and administration using the GUI
493
2. The Modifying virtual disk VDiskname window (where VDiskname is the VDisk you selected in the previous step) opens. See Figure 10-126 on page 495. You can perform the following steps separately or in combination: a. Type a new name for your VDisk. Note: The name can consist of the letters A to Z, a to z, the numbers 0 to 9, and the underscore. It can be between one and 15 characters in length. However, it cannot start with a number or the word VDisk, because this prefix is reserved for SVC assignment only. b. Select an alternate I/O group from the list to alter the I/O group to which it is assigned. c. Set performance throttling for a specific VDisk. In the I/O Governing field, type a number and select either I/O or MB from the list. Note the following items: •
I/O governing effectively throttles the amount of I/Os per second (or MBs per second) to and from a specific VDisk. You might want to do this if you have a VDisk that has an access pattern that adversely affects the performance of other VDisks on the same set of MDisks, for example, if it uses most of the available bandwidth.
•
If this application is highly important, then migrating the VDisk to another set of MDisks might be advisable. However, in some cases, it is an issue with the I/O profile of the application rather than a measure of its use or importance.
•
The choice between I/O and MB as the I/O governing throttle should be based on the disk access profile of the application. Database applications generally issue large amounts of I/O but only transfer a relatively small amount of data. In this case, setting an I/O governing throttle based on MBs per second does not achieve much. It is better for you to use an I/O per second throttle. At the other extreme, a streaming video application generally issues a small amount of I/O, but transfers large amounts of data. In contrast to the database example, setting an I/O governing throttle based on I/Os per second does not achieve much. Therefore, you should use an MB per second throttle.
•
Additionally, you can specify a unit device identifier.
•
The Primary Copy is used to select which VDisk copy is going to be used as the preferred copy for read operations.
•
Mirror Synchronization rate is the I/O governing rate in percentage during initial synchronization. A zero value disables synchronization.
•
The Copy ID section is used for space-efficient VDisks. If you only have a single space-efficient VDisk, the Copy ID drop-down will be greyed out and you can change the warning thresholds and whether the copy will autoexpand. If you have a VDisk mirror and one, or more, of the copies are space-efficient, you can select a copy, or all copies, and change the warning thresholds/autoexpand individually.
Click OK when you are done making changes.
494
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-126 Modifying a VDisk
Migrating a VDisk To migrate a VDisk, perform the following steps: 1. Select the radio button to the left of the VDisk you want to migrate (Figure 10-104 on page 475). Select Migrate a VDisk from the list and click Go. 2. The Migrating Virtual Disk VDiskname window (where VDiskname is the VDisk you selected in the previous step) opens, as shown in Figure 10-127 on page 496. From the MDisk Group Name list,: a. Select the MDG to which you want to reassign the VDisk. You will only be presented with a list of MDisk groups with the same extent size. b. Specify the number of threads to devote to this process (a value from 1 to 4). The optional threads parameter allows you to assign a priority to the migration process. A setting of 4 is the highest priority setting. If you want the process to take a lower priority over other types of I/O, you can specify 3, 2, or 1.
Chapter 10. SVC configuration and administration using the GUI
495
Important: After a migration is started, there is no way to stop it. Migration continues until it is complete unless it is stopped or suspended by an error condition or the VDisk being migrated is deleted. When you are done making your selections, click OK to begin the migration process. 3. You need to manually refresh your browser or close it and return to the Viewing Virtual Disks window periodically to see the MDisk Group Name column in the Viewing Virtual Disks window update to reflect the new MDG name.
Figure 10-127 Migrating a VDisk
Migrating a VDisk to an image mode VDisk Migrating a VDisk to an image mode VDisk allows the SVC to be removed from the data path. This might be useful where the SVC is used as a data mover appliance. To migrate a VDisk to an image mode VDisk, the following rules apply: The destination MDisk must be greater than or equal to the size of the VDisk. The MDisk specified as the target must be in an unmanaged state. Regardless of the mode that the VDisk starts in, it is reported as being in managed mode during the migration. Both of the MDisks involved are reported as being in image mode during the migration. If the migration is interrupted by a cluster recovery, or by a cache problem, then the migration will resume after the recovery completes. To accomplish the migration, perform the following steps: 1. Select a VDisk from the list, choose Migrate to an Image Mode VDisk from the drop-down list (Figure 10-104 on page 475), and click Go. 2. The Migrate to Image Mode VDisk wizard launches (not shown here). Read the steps in this window and click Next. 3. Select the radio button to the left of the MDisk where you want the data to be migrated (Figure 10-128 on page 497). Click Next.
496
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-128 Migrate to image mode VDisk wizard: Select the Target MDisk
4. Select the MDG the MDisk will join (Figure 10-129). Click Next.
Figure 10-129 Migrate to image mode VDisk wizard: Select MDG
Chapter 10. SVC configuration and administration using the GUI
497
5. Select the priority of the migration by selecting the number of threads (Figure 10-130). Click Next.
Figure 10-130 Migrate to image mode VDisk wizard: Select the Threads
6. Verify that the information you specified is correct (Figure 10-131). If you are satisfied, click Finish. If you want to change something, use the Back option.
Figure 10-131 Migrate to image mode VDisk wizard: Verify Migration Attributes
7. Figure 10-132 on page 499 displays the details of the VDisk that you are migrating.
498
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-132 Migrate to image mode VDisk wizard: Progress of migration
10.8.10 Creating a VDisk Mirror from an existing VDisk You can create a mirror of the MDisks from an existing VDisk, that is, it can give you two copies of the underlying disk extents. Note: You can also create a new mirrored VDisk by selecting an option during VDisk creation (see 7.7.2, “Creating a mirrored VDisk” on page 203). Any operation that can be done with a VDisk can be done with a VDisk mirror. It is transparent to higher level operations like Metro Mirror, Global Mirror, or FlashCopy. This is not restricted to the same Managed Disk Group, so it makes an ideal method to protect your data from a disk system or an array failure. If one copy of the mirror fails, it will provide continuous data access to the other. When the failed copy is repaired, they will automatically resynchronize. It can also be used as an alternative migration tool, where you can synchronize the mirror before splitting off the original side of the mirror. The VDisk stays online, and can be used normally, while the data is being synchronized. The copies can also be different structures (that is, striped, image, sequential, or space-efficient) and different extent sizes. To create a mirror copy from within a VDisk, perform the following steps; 1. Select a VDisk from the list, choose Add a Mirrored VDisk Copy from the drop-down list (see Figure 10-104 on page 475), and click Go. 2. The Add Copy to VDisk VDiskname window (where VDiskname is the VDisk you selected in the previous step) opens. See Figure 10-133 on page 500. You can perform the following steps separately or in combination: a. Choose what type of VDisk Copy you want to create, striped or sequential. b. Select the Managed Disk Group you want to put the copy in. We recommend that you choose a different group to maintain higher availability. c. Select the Select MDisk(s) manually button, which will expand the section that has a list of MDisks that are available for adding.
Chapter 10. SVC configuration and administration using the GUI
499
d. Choose the Mirror synchronization rate. This is the I/O governing rate in percentage during initial synchronization. A zero value disables synchronization. You can also select Synchronized, but this should be used only when the VDisk has never been used or is going to be formatted by the host. e. You can make the copy to be space-efficient. This section will expand, giving you options to allocate the virtual size, warning thresholds, autoexpansion, and Grain size. See 7.7.1, “Creating a space-efficient VDisk (SEV Disk)” on page 198 for more information. f. Optionally, format the new VDisk by selecting the Format the new VDisk copy and mark the VDisk synchronized check box. Use this option with care, because if the primary copy goes offline, you may not have the data replicated on the other copy. g. Click OK.
Figure 10-133 Add Copy to VDisk window
You can monitor the MDisk copy synchronization progress from the Manage Progress menu option and then the View Progress link, as shown in Figure 10-134 on page 501.
500
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-134 VDisk copy synchronization status
10.8.11 Migrating to a space-efficient VDisk using VDisk mirroring In this scenario, we are going to migrate from a fully allocated (or an image mode) VDisk to a space-efficient VDisk using VDisk mirroring.
Creating a VDisk mirror This procedure repeats the information from “Migrating a VDisk to an image mode VDisk” on page 496, but here we are selecting space-efficient as the mirrored copy. 1. Select a VDisk from the list, choose Add a Mirrored VDisk Copy from the drop-down list (see Figure 10-104 on page 475), and click Go. 2. The Add Copy to VDisk VDiskname window (where VDiskname is the VDisk you selected in the previous step) opens. See Figure 10-135 on page 502. You can perform the following steps separately or in combination: a. Choose what type of VDisk Copy you want to create, striped or sequential. b. Select the Managed Disk Group you want to put the copy in. c. Select the Select MDisk(s) manually button, which will expand the section with a list of MDisks that are available for adding. d. Choose the Mirror synchronization rate. This is the I/O governing rate in percentage during initial synchronization. A zero value disables synchronization. You can also select Synchronized, but this should be used only when the VDisk has never been used or is going to be formatted by the host. e. Select Space-efficient. This section will expand, and you should do the following: i. Type 100 in the % box for the real size to initially allocate. The SVC will see Copy 0 as 100% utilized, so Copy 1 must be defined as the same size. ii. Uncheck the Warn when used capacity of VDisk reaches check box. iii. Check Autoexpand. iv. Set the Grain size. See 7.7.1, “Creating a space-efficient VDisk (SEV Disk)” on page 198 for more information. f. Click OK.
Chapter 10. SVC configuration and administration using the GUI
501
Figure 10-135 Add a space-efficient Copy to VDisk window
You can monitor the MDisk copy synchronization progress from the Manage Progress menu option and then the View Progress link, as shown in Figure 10-134 on page 501. 502
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-136 VDisk copy synchronization status
Deleting a VDisk Copy from a VDisk mirror Once the VDisk copy has finished synchronizing, you can remove the original VDisk copy (Copy 0). 1. In the Viewing Virtual Disks window, select the mirrored VDisk from the list, choose Delete a Mirrored VDisk Copy from the drop-down list (Figure 10-137), and click Go.
Figure 10-137 Viewing Virtual Disks - Deleting a mirrored VDisk copy
Chapter 10. SVC configuration and administration using the GUI
503
2. Figure 10-138 displays both copies of the VDisk mirror. Select the radio button of the original copy (Copy ID 0) and click OK.
Figure 10-138 Deleting VDisk Copy 0
The VDisk is now a single space-efficient copy. To migrate an SEV to a fully allocated VDisk, follow the same scenario, but add a normal (fully allocated) VDisk as the second copy.
10.8.12 Splitting a VDisk Copy To split off a synchronized VDisk copy to a new VDisk, perform the following steps; 1. Select a mirrored VDisk from the list, choose Split a VDisk Copy from the drop-down list (Figure 10-104 on page 475), and click Go. 2. The Split a Copy from VDisk VDiskname window (where VDiskname is the VDisk you selected in the previous step) opens (See Figure 10-139 on page 505). Do the following steps: a. Select which copy you wish to split. a. Type a name for the new VDisk. b. You can optionally force split the copies even if the copy is not synchronized. This may mean the split copy will not be point-in-time consistent. c. Choose an I/O group and then a preferred node. In our case, we let the system choose. d. Select the cache mode, Read/Write or None. e. If you want, enter a unit device identifier. f. Click OK.
504
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-139 Split a VDisk Copy window
This new VDisk is available to be mapped to a host. Note: Once you split a VDisk mirror, you cannot resynchronize or recombine them. You must create a VDisk copy from scratch.
10.8.13 Shrinking a VDisk The method that the SVC uses to shrink a VDisk is to remove the required number of extents from the end of the VDisk. Depending on where the data actually resides on the VDisk, this can be quite destructive. For example, you might have a VDisk that consists of 128 extents (0 to 127) of 16 MB (2 GB capacity) and you want to decrease the capacity to 64 extents (1 GB capacity). In this case, the SVC simply removes extents 64 to 127. Depending on the operating system, there is no easy way to ensure that your data resides entirely on extents 0 through 63, so be aware that you might lose data. Although easily done using the SVC, you must ensure that your operating system supports shrinking, either natively or by using third-party tools, before using this function. Dynamic shrinking of a VDisk is only supported when the VDisk is in use by: W2K and W2K3 for basic disks W2K and W2K3 with a special fix from Microsoft (Q327020) for dynamic disks Chapter 10. SVC configuration and administration using the GUI
505
In addition, we recommend that you always have a good current backup before you execute this task. Shrinking a VDisk is useful in certain circumstances, such as: Reducing the size of a candidate target VDisk of a PPRC relationship to make it the same size as the source. Releasing space from VDisks to have free extents in the MDG, provided you do not use that space any more and take precautions with the remaining data, as explained earlier. Assuming your operating system supports it, perform the following steps to shrink a VDisk: 1. Perform any necessary steps on your host to ensure that you are not using the space you are about to remove. 2. Select the radio button to the left of the VDisk you want to shrink (Figure 10-104 on page 475). Select Shrink a VDisk from the list and click Go. 3. The Shrinking Virtual Disks VDiskname window (where VDiskname is the VDisk you selected in the previous step) opens, as shown in Figure 10-140. In the Reduce Capacity By field, enter the capacity you want to reduce. Select B, KB, MB, GB, TB, or PB. The final capacity of the VDisk is the Current Capacity minus the capacity that you specify. Note: Be careful with the capacity information. The Current Capacity field shows it in MBs, while you can specify a capacity to reduce in GBs. SVC calculates 1 GB as being 1024 MB. When you are done, click OK. The changes should become apparent on your host.
Figure 10-140 Shrinking a VDisk
10.8.14 Showing the MDisks To show the MDisks that are used by a specific VDisk, perform the following steps: 1. Select the radio button to the left of the VDisk you want to view MDisk information about (Figure 10-104 on page 475). Select Show MDisks This VDisk is Using from the list and click Go. 2. You see a subset (specific to the VDisk you chose in the previous step) of the Viewing Managed Disks window (Figure 10-141 on page 507). 506
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-141 Showing MDisks used by a VDisk
For information about what you can do in this window, see 10.5.3, “Managed disks” on page 433.
10.8.15 Showing the MDisk group To show the MDG to which a specific VDisk belongs, perform the following steps: 1. Select the radio button to the left of the VDisk you want to view MDG information about (Figure 10-104 on page 475). Select Show MDisk Group This Vdisk Belongs To from the list and click Go. 2. You see a subset (specific to the VDisk you chose in the previous step) of the Viewing Managed Disk Groups Belonging to VDiskname window (Figure 10-142).
Figure 10-142 Showing an MDG for a VDisk
10.8.16 Showing the host to which the VDisk is mapped To show the host to which a specific VDisk belongs, select the radio button to the left of the VDisk you want to view MDG information about (Figure 10-104 on page 475). Select Show Hosts This VDisk is Mapped To from the list and click Go. This shows you the Host to which the VDisk is attached (Figure 10-143 on page 508). Alternatively, you can use the procedure described in 10.8.18, “Showing VDisks mapped to a host” on page 509 to see all VDisk to Host mappings.
Chapter 10. SVC configuration and administration using the GUI
507
Figure 10-143 Show host to VDisk mapping
10.8.17 Showing capacity information To show the capacity information of the cluster, perform the following steps: 1. In Figure 10-144, select Show Capacity Information from the drop-down list and click Go. You should then see the capacity information for this cluster in Figure 10-145.
Figure 10-144 Select Show Capacity Information
2. Figure 10-145 shows you the total MDisk capacity, the space in the MDGs, the space allocated to the VDisks, and the total free space.
Figure 10-145 Show capacity information
508
Implementing the IBM System Storage SAN Volume Controller V4.3
10.8.18 Showing VDisks mapped to a host To show the VDisks assigned to a specific host, perform the following steps: 1. From the SVC welcome window, click the Work with Virtual Disks option and then the Virtual Disk to Host Mappings link (Figure 10-146).
Figure 10-146 VDisk to host mapping
2. Now you can see which host that VDisk belongs to. If this is a long list, you can use the Additional Filtering and Sort option from 10.2.1, “Organizing on screen content” on page 407.
10.8.19 Deleting VDisks from a host In the same window where you can view the VDisk to host mapping (Figure 10-146), you can also delete a mapping. Select the radio button to the left of the host and VDisk combination you want to delete. Ensure that Delete a Mapping is selected from the list. Click Go. 1. Confirm the selection you made in Figure 10-147 by clicking the Delete button.
Figure 10-147 Deleting VDisk to Host mapping
2. Now you are back at the window shown in Figure 10-146. Check that this VDisk (MDG_SE_VDisk2) is no longer mapped to this Host (Kanaga). Now you can assign this VDisk to another Host, as described in 10.8.8, “Mapping a VDisk to a host” on page 492. You have now completed the tasks required to manage virtual disks within an SVC environment.
Chapter 10. SVC configuration and administration using the GUI
509
10.9 Managing Copy Services See Chapter 11, “Copy Services: FlashCopy” on page 539, Chapter 12, “Copy Services: Metro Mirror” on page 603, and Chapter 13, “Copy Services: Global Mirror” on page 669 for more information about the tasks related to the management of Copy Services in the SVC environment.
10.10 Service and maintenance using the GUI This section discusses the various service and maintenance tasks that you can perform within the SVC environment. To perform all of the following activities, in the SVC Welcome window (Figure 10-148), select the Service and Maintenance option. Note: You are prompted for a cluster user ID and password for some of the following tasks.
Figure 10-148 Service and Maintenance functions
510
Implementing the IBM System Storage SAN Volume Controller V4.3
10.11 Upgrading software This section explains how to upgrade the SVC software.
10.11.1 Package numbering and version The format for the software upgrade package name ends in four positive integers separated by dots. For example, a software upgrade package may have the name IBM_2145_INSTALL_4.3.0.600.
10.11.2 Upgrade status utility A function of the master console is to check the software levels in the system against recommended levels that will be documented on the support Web site. You are informed if software levels are up-to-date, or if you need to download and install newer levels. This information is provided after you log in to the SVC GUI. In the middle of the Welcome window, you will see that new software is available. Use the link that is provided there to download the new software and get more information about it. Important: To use this feature, the SSPC/Master Console must be able to access the Internet. If the SSPC cannot access the Internet because of restrictions such as a local firewall, you will see the message The update server cannot be reached at this time. Use the Web link provided in the message for the latest software information.
10.11.3 Precautions before upgrade In this section, we describe precautions you should take before attempting an upgrade. Important: Before attempting any SVC code update, you should read and understand the SAN volume controller concurrent compatibility and code cross reference matrix. Go to the following site and click the link for Latest SAN Volume Controller code: http://www-1.ibm.com/support/docview.wss?uid=ssg1S1001707 During the upgrade, each node in your cluster will be automatically shut down and restarted by the upgrade process. Since each node in an I/O group provides an alternate path to VDisks, you need to make sure that all I/O paths between all hosts and SANs are working. If you have not performed this check, then some hosts might lose connectivity to their VDisk and experience I/O errors when the SVC node providing that access is shut down during the upgrade process (Example 10-1). Example 10-1 Using datapath query commands to check all paths are online
C:\Program Files\IBM\SDDDSM>datapath query adapter Active Adapters :2 Adpt# 0 1
Name Scsi Port2 Bus0 Scsi Port3 Bus0
State NORMAL NORMAL
Mode ACTIVE ACTIVE
Select 167 137
Errors 0 0
Paths 4 4
Active 4 4
C:\Program Files\IBM\SDDDSM>datapath query device Chapter 10. SVC configuration and administration using the GUI
511
Total Devices : 2
DEV#: 0 DEVICE NAME: Disk1 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 6005076801A180E9080000000000002A ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port2 Bus0/Disk1 Part0 OPEN NORMAL 37 0 1 Scsi Port2 Bus0/Disk1 Part0 OPEN NORMAL 0 0 2 Scsi Port3 Bus0/Disk1 Part0 OPEN NORMAL 0 0 3 Scsi Port3 Bus0/Disk1 Part0 OPEN NORMAL 29 0 DEV#: 1 DEVICE NAME: Disk2 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 6005076801A180E90800000000000010 ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port2 Bus0/Disk2 Part0 OPEN NORMAL 0 0 1 Scsi Port2 Bus0/Disk2 Part0 OPEN NORMAL 130 0 2 Scsi Port3 Bus0/Disk2 Part0 OPEN NORMAL 108 0 3 Scsi Port3 Bus0/Disk2 Part0 OPEN NORMAL 0 0 You can check the I/O paths by using datapath query commands, as shown in Example 10-1 on page 511. You do not need to check for hosts that have no active I/O operations to the SANs during the software upgrade. Tip: See the Subsystem Device Driver User's Guide for the IBM TotalStorage Enterprise Storage Server and the IBM System Storage SAN Volume Controller, SC26-7540 for more information about datapath query commands. It is well worth double checking that your UPS power configuration is also set up correctly (even if your cluster is running without problems). Specifically: Ensure that your UPSs are all getting their power from an external source, and that they are not daisy chained. In other words, make sure that each UPS is not supplying power to another node’s UPS. Ensure that the power cable, and the serial cable coming from the back of each node, goes back to the same UPS. If the cables are crossed and are going back to different UPSs, then during the upgrade, as one node is shut down, another node might also be mistakenly shut down.
10.11.4 SVC software upgrade test utility This is a SVC software utility that checks for known issues that could cause problems during an SVC software upgrade. It can be run on any SVC cluster running level 4.1.0.0 or above. It is available from the following location: http://www-1.ibm.com/support/docview.wss?uid=ssg1S40005857 The package is installed in the same way as you would upgrade the SVC software code, as described in “Upgrade procedure” on page 513. Example 10-2 on page 513 shows the command to test an upgrade.
512
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 10-2 Run an upgrade test
IBM_2145:ITSO-CLS2:admin>svcupgradetest svcupgradetest version 1.11. Please wait while the tool tests for issues that may prevent a software upgrade from completing successfully. The test will take approximately one minute to complete. The test has not found any problems with the 2145 cluster. Please proceed with the software upgrade.
Upgrade procedure To upgrade the SVC cluster software, perform the following steps: 1. Use the Run Maintenance Procedure in the GUI and correct all open problems first. 2. Back up the SVC Config, as described in “Backup procedure” on page 536. 3. Back up the support data, just in case there is a problem during the upgrade that renders a node unusable. This information could assist IBM Support in determining why the upgrade might have failed and help with a resolution. Example 10-3 shows the necessary commands that need to be run. This command is only available in the CLI. Example 10-3 Creating an SVC snapshot
IBM_2145:ITSO-CLS2:admin>svc_snap Collecting system information... Copying files, please wait... Copying files, please wait... Dumping error log... Creating snap package... Snap data collected in /dumps/snap.100047.080617.002334.tgz
Note: You can ignore the error message No such file or directory.
Chapter 10. SVC configuration and administration using the GUI
513
Select Software Maintenance → List Dumps → Software Dumps, download the dump that was created in Example 10-3 on page 513, and store it in a safe place with the SVC Config that you created previously (see Figure 10-149 and Figure 10-150).
Figure 10-149 Getting software dumps
Figure 10-150 Downloading software dumps
4. From the SVC Welcome window, click the Service and Maintenance option and then the Upgrade Software link. 5. In the Upgrade Software window shown in Figure 10-151 on page 515, you can either upload a new software upgrade file or list the upgrade files. Click the Upload button to upload the latest SVC cluster code.
514
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-151 Update Software window
6. In the Software Upgrade (file upload) window (Figure 10-152), type or browse to the directory on your management workstation (for example, master console) where you stored the latest code level and click Upload.
Figure 10-152 Software upgrade (file upload)
7. The File Upload window (Figure 10-153) is displayed if the file is uploaded. Click Continue.
Figure 10-153 File upload
Chapter 10. SVC configuration and administration using the GUI
515
8. The Select Upgrade File window (Figure 10-154) lists the available software packages. Make sure the radio button next to the package you want to apply is selected. Click the Apply button.
Figure 10-154 Select Upgrade File
9. In the Confirm Upgrade File window (Figure 10-155), click the Confirm button.
Figure 10-155 Confirm Upgrade File
10.After this confirmation, the SVC will check if there are any outstanding errors. If there are no errors, click Continue, as shown in Figure 10-156 on page 517, to proceed to the next upgrade step. Otherwise, the Run Maintenance button is displayed, which is used to check the errors. For more information about how to use the maintenance procedures, see 10.11.5, “Running maintenance procedures” on page 519.
516
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-156 Check Outstanding Errors window
11.The Check Node Status window shows the in-use nodes with their current status displayed, as shown in Figure 10-157. Click Continue to proceed.
Figure 10-157 Check Node Status window
Chapter 10. SVC configuration and administration using the GUI
517
12.The Start Upgrade window is displayed. Click the Start Software Upgrade button to start the software upgrade, as shown in Figure 10-158.
Figure 10-158 Start Upgrade software window
The upgrade will start by upgrading one node in each I/O group. 13.The Software Upgrade Status window (Figure 10-159) opens. Click the Check Upgrade Status button periodically. This process might take a while to complete. If the software is completely upgraded, you should get a software completed message and the code level of the cluster and nodes will show the newly applied software level.
Figure 10-159 Software Upgrade Status
14.During the upgrade process, you can only issue informational commands. All task commands such as the creation of a VDisk (as shown in Figure 10-160 on page 519) are denied. This applies to both the GUI and the CLI. All tasks, such as creation, modifying, mapping, and deleting, are denied.
518
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-160 Denial of a task command during the software update
15.The new code is distributed and applied to each node in the SVC cluster. After installation, each node is automatically restarted in turn. Although unlikely, under some circumstances the concurrent code load (CCL) might fail. For example, if one node fails to accept the new code level, then the update on that one node will be backed out, and the node will revert back to the original code level. Prior to 4.1.0, this caused the update process to run in reverse with all nodes reverting to the old level of software. From 4.1.0 onwards, the update will simply wait for user intervention. For example, if there are two nodes (A and B) in an I/O group, and node A has been upgraded successfully, and then node B then suffers a hardware failure, the upgrade will end with an I/O group that has a single node at the higher code level. If the hardware failure is repaired on node B, the CCL will then complete the code upgrade process. I
Tip: Be patient! After the software update is applied, the first SVC node in a cluster will update and install the new SVC code version shortly afterwards. If there is more than one I/O group (up to four I/O groups are possible) in an SVC cluster, the second node of the second I/O group will load the new SVC code and restart with a 10 minute delay to the first node. A 30 minute delay between the update of the first node and the second node in an I/O group ensures that all paths, from a multipathing point of view, are available again. An SVC cluster update with one I/O group takes approximately one hour. 16.If you run into an error, go to the Analyze Error Log window. Search for Software Install completed. Select the radio button Sort by date with the newest first and then click Perform. This should list the software near the top. For more information about how to work with the Analyze Error Log window, see 10.11.7, “Analyzing the error log” on page 524. You might also find it worthwhile to capture information for IBM support to help you diagnose what went wrong. We covered this in step 3 on page 513. You have now completed the tasks required to upgrade the SVC software. Click the X icon in the upper right corner of the display area to close the Upgrade Software window. Do not close the browser by mistake.
10.11.5 Running maintenance procedures To run the maintenance procedures on the SVC cluster, perform the following steps: 1. From the SVC Welcome window, click the Service and Maintenance option and then the Run Maintenance Procedures link.
Chapter 10. SVC configuration and administration using the GUI
519
2. Click Start Analysis, as shown in Figure 10-161. This will analyze the cluster log and guide you through the maintenance procedures.
Figure 10-161 Maintenance Procedures
3. This generates a new error log file. In this case, the file name is errlog_100048_080701_165233 in the /dumps/elogs/ directory (Figure 10-162), where: – – – –
errlog: This part of the file name is generic for all error log files. 100048: This is the window name of the current configuration node. 080701: This is the date (YYMMDD). 165233: This is the time (HHMMSS).
Figure 10-162 Maintenance error log with unfixed errors
4. Click the error number in the Error Code column in Figure 10-162. This gives you the explanation for this error, as shown in Figure 10-163 on page 521.
520
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-163 Maintenance: error code description
5. To perform problem determination, click Continue. It will now display the details for the error and may give you some options to diagnose/repair the problem. In this case, it asks for you to check an external configuration and then press Continue (Figure 10-164).
Figure 10-164 Maintenance procedures: fixing Stage 2
6. The SVC maintenance procedure will now run a new discovery to confirm the problem is fixed. Press Continue, as shown in Figure 10-165.
Figure 10-165 Maintenance procedure: fixing Stage 3
Chapter 10. SVC configuration and administration using the GUI
521
7. The discovery reported no new errors, so the entry in the error log is now marked as fixed (as shown in Figure 10-166). Click OK.
Figure 10-166 Maintenance procedure: fixed
8. After you have gone through each of the unfixed errors, you will see a window similar to Figure 10-167. You can now click the X icon in the upper right corner of the Run Maintenance Procedures frame to close this window.
Figure 10-167 Maintenance procedures: close
10.11.6 Setting up error notification To set up error notification, perform the following steps: 1. From the SVC Welcome window, click the Service and Maintenance option and then the Set SNMP Error Notifications link. 2. In the Modify Error Notification Settings window (Figure 10-168 on page 523), select the level of notification (default is None) to apply to both SNMP and e-mail alerting. Click Modify Settings.
522
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 10-168 Setting SNMP error notification
3. Type the IP address of your SNMP Manager, (optional) port, and community string to use (Figure 10-169). Click Add. Note: Depending on what IP protocol addressing is configured, it will display options for IPV4, IPV6, or both.
Figure 10-169 Set the SNMP settings
4. The Modify Error Notification Settings window now displays confirmation that it has updated the settings, as shown in Figure 10-170.
Figure 10-170 Error Notification settings confirmation
Chapter 10. SVC configuration and administration using the GUI
523
5. The Modify Error Notification Settings window now displays the current status, as shown in Figure 10-171.
Figure 10-171 Current error notification settings
6. You can now click the X icon in the upper right corner of the Set SNMP Error Notification frame to close this window.
10.11.7 Analyzing the error log The following types of events and errors are logged in the error log: Events: State changes that are detected by the cluster software and that are logged for informational purposes. Events are recorded in the cluster error log. Errors: Hardware or software problems that are detected by the cluster software and that require some sort of repair. Errors are recorded in the cluster error log. Unfixed errors: Errors that were detected and recorded in the cluster error log and that were not yet corrected or repaired. Fixed errors: Errors that were detected and recorded in the cluster error log and that were subsequently corrected or repaired.
524
Implementing the IBM System Storage SAN Volume Controller V4.3
To display the error log for analysis, perform the following steps: 1. From the SVC Welcome window, click the Service and Maintenance options and then the Analyze Error Log link. 2. From the Error Log Analysis window (Figure 10-172), you can choose either the Process or Clear Log button.
Figure 10-172 Analyzing the error log
a. Select the appropriate radio buttons and click the Process button to display the log for analysis. The Analysis Options and Display Options radio button boxes allow you to filter the results of your log inquiry to reduce the output. b. You can display the whole log, or you can filter the log so that only errors, events, or unfixed errors are displayed. You can also sort the results by selecting the appropriate display options. For example, you can sort the errors by error priority (lowest number = most serious error) or by date. If you sort by date, you can specify whether the newest or oldest error displays at the top of the table. You can also specify the number of entries you want to display on each page of the table. c. Click the Log File Options radio button to use the existing log file or to generate a fresh one. Using the existing log file displays entries that exist in the log file that was last generated. If this is the first time you are using this option, no error log exists. To obtain the latest status of your cluster, or if it is the first time you are using this option, select the Generate a new error log file option.
Chapter 10. SVC configuration and administration using the GUI
525
The errlog_100048_080701_215907 error log file is created in the /dumps/elogs/ directory and is ready for analysis (Figure 10-173), where: • • • •
errlog: This part of the file name is generic for all error log files. 100048: This is the window name of the current configuration node. 080701: This is the date (YYMMDD). 215907: This is the time (HHMMSS).
Figure 10-173 Analyzing Error Log: Process
526
Implementing the IBM System Storage SAN Volume Controller V4.3
d. Click an underlined sequence number; this gives you the detailed log of this error (Figure 10-174).
Figure 10-174 Analyzing Error Log: Detailed Error Analysis
Chapter 10. SVC configuration and administration using the GUI
527
e. You can optionally display detailed sense code data by pressing the Sense Expert button shown in Figure 10-175. Press Return to go back to the detailed error Analysis window.
Figure 10-175 Decoding Sense Data
f. If the log entry is an error, you have the option of marking the error as fixed. This does not run through any other checks/processes, so we recommend that you do this as a maintenance procedures task (see 10.11.5, “Running maintenance procedures” on page 519). g. Click the Clear Log button at the bottom of the Error Log Analysis window in Figure 10-172 on page 525 to clear the log. If the error log contains unfixed errors, a warning message is displayed when you click Clear Log. 3. You can now click the X icon in the upper right corner of the Analyze Error Log window.
10.11.8 License settings To change license settings, perform the following steps: 1. From the SVC Welcome window, click the Service and Maintenance options and then the License Settings link. 528
Implementing the IBM System Storage SAN Volume Controller V4.3
2. In the License Settings window (Figure 10-176), consult your license before you make changes in this window. If you purchased additional features (for example, FlashCopy or Global Mirror) or if you increased the capacity of your license, make the appropriate changes. Then click the Update License Settings button.
Figure 10-176 License Settings
3. You now see a license confirmation window, as shown in Figure 10-177. Review this window and ensure that you are in compliance. If you are in compliance, click I Agree to make the requested changes take effect.
Figure 10-177 License agreement
Chapter 10. SVC configuration and administration using the GUI
529
4. You return to the Update License Settings review window (Figure 10-178), where your changes should be reflected.
Figure 10-178 Featurization settings update
5. You can now click the X icon in the upper right corner of the License Settings window.
10.11.9 Viewing the license settings log To view the feature log, which registers the events related to the SVC licensed features, perform the following steps: 1. From the SVC Welcome window, click the Service and Maintenance option and then the View License Settings Log link. 2. The License Log window (Figure 10-179) opens. It displays the current license settings and a log of when changes were made.
Figure 10-179 Feature log
3. You can now click the X icon in the upper right corner of the View License Settings Log window.
530
Implementing the IBM System Storage SAN Volume Controller V4.3
10.11.10 Listing dumps To list the dumps that were generated, perform the following steps: 1. From the SVC Welcome window, click the Service and Maintenance option and then the List Dumps link. 2. In the List Dumps window (Figure 10-180), you see several dumps and log files that were generated over time on this node. They include the configuration dump we generated in Example 10-3 on page 513. Click any of the available links (the underlined text in the table under the List Dumps heading) to go to another window that displays the available dumps. To see the dumps on the other node, you must click Check other nodes. Note: By default, the dump and log information that is displayed is available from the configuration node. In addition to these files, each node in the SVC cluster keeps a local software dump file. Occasionally, other dumps are stored on them. Click the Check Other Nodes button at the bottom of the List Dumps window (Figure 10-180) to see which dumps or logs exist on other nodes in your cluster.
Figure 10-180 List Dumps
Chapter 10. SVC configuration and administration using the GUI
531
3. Figure 10-181 shows the list of dumps from the partner node. You can see a list of the dumps by clicking one of the Dump Types.
Figure 10-181 List Dumps from the partner node
4. To copy a file from this partner node to the config node, click the dump type and then click the file you want to copy, as shown in Figure 10-182.
Figure 10-182 Copy dump files
532
Implementing the IBM System Storage SAN Volume Controller V4.3
5. You will see a confirmation window that the dumps are being retrieved. You can either Continue working with the other node or Cancel back to the original node (Figure 10-183).
Figure 10-183 Retrieve dump confirmation
6. After all the necessary files are copied to the SVC config node, click Cancel to finish the copy operation, and Cancel again to return to the SVC config node. Now, for example, if you click the Error Logs link, you should see information similar to that shown in Figure 10-184.
Figure 10-184 List Dumps: Error Logs
Chapter 10. SVC configuration and administration using the GUI
533
7. From this window, you can perform either of the following tasks: – Click any of the available log file links (indicated by the underlined text) to display the log in complete detail, as shown in Figure 10-185.
Figure 10-185 List Dumps: Error log detail
– Delete one or all of the dump or log files. To delete all, click the Delete All button. To delete some, select the radio button or buttons to the right of the file and click the Delete button. In Figure 10-186, you have to confirm the deletion by clicking Confirm Delete.
Figure 10-186 Confirm Delete
8. You can now click the X icon in the upper right corner of the List Dumps window.
534
Implementing the IBM System Storage SAN Volume Controller V4.3
10.12 Backing up the SVC configuration The SVC configuration data is stored on all the nodes in the cluster. It is specially hardened so that, in normal circumstances, the SVC should never lose its configuration settings. However, in exceptional circumstances, this data could become corrupted or lost. This section details the tasks that you can perform to save the configuration data from an SVC configuration node and restore it. The following configuration information is backed up:
Storage subsystem Hosts Managed disks (MDisks) Managed Disk Groups (MDGs) SVC nodes Virtual disks VDisk-to-host mappings FlashCopy mappings FlashCopy consistency groups Mirror relationships Mirror consistency groups
Backing up the cluster configuration enables you to restore your cluster configuration in the event that it is lost. But only the data that describes the cluster configuration is backed up. In order to back up your application data, you need to use the appropriate backup methods. To begin the restore process, consult IBM Support to determine the cause as to why you cannot access your original configuration data. The prerequisites for having a successful backup are as follows: All nodes in the cluster must be online. No object name can begin with an underscore (_). Do not run any independent operations that could change the cluster configuration while the backup command runs. Do not make any changes to the fabric or cluster between backup and restore. If changes are made, back up your configuration again or you might not be able to restore it later. Note: We recommend that you make a backup of the SVC configuration data after each major change in the environment, such as defining or changing a VDisks, VDisk-to-host mappings, and so on. The output of the SVC configuration backup is a file with the name svc.config.backup.xml that is stored in the C:\Program Files\IBM\svcconsole\cimom\backup\SVCclustername folder in the SSPC or master console (where SVCclustername is the SVC cluster name of the configuration from which you backed up). This differs from backing up the configuration using CLI. The svc.config.backup.xml file is stored in the /tmp folder on the configuration node and must be copied to an external and secure place for backup purposes. Important: We strongly recommend that you change the default names of all objects to non-default names. For objects with a default name, a warning is produced and the object is restored with its original name and “_r” appended to it.
Chapter 10. SVC configuration and administration using the GUI
535
10.12.1 Backup procedure To back up the SVC configuration data, perform the following steps: 1. From the SVC Welcome window, click the Service and Maintenance option and then the Backup Configuration link. 2. In the Backing up a Cluster Configuration window (Figure 10-187), click the Backup button.
Figure 10-187 Backing up a Cluster Configuration data
3. After the configuration backup is successfully done, you see messages similar to the ones shown in Figure 10-188. Make sure you that you read, understand, act upon, and document the warning messages, since they can influence the restore procedure.
Figure 10-188 Configuration backup successful message and warnings
4. You can now click the X icon in the upper right corner of the Backing up a Cluster Configuration window. Info: To avoid getting the CMMVC messages that are shown in Figure 10-188, you need to replace all the default names, for example, mdisk1, vdisk1, and so on.
536
Implementing the IBM System Storage SAN Volume Controller V4.3
10.12.2 Restoring the SVC configuration It is very important that you perform the configuration backup described in 10.12.1, “Backup procedure” on page 536 periodically, and every time after you change the configuration of your cluster. Carry out the restore procedure only under the direction of IBM Level 3 support.
10.12.3 Deleting the configuration backup files This section details the tasks that you can perform to delete the configuration backup files from the default folder in the SVC master console. You can do this if you have already copied them to another external and secure place. To delete the SVC Configuration backup files, perform the following steps: 1. From the SVC Welcome window, click the Service and Maintenance options and then the Delete Backup link. 2. In the Deleting a Cluster Configuration window (Figure 10-189), click the OK button to confirm the deletion. This deletes the C:\Program Files\IBM\svcconsole\cimom\backup\SVCclustername folder (where SVCclustername is the SVC cluster name on which you are working) on the SVC master console and all its contents.
Figure 10-189 Deleting a Cluster Configuration
3. Click Delete to confirm the deletion of the configuration backup data. See Figure 10-190.
Figure 10-190 Deleting a Cluster Configuration confirmation message
4. The cluster configuration is now deleted.
Chapter 10. SVC configuration and administration using the GUI
537
538
Implementing the IBM System Storage SAN Volume Controller V4.3
11
Chapter 11.
Copy Services: FlashCopy The FlashCopy function of the IBM System Storage SAN Volume Controller (SVC) provides the capability to perform a point-in-time (PiT) copy of one or more VDisks. In this chapter, we describe how FlashCopy works on the SVC, and we present examples of how to configure and utilize FlashCopy.
© Copyright IBM Corp. 2003-2008. All rights reserved.
539
11.1 FlashCopy FlashCopy is also known as point-in-time copy (PiT). This technique is used to help solve the problem where it is difficult to make a consistent copy of a data set that is being constantly updated. The FlashCopy source is frozen for several seconds during the PiT copy process. It will be able to accept I/O when the PiT copy bitmap is set up and the FlashCopy function is ready to intercept read/write requests in the IO path. Although the background copy operation takes some time, the resulting data at the target appears as though the copy were made instantaneously. SVC’s FlashCopy service provides the capability to perform a PiT copy of one or more VDisks. Since it is performed at the block level, it is necessary to flush the cache and OS buffers prior to executing the FlashCopy in order to ensure consistency at the application level.
Business requirement The business applications for FlashCopy are many and various. An important use is facilitating consistent backups of constantly changing data, and in these instances a FlashCopy is created to capture a PiT copy. The resulting image is backed up to tertiary storage such as tape. After the copied data is on tape, the FlashCopy target is redundant. Different tasks can benefit from the use of FlashCopy. In the following sections, we describe the most common situations.
Moving and migrating data When you need to move a consistent data set from one host to another, FlashCopy can facilitate this action with a minimum of downtime for the host application dependent on the source VDisk. It is very important to quiesce the application on the host and flush the application and OS buffers so that the new VDisk contains data that is “clean” to the application. Failing to do this might result in the newly created VDisk being a mirrored copy of inconsistent data, and thus it might not be usable by the application. The cache on the SVC is also flushed using the FlashCopy prepare command; see “Preparing” on page 556 prior to performing the FlashCopy. The created data set on the FlashCopy target is immediately available as well as the source VDisk.
Backup FlashCopy does not impact your backup time, but it allows you to create a PiT consistent data set (across VDisks), with a minimum of downtime for your source host. The FlashCopy target can then be mounted on a different host (or the backup server) and backed up. Using this procedure, the backup speed becomes less important, since the backup time does not require downtime for the host dependent on the source VDisks.
Restore You can keep periodically created FlashCopy targets online, to provide very fast restore of specific files from the PiT consistent data set revealed on the FlashCopy targets, which simply can be copied to the source VDisk in case a restore is needed.
540
Implementing the IBM System Storage SAN Volume Controller V4.3
When a background copy process has completed (that is, it has entered the copied state; see “Idle_or_copied” on page 555), and a complete data set restore is needed, it is possible to delete the FlashCopy mappings and create corresponding FlashCopy mappings in the opposite direction. This is often referred to as a FlashBack procedure. This procedure can be used to restore the PiT consistent data set obtained from the preceding FlashCopy very quickly.
Application testing You can test new applications and new operating system releases against a FlashCopy of your production data. The risk of data corruption is eliminated, and your application does not need to be taken offline for an extended period of time to perform the copy of the data. Data mining is a good example of an area where FlashCopy can help you. Data mining can now extract data without affecting your application.
11.2 SVC FlashCopy features The FlashCopy function in SVC supports these features: The target is the time-zero copy of the source (known as FlashCopy mapping targets). The source VDisk and target VDisk are available (almost) immediately. One source VDisk can have up to 256 target VDisks at the same or different PiTs. Consistency groups are supported to enable FlashCopy across multiple VDisks. The target VDisk can be updated independently of the source VDisk. Bitmaps governing I/O redirection (I/O indirection layer) are maintained in both nodes of the SVC I/O group to prevent a single point of failure. FlashCopy mapping can be automatically withdrawn after the completion of background copy. FlashCopy consistency groups can be automatically withdrawn after the completion of background copy. SVC V4.3 has incorporated several enhancement to the FlashCopy functions. Multi-Target FlashCopy: FlashCopy now supports up to 256 target copies from a single source VDisk. Space-efficient FlashCopy (SEFC): SEFC uses disk space only for changes between source and target data and not for the entire capacity of a virtual disk copy. FlashCopy licensing: The FlashCopy previously was licensed by the source and target virtual capacity. It will now be licensed only by source virtual capacity.
Chapter 11. Copy Services: FlashCopy
541
11.3 How it works FlashCopy works by defining a FlashCopy mapping that consists of one source VDisk together with one target VDisk. Multiple FlashCopy mappings can be defined and PiT consistency can be observed across multiple FlashCopy mappings using consistency groups; see “Consistency group with MTFC” on page 546. When FlashCopy is started, it makes a copy of a source VDisk to a target VDisk, and the original contents of the target VDisk are overwritten. When the FlashCopy operation is started, the target VDisk presents the contents of the source VDisk as they existed at the single PiT of FlashCopy starting. This is also referred to as a time-zero copy (T0 ). When a FlashCopy is started, the source and target VDisks are instantaneously available, because when it starts, bitmaps are created to govern and redirect I/O to the source or target VDisk, respectively, depending on where the requested block is present, while the blocks are copied in the background from the source to the target VDisk. For more details on background copy, see “Grains and the FlashCopy bitmap” on page 547. Both the source and target VDisks are available for read and write operations, although the background copy process has not yet completed copying across the data from the source to target volumes. In Figure 11-1, the redirection of the host I/O towards source and target VDisk is explained.
Figure 11-1 Redirection of host I/O
542
Implementing the IBM System Storage SAN Volume Controller V4.3
11.4 Implementation of SVC FlashCopy In the topics that follow, we describe how FlashCopy is implemented in the SVC.
11.4.1 FlashCopy mappings In the SVC, FlashCopy occurs between a source VDisk and a target VDisk. The source and target VDisks must be the same size. The minimum granularity that SVC supports for FlashCopy is an entire VDisk; this means it is not possible to FlashCopy only part of a VDisk. The source and target VDisks must both belong to the same SVC Cluster, but can be in different I/O groups within that Cluster. SVC FlashCopy associates a source VDisk and a target VDisk together in a FlashCopy mapping. VDisks, which are members of a FlashCopy mapping, cannot have their size increased or decreased while they are members of the FlashCopy mapping. The SVC supports the creation of enough FlashCopy mappings to allow every VDisk to be a member of a FlashCopy mapping. A FlashCopy mapping is the act of creating a relationship between a source VDisk and a target VDisk. FlashCopy mappings can be either stand-alone or a member of a consistency group. You can perform the act of preparing, starting, or stopping on either the stand-alone mapping or the consistency group. Note: Once a mapping is in a consistency group, you can only operate on the group and can no longer prepare, start, or stop the individual mapping. Figure 11-2 illustrates the concept of FlashCopy mapping.
Figure 11-2 FlashCopy mapping
Chapter 11. Copy Services: FlashCopy
543
11.4.2 Multiple Target FlashCopy From SVC release 4.2.0 onwards, SVC supports up to 256 target VDisks to be copied from a single source VDisk. Each copy is managed by a unique mapping. In general, each mapping acts independently and is not affected by the fact that other mappings share the same source VDisk. Figure 11-3 illustrates how these can be viewed.
Figure 11-3 Multiple Target FlashCopy implementation
Figure 11-3 shows four targets and mappings taken from a single source. It also shows that there is an ordering to the targets: Target 1 is the oldest (as measured from the time it was started), through to Target 4, which is the newest. The ordering is important because of the way in which data is copied when multiple target VDisks are defined and because of the dependency chain that results. A write to source VDisk does not cause its data to be copied to all the targets; instead, it is copied to the newest target VDisk only (Target 4 above). The older targets will refer to new targets first before referring to the source. From the point of view of an intermediate target disk (either the oldest or the newest), it treats the set of newer target VDisks and the true source VDisk as a type of composite source. It treats all older VDisks as a kind of target (and behaves like a source to them). If the mapping for an intermediate target VDisk shows 100% progress, then its target VDisk contains a complete set of data. In this case, mappings treat the set of newer target VDisks, up to and including the 100% progress target, as a form of composite source. A dependency relationship exists between a particular target and all newer targets (up to and including a target that shows 100% progress) that share the same source until all data has been copied to this target and all older targets. More information about Multiple Target FlashCopy (MTFC) can be find in 11.4.5, “Interaction and dependency between MTFC” on page 549.
11.4.3 Consistency groups Consistency groups address the issue where the objective is to preserve data consistency across multiple VDisks, because the applications have related data that span multiple VDisks. A requirement for preserving the integrity of data being written is to ensure that “dependent writes” are executed in the application's intended sequence. Because the SVC provides PiT semantics, a self consistent data set is obtained.
544
Implementing the IBM System Storage SAN Volume Controller V4.3
FlashCopy mappings must be part of a consistency group, although if no FlashCopy consistency group is specified, upon creation, the FlashCopy mapping will belong to the default group zero. The default consistency group 0 is a pseudo consistency group, and this means that no commands can be directed at FlashCopy consistency group 0, since it is intended for FlashCopy mappings that are to be handled as a single instance. FlashCopy commands can be issued to a FlashCopy consistency group, which affects all FlashCopy mappings in the consistency group, or to a single FlashCopy mapping if it is not part of a defined FlashCopy consistency group. Figure 11-4 illustrates a consistency group consisting of two FlashCopy mappings.
Figure 11-4 FlashCopy consistency group
Dependent writes To illustrate why it is crucial to use consistency groups when a data set spans multiple VDisks, consider the following typical sequence of writes for a database update transaction: 1. A write is executed to update the database log, indicating that a database update is to be performed. 2. A second write is executed to update the database. 3. A third write is executed to update the database log, indicating that the database update has completed successfully. The database ensures the correct ordering of these writes by waiting for each step to complete before starting the next. However, if the database log (updates 1 and 3) and the database itself (update 2) are on different VDisks and a FlashCopy mapping is started during this update, then you need to exclude the possibility that the database itself is copied slightly before the database log resulting in the target VDisks seeing writes (1) and (3) but not (2), since the database was copied before the write was completed. In this case, if the database was restarted using the backup made from the FlashCopy target disks, the database log would indicate that the transaction had completed successfully when, in fact, that is not the case, because the FlashCopy of the VDisk with the database file was started (bitmap was created) before the write was on the disk. Therefore, the transaction is lost and the integrity of the database is in question. To overcome the issue of dependent writes across VDisks and create a consistent image of the client data, it is necessary to perform a FlashCopy operation on multiple VDisks as an atomic operation. To achieve this condition, the SVC supports the concept of consistency groups.
Chapter 11. Copy Services: FlashCopy
545
A FlashCopy consistency group can contain up to 512 FlashCopy mappings (up to the maximum number of FlashCopy mappings supported by the SVC Cluster). FlashCopy commands can then be issued to the FlashCopy consistency group and thereby simultaneously for all FlashCopy mappings defined in the consistency group. For example, when issuing a FlashCopy start command to the consistency group, all of the FlashCopy mappings in the consistency group are started at the same time, resulting in a PiT copy that is consistent across all of the FlashCopy mappings that are contained in the consistency group.
Consistency group with MTFC It is important to note that a consistency group aggregates FlashCopy mappings, not VDisks. Thus, where a source VDisk has multiple FlashCopy mappings, they can be in the same or different consistency groups. If a particular VDisk is the source VDisk for multiple FlashCopy mappings, you need to create separate consistency groups to separate each mapping of the same source VDisk. If the source VDisk with multiple target VDisks is in the same consistency group, then the result will be that when the consistency group is started, multiple identical copies of the VDisk will be created. However, this might be what the user wants, for example, they may want to run multiple simulations on the same set of source data, and this would be one way of obtaining identical sets of source data.
Consistency group zero For FlashCopy mappings where there is no need for the complexity of consistency groups, SVC allows a FlashCopy mapping to be treated as an independent entity. In this case, the FlashCopy mapping will become a member of the pseudo consistency group zero. For FlashCopy mappings that are configured in this way, the prepare and start commands are directed at the FlashCopy mapping name or FlashCopy mapping ID rather than the consistency group ID. A prepare or start command directed toward a FlashCopy mapping, which is a member of any other consistency group, is illegal and fails, and at the same time all operations to the pseudo consistency group will fail. For more information, see 11.5.4, “Preparing (pre-triggering) the FlashCopy mapping” on page 567.
Maximum configurations Table 11-1 shows the FlashCopy properties and maximum configurations. Table 11-1 FlashCopy properties and maximum configuration
546
FlashCopy property
Maximum
Comment
FC mappings per SVC cluster
4096
The number of mappings is no longer limited by the number of VDisks in the cluster and so the FC component limit applies.
FC consistency groups per SVC cluster
255
This is an arbitrary limit set by the SVC software.
FC VDisk per I/O group
1024 TB
There is a per I/O group limit of 1024 TB on the amount of source VDisk capacity that can participate in FC mappings.This maximum configuration will consume all 512 MB of bitmap space for the IO Group and allow no Metro and Global Mirror bitmap space. The default is 40 TB.
FC mappings per consistency group
512
This maximum exists because of the time taken to prepare a consistency group with a large number of mappings.
Implementing the IBM System Storage SAN Volume Controller V4.3
11.4.4 FlashCopy indirection layer The FlashCopy indirection layer governs the I/O to both the source and target VDisks when a FlashCopy mapping is started, which is done using a FlashCopy bitmap. The purpose of the FlashCopy indirection layer is to enable both the source and target VDisks for read and write I/O immediately after the FlashCopy has been started. To illustrate how the FlashCopy indirection layer works, we look at what happens when a FlashCopy mapping is prepared and subsequently started. When a FlashCopy mapping is prepared and started, the following sequence is applied: 1. Flush write data in cache onto a source VDisk or VDisks that are part of a consistency group. 2. Put cache into write-through on the source VDisk(s). 3. Discard cache for the target VDisk(s). 4. Establish a sync point on all source VDisks in the consistency group (creating the FlashCopy bitmap). 5. Ensure that the indirection layer governs all I/O to source and target VDisks. 6. Enable cache on both the source and target VDisks. FlashCopy provides the semantics of a PiT copy, using the indirection layer, which intercepts I/Os targeted at either the source or target VDisks. The act of starting a FlashCopy mapping causes this indirection layer to become active in the I/O path. This occurs as an atomic command across all FlashCopy mappings in the consistency group. The indirection layer makes a decision about each I/O. This decision is based upon: The VDisk and logical block number (LBA) to which the I/O is addressed Its direction (read or write) The state of an internal data structure, the FlashCopy bitmap The indirection layer either allows the I/O to go through the underlying storage, redirects the I/O from the target VDisk to the source VDisk, or stalls the I/O while it arranges for data to be copied from the source VDisk to the target VDisk. To explain in more detail which action is applied for each I/O, we first look at the FlashCopy bitmap.
Grains and the FlashCopy bitmap When data is copied from the source VDisk to the target VDisk or from target to target, it is copied in units of address space known as grains. In the SVC, the default grain size is 256 KB. The FlashCopy bitmap contains one bit for each grain. The bit records whether the associated grain has yet been split or not. Grain is split: The grain is already copied from the source to the target, or from the target to its dependency target. Grain is not split: The grain has not been copied from source to target, or from the target to its dependency target. The rate at which the grains are copied across from the source VDisk to the target VDisk is called the copy rate. By default, the copy rate is 50, although this can be altered. For more information about copy rates, see 11.4.12, “Space-efficient FlashCopy” on page 558.
Chapter 11. Copy Services: FlashCopy
547
The FlashCopy indirection layer algorithm Imagine the FlashCopy indirection layer as the I/O traffic cop when a FlashCopy mapping is active. The I/O is intercepted and handled according to whether it is directed at the source VDisk or the target VDisk, depending on the nature of the I/O (read or write) and the state of the grain (has it been copied or not). In Figure 11-5, we illustrate how the background copy runs while I/Os are handled according to the indirection layer algorithm.
Figure 11-5 I/O processing with FlashCopy
In the following sections, we describe how the FlashCopy indirection layer handles read and write I/O to the source and target VDisks, respectively.
Source reads Reads of the source are always passed through to the underlying source disk.
Target reads In order for FlashCopy to process a read from the target disk, it must consult its bitmap: If the data being read is already copied to the target (grain is split), then the read is sent to the target disk. If the data being read has not yet been copied (grain is unsplit), then the read is sent to the source disk or possibly to another target VDisk if multiple FlashCopy mappings exist for the source VDisk. Clearly, this algorithm requires that while this read is outstanding, no writes are allowed to execute that would change the data being read from the source. The SVC satisfies this requirement by a cluster-wide locking scheme at the grain level.
548
Implementing the IBM System Storage SAN Volume Controller V4.3
Writes to the source or target Where writes occur to source or target to an area (grain) that has not yet been copied, these will usually be held while a copy operation is performed to copy data from the source to the target (step 1 of Figure 11-5 on page 548), to maintain the illusion that the target contains its own copy. After step 1 is finished, the write I/O will be performed, as shown in step 2 in Figure 11-6. A specific optimization is performed where an entire grain is written to the target VDisk. In this case, the new grain contents are written to the target VDisk, and if this succeeds, then the grain is marked as split in the FlashCopy bitmap without a copy from the source to the target having been performed. If the write fails, then the grain is not marked as split. This is described further in “Write to target VDisk” on page 550.
11.4.5 Interaction and dependency between MTFC Figure 11-6 represents a set of four FlashCopy mappings that share a common source. The FlashCopy mappings will target VDisks Target 0, Target 1, Target 2, and Target 3.
Figure 11-6 Interactions between MTFC mappings
Target 0 is not dependent on a source because it has completed copying. Target 0 has two dependent mappings (Target 1 and Target 2). Target 1 is dependent upon Target 0. It will remain dependent until all of Target 1 has been copied. Target 2 is dependent on it since Target 2 is 20% copy complete. Once all of Target 1 has been copied, it can then move to the idle_copied state. Target 2 is dependent upon Target 0 and Target 1 and will remain dependent until all of Target 2 has been copied. No target is dependent on Target 2, so when all of it has been copied to Target 2, it can move to the idle_copied state. Target 3 has actually completed copying, so it is not dependent on any other maps.
Chapter 11. Copy Services: FlashCopy
549
Write to target VDisk A write to an intermediate or newest target VDisk must consider the state of the grain within its own mapping as well as that of the grain of the next oldest mapping: If the grain of the next oldest mapping has not yet been copied, then it must be copied before the write is allowed to proceed in order to preserve the contents of the next oldest mapping. The data written to the next oldest mapping comes from a target or source. If the grain in the target being written has not yet been copied, then the grain is copied from the oldest already copied grain in the mappings that are newer than it, or the source if none are already copied. Once this copy has been done, the write can be applied to the target.
Read to target VDisk If the grain being read has been split, then the read simply returns data from the target being read. If the read is to an uncopied grain on an intermediate target VDisk, then each of the newer mappings is examined in turn to see if the grain has been split. The read is surfaced from the first split grain found or from the source VDisk if none of the newer mappings has a split grain.
Stopping copy process An important scenario arises when a stop command is delivered to a mapping for a target that has dependent mappings. Once a mapping is in the stopped state, it can be deleted or restarted, and this must not be allowed if there are still grains that hold data that other mappings depend upon. To avoid this, when a mapping receives a stop command, rather than immediately moving to the stopped state, it enters the “stopping” state. An automatic copy process is driven that will find and copy all data uniquely held on the target VDisk of the mapping that is being stopped, to the next oldest mapping that is in the copying state. Note: The stopping copy process can be ongoing for several mappings sharing the same source at the same time. At the completion of this process, the mapping will automatically make an asynchronous state transition to the stopped state or the idle_copied state if the mapping was in the copying state with progress = 100%. For example, if the mapping associated with Target 0 was issued a stop command, then Target 0 would enter the stopping state while a process copied the data of Target 0 to Target 1. Once all the data has been copied, Target 0 would enter the stopped state, and Target 1 would no longer be dependent upon Target 0, but would remain dependent upon Target 2.
11.4.6 Summary of the FlashCopy indirection layer algorithm Table 11-2 summarizes the indirection layer algorithm. Table 11-2 Summary table of the FlashCopy indirection layer algorithm
550
VDisk being accessed
Has the grain been split (copied)?
Host I/O operation
Source
No
Read from source VDisk.
Copy grain to most recently started target for this source, then write to the source.
Yes
Read from source VDisk.
Write to source VDisk.
Read
Implementing the IBM System Storage SAN Volume Controller V4.3
Write
VDisk being accessed
Has the grain been split (copied)?
Target
Host I/O operation Read
Write
No
If any newer targets exist for this source in which this grain has already been copied, then read from the oldest of these. Otherwise, read from the source.
Hold the write. Check dependency target VDisks to see if the grain is split. If the grain is not already copied to the next oldest target for this source, then copy the grain to the next oldest target. Then, write to the target.
Yes
Read from target VDisk.
Write to target VDisk.
11.4.7 Interaction with the cache This copy-on-write process can introduce significant latency into write operations. In order to isolate the active application from this latency, the FlashCopy indirection layer is placed logically below the cache. This means that the copy latency is typically only seen when de-staged from the cache, rather than for write operations from an application; otherwise, the copy operation may be blocked waiting for the write to complete. In Figure 11-7, we illustrate the logical placement of the FlashCopy indirection layer.
Figure 11-7 Logical placement of the FlashCopy indirection layer
11.4.8 FlashCopy rules For SVC V4.3, the maximum number of supported FlashCopy mappings are 4096 per SVC cluster. This is slightly less than half of the number of supported VDisks, which is 8192/2=4096 per SVC cluster. This means that the maximum configuration does not allow all the VDisks to be part of a FlashCopy mapping. The following rules have to be considered while defining FlashCopy mappings: There is a one-to-one mapping of the source VDisk to the target VDisk. One source VDisk can have 256 target VDisks. The source and target VDisks can be in different I/O groups of the same cluster.
Chapter 11. Copy Services: FlashCopy
551
The minimum FlashCopy granularity is the entire VDisk. The source and target must be exactly equal in size. The size of a source and target VDisk cannot be altered (increased or decreased) after the FlashCopy mapping is created. There is a per I/O group limit of 1024 TB on the quantity of the source and target VDisk capacity that can participate in FlashCopy mappings.
11.4.9 FlashCopy and image mode disks You can use FlashCopy with an image mode VDisk. Since the source and target VDisks must be exactly the same size when creating a FlashCopy mapping, a VDisk must be created with the exact same size as the image mode VDisk. To accomplish this task, use the command svcinfo lsvdisk -bytes VDiskName. The size in bytes is then used to create the VDisk to be used in the FlashCopy mapping. In Example 11-1 we list the size of the VDisk vdisk_C. Subsequently, the VDisk vdisk_C_copy is created, specifying the same size. Example 11-1 Listing the size of a VDisk in bytes and creating a VDisk of equal size
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk Image_Vdisk_A id 8 name Image_Vdisk_A IO_group_id 0 IO_group_name io_grp0 status online mdisk_grp_id 2 mdisk_grp_name MDG_Image capacity 36.0GB type image formatted no mdisk_id 20 mdisk_name mdisk20 FC_id FC_name RC_id RC_name vdisk_UID 60050768018301BF280000000000000B throttling 0 preferred_node_id 1 fast_write_state empty cache readwrite udid fc_map_count 0 sync_rate 50 copy_count 1 copy_id 0 status online sync yes primary yes mdisk_grp_id 2 mdisk_grp_name MDG_Image type image mdisk_id 20 552
Implementing the IBM System Storage SAN Volume Controller V4.3
mdisk_name mdisk20 fast_write_state empty used_capacity 36.00GB real_capacity 36.00GB free_capacity 0.00MB overallocation 100 autoexpand warning grainsize IBM_2145:ITSO-CLS1:admin>svctask mkvdisk -size 36 -unit gb -name vdisk_A_copy -mdiskgrp MDG_DS47 -vtype striped -iogrp 1 Virtual Disk, id [19], successfully created Tip: Alternatively, the expand and shrink VDisk commands can be used to modify the size of the commands support specification of the size in bytes. See 9.9.7, “Expanding a VDisk” on page 359 and 9.9.14, “Shrinking a VDisk” on page 369 for more information. An image mode VDisk can be used as either a FlashCopy source or target VDisk.
11.4.10 FlashCopy mapping events In this section, we explain the series of events that modify the states of a FlashCopy. In Figure 11-8 on page 554, the FlashCopy mapping state diagram shows an overview of the states that apply to a FlashCopy mapping. Overview of a FlashCopy sequence of events: 1. Associate the source data set with a target location (one or more source and target VDisks). 2. Create a FlashCopy mapping for each source VDisk to the corresponding target VDisk. The target VDisk must be equal in size to the source VDisk. 3. Discontinue access to the target (application dependent). 4. Prepare (pre-trigger) the FlashCopy: a. Flush cache for the source. b. Discard cache for the target. 5. Start (trigger) the FlashCopy: a. Pause I/O (briefly) on the source. b. Resume I/O on the source. c. Start I/O on the target.
Chapter 11. Copy Services: FlashCopy
553
Figure 11-8 FlashCopy mapping state diagram Table 11-3 Mapping events
554
Mapping event
Description
Create
A new FlashCopy mapping is created between the specified source virtual disk (VDisk) and the specified target VDisk. The operation fails if any of the following is true: For SAN Volume Controller software Version 4.1.0 or earlier, the source or target VDisk is already a member of a FlashCopy mapping. For SAN Volume Controller software Version 4.2.0 or later, the source or target VDisk is already a target VDisk of a FlashCopy mapping. For SAN Volume Controller software Version 4.2.0 or later, the source VDisk is already a member of 16 FlashCopy mappings. For SAN Volume Controller software Version 4.3.0 or later, the source VDisk is already a member of 256 FlashCopy mappings. The node has insufficient bitmap memory. The source and target VDisks are different sizes.
Prepare
The prepare command is directed to either a consistency group for FlashCopy mappings that are members of a normal consistency group or to the mapping name for FlashCopy mappings that are stand-alone mappings. The prepare command places the FlashCopy mapping into the preparing state. Attention: The prepare command can corrupt any data that previously resided on the target VDisk because cached writes are discarded. Even if the FlashCopy mapping is never started, the data from the target might have logically changed during the act of preparing to start the FlashCopy mapping.
Flush done
The FlashCopy mapping automatically moves from the preparing state to the prepared state after all cached data for the source is flushed and all cached data for the target is no longer valid.
Implementing the IBM System Storage SAN Volume Controller V4.3
Mapping event
Description
Start
When all the FlashCopy mappings in a consistency group are in the prepared state, the FlashCopy mappings can be started. To preserve the cross volume consistency group, the start of all of the FlashCopy mappings in the consistency group must be synchronized correctly with respect to I/Os that are directed at the VDisks. This is achieved with the start command. The following occurs during the start command’s run: New reads and writes to all source VDisks in the consistency group are paused in the cache layer until all ongoing reads and writes below the cache layer are completed. After all FlashCopy mappings in the consistency group are paused, the internal cluster state is set to allow FlashCopy operations. After the cluster state is set for all FlashCopy mappings in the consistency group, read and write operations are unpaused on the source VDisks. The target VDisks are brought online. As part of the start command, read and write caching is enabled for both the source and target VDisks.
Modify
The following FlashCopy mapping properties can be modified: FlashCopy mapping name. Clean rate. Consistency group. Copy rate (for background copy). Automatic deletion of the mapping when the background copy is complete.
Stop
There are two separate mechanisms by which a FlashCopy mapping can be stopped: You have issued a command. An I/O error has occurred.
Delete
This command requests that the specified FlashCopy mapping be deleted. If the FlashCopy mapping is in the stopped state, the force flag must be used.
Flush failed
If the flush of data from the cache cannot be completed, the FlashCopy mapping enters the stopped state.
Copy complete
After all of the source data has been copied to the target and there are no dependent mappings, the state is set to copied. If the option to automatically delete the mapping after the background copy completes is specified, the FlashCopy mapping is automatically deleted. If this option is not specified, the FlashCopy mapping is not automatically deleted and can be reactivated by preparing and starting again.
Bitmap Online/Offline
The node has failed.
11.4.11 FlashCopy mapping states In this section, we explain the states of a FlashCopy mapping in more detail.
Idle_or_copied Read and write caching is enabled for both the source and the target. A FlashCopy mapping exists between the source and target, but they behave as independent VDisks in this state.
Chapter 11. Copy Services: FlashCopy
555
Copying The FlashCopy indirection layer governs all I/O to the source and target VDisks while the background copy is running. Reads and writes are executed on the target as though the contents of the source were instantaneously copied to the target during the start command. The source and target can be independently updated. Internally, the target depends on the source for some tracks. Read and write caching is enabled on the source and the target.
Stopped The FlashCopy was stopped either by user command or by an I/O error. When a FlashCopy mapping is stopped, any useful data in the target VDisk is lost. Because of this, while the FlashCopy mapping is in this state, the target VDisk is in the offline state. In order to regain access to the target, the mapping must be started again (the previous PiT will be lost) or the FlashCopy mapping must be deleted. The source VDisk is accessible and read/write caching is enabled for the source. In the stopped state, a mapping can be prepared again or it can be deleted.
Stopping The mapping is in the process of transferring data to an dependency mapping. The behavior of the target VDisk depends on whether the background copy process had completed while the mapping was in the copying state. If the copy process had completed, then the target VDisk remains online while the stopping copy process completes. If the copy process had not completed, then data in the cache is discarded for the target VDisk. The target VDisk is taken offline and the stopping copy process runs. When the data has been copied, then a stop complete asynchronous event is notified. The mapping will move to idle/copied state if the background copy has completed or to stopped if it has not. The source VDisk remains accessible for I/O.
Suspended The target has been “flashed” from the source, and was in the copying or stopping state. Access to the metadata has been lost, and as a consequence, both source and target VDisks are offline. The background copy process has been halted. When the metadata becomes available again, the FlashCopy mapping will return to the copying or stopping state, access to the source and target VDisks will be restored, and the background copy or stopping process resumed. Unflushed data that was written to the source or target before the FlashCopy was suspended is pinned in the cache, consuming resources, until the FlashCopy mapping leaves the suspended state.
Preparing Since the FlashCopy function is placed logically below the cache to anticipate any write latency problem, it demands no read or write data for the target and no write data for the source in the cache at the time that the FlashCopy operation is started. This ensures that the resulting copy is consistent. Performing the necessary cache flush as part of the start command unnecessarily delays the I/Os received after the start command is executed, since these I/Os must wait for the cache flush to complete.
556
Implementing the IBM System Storage SAN Volume Controller V4.3
To overcome this problem, SVC FlashCopy supports the prepare command, which prepares for a FlashCopy start while still allowing I/Os to continue to the source VDisk. In the preparing state, the FlashCopy mapping is prepared by the following steps: 1. Flushing any modified write data associated with the source VDisk from the cache. Read data for the source will be left in the cache. 2. Placing the cache for the source VDisk into write through mode, so that subsequent writes wait until data has been written to disk before completing the write command received from the using host. 3. Discarding any read or write data associated with the target VDisk from the cache. While in this state, writes to the source VDisk will experience additional latency because the cache is operating in write through mode. While the FlashCopy mapping is in this state, the target VDisk is reported as online, but will not perform reads or writes. These are failed by the SCSI front end. Before starting the FlashCopy mapping, it is important that any cache at the host level, for example, buffers in the host OSs or applications, are also instructed to flush any outstanding writes to the source VDisk.
Prepared When in the prepared state, the FlashCopy mapping is ready to perform a start. While the FlashCopy mapping is in this state, the target VDisk is in the offline state. In the prepared state, writes to the source VDisk experience additional latency because the cache is operating in write through mode.
Summary of FlashCopy mapping states Table 11-4 lists the various FlashCopy mapping states and the corresponding state of the source and target VDisks. Table 11-4 FlashCopy mapping state summary due to FlashCopy State
Source
Target
Online/offline
Cache state
Online/offline
Cache state
Idling/Copied
Online
Write-back
Online
Write-back
Copying
Online
Write-back
Online
Write-back
Stopped
Online
Write-back
Offline
-
Stopping
Online
Write-back
Online if copy complete Offline if copy not complete
-
Suspended
Offline
Write-back
Offline
-
Preparing
Online
Write-through
Online but not accessible
-
Prepared
Online
Write-through
Online but not accessible
-
Chapter 11. Copy Services: FlashCopy
557
11.4.12 Space-efficient FlashCopy You can have a mix of space-efficient and fully allocated VDisks in FlashCopy mappings. One common combination is a fully allocated source with a space-efficient target, which allows the target to consume a smaller amount of real storage than the source For best performance, the grain size of the space-efficient VDisk must match the grain size of the FlashCopy mapping. However, if the grain sizes are different, the mapping still proceeds. Consider the following information when you create your FlashCopy mappings: If you are using a fully allocated source with a space-efficient target, disable the background copy and cleaning mode on the FlashCopy map by setting both the background copy rate and cleaning rate to zero. Otherwise, if these features are enabled, all of the source is copied onto the target VDisk. This causes the space-efficient VDisk to either go offline or to grow as large as the source. If you are using only a space-efficient source, only the space that is used on the source VDisk is copied to the target VDisk. For example, if the source VDisk has a virtual size of 800 GB and a real size of 100 GB, of which 50 GB has been used, only the used 50 GB is copied.
Multiple space-efficient targets for FlashCopy The SVC implementation of multiple target FlashCopy ensures that when new data is written to a source or target, that data is copied to zero or one other targets. A consequence of this implementation is that space-efficient VDisks can be used in conjunction with multiple target FlashCopy without causing allocations to occur on multiple targets when data is written to the source.
Space-efficient incremental FlashCopy The implementation of space-efficient VDisks does not preclude the use of incremental FlashCopy on the same VDisks. It does not make much sense to have a fully allocated source VDisk and to use incremental FlashCopy to copy this to a space-efficient target VDisk; however, this combination is not prohibited. Two more interesting combinations of incremental FlashCopy and space-efficient VDisks are: A space-efficient source VDisk can be incrementally FlashCopied to a space-efficient target VDisk. Whenever the FlashCopy is retriggered, only data that has been modified is re-copied to the target. Note that if space is allocated on the target because of I/O to the target VDisk, this space is not reclaimed when the FlashCopy is retriggered. A fully allocated source VDisk can be incrementally FlashCopied to another fully allocated VDisk at the same time as being copied to multiple space-efficient targets (taken at different points in time). This allows a single full backup to be kept for recovery purposes and to separate the backup workload from the production workload and at the same time allowing older space-efficient backups to be retained.
Migration from and to a space-efficient VDisk There are two scenarios to migrate a non-space-efficient VDisk to a space-efficient VDisk and vice-versa. One is to migrate it by adding a VDisk copy mirror, as shown in 9.9.4, “Adding a mirrored VDisk copy” on page 351. The other possibility is to use FlashCopy to migrate a VDisk to a space-efficient VDisk or vice-versa. Refer to 11.5.13, “Migrate a VDisk to a space-efficient VDisk” on page 572 and 11.6.11, “Migration from a fully allocated VDisk to SEV and vice versa using a GUI” on page 597 to see the scenario of the migration process by FlashCopy.
558
Implementing the IBM System Storage SAN Volume Controller V4.3
11.4.13 Background copy The FlashCopy background feature enables you to copy all data in a source VDisk to the corresponding target VDisk. Without the FlashCopy background feature, only data that changed on the source VDisk can be copied to the target VDisk. The benefit of using FlashCopy mapping with background copy enabled is that the target VDisk becomes a real clone (independent from the source VDisk) of the FlashCopy mapping source VDisk. The background copy rate is a property of a FlashCopy mapping that is expressed as a value between 0 and 100. It can be changed in any FlashCopy mapping state and can be different in the mappings of one consistency group. A value of 0 disables background copy. The relationship of the background copy rate value to the attempted number of grains to be split (copied) per second is shown in Table 11-5. Table 11-5 Background copy rate Value
Data copied per second
Grains per second
1-10
128 KB
0.5
11-20
256 KB
1
21-30
512 KB
2
31-40
1 MB
4
41-50
2 MB
8
51-60
4 MB
16
61-70
8 MB
32
71-80
16 MB
64
81-90
32 MB
128
91-100
64 MB
256
The grains per second numbers represent the maximum number of grains the SVC will copy per second, assuming that the bandwidth to the MDisks can accommodate this rate. The SVC is unable to achieve these copy rates if insufficient bandwidth is available from the SVC nodes to the physical disks making up the managed disks, after taking into account the requirements of foreground I/O. If this situation arises, then background copy I/O contends for resources on an equal basis with I/O arriving from hosts. Both tend to see an increase in latency, and a consequential reduction in throughput with respect to the situation had the bandwidth not been limited. Degradation is graceful. Both background copy and foreground I/O continue to make forward progress, and do not stop, hang, or cause the node to fail. The background copy is performed by both nodes of the I/O group in which the source VDisk resides.
11.4.14 Synthesis The FlashCopy functionality in SVC simply creates copy VDisks. All the data in the source VDisk is copied to the destination VDisk. This includes operating system control information as well as application data and metadata.
Chapter 11. Copy Services: FlashCopy
559
Some operating systems are unable to use FlashCopy without an additional step, which is termed synthesis. In general, synthesis performs some transformation on the operating system metadata in the target VDisk so that the operating system can use the disk. Operating system specifics are discussed in Appendix A, “Copy Services and open systems” on page 843.
11.4.15 Serialization of I/O by FlashCopy In general, the FlashCopy function in the SVC introduces no explicit serialization into the I/O path. Therefore, many concurrent I/Os are allowed to the source and target VDisks. However, there is a lock for each grain. The lock can be taken shared or exclusive. For multiple targets, a common lock is shared and the mappings are derived from a particular source VDisk. The lock is taken in the following modes under the following conditions: The lock is taken shared for the duration of a read from the target VDisk which touches a grain that is not split. The lock is taken exclusive during a grain split. This happens prior to FlashCopy actioning any destage (or write through) from the cache to a grain that is going to be split (the destage waits for the grain to be split). The lock is held during the grain split and released before the destage is processed. If the lock is held shared, and another process wants to take the lock shared, then this request is granted unless a process is already waiting to take the lock exclusive. If the lock is held shared and it is requested exclusive, then the requesting process must wait until all holders of the shared lock free it. Similarly, if the lock is held exclusive, then a process wanting to take the lock in either shared or exclusive mode must wait for it to be freed.
11.4.16 Error handling When a FlashCopy mapping is not copying or stopping, the FlashCopy function does not affect the error handling or reporting of errors in the I/O path. Only when a FlashCopy mapping is copying or stopping are error handling and reporting affected by FlashCopy. We describe these scenarios in the following sections.
Node failure Normally, two copies of the FlashCopy bitmaps are maintained, one on each of the two nodes making up the I/O group of the source VDisk. When a node fails, one copy of the bitmaps, for all FlashCopy mappings whose source VDisk is a member of the failing node’s I/O group, will become inaccessible. FlashCopy will continue with a single copy of the FlashCopy bitmap being stored as non-volatile in the remaining node in the source I/O group. The cluster metadata is updated to indicate that the missing node no longer holds up-to-date bitmap information. When the failing node recovers, or a replacement node is added to the I/O group, up-to-date bitmaps will be re-established on the new node, and it will once again provide a redundant location for the bitmaps. When the FlashCopy bitmap becomes available again (at least one of the SVC nodes in the I/O group is accessible), the FlashCopy mapping will return to the copying state, access to the source and target VDisks will be restored, and the background copy process resumed. Unflushed data that was written to the source or target before the FlashCopy 560
Implementing the IBM System Storage SAN Volume Controller V4.3
was suspended is pinned in the cache until the FlashCopy mapping leaves the suspended state. Normally, two copies of the FlashCopy bitmaps are maintained (in non-volatile memory), one on each of the two SVC nodes making up the I/O group of the source VDisk. If only one of the SVC nodes in the I/O group that the source VDisk belongs to goes offline, then the FlashCopy mapping will continue in the copying state, with a single copy of the FlashCopy bitmap. When the failed SVC node recovers, or a replacement SVC node is added to the I/O group, up-to-date FlashCopy bitmaps will be re-established on the resuming SVC node, and again provide a redundant location for the FlashCopy bitmaps. Note: If both nodes in the I/O group to which the target VDisk belongs become unavailable, then host cannot access the target VDisk.
Path failure (path offline state) In a fully functioning cluster, all nodes have a software representation of every VDisk in the cluster within their application hierarchy. Since the SAN that links the SVC nodes to each other and to the managed disks is made up of many independent links, it is possible for some subset of the nodes to be temporarily isolated from some of the managed disks. When this happens, the managed disks are said to be path offline on some nodes. Note: Other nodes might see the managed disks as online, because their connection to the managed disks is still functioning. When a managed disk enters the path offline state on an SVC node, all the VDisks that have any extents on the managed disk also become path offline. Again, this happens only on the affected nodes. When a VDisk is path offline on a particular SVC node, this means that host access to that VDisk through the node will fail with the SCSI sense indicating offline.
Path offline for the source VDisk If a FlashCopy mapping is in the copying state and the source VDisk goes path offline, then this path offline state is propagated to all target VDisks up to but not including the target VDisk for the newest mapping that is 100% copied but remains in the copying state. If no mappings are 100% copied, then all target VDisks will be taken offline. Again, note that path offline is a state that exists on a per-node basis. Other nodes may not be affected. If the source VDisk comes online, then the target and source VDisks are brought back online.
Path offline for the target VDisk If a target VDisk goes path offline, but the source VDisk is still online, and if there are any dependent mappings, then those target VDisks will also go path offline. The source VDisk will remain online.
Chapter 11. Copy Services: FlashCopy
561
11.4.17 Asynchronous notifications FlashCopy raises informational error logs when mappings or consistency groups make certain state transitions. These state transitions occur as a result of configuration events that complete asynchronously, and the informational errors can be used to generate Simple Network Management Protocol (SNMP) traps to notify the user. Other configuration events complete synchronously, and no informational errors are logged as a result of these events: PREPARE_COMPLETED: This is logged when the FlashCopy mapping or consistency group enters the prepared state as a result of a user request to prepare. The user can now start (or stop) the mapping or consistency group. COPY_COMPLETED: This is logged when the FlashCopy mapping or consistency group enters the idle_or_copied state when it was previously in the copying or stopping state. This indicates that the target disk now contains a complete copy and no longer depends on the source. STOP_COMPLETED: This is logged when the FlashCopy mapping or consistency group has entered the stopped state as a result of a user request to stop. It will be logged once the automatic copy process has completed. This includes mappings where no copying needed to be performed. It is different from the error that is logged when a mapping or group enters the stopped state as a result of an IO error.
11.4.18 Interoperation with Metro Mirror and Global Mirror FlashCopy can work together with Metro Mirror and Global Mirror to provide better protection of data. For example, we can perform a Metro Mirror to duplicate data from Site_A to Site_B, then do a daily FlashCopy and copy it to tape. Table 11-6 details which combinations of FlashCopy and Remote Copy are supported. In the table, Remote Copy refers to Metro Mirror and Global Mirror. Table 11-6 FlashCopy Remote Copy interaction Component
Remote Copy Primary
Remote Copy Secondary
FlashCopy Source
Supported
Supported Note: When the FlashCopy relationship is in the preparing and prepared states, the cache at the Remote Copy secondary site will be operating in write through mode. This will add additional latency to the already latent Remote Copy Relationship.
FlashCopy Destination
Not supported
Not Supported
11.4.19 Recovering data from FlashCopy FlashCopy PiT can be used to recover the data if some form of corruption has happened. For example, if a user deletes some data by mistake, you can map the FlashCopy target VDisks to the application server, and import all the logical volume level configurations, start the application, and restore the data back to a given point in time.
562
Implementing the IBM System Storage SAN Volume Controller V4.3
Tip: It is better to map a FlashCopy target VDisk to a backup machine with the same application installed. We do not recommend that you map a FlashCopy target VDisk to the same application server that the FlashCopy source VDisk is mapped. The reason is that the FlashCopy target and source VDisk have same signature, pvid, vgda, and so on. Special steps will be necessary to handle the conflict at OS level. For example, you can use the command recreatevg in AIX to generate different vg, lv, file system, and so on, names in order to avoid a naming conflict. FlashCopy backup is a disk-based backup copy that can be used to restore service more quickly than other backup techniques. This application is further enhanced by the ability to maintain multiple backup targets, spread over a range of time, allowing the user to choose a backup from before the time of corruption.
11.5 Using the command line to perform FlashCopy In this section, we use a scenario to illustrate how to use commands with PuTTY to perform FlashCopy. Refer to the IBM System Storage SAN Volume Controller: Command-Line Interface User’s Guide for more commands, which is available at: http://www-1.ibm.com/support/docview.wss?rs=591&context=STCCCXR&context=STCCCYH&dc =DA400&uid=ssg1S7002157&loc=en_US&cs=utf-8&lang=en
11.5.1 Scenario description We use the following scenario in both the command line section and the GUI section. In the following scenario, we want to FlashCopy the following VDisks: DB_Source
Database files
Log_Source
Database log files
App_Source
Application files
Since data integrity must be kept on DB_Source and Log_Source, we create consistency groups to handle the FlashCopy of DB_Source and Log_Source.
Chapter 11. Copy Services: FlashCopy
563
In our scenario, the application files are independent of the database, so we create a single FlashCopy mapping for App_Source. We will make two FlashCopy targets for DB_Source and Log_Source, and thereby two consistency groups. The scenario is shown in Example 11-9 on page 568.
Figure 11-9 FlashCopy scenario using the CLI
Setting up FlashCopy We have already created the source and target VDisks, and the source and target VDisks are identical in size, which is a requirement of the FlashCopy function: DB_Source, DB_Target1, and DB_Target2 Log_Source, Log_Target1, and Log_Target2 App_Source and App_Target1 To set up the FlashCopy, we performed the following steps: 1. Create a FlashCopy consistency group: – Named FCCG1 – Named FCCG2 2. Create FlashCopy mapping for Source VDisks: – DB_Source FlashCopy to DB_Target1, the mapping name is DB_Map1 – DB_Source FlashCopy to DB_Target2, the mapping name is DB_Map2 – Log_Source FlashCopy to Log_Target1, the mapping name is Log_Map1 – Log_Source FlashCopy to Log_Target2, the mapping name is Log_Map2 – App_Source FlashCopy to App_Target1, the mapping name is App_Map1 – Copyrate 50
11.5.2 Creating a FlashCopy consistency group To create a FlashCopy consistency group, we use the command svctask mkfcconsistgrp to create a new consistency group. The ID of the new group is returned. If you have created several FlashCopy mappings for a group of VDisks that contain elements of data for the same application, you may find it convenient to assign these mappings to a single FlashCopy consistency group. Then you can issue a single prepare or start command for the whole group, so that, for example, all the files for a particular database are copied at the same time. 564
Implementing the IBM System Storage SAN Volume Controller V4.3
In Example 11-2, the consistency group FCCG1 and FCCG2 are created, which will hold the FlashCopy maps of DB and Log together. This is a very important step in doing FlashCopy on database applications. It helps to keep data integrity during FlashCopy. Example 11-2 Creating two FlashCopy consistency groups
IBM_2145:ITSO-CLS1:admin>svctask mkfcconsistgrp -name FCCG1 FlashCopy Consistency Group, id [1], successfully created IBM_2145:ITSO-CLS1:admin>svctask mkfcconsistgrp -name FCCG2 FlashCopy Consistency Group, id [2], successfully created In Example 11-3, we checked the status of consistency groups. Each has a status of Idle_or_copied. Example 11-3 Check FlashCopy consistency group
IBM_2145:ITSO-CLS1:admin>svcinfo lsfcconsistgrp id name status 1 FCCG1 idle_or_copied 2 FCCG2 idle_or_copied If you would like to change the name of a consistency group, you can use the svctask chfcconsistgrp command. Type svctask chfcconsistgrp -h for help with this command.
11.5.3 Creating a FlashCopy mapping To create a FlashCopy mapping, we use the command svctask mkfcmap. This command creates a new FlashCopy mapping, which maps a source virtual disk to a target virtual disk to prepare for subsequent copying. When executed, this command creates a new FlashCopy mapping logical object. This mapping persists until it is deleted. The mapping specifies the source and destination virtual disks. The destination must be identical in size to the source, or the mapping will fail. Issue the command svcinfo lsvdisk -bytes to find the exact size of the source VDisk for which you want to create a target disk of the same size. In a single mapping, source and destination cannot be on the same VDisk. A mapping is triggered at the point in time when the copy is required. The mapping can optionally be given a name and assigned to a consistency group. These are groups of mappings that can be triggered at the same time. This enables multiple virtual disks to be copied at the same time, which creates a consistent copy of multiple disks. This is required for database products in which the database and log files reside on different disks. If no consistency group is defined, the mapping is assigned into the default group 0. This is a special group that cannot be started as a whole. Mappings in this group can only be started on an individual basis. The background copy rate specifies the priority that should be given to completing the copy. If 0 is specified, the copy will not proceed in the background. The default is 50.
Chapter 11. Copy Services: FlashCopy
565
Tip: There is a parameter to delete FlashCopy mappings automatically after completion of a background copy (when the mapping gets to the idle_or_copied state). Use the command: svctask mkfcmap -autodelete This command does not delete mappings in cascade with dependent mappings because it would not get to the idle_or_copied state. In Example 11-4, the first FlashCopy mapping for DB_Source and Log_Source is created. Example 11-4 Create the first FlashCopy mapping for DB_Source, Log_Source, and App_Source
IBM_2145:ITSO-CLS1:admin>svctask mkfcmap -source DB_Source -target DB_Target1 -name DB_Map1 -consistgrp FCCG1 FlashCopy Mapping, id [0], successfully created IBM_2145:ITSO-CLS1:admin>svctask mkfcmap -source Log_Source -target Log_Target1 -name Log_Map1 -consistgrp FCCG1 FlashCopy Mapping, id [1], successfully created IBM_2145:ITSO-CLS1:admin>svctask mkfcmap -source App_Source -target App_Target1 -name App_Map1 FlashCopy Mapping, id [2], successfully created Example 11-5 shows the command to create a second FlashCopy mapping for VDisk DB_Source and Log_Source. Example 11-5 Create additional FlashCopy mappings
IBM_2145:ITSO-CLS1:admin>svctask mkfcmap -source DB_Source -target DB_Target2 -name DB_Map2 -consistgrp FCCG2 FlashCopy Mapping, id [3], successfully created IBM_2145:ITSO-CLS1:admin>svctask mkfcmap -source Log_Source -target Log_Target2 -name Log_Map2 -consistgrp FCCG2 FlashCopy Mapping, id [4], successfully created Example 11-6 shows the result of these FlashCopy mappings. The status of the mapping is idle_or_copied. Example 11-6 Check the result of Multi-Target FlashCopy mappings
IBM_2145:ITSO-CLS1:admin>svcinfo lsfcmap -delim , id,name,source_vdisk_id,source_vdisk_name,target_vdisk_id,target_vdisk_name,group_ id,group_name,status,progress,copy_rate,clean_progress,incremental 0,DB_Map1,20,DB_Source,21,DB_Target1,1,FCCG1,idle_or_copied,0,50,100,off 1,Log_Map1,23,Log_Source,24,Log_Target1,1,FCCG1,idle_or_copied,0,50,100,off 2,App_Map1,28,App_Source,27,App_Target1,,,idle_or_copied,0,50,100,off 3,DB_Map2,20,DB_Source,22,DB_Target2,2,FCCG2,idle_or_copied,0,50,100,off 4,Log_Map2,23,Log_Source,25,Log_Target2,2,FCCG2,idle_or_copied,0,50,100,off If you would like to change the FlashCopy mapping, you can use the svctask chfcmap command. Type svctask chfcmap -h to get help with this command.
566
Implementing the IBM System Storage SAN Volume Controller V4.3
11.5.4 Preparing (pre-triggering) the FlashCopy mapping At this point, the mapping has been created, but the cache is still accepting data for the source VDisks. You can only trigger the mapping when the cache does not contain any data for FlashCopy source VDisks. You must issue an svctask prestartfcmap command to prepare a FlashCopy mapping to start. This command tells SVC to flush the cache of any content for the source VDisk and to pass through any further write data for this VDisk. When svctask prestartfcmap is executed, the mapping enters the preparing state. After the preparation is complete, it changes to the prepared state. At this point, the mapping is ready for triggering. Preparing and the subsequent triggering is usually performed on a consistency group basis. Only mappings belonging to consistency group 0 can be prepared on their own, since consistency group 0 is a special group, which contains the FlashCopy mappings that do not belong to any consistency group. A FlashCopy must be prepared before it can be triggered. In our scenario, App_Map1 is not in a consistency group. In Example 11-7, we show how we initialize the preparation for App_Map1. Another option is that you add the -prep parameter to the svctask startfcmap command, which will first prepare the mapping, then start the FlashCopy. In the example, we also show how to check the status of the current FlashCopy mapping. App_Map1’s status is prepared. Example 11-7 Prepare a FlashCopy without consistency group
IBM_2145:ITSO-CLS1:admin>svctask prestartfcmap App_Map1 IBM_2145:ITSO-CLS1:admin>svcinfo lsfcmap -delim , id,name,source_vdisk_id,source_vdisk_name,target_vdisk_id,target_vdisk_name,group_ id,group_name,status,progress,copy_rate,clean_progress,incremental 0,DB_Map1,20,DB_Source,21,DB_Target1,1,FCCG1,idle_or_copied,0,50,100,off 1,Log_Map1,23,Log_Source,24,Log_Target1,1,FCCG1,idle_or_copied,0,50,100,off 2,App_Map1,28,App_Source,27,App_Target1,,,prepared,0,50,100,off 3,DB_Map2,20,DB_Source,22,DB_Target2,2,FCCG2,idle_or_copied,0,50,100,off 4,Log_Map2,23,Log_Source,25,Log_Target2,2,FCCG2,idle_or_copied,0,50,100,off
11.5.5 Preparing (pre-triggering) the FlashCopy consistency group We use the command svctask prestartfcconsistsgrp to prepare a FlashCopy consistency group. As with 11.5.4, “Preparing (pre-triggering) the FlashCopy mapping” on page 567, this command flushes the cache of any data destined for the source VDisks and forces the cache into the write-through mode until the mapping is started. The difference is that this command prepares a group of mappings (at a consistency group level) instead of one mapping. When you have assigned several mappings to a FlashCopy consistency group, you only have to issue a single prepare command for the whole group to prepare all the mappings at once.
Chapter 11. Copy Services: FlashCopy
567
Example 11-8 shows how we prepare the consistency groups for DB and Log and check the result. After the command has executed all the FlashCopy maps we have, all of them are in prepared status, and all the consistency groups are in prepared status too. Now we are ready to start the FlashCopy. Example 11-8 Prepare a FlashCopy with consistency group
IBM_2145:ITSO-CLS1:admin>svctask prestartfcconsistgrp FCCG1 IBM_2145:ITSO-CLS1:admin>svctask prestartfcconsistgrp FCCG2 IBM_2145:ITSO-CLS1:admin>svcinfo lsfcmap -delim , id,name,source_vdisk_id,source_vdisk_name,target_vdisk_id,target_vdisk_name,group_ id,group_name,status,progress,copy_rate,clean_progress,incremental 0,DB_Map1,20,DB_Source,21,DB_Target1,1,FCCG1,prepared,0,50,100,off 1,Log_Map1,23,Log_Source,24,Log_Target1,1,FCCG1,prepared,0,50,100,off 2,App_Map1,28,App_Source,27,App_Target1,,,prepared,0,50,100,off 3,DB_Map2,20,DB_Source,22,DB_Target2,2,FCCG2,prepared,0,50,100,off 4,Log_Map2,23,Log_Source,25,Log_Target2,2,FCCG2,prepared,0,50,100,off IBM_2145:ITSO-CLS1:admin>svcinfo lsfcconsistgrp id name status 1 FCCG1 prepared 2 FCCG2 prepared
11.5.6 Starting (triggering) FlashCopy mappings The command svctask startfcmap is used to start a single FlashCopy mapping. When invoked, a PiT copy of the source VDisk is created on the target VDisk. When the FlashCopy mapping is triggered, it enters the copying state. The way the copy proceeds depends on the background copy rate attribute of the mapping. If the mapping is set to 0 (NOCOPY), only data that is subsequently updated on the source will be copied to the destination. This operation means that the destination can only be used as a backup copy while the mapping exists in the copying state. If the copy is stopped, the destination will not be usable. If you want to end up with a duplicate copy of the source at the destination, you should set the background copy rate greater than 0. This means that the system copies all the data (even unchanged data) to the destination and eventually reaches the idle or copied state. After this data is copied, you can delete the mapping and have a usable point-in-time copy of the source at the destination. Immediately after the quiesce, we execute the command svctask startfcmap, as shown in Example 11-9. After the FlashCopy is started, App_Map1 changes to copying status. Example 11-9 Start App_Map1
IBM_2145:ITSO-CLS1:admin>svctask startfcmap App_Map1 IBM_2145:ITSO-CLS1:admin>svcinfo lsfcmap -delim , id,name,source_vdisk_id,source_vdisk_name,target_vdisk_id,target_vdisk_name,group_ id,group_name,status,progress,copy_rate,clean_progress,incremental 0,DB_Map1,20,DB_Source,21,DB_Target1,1,FCCG1,prepared,0,50,100,off 1,Log_Map1,23,Log_Source,24,Log_Target1,1,FCCG1,prepared,0,50,100,off 2,App_Map1,28,App_Source,27,App_Target1,,,copying,3,50,100,off 3,DB_Map2,20,DB_Source,22,DB_Target2,2,FCCG2,prepared,0,50,100,off
568
Implementing the IBM System Storage SAN Volume Controller V4.3
4,Log_Map2,23,Log_Source,25,Log_Target2,2,FCCG2,prepared,0,50,100,off
11.5.7 Starting (triggering) FlashCopy consistency group We execute the command svctask startfcconsistgrp, as shown in Example 11-10, and afterwards the database can be resumed. We have created two PiT consistent copies of the DB and Log VDisks. After execution, the status of consistency group and FlashCopy maps are all in copying status. Example 11-10 Start FlashCopy consistency group
IBM_2145:ITSO-CLS1:admin>svctask startfcconsistgrp FCCG1 IBM_2145:ITSO-CLS1:admin>svctask startfcconsistgrp FCCG2 IBM_2145:ITSO-CLS1:admin>svcinfo lsfcconsistgrp id name status 1 FCCG1 copying 2 FCCG2 copying IBM_2145:ITSO-CLS1:admin>svcinfo lsfcmap -delim , id,name,source_vdisk_id,source_vdisk_name,target_vdisk_id,target_vdisk_name,group_ id,group_name,status,progress,copy_rate,clean_progress,incremental 0,DB_Map1,20,DB_Source,21,DB_Target1,1,FCCG1,copying,12,50,100,off 1,Log_Map1,23,Log_Source,24,Log_Target1,1,FCCG1,copying,3,50,100,off 2,App_Map1,28,App_Source,27,App_Target1,,,copying,3,50,100,off 3,DB_Map2,20,DB_Source,22,DB_Target2,2,FCCG2,copying,12,50,100,off 4,Log_Map2,23,Log_Source,25,Log_Target2,2,FCCG2,copying,3,50,100,off
11.5.8 Monitoring the FlashCopy progress To monitor the background copy progress of the FlashCopy mappings, we issue the command svcinfo lsfcmapprogress for each FlashCopy mapping. Alternatively, the copy progress can also be queried using the command svcinfo lsfcmap. As shown in Example 11-11, both DB_Map1 and Log_Map1 return that the background copy is 21% completed, and both DB_Map2 and Log_Map2 return that the background copy is 18% completed. Example 11-11 Monitoring background copy progress
IBM_2145:ITSO-CLS1:admin>svcinfo lsfcmapprogress DB_Map1 id progress 0 21 IBM_2145:ITSO-CLS1:admin>svcinfo lsfcmapprogress Log_Map1 id progress 1 18 IBM_2145:ITSO-CLS1:admin>svcinfo lsfcmapprogress DB_Map2 id progress 3 21 IBM_2145:ITSO-CLS1:admin>svcinfo lsfcmapprogress Log_Map2 id progress
Chapter 11. Copy Services: FlashCopy
569
4
18
IBM_2145:ITSO-CLS1:admin>svcinfo lsfcmapprogress App_Map1 id progress 2 50 When the background copy has completed, the FlashCopy mapping enters the idle_or_copied state, and when all FlashCopy mappings in a consistency group enter this status, the consistency group will be at idle_or_copied status. When in this state, the FlashCopy mapping can be deleted, and the target disk can be used independently, if, for example, another target disk is to be used for the next FlashCopy of the particular source VDisk.
11.5.9 Stopping the FlashCopy mapping The command svctask stopfcmap is used to stop a FlashCopy mapping. This command allows you to stop an active (copying) or suspended mapping. When executed, this command stops a single FlashCopy mapping. When a FlashCopy mapping is stopped, the target VDisk becomes invalid and is set offline by the SVC. The FlashCopy mapping needs to be prepared again, or re-triggered to bring the target VDisk online again. Tip: In a multi-target FlashCopy environment, if you want to stop a mapping or group, consider whether you want to keep any of the dependent mappings. If not, you should issue the stop command with the force parameter, which will stop all of the dependent maps and negate the need for the stopping copy process to run.
Note: Stopping a FlashCopy mapping should only be done when the data on the target VDisk is of no use, or you want to modify the FlashCopy mapping. When a FlashCopy mapping is stopped, the target VDisk becomes invalid and is set offline by the SVC, if the mapping is in the copying state and progress !=100. As shown in Example 11-12, we stop App_Map1 FlashCopy. The status of App_Map1 has changed to idle_or_copied. Example 11-12 Stop APP_Map1 FlashCopy
IBM_2145:ITSO-CLS1:admin>svctask stopfcmap App_Map1 IBM_2145:ITSO-CLS1:admin>svcinfo lsfcmap -delim , id,name,source_vdisk_id,source_vdisk_name,target_vdisk_id,target_vdisk_name,group_ id,group_name,status,progress,copy_rate,clean_progress,incremental 0,DB_Map1,20,DB_Source,21,DB_Target1,1,FCCG1,idle_or_copied,100,50,100,off 1,Log_Map1,23,Log_Source,24,Log_Target1,1,FCCG1,copying,3,50,100,off 2,App_Map1,28,App_Source,27,App_Target1,,,stopped,0,50,100,off 3,DB_Map2,20,DB_Source,22,DB_Target2,2,FCCG2,idle_or_copied,100,50,100,off 4,Log_Map2,23,Log_Source,25,Log_Target2,2,FCCG2,copying,3,50,100,off
570
Implementing the IBM System Storage SAN Volume Controller V4.3
11.5.10 Stopping the FlashCopy consistency group The command svctask stopfcconsistgrp is used to stop any active FlashCopy consistency group. It stops all mappings in a consistency group. When a FlashCopy consistency group is stopped for all mappings that are not 100% copied, the target VDisks become invalid and are set offline by the SVC. The FlashCopy consistency group needs to be prepared again and restarted to bring the target VDisks online again. Note: Stopping a FlashCopy mapping should only be done when the data on the target VDisk is of no use, or you want to modify the FlashCopy consistency group. When a consistency group is stopped, the target VDisk might become invalid and set offline by the SVC depending on the state of the mapping. As shown in Example 11-13, we stop FCCG1 and FCCG2 consistency groups. The status of the two consistency groups has changed to stopped. Most of the FC mapping relations now have the status stopped. As you can see, some of them have already completed the copy operation and are now in a status of idle_or_copied. Example 11-13 Stop FCCG1 and FCCG2 consistency group
IBM_2145:ITSO-CLS1:admin>svctask stopfcconsistgrp FCCG1 IBM_2145:ITSO-CLS1:admin>svctask stopfcconsistgrp FCCG2 IBM_2145:ITSO-CLS1:admin>svcinfo lsfcconsistgrp id name status 1 FCCG1 stopped 2 FCCG2 stopped IBM_2145:ITSO-CLS1:admin>svcinfo lsfcmap -delim , id,name,source_vdisk_id,source_vdisk_name,target_vdisk_id,target_vdisk_name,group_ id,group_name,status,progress,copy_rate,clean_progress,incremental 0,DB_Map1,20,DB_Source,21,DB_Target1,1,FCCG1,idle_or_copied,100,50,100,off 1,Log_Map1,23,Log_Source,24,Log_Target1,1,FCCG1,stopped,0,50,100,off 2,App_Map1,28,App_Source,27,App_Target1,,,stopped,0,50,100,off 3,DB_Map2,20,DB_Source,22,DB_Target2,2,FCCG2,idle_or_copied,100,50,100,off 4,Log_Map2,23,Log_Source,25,Log_Target2,2,FCCG2,stopped,0,50,100,off
11.5.11 Deleting the FlashCopy mapping To delete a FlashCopy mapping, we use the command svctask rmfcmap. When the command is executed, it attempts to delete the FlashCopy mapping specified. If the FlashCopy mapping is stopped, the command fails unless the -force flag is specified. If the mapping is active (copying), then it must first be stopped before it can be deleted. Deleting a mapping only deletes the logical relationship between the two VDisks. However, when issued on an active FlashCopy mapping using the -force flag, the delete renders the data on the FlashCopy mapping target VDisk as inconsistent. Tip: If you want to use the target VDisk as normal VDisks, monitor the background copy progress until it is complete (100% copied), and then delete the FlashCopy mapping. Another option is to set -autodelete option when creating the FlashCopy mapping.
Chapter 11. Copy Services: FlashCopy
571
As shown in Example 11-14, we delete App_Map1. Example 11-14 Delete App_Map1
IBM_2145:ITSO-CLS1:admin>svctask rmfcmap App_Map1 IBM_2145:ITSO-CLS1:admin>svcinfo lsfcmap -delim , id,name,source_vdisk_id,source_vdisk_name,target_vdisk_id,target_vdisk_name,group_ id,group_name,status,progress,copy_rate,clean_progress,incremental 0,DB_Map1,20,DB_Source,21,DB_Target1,1,FCCG1,idle_or_copied,100,50,100,off 1,Log_Map1,23,Log_Source,24,Log_Target1,1,FCCG1,idle_or_copied,100,50,100,off 3,DB_Map2,20,DB_Source,22,DB_Target2,2,FCCG2,idle_or_copied,100,50,100,off 4,Log_Map2,23,Log_Source,25,Log_Target2,2,FCCG2,idle_or_copied,100,50,100,off
11.5.12 Deleting the FlashCopy consistency group The command svctask rmfcconsistgrp is used to delete a FlashCopy consistency group. When executed, this command deletes the consistency group specified. If there are mappings that are members of the group, the command fails unless the -force flag is specified. If you want to delete all the mappings in the consistency group as well, you must first delete the mappings, and then delete the consistency group. As shown in Example 11-15, we delete all the maps and consistency groups, and then we check the result. Example 11-15 remove fcmaps and fcconsistgrp
IBM_2145:ITSO-CLS1:admin>svctask rmfcmap DB_Map1 IBM_2145:ITSO-CLS1:admin>svctask rmfcmap DB_Map2 IBM_2145:ITSO-CLS1:admin>svctask rmfcmap Log_Map1 IBM_2145:ITSO-CLS1:admin>svctask rmfcmap Log_Map2 IBM_2145:ITSO-CLS1:admin>svctask rmfcconsistgrp FCCG1 IBM_2145:ITSO-CLS1:admin>svctask rmfcconsistgrp FCCG2 IBM_2145:ITSO-CLS1:admin>svcinfo lsfcconsistgrp IBM_2145:ITSO-CLS1:admin>svcinfo lsfcmap IBM_2145:ITSO-CLS1:admin>
11.5.13 Migrate a VDisk to a space-efficient VDisk Use the following scenario to migrate a VDisk to a space-efficient VDisk. 1. Create a space-efficient target VDisk with exactly the same size as the VDisk you want to migrate, as shown in Example 11-16 on page 573.
572
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 11-16 svcinfo lsvdisk IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk vdisk_B id 1 name vdisk_B IO_group_id 1 IO_group_name io_grp1 status online mdisk_grp_id 1 mdisk_grp_name MDG_DS47 capacity 3.0GB type striped formatted yes mdisk_id mdisk_name FC_id FC_name RC_id RC_name vdisk_UID 60050768018301BF2800000000000001 throttling 0 preferred_node_id 6 fast_write_state empty cache readwrite udid fc_map_count 0 sync_rate 50 copy_count 1 copy_id 0 status online sync yes primary yes mdisk_grp_id 1 mdisk_grp_name MDG_DS47 type striped mdisk_id mdisk_name fast_write_state empty used_capacity 3.00GB real_capacity 3.00GB free_capacity 0.00MB overallocation 100 autoexpand warning grainsize IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk vdisk_N id 2 name vdisk_N IO_group_id 0 IO_group_name io_grp0 status online mdisk_grp_id 0 mdisk_grp_name MDG_DS45 capacity 3.0GB type striped formatted no mdisk_id mdisk_name FC_id Chapter 11. Copy Services: FlashCopy
573
FC_name RC_id RC_name vdisk_UID 60050768018301BF280000000000002F throttling 0 preferred_node_id 2 fast_write_state empty cache readwrite udid fc_map_count 0 sync_rate 50 copy_count 1 copy_id 0 status online sync yes primary yes mdisk_grp_id 0 mdisk_grp_name MDG_DS45 type striped mdisk_id mdisk_name fast_write_state empty used_capacity 3.00GB real_capacity 3.01GB free_capacity 12.56MB overallocation 99 autoexpand on warning 80 grainsize 64
2. Define a FlashCopy mapping in which the non-space-efficient VDisk is the source and the space-efficient VDisk is the target. Specify a copy_rate as high as possible and activate the autodelete option for the mapping. See Example 11-17. Example 11-17 svctask mkfcmap IBM_2145:ITSO-CLS1:admin>svctask mkfcmap -source vdisk_B -target vdisk_N -name migrtosev -copyrate 100 -autodelete FlashCopy Mapping, id [0], successfully created IBM_2145:ITSO-CLS1:admin>svcinfo lsfcmap migrtosev id 0 name migrtosev source_vdisk_id 1 source_vdisk_name vdisk_B target_vdisk_id 2 target_vdisk_name vdisk_N group_id group_name status idle_or_copied progress 0 copy_rate 100 start_time dependent_mappings 0 autodelete on clean_progress 100 clean_rate 50 incremental off difference 100
574
Implementing the IBM System Storage SAN Volume Controller V4.3
grain_size 256 IO_group_id 1 IO_group_name io_grp1
3. Run the svctask prestartfcmap command, as shown in Example 11-18. Example 11-18 svctask prestartfcmap IBM_2145:ITSO-CLS1:admin>svctask prestartfcmap migrtosev IBM_2145:ITSO-CLS1:admin>svcinfo lsfcmap migrtosev id 0 name migrtosev source_vdisk_id 1 source_vdisk_name vdisk_B target_vdisk_id 2 target_vdisk_name vdisk_N group_id group_name status prepared progress 0 copy_rate 100 start_time dependent_mappings 0 autodelete on clean_progress 100 clean_rate 50 incremental off difference 100 grain_size 256 IO_group_id 1 IO_group_name io_grp1
4. Run the svctask startfcmap command, as shown in Example 11-19. Example 11-19 svctask startfcmap IBM_2145:ITSO-CLS1:admin>svctask startfcmap migrtosev
5. Monitor the copy process using the svcinfo lsfcmapprogress command, as shown in Example 11-20. Example 11-20 svcinfo lsfcmapprogress IBM_2145:ITSO-CLS1:admin>svcinfo lsfcmapprogress migrtosev id progress 0 63
6. The FlashCopy mapping has been deleted automatically, as shown in Example 11-21. Example 11-21 svcinfo lsfcmap IBM_2145:ITSO-CLS1:admin>svcinfo lsfcmap migrtosev CMMVC5754E The object specified does not exist, or the name supplied does not meet the naming rules
Chapter 11. Copy Services: FlashCopy
575
An independent copy of the source VDisk (vdisk_B) has been created. The migration has completed, as shown in Example 11-22. Example 11-22 svcinfo lsvdisk IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk vdisk_N id 2 name vdisk_N IO_group_id 0 IO_group_name io_grp0 status online mdisk_grp_id 0 mdisk_grp_name MDG_DS45 capacity 3.0GB type striped formatted no mdisk_id mdisk_name FC_id FC_name RC_id RC_name vdisk_UID 60050768018301BF280000000000002F throttling 0 preferred_node_id 2 fast_write_state empty cache readwrite udid fc_map_count 0 sync_rate 50 copy_count 1 copy_id 0 status online sync yes primary yes mdisk_grp_id 0 mdisk_grp_name MDG_DS45 type striped mdisk_id mdisk_name fast_write_state empty used_capacity 3.00GB real_capacity 3.01GB free_capacity 12.50MB overallocation 99 autoexpand on warning 80 grainsize 64
Note: Independent of what you defined as the real size of the target SEV, the real size will be at least the capacity of the source VDisk. To migrate a space-efficient VDisk to a fully allocated VDisk, you can follow the same scenario.
576
Implementing the IBM System Storage SAN Volume Controller V4.3
11.6 Using the GUI to perform FlashCopy In the following example, we use the same scenario described in 11.5.1, “Scenario description” on page 563. We follow the same procedures to perform the task using the GUI.
11.6.1 Creating a FlashCopy consistency group To create a FlashCopy consistency group in the SVC GUI, expand Manage Copy Services in the Task pane and select FlashCopy Consistency Groups (Figure 11-10).
Figure 11-10 Select FlashCopy Consistency Groups
Then, from the drop-down menu, select Create a Consistency Group and Go; this can be seen in Figure 11-11.
Figure 11-11 Create consistency group
Chapter 11. Copy Services: FlashCopy
577
Enter the desired name, and then click OK (Figure 11-12).
Figure 11-12 Create consistency group
Click Close when finished (Figure 11-13).
Figure 11-13 Close Create consistency group process
Check the result (Figure 11-14).
Figure 11-14 View consistency group
Repeat the previous steps to create another FC consistency group (Figure 11-15 on page 579). The FlashCopy consistency groups are now ready to use.
578
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 11-15 View consistency group
11.6.2 Creating a FlashCopy mapping Here we create the FlashCopy mappings for each of our VDisks for their respective targets. In the SVC GUI, we expand Manage Copy Services in the Task pane and select FlashCopy mappings. When prompted for filtering, we select Bypass Filter. (This will show us all the defined FlashCopy mappings, if there were any created previously.) As shown in Figure 11-16, we select Create a Mapping from the drop-down menu and click Go, as shown in Figure 11-16, to start the creation process of a FlashCopy mapping.
Figure 11-16 Create FC mapping
Chapter 11. Copy Services: FlashCopy
579
We are then presented with the FlashCopy creation wizard overview of the creation process for a FlashCopy mapping, as shown in Figure 11-17, and we click Next to proceed.
Figure 11-17 FC mapping wizard
We name the first FlashCopy mapping DB_Map1, select the previously created consistency group FCCG1, set the background copy priority to 50 and the Grain Size to 64, and click Next to proceed, as shown in Figure 11-18.
Figure 11-18 Define FC mapping properties
580
Implementing the IBM System Storage SAN Volume Controller V4.3
The next step is to select the source VDisk. If there were many source VDisks that were not already defined in a FlashCopy mapping, then we can filter that list here. In Figure 11-19, we define the filter * (which will show us all our VDisks) for the source VDisk and click Next to proceed.
Figure 11-19 Filter source VDisk candidates
We select DB_Source as the source disk and click Next to proceed (Figure 11-20).
Figure 11-20 Select source VDisk
Chapter 11. Copy Services: FlashCopy
581
The next step is to select our target VDisk. The FlashCopy mapping wizard will only present a list of VDisks that are the same size as the source VDisks and not already in a FlashCopy mapping or defined in a Metro Mirror relationship. In Figure 11-21, we select the target DB_Target1 and click Next to proceed.
Figure 11-21 Select target VDisk
In the next step, we select an I/O group for this mapping (Figure 11-22).
Figure 11-22 Select IO group
Finally, we verify our FlashCopy mapping (Figure 11-23 on page 583) and click Finish to create it.
582
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 11-23 FC mapping verification
We check the result of this creation, as shown in Figure 11-24.
Figure 11-24 View FC mapping
We repeat the procedure to create other FC mappings on the second FlashCopy target VDisk of DB_Source. We give it a different FC mapping name and choose a different FC consistency group, as shown in Figure 11-25 on page 584. As you can see in this example, we changed the background copy rate to 30, which will slow down the background copy process. The clearing rate of 60 will extend the stopping process if we had to stop the mapping during a copy process. An incremental mapping copies only
parts of the source or target VDisk that have changed since the last FlashCopy process. Note: Even if the type of the FC mapping is incremental, the first copy process will copy all data from the source to the target VDisk.
Chapter 11. Copy Services: FlashCopy
583
Figure 11-25 Create FC mapping type incremental
In Figure 11-26, you can see that DB_Source is still available.
Figure 11-26 Viewing FC mapping
We select DB_Target2 as the destination VDisk, as shown in Figure 11-27 on page 585.
584
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 11-27 Select a second target VDisk
On the last page of the wizard, as shown in Figure 11-28, we click Finish after verifying all the parameters.
Figure 11-28 Verification of FC mapping
Chapter 11. Copy Services: FlashCopy
585
We repeat the previous steps to create an FC mapping for Log_Source and Log_Target1, and also create an FC mapping for Log_Source and Log_Target2. We check the result in Figure 11-29.
Figure 11-29 View FC mappings
When creating the FC mapping for App_Source, we check the Automatically delete mappings when the background copy completes check box. This is shown in Figure 11-30.
Figure 11-30 Set FC mapping properties
586
Implementing the IBM System Storage SAN Volume Controller V4.3
We confirm the parameter settings by clicking Finish, as shown in Figure 11-31.
Figure 11-31 Verification of FC mapping
After the FlashCopy mapping is successfully created, we are returned to the FlashCopy mapping list (Figure 11-32) and we are presented with all the currently defined FlashCopy mappings.
Figure 11-32 View FC mapping
Chapter 11. Copy Services: FlashCopy
587
Click each mapping’s name to check the parameters, as shown in Figure 11-33.
Figure 11-33 View FC mapping details
If no consistency group is defined, the mapping is assigned into the default consistency group 0. This is a special group that cannot be started as a whole. Mappings in this group can only be started on an individual basis. The background copy rate specifies the priority that should be given to complete the copy. If 0 is specified, the copy will not proceed in the background. The default is 50. Tip: FlashCopy can be invoked from the SVC graphical user interface (GUI), but this might not make much sense if you plan to handle a large number of FlashCopy mappings or consistency groups periodically, or at varying times. In this case, creating a script by using the CLI may be more convenient.
11.6.3 Preparing (pre-triggering) the FlashCopy mapping At this point, the mapping has been created but the cache is still accepting data for the source VDisks. To ensure a consistent data set is created, it is crucial to flush application and OS buffers, and quiesce the application. To do this, we need to integrate our SVC commands into host scripts. Note: A FlashCopy must be prepared before it can be triggered. In Figure 11-34 on page 589, we select the FlashCopy mapping that is not in a consistency group. Select Prepare a mapping from the action list and click Go. The status will go to Preparing, and then finally to Prepared. Press the Refresh button several times until it is in the Prepared state.
588
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 11-34 Prepare FC mapping
Check the mapping status, as shown in Figure 11-35. The App_Map1 status is changed from Idle_or_Copied to Prepared.
Figure 11-35 View FC mapping state as prepared
11.6.4 Preparing (pre-triggering) the FlashCopy consistency group When performing the FlashCopy on the VDisks with the database, we want to be able to control the PiT when the FlashCopy is triggered, in order to keep our quiesce time at a minimum and preserve data integrity. We put the VDisks in a consistency group, and then we prepare the consistency group in order to flush the cache for all source VDisks. When you have assigned several mappings to a FlashCopy consistency group, you only have to issue a single prepare command for the whole group, to prepare all the mappings at once.
Chapter 11. Copy Services: FlashCopy
589
In Figure 11-36, we select the FlashCopy consistency group and Prepare a consistency group from the action list and click Go. The status will go to Preparing, and then finally to Prepared. Press the Refresh button several times until it is in the Prepared state.
Figure 11-36 Prepare FC consistency group
Figure 11-37 shows how we check the result. The status of the consistency group has changed to Prepared.
Figure 11-37 View prepared state of consistency groups
11.6.5 Starting (triggering) FlashCopy mappings When the FlashCopy mapping enters the Prepared state, we can start FlashCopy. As shown in Figure 11-38, we select the FlashCopy mapping App_Map1, select Start a Mapping from the scroll menu, and click Go to proceed.
Figure 11-38 Start a FC mapping
590
Implementing the IBM System Storage SAN Volume Controller V4.3
Because we have already prepared the FlashCopy mapping, the prepare box is grayed out, as shown in Figure 11-39, and we click OK to start the FlashCopy mapping.
Figure 11-39 Starting FC mapping
11.6.6 Starting (triggering) a FlashCopy consistency group As shown in Figure 11-40, the FlashCopy consistency group enters the prepared state. To start the FlashCopy consistency group, we select the consistency group and select Start a Consistency Group from the scroll menu and click Go.
Figure 11-40 Start the consistency group
In Figure 11-41, we are prompted to confirm starting the FlashCopy consistency group. We now flush the database and OS buffers and quiesce the database, then click OK to start the FlashCopy consistency group. Note: Since we have already prepared the FlashCopy consistency group, this option is grayed out when you are prompted to confirm starting the FlashCopy consistency group.
Figure 11-41 Start consistency group message
As shown in Figure 11-42 on page 592, we verified that the consistency group is in the Copying state, and subsequently, we resume the database I/O.
Chapter 11. Copy Services: FlashCopy
591
Figure 11-42 Consistency group status
11.6.7 Monitoring the FlashCopy progress To monitor the copy progress, you can click the Refresh button (Figure 11-43). Tips: Even if you click the Refresh button several times, the SVC only updates the progress of the background copy once a minute.
Figure 11-43 Viewing FC mapping state (background copy status)
Another option is to select Manage Progress, then FlashCopy, and then you can monitor the progress. This is shown in Figure 11-44 on page 593.
592
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 11-44 FC background copy progress
When the background copy is completed for all FlashCopy mappings in the consistency group, the status is changed to “Idle or Copied”.
11.6.8 Stopping the FlashCopy consistency group When a FlashCopy consistency group is stopped, the target VDisks become invalid and are set offline by the SVC. The FlashCopy mapping or consistency group needs to be prepared again or re-triggered to bring the target VDisks online again. Tip: If you want to stop a mapping or group in a multiple target FlashCopy environment, consider whether you want to keep any of the dependent mappings. If not, you should issue the stop command with the force parameter; this will stop all of the dependent maps too and negate the need for the stopping copy process to run.
Note: Stopping a FlashCopy mapping should only be done when the data on the target VDisk is of no use, or if you want to modify the FlashCopy mapping. When a FlashCopy mapping is stopped, the target VDisk becomes invalid and is set offline by the SVC, if the mapping is in the copying state and progress !=100.
Chapter 11. Copy Services: FlashCopy
593
As shown in Figure 11-45, we stop the FCCG1 and FCCG2 consistency groups. All mappings are now in the Copying state.
Figure 11-45 Stop FC consistency group
We click the Forced Stop button to proceed, as shown in Figure 11-46.
Figure 11-46 Stopping FC consistency group
The status of the FlashCopy consistency groups is Stopped, as shown in Figure 11-47.
Figure 11-47 FC consistency group status
594
Implementing the IBM System Storage SAN Volume Controller V4.3
11.6.9 Deleting the FlashCopy mapping As we have already enabled the function that will automatically delete the FC mapping when the background copy process has finished, as shown in Figure 11-48, App_Map1 has already been deleted.
Figure 11-48 Autodelete FC mapping
11.6.10 Deleting the FlashCopy consistency group If you want to delete all the mappings in the consistency group as well, first delete the mappings, then delete the consistency group. Tip: If you want to use the target VDisks in a consistency group as normal VDisks, you can monitor the background copy progress until it is complete (100% copied), and then delete the FlashCopy mapping. As shown in Figure 11-49, we delete all the maps and consistency groups and then check the result.
Figure 11-49 Delete FC mapping
Chapter 11. Copy Services: FlashCopy
595
Confirm the delete by clicking OK (Figure 11-50).
Figure 11-50 Confirm deletion of FC mapping
We repeat the above two steps to delete all FC mappings in the FCCG1 consistency group. Then we can delete the consistency group, as shown in Figure 11-51.
Figure 11-51 Delete FC consistency group
Confirm the deletion, as shown in Figure 11-52.
Figure 11-52 FC consistency group deletion confirmation
If you delete the consistency group before deleting all the FC mappings in it, you will be prompted for a forced deletion. We chose to delete FCCG2 directly, as shown in Figure 11-53.
Figure 11-53 Delete consistency group directly
596
Implementing the IBM System Storage SAN Volume Controller V4.3
There is a prompt for you to confirm your choice, and once you click Forced Delete (Figure 11-54), the consistency group will be deleted.
Figure 11-54 Force deletion of consistency group
11.6.11 Migration from a fully allocated VDisk to SEV and vice versa using a GUI Follow these steps to migrate from a fully allocated VDisk to a space-efficient VDisk: 1. Create a FlashCopy mapping with the fully allocated VDisk as the source and the SEV as the target. Important: The copy process will overwrite all the data on the target VDisk. You must back up all the data you may need before you start the copy process.
Figure 11-55 Create FlashCopy mapping
Chapter 11. Copy Services: FlashCopy
597
Specify the copy rate to be as high as possible and activate the Automatically delete mapping when the background copy completes option, as shown in Figure 11-56.
Figure 11-56 Set properties with Automatically delete mapping... option
Select the fully allocated VDisk (vdisk_B) as the source, as shown in Figure 11-57.
Figure 11-57 Select source VDisk
Select the space-efficient VDisk (vdisk_N) as the target, as shown in Figure 11-58 on page 599.
598
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 11-58 Select target VDisk
Verify the new FlashCopy mapping named migrtosev, as shown in Figure 11-59.
Figure 11-59 Verify FlashCopy mapping
Chapter 11. Copy Services: FlashCopy
599
View the new FlashCopy mapping, as shown in Figure 11-60.
Figure 11-60 View FlashCopy mapping
2. Prepare the FlashCopy mapping, as shown in Figure 11-61.
Figure 11-61 Prepare FlashCopy mapping
Wait until the FlashCopy mapping is in the prepared status, as shown in Figure 11-62.
Figure 11-62 FlashCopy mapping prepared
3. Start the FlashCopy mapping, as shown in Figure 11-63 on page 601.
600
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 11-63 Start FlashCopy mapping
4. Monitor the copy process, as shown in Figure 11-64.
Figure 11-64 View FlashCopy progress
5. After the copy process has completed, the FC mapping is deleted. An independent copy of the source on a space-efficient VDisk is available for use, as shown in Figure 11-65 on page 602.
Chapter 11. Copy Services: FlashCopy
601
Figure 11-65 View VDisk details
If you want to migrate a space-efficient VDisk to a fully allocated VDisk, follow the same scenario, but define the space-efficient VDisk as the source and the fully allocated VDisk as the target. We have now completed copying services with the FlashCopy GUI.
602
Implementing the IBM System Storage SAN Volume Controller V4.3
12
Chapter 12.
Copy Services: Metro Mirror In this chapter, we describe the Metro Mirror copy service, which is a synchronous remote copy function. Metro Mirror in SVC is similar to Metro Mirror in the IBM System Storage DS® family. Prior to SVC V2.1, this function was called PPRC. SVC provides a single point of control while enabling Metro Mirror in your SAN regardless of the disk subsystems used.
© Copyright IBM Corp. 2003-2008. All rights reserved.
603
12.1 Metro Mirror The general application of Metro Mirror is to maintain two real-time synchronized copies of a data set. Often, two copies are geographically dispersed to two SVC clusters, although it is possible to use Metro Mirror in a single cluster (within an I/O group). If the primary copy fails, a secondary copy can be enabled for I/O operation. Tips: Intracluster Metro Mirror will consume more resources for a specific cluster, compared to an intercluster Metro Mirror relationship. We recommend intercluster Metro Mirror when possible. A typical application of this function is to set up a dual-site solution using two SVC clusters where the first site is considered the primary or production site, and the second site is considered the backup site or failover site, which is activated when a failure at the first site is detected.
12.1.1 Metro Mirror overview Metro Mirror works by establishing a Metro Mirror relationship between two VDisks of equal size. To maintain data integrity for dependency writes, you can use consistency groups to group a number of Metro Mirror relationships together, similar to FlashCopy consistency groups. SVC provides both intracluster and intercluster Metro Mirror.
Intracluster Metro Mirror Intracluster Metro Mirror can be applied within any single I/O group. Metro Mirror across I/O groups in the same SVC cluster is not supported, since intracluster Metro Mirror can only be performed between VDisks in the same I/O group.
Intercluster Metro Mirror Intercluster Metro Mirror operations requires a pair of SVC clusters that are separated by a number of moderately high bandwidth links. Two SVC clusters must be defined in an SVC partnership, which must be performed on both SVC clusters to establish a fully functional Metro Mirror partnership. Using standard single mode connections, the supported distance between two SVC clusters in a Metro Mirror partnership is 10 km, although greater distances can be achieved by using extenders. For extended distance solutions, contact your IBM representative. Note: When a local and a remote fabric are connected together for Metro Mirror purposes, then ISL hop count between a local node and a remote node cannot exceed seven.
12.1.2 Remote copy techniques Metro Mirror is a synchronous remote copy, which is briefly explained below. To illustrate the differences between synchronous and asynchronous remote copy, asynchronous remote copy is also explained.
Synchronous remote copy Metro Mirror is a fully synchronous remote copy technique that ensures that writes are committed at both primary and secondary VDisks before the application is given acknowledgement of completion of a write.
604
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 12-1 illustrates how a write to the master VDisk is mirrored to the cache of the auxiliary VDisk before an acknowledgement of write is sent back to the host that issued the write. This ensures that the secondary is real-time synchronized, in case it is needed in a failover situation. However, this also means that the application is fully exposed to the latency and bandwidth limitations (if any) of the communication link to the secondary site. This might lead to unacceptable application performance, particularly when placed under peak load. This is the reason for distance limitations when using Metro Mirror.
Host
1 W r it e
4 Ack 3 A c k n o w le d g e W r ite
C ache
C ache 2 M ir r o r w r ite
M a s te r V D is k
A u x ilia r y V D is k M e tr o M ir r o r R e la tio n s h ip
Figure 12-1 Write on VDisk in Metro Mirror relationship
Chapter 12. Copy Services: Metro Mirror
605
Asynchronous remote copy In asynchronous remote copy, the application is given acknowledgement of the completion of the write to the issuing host as soon as it is committed to cache on the primary site. But an update is not necessarily sent or committed to secondary site at that time. This provides the capability of performing remote copy over distances that exceed the limitations of synchronous remote copy. This is shown in Figure 12-2.
Host
1 W r it e
2 Ack
C ache
3 W r ite to r e m o te
M a s te r V D is k
C ache
A u x ilia r y V D is k G lo b a l M ir r o r R e la tio n s h ip
Figure 12-2 Write on VDisk in Global Mirror relationship
Be aware that in a failover situation, some updates might be missing at the secondary site, and therefore the application must have some external mechanism for recovering missing updates and reapplying them. This mechanism might involve user intervention.
12.1.3 SVC Metro Mirror features SVC Metro Mirror supports following features: Synchronous remote copy of VDisks dispersed over metropolitan scale distances is supported. SVC implements Metro Mirror relationships between VDisk pairs, with each VDisk in a pair managed by an SVC cluster. SVC supports intracluster Metro Mirror, where both VDisks belong to the same cluster (and IO group). SVC supports intercluster Metro Mirror, where each VDisk belongs to their separate SVC cluster. A given SVC cluster can be configured for partnership with another cluster. A given SVC cluster can only communicate with one other cluster. All intercluster Metro Mirror processing takes place between two SVC clusters configured in a partnership. Intercluster and intracluster Metro Mirror can be used concurrently within a cluster for different relationships. SVC does not require a control network or fabric to be installed to manage Metro Mirror. For intercluster Metro Mirror, SVC maintains a control link between two clusters. This control link is used to control state and coordinate updates at either end. The control link is 606
Implementing the IBM System Storage SAN Volume Controller V4.3
implemented on top of the same FC fabric connection that the SVC uses for Metro Mirror I/O. SVC implements a configuration model that maintains the Metro Mirror configuration and state through major events, such as failover, recovery, and resynchronization, to minimize user configuration action through these events. SVC maintains and polices a strong concept of consistency and makes this available to guide configuration activity. SVC implements flexible resynchronization support enabling it to re-synchronize VDisk pairs that have suffered write I/O to both disks and to resynchronize only those regions that are known to have changed.
12.1.4 Metro Mirror relationship A Metro Mirror relationship is composed of two VDisks equal in size. The master VDisk and auxiliary VDisk can be in same I/O group, within the same SVC cluster (intracluster Metro Mirror), or can be on separate SVC clusters that are defined as SVC partners (intercluster Metro Mirror). Note: Be aware that: A VDisk can only be part of one Metro Mirror relationship at a time. A VDisk that is a FlashCopy target cannot be part of a Metro Mirror relationship. Figure 12-3 illustrates the Metro Mirror relationship.
VDisk1M MM_Master
MM_Relationship
VDisk1A MM_Auxiliary
Figure 12-3 Metro Mirror relationship
Metro Mirror relationship between primary and secondary VDisks When creating a Metro Mirror relationship, one VDisk should be defined as the master, and the other as the auxiliary. The relationship between two copies is symmetric. When a Metro Mirror relationship is created, the master VDisk is initially considered the primary copy (often referred to as the source), and the auxiliary VDisk is considered the secondary copy (often referred to as target). This implies that the initial copy direction is mirroring the master VDisk to the auxiliary VDisk. After the initial synchronization is complete, the copy direction can be changed if appropriate. In most common applications of Metro Mirror, the master VDisk contains the production copy of the data, and is used by the host application, while the auxiliary VDisk contains a mirrored copy of the data and is used for failover in disaster recovery scenarios. The terms master and auxiliary help support this use. However, if Metro Mirror is applied differently, the terms master and auxiliary VDisks need to be interpreted appropriately.
Chapter 12. Copy Services: Metro Mirror
607
Importance of write ordering Many applications that use block storage have a requirement to survive failures, such as loss of power or a software crash, and not lose data that existed prior to the failure. Since many applications need to perform large numbers of update operations in parallel with storage, maintaining write ordering is key to ensuring correct operation of applications following a disruption. An application, for example, databases, that is performing a large set of updates is usually designed with the concept of dependent writes. These are writes where it is important to ensure that an earlier write has completed before a later write is started. Reversing the order of dependent writes can undermine applications algorithms and can lead to problems, such as detected, or undetected, data corruption.
Dependent writes that span multiple VDisks The following scenario illustrates a simple example of a sequence of dependent writes, and in particular, what can happen if they span multiple VDisks. Consider the following typical sequence of writes for a database update transaction: 1. A write is executed to update the database log, indicating that a database update will be performed. 2. A second write is executed to update the database. 3. A third write is executed to update the database log, indicating that a database update has completed successfully. We illustrate the write sequence in Figure 12-4. Time Log
Step 1
DB file
Log
Step 2 DB file
Log
Step 3
Log: Update record xyz ... started
DB file
Log: Update record xyz ... started
DB: record xyz ...
Log: Update record xyz ... started Log: Update record xyz ... completed
DB: record xyz ...
Figure 12-4 Dependent writes for a database
608
Implementing the IBM System Storage SAN Volume Controller V4.3
The database ensures correct ordering of these writes by waiting for each step to complete before starting the next. Note: All databases have logs associated with them. These logs keep records of database changes. If a database needs to be restored to a point beyond the last full, offline backup, logs are required to roll the data forward to the point of failure. But imagine if the database log and database itself are on different VDisks and a Metro Mirror relationship is stopped during this update. In this case, you need to consider the possibility that the Metro Mirror relationship for the VDisk with the database file is stopped slightly before the VDisk containing the database log. If this were the case, then it could be possible that the secondary VDisks see writes (1) and (3), but not (2). Then, if the database was restarted using data available from secondary disks, the database log would indicate that the transaction had completed successfully, when that is not the case. In this scenario, the integrity of the database is in question.
Metro Mirror consistency groups Metro Mirror consistency groups address the issue of dependent writes across VDisks, where the objective is to preserve data consistency across multiple Metro Mirrored VDisks. Consistency groups ensure a consistent data set, because applications have relational data spanning across multiple VDisks. A Metro Mirror consistency group can contain an arbitrary number of relationships up to the maximum number of Metro Mirror relationships supported by the SVC Cluster. Metro Mirror commands can be issued to a Metro Mirror consistency group, and thereby simultaneously for all Metro Mirror relationships defined within that consistency group, or to a single Metro Mirror relationship that is not part of a Metro Mirror consistency group. For example, when issuing a Metro Mirror start command to the consistency group, all of the Metro Mirror relationships in the consistency group are started at the same time.
Chapter 12. Copy Services: Metro Mirror
609
The concept of Metro Mirror consistency groups is illustrated In Figure 12-5. Since the MM_Relationship 1 and 2 are part of the consistency group, they can be handled as one entity, while the stand-alone MM_Relationship 3 is handled separately.
Consistency Group 1
VDisk1M MM_Master
MM_Relationship 1
VDisk1A MM_Auxiliary
VDisk2M MM_Master
MM_Relationship 2
VDisk2A MM_Auxiliary
VDisk3M MM_Master
MM_Relationship 3
VDisk3A MM_Auxiliary
Figure 12-5 Metro Mirror consistency group
Certain uses of Metro Mirror require manipulation of more than one relationship. Metro Mirror consistency groups can provide the ability to group relationships, so that they are manipulated in unison. Metro Mirror relationships within a consistency group can be in any form: Metro Mirror relationships can be part of a consistency group, or be stand-alone and therefore handled as single instances. A consistency group can contain zero or more relationships. An empty consistency group, with zero relationships in it, has little purpose until it is assigned its first relationship, except that it has a name. All the relationships in a consistency group must have matching master and auxiliary SVC clusters. Although it is possible to use consistency groups to manipulate sets of relationships that do not need to satisfy these strict rules, such manipulation can lead to some undesired side effects. The rules behind a consistency group means that certain configuration commands are prohibited. Where this would not be the case was if the relationship was not part of a consistency group. For example, consider the case of two applications that are completely independent, yet they are placed into a single consistency group. In the event of an error, there is a loss of synchronization, and a background copy process is required to recover synchronization.
610
Implementing the IBM System Storage SAN Volume Controller V4.3
While this process is in progress, Metro Mirror rejects attempts to enable access to secondary VDisks of either application. If one application finishes its background copy much more quickly than the other, Metro Mirror still refuses to grant access to its secondary, even though it is safe in this case, because Metro Mirror policy is to refuse access to the entire consistency group if any part of it is inconsistent. Stand-alone relationships and consistency groups share a common configuration and state model. All the relationships in a non-empty consistency group have same state as the consistency group.
12.1.5 How Metro Mirror works In the sections that follow, we describe how Metro Mirror works.
Intercluster communication and zoning All intercluster communication is performed over the SAN. Prior to creating intercluster Metro Mirror relationships, you must create a partnership between the two clusters. All SVC node ports on each SVC cluster must be able to access each other to facilitate the partnership creation. Therefore, a zone in each fabric must be defined for intercluster communication (see Chapter 3, “Planning and configuration” on page 25).
SVC cluster partnership Each SVC cluster can only be in a partnership with one other SVC cluster. When a SVC cluster partnership has been defined on both clusters, further communication facilities between nodes in each of the cluster are established. This comprises: A single control channel, which is used to exchange and coordinate configuration information I/O channels between each of these nodes in clusters These channels are maintained and updated as nodes appear and disappear and as links fail, and are repaired to maintain operation where possible. If communication between SVC clusters is interrupted or lost, an error is logged (and consequently Metro Mirror relationships will stop). To handle error conditions, SVC can be configured to raise SNMP traps to the enterprise monitoring system.
Maintenance of intercluster link All SVC nodes maintain a database of other devices that are visible on the fabric. This is updated as devices appear and disappear. Devices that advertise themselves as SVC nodes are categorized according to the SVC cluster to which they belong. SVC nodes that belong to the same cluster establish communication channels between themselves and begin to exchange messages to implement clustering and functional protocols of SVC. Nodes that are in different clusters do not exchange messages after initial discovery is complete, unless they have been configured together to perform Metro Mirror.
Chapter 12. Copy Services: Metro Mirror
611
The intercluster link carries control traffic to coordinate activity between two clusters. It is formed between one node in each cluster, which is termed the focal point. The traffic between focal point nodes is distributed among logins that exist between those nodes. If the focal point node should fail (or all its logins to the remote cluster fail), then a new focal point is chosen to carry control traffic. Changing the focal point causes I/O to pause, but does not cause relationships to become Consistent Stopped.
12.1.6 Metro Mirror process There are several major steps in the Metro Mirror process: 1. An SVC cluster partnership is created between two SVC clusters (for intercluster Metro Mirror). 2. A Metro Mirror relationship is created between two VDisks of the same size. 3. To manage multiple Metro Mirror relationships as one entity, relationships can be made part of a Metro Mirror consistency group. This ensures data consistency across multiple Metro Mirror relationships, or simply for ease of management. 4. When a Metro Mirror relationship is started, and when the background copy has completed, the relationship becomes consistent and synchronized. 5. Once synchronized, the secondary VDisk holds a copy of the production data at the primary, which can be used for disaster recovery. 6. To access the auxiliary VDisk, the Metro Mirror relationship must be stopped with the access option enabled before write I/O is submitted to the secondary. 7. The remote host server is mapped to the auxiliary VDisk and the disk is available for I/O.
12.1.7 Methods of synchronization This section describes three methods that can be used to establish a relationship.
Full synchronization after creation This is the default method. It is the simplest in that it requires no administrative activity apart from issuing the necessary commands. However, in some environments, the bandwidth available will make this method unsuitable. The command sequence for a single relationship is as follows: 1. Run mkrcrelationship without specifying the -sync option. 2. Run startrcrelationship without specifying the -clean option
Synchronized before creation In this method, the administrator must ensure that the master and auxiliary VDisks contain identical data before creating the relationship. There are two ways in which this might be done: Both disks are created with the security delete feature so as to make all data zero. A complete tape image (or other method of moving data) is copied from one disk to the other. In either technique, no write I/O must take place to either the master or the auxiliary before the relationship is established.
612
Implementing the IBM System Storage SAN Volume Controller V4.3
Then, the administrator must: Run mkrcrelationship with the -sync flag. Run startrcrelationship without the -clean flag. If these steps are not performed correctly, then Metro Mirror will report the relationship as being consistent when it is not. This is likely to make any secondary disk useless. This method has an advantage over full synchronization, in that it does not require all the data to be copied over a constrained link. However, if data needs to be copied, the master and auxiliary disks cannot be used until the copy is complete, which might be unacceptable.
Quick synchronization after creation In this method, the administrator must still copy data from the master to the auxiliary, but it can be used without stopping the application at the master. The administrator must ensure that: A mkrcrelationship command is issued with the -sync flag. A stoprcrelationship command is issued with the -access flag. A tape image (or other method of transferring data) is used to copy the entire master disk to the auxiliary disk. Once the copy is complete, administrator must ensure that: A startrcrelationship command is issued with the -clean flag. With this technique, only data that has changed since the relationship was created, including all regions that were incorrect in the tape image, are copied from the master and the auxiliary. As with “Synchronized before creation” on page 612, the copy step must be performed correctly or the auxiliary will be useless, although the copy operation will report it as being synchronized.
Metro Mirror states and events In this section, we explain the different states of a Metro Mirror relationship, and the series of events that modify these states.
Chapter 12. Copy Services: Metro Mirror
613
In Figure 12-6, the Metro Mirror relationship state diagram shows an overview of states that can apply to a Metro Mirror relationship in a connected state.
1a
c) yn s (in
Consistent Stopped
2a 4b
Stop enable access
Stop or Error
Start
Create Metro Mirror Relationship (ou t o 1b fs yn c)
Fo rce dS (ou ta to f sy rt nc)
3 Consistent Synchronized
4a
Background copy complete
Stop enable access
5a
Start (in sync)
Inconsistent Stopped
2b Start
Stop or Error
Inconsistent Copying
t tar S ) d nc ce y r s Fo t of u (o
Idling
Figure 12-6 Metro Mirror mapping state diagram
When creating the Metro Mirror relationship, you can specify if the auxiliary VDisk is already in sync with the master VDisk, and the background copy process is then skipped. This is especially useful when creating Metro Mirror relationships for VDisks that have been created with the format option. Create the relationship as follows: 1. Step 1 is done as follows: a. The Metro Mirror relationship is created with the -sync option and the Metro Mirror relationship enters the Consistent stopped state. b. The Metro Mirror relationship is created without specifying that the master and auxiliary VDisks are in sync, and the Metro Mirror relationship enters the Inconsistent stopped state. 2. Step 2 is done as follows: a. When starting a Metro Mirror relationship in the Consistent stopped state, it enters the Consistent synchronized state. This implies that no updates (write I/O) have been performed on the primary VDisk while in the Consistent stopped state, otherwise the -force option must be specified, and the Metro Mirror relationship then enters the Inconsistent copying state, while the background copy is started. 614
Implementing the IBM System Storage SAN Volume Controller V4.3
b. When starting a Metro Mirror relationship in the Inconsistent stopped state, it enters the Inconsistent copying state, while the background copy is started. 3. Step 3 is done as follows: When the background copy completes, the Metro Mirror relationship transits from the Inconsistent copying state to the Consistent synchronized state. 4. Step 4 is done as follows: a. When stopping a Metro Mirror relationship in the Consistent synchronized state, specifying the -access option, which enables write I/O on the secondary VDisk, the Metro Mirror relationship enters the Idling state. b. To enable write I/O on the secondary VDisk, when the Metro Mirror relationship is in the Consistent stopped state, issue the command svctask stoprcrelationship specifying the -access option, and the Metro Mirror relationship enters the Idling state. Step 5 is done as follows: a. When starting a Metro Mirror relationship that is in the Idling state, you must specify the -primary argument to set the copy direction. Given that no write I/O has been performed (to either the master or auxiliary VDisk) while in the Idling state, the Metro Mirror relationship enters the Consistent synchronized state. b. In case write I/O has been performed to either the master or the auxiliary VDisk, then the -force option must be specified, and the Metro Mirror relationship then enters the Inconsistent copying state, while the background copy is started. Stop or Error: When a Metro Mirror relationship is stopped (either intentionally or due to an error), a state transition is applied: For example, this means that the Metro Mirror relationships in the Consistent synchronized state enter the Consistent stopped state and the Metro Mirror relationships in the Inconsistent copying state enter the Inconsistent stopped state. In case the connection is broken between the SVC clusters in a partnership, then all (intercluster) Metro Mirror relationships enter a disconnected state. For further information, refer to “Connected versus disconnected” on page 615. Note: Stand-alone relationships and consistency groups share a common configuration and state model. This means that all Metro Mirror relationships in a non-empty consistency group have the same state as the consistency group.
12.1.8 State overview SVC defined concepts of state are key to understanding configuration concepts and are therefore explained in more detail below.
Connected versus disconnected This distinction can arise when a Metro Mirror relationship is created with the two VDisks in different clusters. Under certain error scenarios, communications between the two clusters might be lost. For example, power might fail, causing one complete cluster to disappear. Alternatively, the fabric connection between the two clusters might fail, leaving the two clusters running but unable to communicate with each other.
Chapter 12. Copy Services: Metro Mirror
615
When the two clusters can communicate, the clusters and the relationships spanning them are described as connected. When they cannot communicate, the clusters and the relationships spanning them are described as disconnected. In this scenario, each cluster is left with half the relationship and has only a portion of the information that was available to it before. Some limited configuration activity is possible, and is a subset of what was possible before. The disconnected relationships are portrayed as having a changed state. The new states describe what is known about the relationship, and what configuration commands are permitted. When the clusters can communicate again, the relationships become connected once again. Metro Mirror automatically reconciles the two state fragments, taking into account any configuration or other event that took place while the relationship was disconnected. As a result, the relationship can either return to the state it was in when it became disconnected or it can enter a different connected state. Relationships that are configured between VDisks in the same SVC cluster (intracluster) will never be described as being in a disconnected state.
Consistent versus inconsistent Relationships that contain VDisks operating as secondaries can be described as being consistent or inconsistent. Consistency groups that contain relationships can also be described as being consistent or inconsistent. The consistent or inconsistent property describes the relationship of the data on the secondary to the one on the primary VDisk. It can be considered a property of the secondary VDisk itself. A secondary is described as consistent if it contains data that could have been read by a host system from the primary if power had failed at some imaginary point in time while I/O was in progress and power was later restored. This imaginary point in time is defined as the recovery point. The requirements for consistency are expressed with respect to activity at the primary up to the recovery point: The secondary VDisk contains the data from all writes to the primary for which the host received good completion and that data had not been overwritten by a subsequent write (before the recovery point). For writes for which the host did not receive good completion (that is, it received bad completion or no completion at all), and the host subsequently performed a read from the primary of that data and that read returned good completion and no later write was sent (before the recovery point), the secondary contains the same data as that returned by the read from the primary. From the point of view of an application, consistency means that a secondary VDisk contains the same data as the primary VDisk at the recovery point (the time at which the imaginary power failure occurred). If an application is designed to cope with unexpected power failure, this guarantee of consistency means that the application will be able to use the secondary and begin operation just as though it had been restarted after the hypothetical power failure. Again, the application is dependent on the key properties of consistency: Write ordering Read stability for correct operation at the secondary
616
Implementing the IBM System Storage SAN Volume Controller V4.3
If a relationship, or set of relationships, is inconsistent and an attempt is made to start an application using the data in the secondaries, a number of outcomes are possible: The application might decide that the data is corrupt and crash or exit with an error code. The application might fail to detect that the data is corrupt and return erroneous data. The application might work without a problem. Because of the risk of data corruption, and in particular undetected data corruption, Metro Mirror strongly enforces the concept of consistency and prohibits access to inconsistent data. Consistency as a concept can be applied to a single relationship or a set of relationships in a consistency group. Write ordering is a concept that an application can maintain across a number of disks accessed through multiple systems; therefore, consistency must operate across all those disks. When deciding how to use consistency groups, the administrator must consider the scope of an application’s data, taking into account all the interdependent systems that communicate and exchange information. If two programs or systems communicate and store details as a result of the information exchanged, then either of the following actions might occur: All the data accessed by the group of systems must be placed into a single consistency group. The systems must be recovered independently (each within its own consistency group). Then, each system must perform recovery with the other applications to become consistent with them.
Consistent versus synchronized A copy that is consistent and up-to-date is described as synchronized. In a synchronized relationship, the primary and secondary VDisks are only different in regions where writes are outstanding from the host. Consistency does not mean that the data is up-to-date. A copy can be consistent and yet contain data that was frozen at some point in time in the past. Write I/O might have continued to a primary and not have been copied to the secondary. This state arises when it becomes impossible to keep up-to-date and maintain consistency. An example is a loss of communication between clusters when writing to the secondary. When communication is lost for an extended period of time, Metro Mirror tracks the changes that happen at the primary, but not the order of such changes, or the details of such changes (write data). When communication is restored, it is impossible to make the secondary synchronized without sending write data to the secondary out-of-order, and therefore losing consistency. Two policies can be used to cope with this: Make a point-in-time copy of the consistent secondary before allowing the secondary to become inconsistent. In the event of a disaster before consistency is achieved again, the point-in-time copy target provides a consistent, though out-of-date, image. Accept the loss of consistency, and loss of useful secondary, while making it synchronized.
Chapter 12. Copy Services: Metro Mirror
617
12.1.9 Detailed states The following sections detail the states that are portrayed to the user, for either consistency groups or relationships. It also details the extra information available in each state. The different major states are constructed to provide guidance on the configuration commands that are available.
InconsistentStopped This is a connected state. In this state, the primary is accessible for read and write I/O, but the secondary is not accessible for either. A copy process needs to be started to make the secondary consistent. This state is entered when the relationship or consistency group was InconsistentCopying and has either suffered a persistent error or received a stop command that has caused the copy process to stop. A start command causes the relationship or consistency group to move to the InconsistentCopying state. A stop command is accepted, but has no effect. If the relationship or consistency group becomes disconnected, the secondary side transits to InconsistentDisconnected. The primary side transits to IdlingDisconnected.
InconsistentCopying This is a connected state. In this state, the primary is accessible for read and write I/O, but the secondary is not accessible for either read or write I/O. This state is entered after a start command is issued to an InconsistentStopped relationship or consistency group. It is also entered when a forced start is issued to an Idling or ConsistentStopped relationship or consistency group. In this state, a background copy process runs that copies data from the primary to the secondary virtual disk. In the absence of errors, an InconsistentCopying relationship is active, and the Copy Progress increases until the copy process completes. In some error situations, the copy progress might freeze or even regress. A persistent error or stop command places the relationship or consistency group into an InconsistentStopped state. A start command is accepted, but has no effect. If the background copy process completes on a stand-alone relationship, or on all relationships for a consistency group, the relationship or consistency group transits to ConsistentSynchronized state. If the relationship or consistency group becomes disconnected, then the secondary side transits to InconsistentDisconnected. The primary side transitions to IdlingDisconnected.
ConsistentStopped This is a connected state. In this state, the secondary contains a consistent image, but it might be out-of-date with respect to the primary. This state can arise when a relationship was in a Consistent Synchronized state and suffers an error that forces a Consistency Freeze. It can also arise when a relationship is created with a CreateConsistentFlag set to TRUE.
618
Implementing the IBM System Storage SAN Volume Controller V4.3
Normally, following an I/O error, subsequent write activity cause updates to the primary and the secondary is no longer synchronized (set to FALSE). In this case, to re-establish synchronization, consistency must be given up for a period. A start command with the -force option must be used to acknowledge this, and the relationship or consistency group transits to InconsistentCopying. Do this only after all outstanding errors are repaired. In the unusual case where the primary and secondary are still synchronized (perhaps following a user stop, and no further write I/O was received), a start command takes the relationship to ConsistentSynchronized. No -force option is required. Also, in this unusual case, a switch command is permitted that moves the relationship or consistency group to ConsistentSynchronized and reverses the roles of the primary and secondary. If the relationship or consistency group becomes disconnected, then the secondary side transits to ConsistentDisconnected. The primary side transitions to IdlingDisconnected. An informational status log is generated every time a relationship or consistency group enters the ConsistentStopped with a status of Online state. This can be configured to enable an SNMP trap and provide a trigger to automation software to consider issuing a start command following a loss of synchronization.
ConsistentSynchronized This is a connected state. In this state, the primary VDisk is accessible for read and write I/O. The secondary VDisk is accessible for read-only I/O. Writes that are sent to the primary VDisk are sent to both primary and secondary VDisks. Either good completion must be received for both writes, the write must be failed to the host, or a state must transit out of the ConsistentSynchronized state before a write is completed to the host. A stop command takes the relationship to the ConsistentStopped state. A stop command with the -access parameter takes the relationship to the Idling state. A switch command leaves the relationship in the ConsistentSynchronized state, but reverses the primary and secondary roles. A start command is accepted, but has no effect. If the relationship or consistency group becomes disconnected, the same transitions are made as for ConsistentStopped.
Idling This is a connected state. Both master and auxiliary disks are operating in the primary role. Consequently, both are accessible for write I/O. In this state, the relationship or consistency group accepts a start command. Metro Mirror maintains a record of regions on each disk that received write I/O while Idling. This is used to determine what areas need to be copied following a start command. The start command must specify the new copy direction. A start command can cause a loss of consistency if either VDisk in any relationship has received write I/O. This is indicated by the synchronized status. If the start command leads to loss of consistency, then a -force parameter must be specified. Following a start command, the relationship or consistency group transits to ConsistentSynchronized if there is no loss of consistency, or to InconsistentCopying if there is such a loss.
Chapter 12. Copy Services: Metro Mirror
619
Also, while in this state, the relationship or consistency group accepts a -clean option on the start command. If the relationship or consistency group becomes disconnected, then both sides change their state to IdlingDisconnected.
IdlingDisconnected This is a disconnected state. The VDisk or disks in this half of the relationship or consistency group are all in the primary role and accept read or write I/O. The main priority in this state is to recover the link and make the relationship or consistency group connected once more. No configuration activity is possible (except for deletes or stops) until the relationship becomes connected again. At that point, the relationship transits to a connected state. The exact connected state that is entered depends on the state of the other half of the relationship or consistency group, which depends on: The state when it became disconnected The write activity since it was disconnected The configuration activity since it was disconnected If both halves are IdlingDisconnected, then the relationship becomes Idling when reconnected. While IdlingDisconnected, if a write I/O is received that causes loss of synchronization (synchronized attribute transits from TRUE to FALSE) and the relationship was not already stopped (either through user stop or a persistent error), then an error log is raised to notify you of this situation. This error log is the same as that raised when the same situation arises for ConsistentSynchronized.
InconsistentDisconnected This is a disconnected state. The virtual disks in this half of the relationship or consistency group are all in the secondary role and do not accept read or write I/O. No configuration activity except for deletes is permitted until the relationship becomes connected again. When the relationship or consistency group becomes connected again, the relationship becomes InconsistentCopying automatically unless either: The relationship was InconsistentStopped when it became disconnected. The user issued a stop while disconnected. In either case, the relationship or consistency group becomes InconsistentStopped.
ConsistentDisconnected This is a disconnected state. The VDisks in this half of the relationship or consistency group are all in the secondary role and accept read I/O but not write I/O. This state is entered from ConsistentSynchronized or ConsistentStopped when the secondary side of a relationship becomes disconnected. In this state, the relationship or consistency group displays an attribute of FreezeTime, which is the point in time that Consistency was frozen. When entered from ConsistentStopped, it retains the time it had in that state. When entered from ConsistentSynchronized, the FreezeTime shows the last time at which the relationship or consistency group was known to be consistent. This corresponds to the time of the last successful heartbeat to the other cluster. 620
Implementing the IBM System Storage SAN Volume Controller V4.3
A stop command with the -access flag set to TRUE transits the relationship or consistency group to the IdlingDisconnected state. This allows write I/O to be performed to the secondary VDisk and is used as part of a disaster recovery scenario. When the relationship or consistency group becomes connected again, the relationship or consistency group becomes ConsistentSynchronized only if this does not lead to a loss of Consistency. This is the case provided: The relationship was ConsistentSynchronized when it became disconnected. No writes received successful completion at the primary while disconnected. Otherwise, the relationship become ConsistentStopped. The FreezeTime setting is retained.
Empty This state only applies to consistency groups. It is the state of a consistency group that has no relationships and no other state information to show. It is entered when a consistency group is first created. It is exited when the first relationship is added to the consistency group, at which point the state of the relationship becomes the state of the consistency group.
Background copy Metro Mirror paces the rate at which background copy is performed by the appropriate relationships. Background copy takes place on relationships that are in the InconsistentCopying state with a Status of Online. The quota of background copy (configured on the intercluster link) is divided evenly between all the nodes that are performing background copy for one of the eligible relationships. This allocation is made irrespective of the number of disks that the node is responsible for. Each node in turn divides its allocation evenly between the multiple relationships performing a background copy. For intracluster relationships, each node is assigned a static quota of 25 MBps.
12.1.10 Practical use of Metro Mirror The master VDisk is the production VDisk and updates to this copy are real time mirrored to the auxiliary VDisk. The contents of the auxiliary VDisk that existed when the relationship was created are destroyed. Note: The copy direction for a Metro Mirror relationship can be switched so the auxiliary VDisk becomes the primary, and the master VDisk becomes the secondary. While the Metro Mirror relationship is active, the secondary copy (VDisk) is not accessible for host application write I/O at any time. The SVC allows read-only access to the secondary VDisk when it contains a “consistent” image. This is only intended to allow boot time operating system discovery to complete without error, so that any hosts at the secondary site can be ready to start up the applications with minimum delay if required. For example, many operating systems need to read Logical Block Address (LBA) zero to configure a logical unit. Although read access is allowed at the secondary in practice, the data on the secondary volumes cannot be read by a host. The reason for this is that most operating systems write a “dirty bit” to the file system when it is mounted. Because this write operation is not allowed on the secondary volume, the volume cannot be mounted.
Chapter 12. Copy Services: Metro Mirror
621
This access is only provided where consistency can be guaranteed. However, there is no way in which coherency can be maintained between reads performed at the secondary and later write I/Os performed at the primary. To enable access to the secondary VDisk for host operations, the Metro Mirror relationship must be stopped by specifying the -access parameter. While access to the secondary VDisk for host operations is enabled, the host must be instructed to mount the VDisk and related tasks before the application can be started, or instructed to perform a recovery process. For example, the Metro Mirror requirement to enable the secondary copy for access differentiates it from third-party mirroring software on the host, which aims to emulate a single, reliable disk regardless of what system is accessing it. Metro Mirror retains the property that there are two volumes in existence, but suppresses one while the copy is being maintained. Using a secondary copy demands a conscious policy decision by the administrator, that a failover is required, and the tasks to be performed on the host involved in establishing operation on the secondary copy are substantial. The goal is to make this rapid (much faster when compared to recovering from a backup copy) but not seamless. The failover process can be automated through failover management software. The SVC provides Simple Network Management Protocol (SNMP) traps and programming (or scripting) for the command-line interface (CLI) to enable this automation.
12.1.11 Metro Mirror configuration limits Table 12-1 lists the Metro Mirror configuration limits. Table 12-1 Metro Mirror configuration limits Parameter
Value
Number of Metro Mirror consistency groups
256 per SVC cluster.
Number of Metro Mirror relationships
1024 per SVC cluster.
Total VDisk size per I/O group
1024 TB is the per I/O group limit on the quantity of primary and secondary VDisk address space that can participate in Global Mirror relationships.
12.2 Metro Mirror commands For all the details about the Metro Mirror Commands, refer to IBM System Storage SAN Volume Controller Command-Line Interface User’s Guide, SC26-7903 The command set for Metro Mirror contains two broad groups: Commands to create, delete, and manipulate relationships and consistency groups Commands to cause state changes Where a configuration command affects more than one cluster, Metro Mirror performs the work to coordinate configuration activity between the clusters. Some configuration commands can only be performed when the clusters are connected and fail with no effect when they are disconnected. 622
Implementing the IBM System Storage SAN Volume Controller V4.3
Other configuration commands are permitted even though the clusters are disconnected. The state is reconciled automatically by Metro Mirror when the clusters become connected once more. For any given command, with one exception, a single cluster actually receives the command from the administrator. This is significant for defining the context for a CreateRelationShip mkrcrelationship or CreateConsistencyGroup mkrcconsistgrp command, in which case, the cluster receiving the command is called the local cluster. The exception mentioned previously is the command that sets clusters into a Metro Mirror partnership. The mkpartnership command must be issued to both the local and remote clusters. The commands here are described as an abstract command set. These are implemented as: A command-line interface (CLI), which can be used for scripting and automation A graphical user interface (GUI), which can be used for one-off tasks
12.2.1 Listing available SVC cluster partners To create an SVC cluster partnership, use the command svcinfo lsclustercandidate.
svcinfo lsclustercandidate The svcinfo lsclustercandidate command is used to list the clusters that are available for setting up a two-cluster partnership. This is a prerequisite for creating Metro Mirror relationships.
12.2.2 Creating SVC cluster partnership To create an SVC cluster partnership, use the command svctask mkpartnership.
svctask mkpartnership The svctask mkpartnership command is used to establish a one-way Metro Mirror partnership between the local cluster and a remote cluster. To establish a fully functional Metro Mirror partnership, you must issue this command to both clusters. This step is a prerequisite to creating Metro Mirror relationships between VDisks on the SVC clusters. When creating the partnership, you can specify the bandwidth to be used by the background copy process between the local and the remote SVC cluster, and if it is not specified, the bandwidth defaults to 50 MBps. The bandwidth should be set to a value that is less than or equal to the bandwidth that can be sustained by the intercluster link.
Background copy bandwidth impact on foreground I/O latency The background copy bandwidth determines the rate at which the background copy for the IBM System Storage Metro Mirror for SAN Volume Controller will be attempted. The background copy bandwidth can affect foreground I/O latency in one of three ways: The following results can occur if the background copy bandwidth is set too high for the Metro Mirror intercluster link capacity: – The background copy I/Os can back up on the Metro Mirror intercluster link. – There is a delay in the synchronous secondary writes of foreground I/Os. – The foreground I/O latency will increase as perceived by applications.
Chapter 12. Copy Services: Metro Mirror
623
If the background copy bandwidth is set too high for the storage at the primary site, and background copy read I/Os overload the primary storage and delay foreground I/Os. If the background copy bandwidth is set too high for the storage at the secondary site, background copy writes at the secondary overload the secondary storage and again delay the synchronous secondary writes of foreground I/Os. In order to set the background copy bandwidth optimally, make sure that you consider all three resources (the primary storage, the intercluster link bandwidth, and the secondary storage). Provision the most restrictive of these three resources between the background copy bandwidth and the peak foreground I/O workload. This provisioning can be done by calculation (as above) or alternatively by determining experimentally how much background copy can be allowed before the foreground I/O latency becomes unacceptable, and then backing off to allow for peaks in workload and some safety margin.
svctask chpartnership In case it is needed to change the bandwidth available for background copy in an SVC cluster partnership, the command svctask chpartnership can be used to specify the new bandwidth.
12.2.3 Creating a Metro Mirror consistency group To create a Metro Mirror consistency group, use the command svctask mkrcconsistgrp.
svctask mkrcconsistgrp The svctask mkrcconsistgrp command is used to create a new empty Metro Mirror consistency group. The Metro Mirror consistency group name must be unique across all consistency groups known to the clusters owning this consistency group. If the consistency group involves two clusters, the clusters must be in communication throughout the creation process. The new consistency group does not contain any relationships and will be in the empty state. Metro Mirror relationships can be added to the group either upon creation or afterwards using the svctask chrelationship command.
12.2.4 Creating a Metro Mirror relationship To create a Metro Mirror relationship, use the command svctask mkrcrelationship.
svctask mkrcrelationship The svctask mkrcrelationship command is used to create a new Metro Mirror relationship. This relationship persists until it is deleted. The auxiliary VDisk must be equal in size to the master VDisk or the command will fail, and if both VDisks are in the same cluster, they must both be in the same I/O group. The master and auxiliary VDisk cannot be in an existing relationship, and cannot be the target of a FlashCopy mapping. This command returns the new relationship (relationship_id) when successful. When creating the Metro Mirror relationship, it can be added to an already existing consistency group, or be a stand-alone Metro Mirror relationship if no consistency group is specified.
624
Implementing the IBM System Storage SAN Volume Controller V4.3
To check whether the master or auxiliary VDisks comply with the prerequisites to participate in a Metro Mirror relationship, use the command svcinfo lsrcrelationshipcandidate.
svcinfo lsrcrelationshipcandidate The svcinfo lsrcrelationshipcandidate command is used to list available VDisks that are eligible for a Metro Mirror relationship. When issuing the command, you can specify the master VDisk name and auxiliary cluster to list candidates that comply with prerequisites to create a Metro Mirror relationship. If the command is issued with no flags, all VDisks that are not disallowed by some other configuration state, such as being a FlashCopy target, are listed.
12.2.5 Changing a Metro Mirror relationship To modify the properties of a Metro Mirror relationship, use the command svctask chrcrelationship.
svctask chrcrelationship The svctask chrcrelationship command is used to modify the following properties of a Metro Mirror relationship: Change the name of a Metro Mirror relationship. Add a relationship to a group. Remove a relationship from a group using the -force flag. Note: When adding a Metro Mirror relationship to a consistency group that is not empty, the relationship must have the same state and copy direction as the group in order to be added to it.
12.2.6 Changing a Metro Mirror consistency group To change the name of a Metro Mirror consistency group, use the command svctask chrcconsistgrp.
svctask chrcconsistgrp The svctask chrcconsistgrp command is used to change the name of a Metro Mirror consistency group.
12.2.7 Starting a Metro Mirror relationship To start a stand-alone Metro Mirror relationship, use the command svctask startrcrelationship.
svctask startrcrelationship The svctask startrcrelationship command is used to start the copy process of a Metro Mirror relationship. When issuing the command, the copy direction can be set, if it is undefined, and optionally mark the secondary VDisk of the relationship as clean. The command fails it if it is used to attempt to start a relationship that is part of a consistency group.
Chapter 12. Copy Services: Metro Mirror
625
This command can only be issued to a relationship that is connected. For a relationship that is idling, this command assigns a copy direction (primary and secondary roles) and begins the copy process. Otherwise, this command restarts a previous copy process that was stopped either by a stop command or by some I/O error. If the resumption of the copy process leads to a period when the relationship is not consistent, then you must specify the -force flag when restarting the relationship. This situation can arise if, for example, the relationship was stopped, and then further writes were performed on the original primary of the relationship. The use of the -force flag here is a reminder that the data on the secondary will become inconsistent while resynchronization (background copying) occurs, and therefore the date is not usable for disaster recovery purposes before the background copy has completed. In the idling state, you must specify the primary VDisk to indicate the copy direction. In other connected states, you can provide the primary argument, but it must match the existing setting.
12.2.8 Stopping a Metro Mirror relationship To stop a stand-alone Metro Mirror relationship, use the command svctask stoprcrelationship.
svctask stoprcrelationship The svctask stoprcrelationship command is used to stop the copy process for a relationship. It can also be used to enable write access to a consistent secondary VDisk by specifying the -access flag. This command applies to a stand-alone relationship. It is rejected if it is addressed to a relationship that is part of a consistency group. You can issue this command to stop a relationship that is copying from primary to secondary. If the relationship is in an inconsistent state, any copy operation stops and does not resume until you issue a svctask startrcrelationship command. Write activity is no longer copied from the primary to the secondary VDisk. For a relationship in the ConsistentSynchronized state, this command causes a consistency freeze. When a relationship is in a consistent state (that is, in the ConsistentStopped, ConsistentSynchronized, or ConsistentDisconnected state) then the -access argument can be used with the stoprcrelationship command to enable write access to the secondary VDisk.
12.2.9 Starting a Metro Mirror consistency group To start a Metro Mirror consistency group, use the command svctask startrcconsistgrp.
svctask startrcconsistgrp The svctask startrcconsistgrp command is used to start a Metro Mirror consistency group. This command can only be issued to a consistency group that is connected. For a consistency group that is idling, this command assigns a copy direction (primary and secondary roles) and begins the copy process. Otherwise, this command restarts a previous copy process that was stopped either by a stop command or by some I/O error.
626
Implementing the IBM System Storage SAN Volume Controller V4.3
12.2.10 Stopping a Metro Mirror consistency group To stop a Metro Mirror consistency group, use the command svctask stoprcconsistgrp.
svctask stoprcconsistgrp The svctask startrcconsistgrp command is used to stop the copy process for a Metro Mirror consistency group. It can also be used to enable write access to the secondary VDisks in the group if the group is in a consistent state. If the consistency group is in an inconsistent state, any copy operation stops and does not resume until you issue the svctask startrcconsistgrp command. Write activity is no longer copied from the primary to the secondary VDisks belonging to the relationships in the group. For a consistency group in the ConsistentSynchronized state, this command causes a consistency freeze. When a consistency group is in a consistent state (for example, in the ConsistentStopped, ConsistentSynchronized, or ConsistentDisconnected state), then the -access argument can be used with the svctask stoprcconsistgrp command to enable write access to the secondary VDisks within that group.
12.2.11 Deleting a Metro Mirror relationship To delete a Metro Mirror relationship, use the command svctask rmrcrelationship.
svctask rmrcrelationship The svctask rmrcrelationship command is used to delete the relationship that is specified. Deleting a relationship only deletes the logical relationship between the two VDisks. It does not affect the VDisks themselves. If the relationship is disconnected at the time that the command is issued, then the relationship is only deleted on the cluster on which the command is being run. When the clusters reconnect, then the relationship is automatically deleted on the other cluster. Alternatively, if the clusters are disconnected, and you still wish to remove the relationship on both clusters, you can issue the rmrcrelationship command independently on both of the clusters. A relationship cannot be deleted if it is part of a consistency group. You must first remove the relationship from the consistency group. If you delete an inconsistent relationship, the secondary VDisk becomes accessible even though it is still inconsistent. This is the one case in which Metro Mirror does not inhibit access to inconsistent data.
12.2.12 Deleting a Metro Mirror consistency group To delete a Metro Mirror consistency group, use the command svctask rmrcconsistgrp.
svctask rmrcconsistgrp The svctask rmrcconsistgrp command is used to delete a Metro Mirror consistency group. This command deletes the specified consistency group. You can issue this command for any existing consistency group.
Chapter 12. Copy Services: Metro Mirror
627
If the consistency group is disconnected at the time that the command is issued, then the consistency group is only deleted on the cluster on which the command is being run. When the clusters reconnect, the consistency group is automatically deleted on the other cluster. Alternatively, if the clusters are disconnected, and you still want to remove the consistency group on both clusters, you can issue the svctask rmrcconsistgrp command separately on both of the clusters. If the consistency group is not empty, then the relationships within it are removed from the consistency group before the group is deleted. These relationships then become stand-alone relationships. The state of these relationships is not changed by the action of removing them from the consistency group.
12.2.13 Reversing a Metro Mirror relationship To reverse a Metro Mirror relationship, use the command svctask switchrcrelationship.
svctask switchrcrelationship The svctask switchrcrelationship command is used to reverse the roles of primary and secondary VDisk when a stand-alone relationship is in a consistent state. When issuing the command, the desired primary is specified.
12.2.14 Reversing a Metro Mirror consistency group To reverse a Metro Mirror consistency group, use the command svctask switchrcconsistgrp.
svctask switchrcconsistgrp The svctask switchrcconsistgrp command is used to reverse the roles of primary and secondary VDisk when a consistency group is in a consistent state. This change is applied to all the relationships in the consistency group, and when issuing the command, the desired primary is specified.
12.2.15 Background copy Metro Mirror paces the rate at which background copy is performed by the appropriate relationships. Background copy takes place on relationships that are in the InconsistentCopying state with a Status of Online. The quota of background copy (configured on the intercluster link) is divided evenly between the nodes that are performing background copy for one of the eligible relationships. This allocation is made without regard for the number of disks that the node is responsible for. Each node in turn divides its allocation evenly between the multiple relationships performing a background copy. For intracluster relationships, each node is assigned a static quota of 25 MBps.
628
Implementing the IBM System Storage SAN Volume Controller V4.3
12.3 Metro Mirror scenario using the CLI Note: This example is for intercluster only. If you want to set up intracluster, we highlight those parts of the following procedure that you do not need to perform. In the following scenario, we will be setting up an intercluster Metro Mirror relationship between the SVC cluster ITSO-CLS1 primary site and SVC cluster ITSO-CLS2 at the secondary site. Details of the VDisks are shown in Table 12-2. Table 12-2 VDisk details Content of Vdisk
VDisks at primary site
VDisks at secondary site
Database Files
MM_DB_Pri
MM_DB_Sec
Database Log Files
MM_DBLog_Pri
MM_DBLog_Sec
Application Files
MM_App_Pri
MM_App_Sec
Since data consistency is needed across the VDisks MM_DB_Pri and MM_DBLog_Pri, a consistency group CG_WIN2K3_MM is created to handle Metro Mirror relationships for them. While, in this scenario, application files are independent of the database, a stand-alone Metro Mirror relationship is created for the VDisk MM_App_Pri. The Metro Mirror setup is illustrated in Figure 12-7.
Figure 12-7 Metro Mirror scenario
Chapter 12. Copy Services: Metro Mirror
629
12.3.1 Setting up Metro Mirror In the following section, we assume that the source and target VDisks have already been created and that the ISLs and zoning are in place, enabling the SVC clusters to communicate. To set up the Metro Mirror, the following steps must be performed: 1. Create an SVC partnership between ITSO-CLS1 and ITSO-CLS2, on both SVC clusters. 2. Create a Metro Mirror consistency group: – Name CG_W2K3_MM 3. Create the Metro Mirror relationship for MM_DB_Pri: – Master MM_DB_Pri – Auxiliary MM_DB_Sec – Auxiliary SVC cluster ITSO-CLS2 – Name MMREL1 – Consistency group CG_W2K3_MM 4. Create the Metro Mirror relationship for MM_DBLog_Pri: – Master MM_DBLog_Pri – Auxiliary MM_DBLog_Sec – Auxiliary SVC cluster ITSO-CLS2 – Name MMREL2 – Consistency group CG_W2K3_MM 5. Create the Metro Mirror relationship for MM_App_Pri: – Master MM_App_Pri – Auxiliary MM_App_Sec – Auxiliary SVC cluster ITSO-CLS2 – Name MMREL3 In the following section, each step is carried out using the CLI.
Creating an SVC partnership between ITSO-CLS1 and ITSO-CLS2 We create the SVC partnership on both clusters. Note: If you are creating an intracluster Metro Mirror, do not perform the next step; instead, go to “Creating a Metro Mirror Consistency Group” on page 632.
Pre-verification To verify that both clusters can communicate with each other, use the svcinfo lsclustercandidate command. As shown in Example 12-1, ITSO-CLS2 is an eligible SVC cluster candidate at ITSO-CLS1, for the SVC cluster partnership, and vice-versa. This confirms that both clusters are communicating with each other. Example 12-1 Listing the available SVC cluster for partnership
IBM_2145:ITSO-CLS1:admin>svcinfo lsclustercandidate id configured 0000020068603A42 no
cluster_name ITSO-CLS2
IBM_2145:ITSO-CLS2:admin>svcinfo lsclustercandidate id configured 0000020060C06FCA no
630
cluster_name ITSO-CLS1
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 12-2 shows the output of the svcinfo lscluster command, before setting up the Metro Mirror relationship. We show it so you can compare with the same relationship after setting up the Metro Mirror relationship. Example 12-2 Pre-verification of cluster configuration
IBM_2145:ITSO-CLS1:admin>svcinfo lscluster -delim : id:name:location:partnership:bandwidth:cluster_IP_address:cluster_service_IP_addre ss:cluster_IP_address_6:cluster_service_IP_address_6:id_alias 0000020060C06FCA:ITSO-CLS1:local:::9.43.86.117:9.43.86.118:::0000020060C06FCA IBM_2145:ITSO-CLS2:admin>svcinfo lscluster -delim : id:name:location:partnership:bandwidth:cluster_IP_address:cluster_service_IP_addre ss:cluster_IP_address_6:cluster_service_IP_address_6:id_alias 0000020068603A42:ITSO-CLS2:local:::9.43.86.119:9.43.68.120:::0000020068603A42
Partnership between clusters In Example 12-3, a partnership is created between ITSO-CLS1 and ITSO-CL2, specifying 50 MBps bandwidth to be used for the background copy. To check the status of the newly created partnership, issue the command svcinfo lscluster. Also notice that the new partnership is only partially configured. It will remain partially configured until the Metro Mirror relationship is created on the other node. Example 12-3 Creating the partnership from ITSO-CLS1 to ITSO-CLS2 and verifying partnership
IBM_2145:ITSO-CLS1:admin>svctask mkpartnership -bandwidth 50 ITSO-CLS2 IBM_2145:ITSO-CLS1:admin>svcinfo lscluster -delim : id:name:location:partnership:bandwidth:cluster_IP_address:cluster_service_IP_addre ss:cluster_IP_address_6:cluster_service_IP_address_6:id_alias 0000020060C06FCA:ITSO-CLS1:local:::9.43.86.117:9.43.86.118:::0000020060C06FCA 0000020068603A42:ITSO-CLS2:remote:partially_configured_local:50:9.43.86.119:9.43.6 8.120:::0000020068603A42 In Example 12-4, the partnership is created between ITSO-CLS2 back to ITSO-CLS1, specifying the bandwidth to be used for a background copy of 50 MBps. After creating the partnership, verify that the partnership is fully configured on both clusters by re-issuing the svcinfo lscluster command. Example 12-4 Creating the partnership from ITSO-CLS2 to ITSO-CLS1 and verifying partnership
IBM_2145:ITSO-CLS2:admin>svctask mkpartnership -bandwidth 50 ITSO-CLS1 IBM_2145:ITSO-CLS2:admin>svcinfo lscluster -delim : id:name:location:partnership:bandwidth:cluster_IP_address:cluster_service_IP_addre ss:cluster_IP_address_6:cluster_service_IP_address_6:id_alias 0000020068603A42:ITSO-CLS2:local:::9.43.86.119:9.43.68.120:::0000020068603A42 0000020060C06FCA:ITSO-CLS1:remote:fully_configured:50:9.43.86.117:9.43.86.118:::00 00020060C06FCA IBM_2145:ITSO-CLS1:admin>svcinfo lscluster -delim : id:name:location:partnership:bandwidth:cluster_IP_address:cluster_service_IP_addre ss:cluster_IP_address_6:cluster_service_IP_address_6:id_alias Chapter 12. Copy Services: Metro Mirror
631
0000020060C06FCA:ITSO-CLS1:local:::9.43.86.117:9.43.86.118:::0000020060C06FCA 0000020068603A42:ITSO-CLS2:remote:fully_configured:50:9.43.86.119:9.43.68.120:::00 00020068603A42
Creating a Metro Mirror Consistency Group In Example 12-5, we create the Metro Mirror consistency group using the svctask mkrcconsistgrp command. This consistency group will be used for the Metro Mirror relationships of database VDisks, namely MM_DB_Pri and MM_DBLog_Pri, and is named CG_W2K3_MM. Example 12-5 Creating the Global Mirror consistency group CG_W2K3_MM
IBM_2145:ITSO-CLS1:admin>svctask mkrcconsistgrp -cluster ITSO-CLS2 -name CG_W2K3_MM RC Consistency Group, id [255], successfully created IBM_2145:ITSO-CLS1:admin>svcinfo lsrcconsistgrp -delim : id:name:master_cluster_id:master_cluster_name:aux_cluster_id:aux_cluster_name:prim ary:state:relationship_count:copy_type 255:CG_W2K3_MM:0000020060C06FCA:ITSO-CLS1:0000020068603A42:ITSO-CLS2::empty:0:empt y_group
Creating Metro Mirror relationships for MM_DB_Pri and MM_DBLog_Pri In Example 12-6, we create the Metro Mirror relationships MMREL1 and MMREL2, respectively for MM_DB_Pri and MM_DBLog_Pri. Also we make them members of the Metro Mirror consistency group CG_W2K3_MM. We use the svcinfo lsvdisk command to list all the VDisks in the ITSO-CLS1 cluster, and then use the svcinfo lsrcrelationshipcandidate command to show the VDisks in ITSO-CLS2. By using this command, we check the possible candidates for MM_DB_Pri. After checking all the above conditions, use the command svctask mkrcrelationship to create the Metro Mirror relationship. To verify the newly created Metro Mirror relationships, list them with the command svcinfo lsrcrelationship. Example 12-6 Creating Metro Mirror relationships MMREL1 and MMREL2
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk -delim : id:name:IO_group_id:IO_group_name:status:mdisk_grp_id:mdisk_grp_name:capacity:type:FC_id:FC_name :RC_id:RC_name:vdisk_UID:fc_map_count:copy_count 4:MM_DBLog_Pri:0:io_grp0:online:0:MDG_DS45:10.0GB:striped:::::60050768018301BF2800000000000004:0 :1 5:MM_DB_Pri:0:io_grp0:online:0:MDG_DS45:10.0GB:striped:::::60050768018301BF2800000000000005:0:1 6:MM_App_Pri:1:io_grp1:online:1:MDG_DS47:10.0GB:striped:::::60050768018301BF2800000000000006:0:1 IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationshipcandidate -aux ITSO-CLS2 id vdisk_name 4 MM_DB_Sec 5 MM_DBLog_Sec 6 MM_App_Sec IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationshipcandidate -aux ITSO-CLS2 -master MM_DB_Pri id vdisk_name 632
Implementing the IBM System Storage SAN Volume Controller V4.3
4
MM_DB_Sec
IBM_2145:ITSO-CLS1:admin>svctask mkrcrelationship -master MM_DB_Pri -aux MM_DB_Sec -cluster ITSO-CLS2 -consistgrp CG_W2K3_MM -name MMREL1 RC Relationship, id [5], successfully created IBM_2145:ITSO-CLS1:admin>svctask mkrcrelationship -master MM_DBLog_Pri -aux MM_DBLog_Sec -cluster ITSO-CLS2 -consistgrp CG_W2K3_MM -name MMREL2 RC Relationship, id [4], successfully created IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationship -delim : id:name:master_cluster_id:master_cluster_name:master_vdisk_id:master_vdisk_name:aux_cluster_id:a ux_cluster_name:aux_vdisk_id:aux_vdisk_name:primary:consistency_group_id:consistency_group_name: state:bg_copy_priority:progress:copy_type 4:MMREL2:0000020060C06FCA:ITSO-CLS1:4:MM_DBLog_Pri:0000020068603A42:ITSO-CLS2:5:MM_DBLog_Sec:mas ter:255:CG_W2K3_MM:inconsistent_stopped:50:0:metro 5:MMREL1:0000020060C06FCA:ITSO-CLS1:5:MM_DB_Pri:0000020068603A42:ITSO-CLS2:4:MM_DB_Sec:master:25 5:CG_W2K3_MM:inconsistent_stopped:50:0:metro
Creating stand-alone Metro Mirror relationship for MM_App_Pri In Example 12-7, we create the stand-alone Metro Mirror relationship MMREL3 for MM_App_Pri. Once it is created, we check the status of this Metro Mirror relationship. Notice the state of MMREL3 is consistent_stopped. This is because it was created with the -sync option. The -sync option indicates that the secondary (auxiliary) VDisk is already synchronized with the primary (master) VDisk. Initial background synchronization is skipped when this option is used, even though they are not actually synchronized in this scenario. Here it is used to illustrate the option of presynchronized master and auxiliary VDisks, before the setting up of the relationship, and we have created the new relationship for MM_App_Sec using -sync option. Tip: The option -sync is only used when the target VDisk has already mirrored all the data from the source VDisk. By using this option, there will be no initial background copy between the primary VDisk and the secondary VDisk. MMREL2 and MMREL1 are in the inconsistent_stopped state, because they were not created with the -sync option, so their auxiliary VDisks need to be synchronized with their primary VDisks. Example 12-7 Creating a stand-alone Global Mirror relationship and verifying it
IBM_2145:ITSO-CLS1:admin>svctask mkrcrelationship -master MM_App_Pri -aux MM_App_Sec -sync -cluster ITSO-CLS2 -name MMREL3 RC Relationship, id [6], successfully created IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationship 6 id 6 name MMREL3 master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 master_vdisk_id 6 master_vdisk_name MM_App_Pri aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2
Chapter 12. Copy Services: Metro Mirror
633
aux_vdisk_id 6 aux_vdisk_name MM_App_Sec primary master consistency_group_id consistency_group_name state consistent_stopped bg_copy_priority 50 progress freeze_time status online sync in_sync copy_type metro
12.3.2 Starting Metro Mirror Now that the Metro Mirror consistency group and relationships are in place, we are ready to use Metro Mirror relationships in our environment. When implementing Metro Mirror, the goal is to reach a consistent and synchronized state that can provide redundancy for a dataset if a failure occurs that affects the production site. In the following section, we show how to stop and start stand-alone Metro Mirror relationships and consistency groups.
Starting a stand-alone Metro Mirror relationship In Example 12-8, we start a stand-alone Metro Mirror relationship MMREL3. Because the Metro Mirror relationship was in the Consistent stopped state and no updates have been made to the primary VDisk, the relationship quickly enters the Consistent synchronized state. Example 12-8 Starting the stand-alone Metro Mirror relationship
IBM_2145:ITSO-CLS1:admin>svctask startrcrelationship MMREL3 IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationship 6 id 6 name MMREL3 master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 master_vdisk_id 6 master_vdisk_name MM_App_Pri aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 aux_vdisk_id 6 aux_vdisk_name MM_App_Sec primary master consistency_group_id consistency_group_name state consistent_synchronized bg_copy_priority 50 progress freeze_time status online sync copy_type metro
634
Implementing the IBM System Storage SAN Volume Controller V4.3
Starting a Metro Mirror consistency group In Example 12-9, we start the Metro Mirror consistency group CG_W2K3_MM. Because the consistency group was in the Inconsistent stopped state, it enters the Inconsistent copying state until the background copy has completed for all the relationships in the consistency group. Upon completion of the background copy, it enters the Consistent synchronized state. Example 12-9 Starting the Metro Mirror consistency group
IBM_2145:ITSO-CLS1:admin>svctask startrcconsistgrp CG_W2K3_MM IBM_2145:ITSO-CLS1:admin>svcinfo lsrcconsistgrp CG_W2K3_MM id 255 name CG_W2K3_MM master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 primary master state inconsistent_copying relationship_count 2 freeze_time status online sync copy_type metro RC_rel_id 4 RC_rel_name MMREL2 RC_rel_id 5 RC_rel_name MMREL1
Monitoring the background copy progress To monitor the background copy progress, we can use the svcinfo lsrcrelationship command. This command will show us all the defined Metro Mirror relationships if used without any arguments. In the command output, progress indicates the current background copy progress. Our Metro Mirror relationship is shown in Example 12-10. Note: Setting up SNMP traps for the SVC enables automatic notification when Metro Mirror consistency groups or relationships change state. Example 12-10 Monitoring background copy progress example
IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationship MMREL1 id 5 name MMREL1 master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 master_vdisk_id 5 master_vdisk_name MM_DB_Pri aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 aux_vdisk_id 4 aux_vdisk_name MM_DB_Sec
Chapter 12. Copy Services: Metro Mirror
635
primary master consistency_group_id 255 consistency_group_name CG_W2K3_MM state inconsistent_copying bg_copy_priority 50 progress 36 freeze_time status online sync copy_type metro IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationship MMREL2 id 4 name MMREL2 master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 master_vdisk_id 4 master_vdisk_name MM_DBLog_Pri aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 aux_vdisk_id 5 aux_vdisk_name MM_DBLog_Sec primary master consistency_group_id 255 consistency_group_name CG_W2K3_MM state inconsistent_copying bg_copy_priority 50 progress 47 freeze_time status online sync copy_type metro When all Metro Mirror relationships have completed the background copy, the consistency group enters the consistent synchronized state, as shown in Example 12-11. Example 12-11 Listing the Metro Mirror consistency group
IBM_2145:ITSO-CLS1:admin>svcinfo lsrcconsistgrp CG_W2K3_MM id 255 name CG_W2K3_MM master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 primary master state consistent_synchronized relationship_count 2 freeze_time status online sync copy_type metro RC_rel_id 4 RC_rel_name MMREL2 RC_rel_id 5
636
Implementing the IBM System Storage SAN Volume Controller V4.3
RC_rel_name MMREL1
12.3.3 Stopping and restarting Metro Mirror Now that the Metro Mirror consistency group and relationships are running, in this and the following sections, we describe how to stop, restart, and change the direction of the stand-alone Metro Mirror relationships, as well as the consistency group. In this section, we show how to stop and restart the stand-alone Metro Mirror relationships and the consistency group.
Stopping a stand-alone Metro Mirror relationship In Example 12-12, we stop the stand-alone Metro Mirror relationship, while enabling access (write I/O) to both the primary and secondary VDisk, and the relationship enters the Idling state. Example 12-12 Stopping stand-alone Metro Mirror relationship & enabling access to secondary VDisk
IBM_2145:ITSO-CLS1:admin>svctask stoprcrelationship -access MMREL3 IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationship MMREL3 id 6 name MMREL3 master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 master_vdisk_id 6 master_vdisk_name MM_App_Pri aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 aux_vdisk_id 6 aux_vdisk_name MM_App_Sec primary consistency_group_id consistency_group_name state idling bg_copy_priority 50 progress freeze_time status sync in_sync copy_type metro
Stopping a Metro Mirror consistency group In Example 12-13, we stop the Metro Mirror consistency group without specifying the -access flag. This means that the consistency group enters the Consistent stopped state. Example 12-13 Stopping a Metro Mirror consistency group
IBM_2145:ITSO-CLS1:admin>svctask stoprcconsistgrp CG_W2K3_MM IBM_2145:ITSO-CLS1:admin>svcinfo lsrcconsistgrp CG_W2K3_MM id 255 name CG_W2K3_MM master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 aux_cluster_id 0000020068603A42 Chapter 12. Copy Services: Metro Mirror
637
aux_cluster_name ITSO-CLS2 primary master state consistent_stopped relationship_count 2 freeze_time 2008/06/19/18/35/32 status online sync in_sync copy_type metro RC_rel_id 4 RC_rel_name MMREL2 RC_rel_id 5 RC_rel_name MMREL1 If, afterwards, we want to enable access (write I/O) to the secondary VDisk, re-issue svctask stoprcconsistgrp, specifying the -access flag, and the consistency group transits to the Idling state, as shown in Example 12-14. Example 12-14 Stopping a Metro Mirror consistency group and enabling access to the secondary
IBM_2145:ITSO-CLS1:admin>svctask stoprcconsistgrp -access CG_W2K3_MM IBM_2145:ITSO-CLS1:admin>svcinfo lsrcconsistgrp CG_W2K3_MM id 255 name CG_W2K3_MM master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 primary state idling relationship_count 2 freeze_time status sync in_sync copy_type metro RC_rel_id 4 RC_rel_name MMREL2 RC_rel_id 5 RC_rel_name MMREL1
Restarting a Metro Mirror relationship in the Idling state When restarting a Metro Mirror relationship in the Idling state, we must specify the copy direction. If any updates have been performed on either the master or auxiliary VDisk, consistency will be compromised. Therefore, we must issue the command with the -force flag to re-start a relationship, as shown in Example 12-15. Example 12-15 Restarting a Metro Mirror relationship after updates in the Idling state
IBM_2145:ITSO-CLS1:admin>svctask startrcrelationship -primary master -force MMREL3 IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationship MMREL3 id 6 name MMREL3 master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1
638
Implementing the IBM System Storage SAN Volume Controller V4.3
master_vdisk_id 6 master_vdisk_name MM_App_Pri aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 aux_vdisk_id 6 aux_vdisk_name MM_App_Sec primary master consistency_group_id consistency_group_name state consistent_synchronized bg_copy_priority 50 progress freeze_time status online sync copy_type metro
Restarting a Metro Mirror consistency group in the Idling state When restarting a Metro Mirror consistency group in the Idling state, we must specify the copy direction. If any updates have been performed on either the master or the auxiliary VDisk in any of the Metro Mirror relationships in the consistency group, then consistency will be compromised. Therefore, we must use the -force flag to start a relationship. If the -force flag is not used, then the command will fail. In Example 12-16, we change the copy direction by specifying the auxiliary VDisks to be primaries. Example 12-16 Restarting a Metro Mirror relationship while changing the copy direction
IBM_2145:ITSO-CLS1:admin>svctask startrcconsistgrp -primary aux CG_W2K3_MM IBM_2145:ITSO-CLS1:admin>svcinfo lsrcconsistgrp CG_W2K3_MM id 255 name CG_W2K3_MM master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 primary aux state consistent_synchronized relationship_count 2 freeze_time status online sync copy_type metro RC_rel_id 4 RC_rel_name MMREL2 RC_rel_id 5 RC_rel_name MMREL1
Chapter 12. Copy Services: Metro Mirror
639
12.3.4 Changing copy direction for Metro Mirror In this section, we show how to change the copy direction of the stand-alone Metro Mirror relationships and the consistency group.
Switching copy direction for a Metro Mirror relationship When a Metro Mirror relationship is in the consistent synchronized state, we can change the copy direction for the relationship using the command svctask switchrcrelationship, specifying the primary VDisk. If the VDisk specified as the primary when issuing this command is already a primary, then the command has no effect. In Example 12-17, we change the copy direction for the stand-alone Metro Mirror relationship by specifying the auxiliary VDisk to be the primary. Note: When the copy direction is switched, it is crucial that there is no outstanding I/O to the VDisk that is being transited from primary to secondary, since all I/O will be inhibited to that VDisk when it becomes the secondary. Therefore, careful planning is required prior to using the svctask switchrcrelationship command. Example 12-17 Switching the copy direction for a Metro Mirror consistency group
IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationship MMREL3 id 6 name MMREL3 master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 master_vdisk_id 6 master_vdisk_name MM_App_Pri aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 aux_vdisk_id 6 aux_vdisk_name MM_App_Sec primary master consistency_group_id consistency_group_name state consistent_synchronized bg_copy_priority 50 progress freeze_time status online sync copy_type metro IBM_2145:ITSO-CLS1:admin>svctask switchrcrelationship -primary aux MMREL3 IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationship MMREL3 id 6 name MMREL3 master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 master_vdisk_id 6 master_vdisk_name MM_App_Pri aux_cluster_id 0000020068603A42
640
Implementing the IBM System Storage SAN Volume Controller V4.3
aux_cluster_name ITSO-CLS2 aux_vdisk_id 6 aux_vdisk_name MM_App_Sec primary aux consistency_group_id consistency_group_name state consistent_synchronized bg_copy_priority 50 progress freeze_time status online sync copy_type metro
Switching copy direction for a Metro Mirror consistency group When a Metro Mirror consistency group is in the consistent synchronized state, we can change the copy direction for the consistency group, using the command svctask switchrcconsistgrp, specifying the primary VDisk. If the VDisk specified is already a primary when issuing this command, then the command has no effect. In Example 12-18, we change the copy direction for the Metro Mirror consistency group by specifying the auxiliary VDisk to become the primary. Note: When the copy direction is switched, it is crucial that there is no outstanding I/O to the VDisks that transitions from primary to secondary, since all I/O will be inhibited when they become the secondary. Therefore, careful planning is required prior to using the svctask switchrcconsistgrp command. Example 12-18 Switching the copy direction for a Metro Mirror consistency group
IBM_2145:ITSO-CLS1:admin>svcinfo lsrcconsistgrp CG_W2K3_MM id 255 name CG_W2K3_MM master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 primary master state consistent_synchronized relationship_count 2 freeze_time status online sync copy_type metro RC_rel_id 4 RC_rel_name MMREL2 RC_rel_id 5 RC_rel_name MMREL1 IBM_2145:ITSO-CLS1:admin>svctask switchrcconsistgrp -primary aux CG_W2K3_MM IBM_2145:ITSO-CLS1:admin>svcinfo lsrcconsistgrp CG_W2K3_MM
Chapter 12. Copy Services: Metro Mirror
641
id 255 name CG_W2K3_MM master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 primary aux state consistent_synchronized relationship_count 2 freeze_time status online sync copy_type metro RC_rel_id 4 RC_rel_name MMREL2 RC_rel_id 5 RC_rel_name MMREL1
12.4 Metro Mirror scenario using the GUI Next, we show how to set up Metro Mirror using the GUI. Note: This example is for intercluster only, so if you want to set up intracluster, we highlight those parts of the following procedure that you do not need to perform. In the following scenario, we will be setting up an intercluster Metro Mirror relationship between the SVC cluster ITSO-CLS1 at the primary site and the SVC cluster ITSO-CLS2 at the secondary site. Details of the VDisks are shown in Table 12-3. Table 12-3 VDisk details Content of Vdisk
VDisks at primary site
VDisks at secondary site
Database Files
MM_DB_Pri
MM_DB_Sec
Database Log Files
MM_DBLog_Pri
MM_DBLog_Sec
Application Files
MM_App_Pri
MM_App_Sec
Since data consistency is needed across VDisks MM_DB_Pri and MM_DBLog_Pri, a consistency group CG_WIN2K3_MM is created to handle the Metro Mirror relationships for them. While, in this scenario, application files are independent of the database, a stand-alone Metro Mirror relationship is created for VDisk MM_App_Pri. The Metro Mirror setup is illustrated in Figure 12-8 on page 643.
642
Implementing the IBM System Storage SAN Volume Controller V4.3
Primary Site
Secondary Site
SVC Cluster – ITSO-CLS1
SVC Cluster – ITSO-CLS2
Consistency Group CG_W2K3_MM
MM_DB_Pri
MM_Relationship 1
MM_DB_Sec
MM_DBLog_Pri
MM_Relationship 2
MM_DBLog_Sec
MM_App_Pri
MM_Relationship 3
MM_App_Sec
Figure 12-8 Metro Mirror scenario
12.4.1 Setting up Metro Mirror In the following section, we assume that the source and target VDisks have already been created and that the ISLs and zoning are in place, enabling the SVC clusters to communicate. To set up the Metro Mirror, the following steps must be performed: 1. Create SVC partnership between ITSO-CLS1 and ITSO-CLS2, on both SVC clusters. 2. Create a Metro Mirror consistency group: – Name CG_W2K3_MM 3. Create the Metro Mirror relationship for MM_DB_Pri: – Master MM_DB_Pri – Auxiliary MM_DB_Sec – Auxiliary SVC cluster ITSO-CLS2 – Name MMREL1 – Consistency group CG_W2K3_MM 4. Create the Metro Mirror relationship for MM_DBLog_Pri: – Master MM_DBLog_Pri – Auxiliary MM_DBLog_Sec – Auxiliary SVC cluster ITSO-CLS2 – Name MMREL2 – Consistency group CG_W2K3_MM 5. Create the Metro Mirror relationship for MM_App_Pri: – Master MM_App_Pri – Auxiliary MM_App_Sec – Auxiliary SVC cluster ITSO-CLS2 – Name MMREL3 Chapter 12. Copy Services: Metro Mirror
643
Creating the SVC partnership between ITSO-CLS1 and ITSO-CLS2 In this section, each step is carried out using the GUI. We perform this operation on both clusters. Note: If you are creating an intracluster Metro Mirror, do not perform the next step, “Creating Cluster Partnership”; instead go to “Creating a Metro Mirror consistency group” on page 646. To create a Metro Mirror partnership between the SVC clusters using the GUI, we launch the SVC GUI for ITSO-CLS1. Then we select Manage Copy Services and click Metro & Mirror Cluster Partnership, as shown in Figure 12-9.
Figure 12-9 Selecting Metro Mirror Cluster Partnership on ITSO-CLS1
In Figure 12-10 on page 645, the available SVC cluster candidates are listed, which in our case is only ITSO-CLS2. Select ITSO-CLS2 and specify the available bandwidth for the background copy, in this case 50 MBps and then click OK. There are two options available during creation: Inter-Cluster Delay Simulation, which simulates the Global Mirror round trip delay between the two clusters, in milliseconds. The default is 0, and the valid range is 0 to 100 milliseconds. Intra-Cluster Delay Simulation, which simulates the Global Mirror round trip delay in milliseconds. The default is 0, and the valid range is 0 to 100 milliseconds.
644
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 12-10 Selecting the SVC cluster partner and specifying the bandwidth for background copy
In the window that appears (Figure 12-11), the newly created Metro Mirror cluster partnership is shown as Partially Configured.
Figure 12-11 Metro Mirror cluster partnership is partially configured
To fully configure the Metro Mirror cluster partnership, we must carry out the same steps on ITSO-CLS2 as we did on ITSO-CLS1. For simplicity and brevity, only the last two windows are shown.
Chapter 12. Copy Services: Metro Mirror
645
Launching the SVC GUI for ITSO-SVCLS2, we select ITSO-SVCLS1 for the Metro Mirror cluster partnership and specify the available bandwidth for background copy, again 50 MBps, and then click OK, as shown in Figure 12-12.
Figure 12-12 Selecting SVC partner and specify bandwidth for background copy
Now that both sides of the SVC Cluster Partnership are defined, the resulting window shown in Figure 12-13 confirms that our Metro Mirror cluster partnership is Fully Configured.
Figure 12-13 Metro Mirror cluster partnership fully configured
The GUI for ITSO-CLS2 is no longer needed. Close this and use the GUI for cluster ITSO-CLS1 for all further steps.
Creating a Metro Mirror consistency group To create the consistency group to be used for the Metro Mirror relationships of VDisks with database and database log files, we select Manage Copy Services and click Metro Mirror Consistency Groups, as shown in Figure 12-14 on page 647.
646
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 12-14 Selecting Metro Mirror consistency groups
To start the creation process, select Create Consistency Group from drop-down menu and click Go, as shown in Figure 12-15.
Figure 12-15 Create a consistency group
We are presented with the wizard that helps create the Metro Mirror consistency group. The first step in the wizard gives an introduction to the steps involved in the creation of a Metro Mirror consistency group, as shown in Figure 12-16. Click Next to proceed.
Figure 12-16 Introduction to Metro Mirror consistency group creation wizard
Chapter 12. Copy Services: Metro Mirror
647
As shown in Figure 12-17, specify the name for the consistency group and whether it is to be used for intercluster or intracluster relationships. In our scenario, we select intercluster and click Next.
Figure 12-17 Specifying consistency group name and type
Figure 12-18 shows the already existing Metro Mirror relationships that need to be included in the Metro Mirror consistency group. Since we do not have any existing relationships at this point to be included in the Metro Mirror consistency group, we will create a blank group by clicking Next to proceed.
Figure 12-18 Select existing Metro Mirror relationship
Verify the setting for the consistency group and click Finish to create the Metro Mirror consistency group, as shown in Figure 12-19 on page 649.
648
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 12-19 Verify settings for Metro Mirror consistency group
After creation of the consistency group, the GUI returns to the Viewing Metro & Global Mirror Consistency Groups window, as shown in Figure 12-20. This page lists the newly created consistency group.
Figure 12-20 Viewing the newly created consistency group
Creating Metro Mirror relationships for MM_DB_Pri and MM_DBLog_Pri To create the Metro Mirror relationships for VDisks MM_DB_Pri and MM_DBLog_Pri, select Manage Copy Services and click Metro Mirror Cluster Relationships, as shown in Figure 12-21.
Figure 12-21 Selecting Metro Mirror relationships
Chapter 12. Copy Services: Metro Mirror
649
To start the creation process, select Create a Relationship from the drop-down menu and click Go, as shown in Figure 12-22.
Figure 12-22 Create a relationship
We are presented with the wizard that will help us create the Metro Mirror relationship. The first step in wizard gives an introduction to the steps involved in the creation of the Metro Mirror relationship, as shown in Figure 12-23. Click Next to proceed.
Figure 12-23 Introduction to the Metro Mirror relationship creation wizard
As shown in Figure 12-24 on page 651, we name the first Metro Mirror relationship MMREL1 and the type of cluster relationship (in this case intercluster) as per the scenario shown in Figure 12-8 on page 643. It also gives us the option to select the type of copy service, which in our case is Metro Mirror.
650
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 12-24 Naming the Metro Mirror relationship and selecting the type of cluster relationship
The next step will enable us to select a master VDisk. As this list could potentially be large, the Filtering Master VDisks Candidates window appears, which will enable us to reduce the list of eligible VDisks based on a defined filter. In Figure 12-25, use the filter for * (to list all VDisks) and click Next. Tip: In our scenario, we can also input MM* as a filter to avoid listing all the VDisks.
Figure 12-25 Define filter for VDisk candidates
Chapter 12. Copy Services: Metro Mirror
651
As shown in Figure 12-26, we select MM_DB_Pri to be a master VDisk for this relationship, and click Next to proceed.
Figure 12-26 Selecting the master VDisk
The next step requires us to select an auxiliary VDisk. The Metro Mirror relationship wizard will automatically filter this list, so that only eligible VDisks are shown. Eligible VDisks are those that have the same size as the master VDisk and are not already part of a Metro Mirror relationship. As shown in Figure 12-27, we select MM_DB_Sec as the auxiliary VDisk for this relationship, and click Next to proceed.
Figure 12-27 Selecting auxiliary VDisk
As shown in Figure 12-28 on page 653, we select the consistency group that we created and click Next to proceed.
652
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 12-28 Selecting relationship to be a part of consistency group
Finally, in Figure 12-29, we verify the attributes for our Metro Mirror relationship and click Finish to create it.
Figure 12-29 Verifying Metro Mirror relationship
Once the relationship is successfully created, we are returned to the Metro Mirror relationship list. After successful creation of the relationship, the GUI returns to the Viewing Metro & Global Mirror Relationships window, as shown in Figure 12-30. This window lists the newly created relationship.
Figure 12-30 Viewing the Metro Mirror relationship
Chapter 12. Copy Services: Metro Mirror
653
By following a similar process, we create the second Metro Mirror relationship MMREL2, which is shown in Figure 12-31.
Figure 12-31 Viewing the second Metro Mirror relationship MMREL2
Creating a stand-alone Metro Mirror relationship for MM_App_Pri To create stand-alone Metro Mirror relationship, we start the creation process by selecting Create a Relationship from the scroll menu and click Go. Next, we are presented with the wizard that shows the steps involved in the process of creating a consistency group, and we click Next to proceed. As shown in Figure 12-32, we name the relationship (MMREL3), specify that it is an intercluster relationship, and click Next.
Figure 12-32 Specifying the Metro Mirror relationship name and auxiliary cluster
As shown in Figure 12-33 on page 655, we are prompted for a filter prior to presenting the master VDisk candidates. We select the MM* filter and click Next.
654
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 12-33 Filtering VDisk candidates
As shown in Figure 12-34, we select MM_App_Pri to be the master VDisk of the relationship, and click Next to proceed.
Figure 12-34 Selecting the master VDisk
As shown in Figure 12-35, we select MM_DB_Sec as the auxiliary VDisk of the relationship, and click Next to proceed.
Figure 12-35 Selecting the auxiliary VDisk
Chapter 12. Copy Services: Metro Mirror
655
As shown in Figure 12-36, as we did not select a consistency group, we are creating a stand-alone Metro Mirror relationship.
Figure 12-36 Selecting options for the Metro Mirror relationship
Note: To add a Metro Mirror relationship to a consistency group, it must be in the same state as the consistency group. Even if we intended to make Metro Mirror relationship MMREL3 part of the consistency group CG_W2K3_MM, we are not offered this option since the state of relationship MMREL3 is Consistent stopped, because we selected the synchronized option, and the state of consistency group CG_W2K3_MM is currently Inconsistent stopped. This is shown in Figure 12-37.
Figure 12-37 Selecting the Synchronized option for a Metro Mirror relationship
656
Implementing the IBM System Storage SAN Volume Controller V4.3
Finally, Figure 12-38 shows the actions that will be performed. We click Finish to create this new relationship.
Figure 12-38 Verifying the Metro Mirror relationship
After successful creation, we are returned to the Metro Mirror relationship window. Figure 12-39 now shows all our defined Metro Mirror relationships.
Figure 12-39 Viewing Metro Mirror relationships
12.4.2 Starting Metro Mirror Now that we have created the Metro Mirror consistency group and relationships, we are ready to use Metro Mirror relationships in our environment. When performing Metro Mirror, the goal is to reach a consistent and synchronized state that can provide redundancy if a failure occurs that affects the SAN at the production site. In the following section, we show how to stop and start a stand-alone Metro Mirror relationship and consistency group.
Chapter 12. Copy Services: Metro Mirror
657
Starting a stand-alone Metro Mirror relationship In Figure 12-40, we select the stand-alone Metro Mirror relationship MMREL3, and from the drop-down menu, we select Start Copy Process and click Go.
Figure 12-40 Starting a stand-alone Metro Mirror relationship
In Figure 12-41, we do not need to change Forced start, Mark as clean, or Copy direction parameters, as this is the first time we are invoking this Metro Mirror relationship (and we defined the relationship as being already synchronized in Figure 12-37 on page 656). We click OK to start the stand-alone Metro Mirror relationship MMREL3.
Figure 12-41 Selecting options and starting the copy process
Because the Metro Mirror relationship was in the Consistent stopped state and no updates have been made to the primary VDisk, the relationship quickly enters the Consistent synchronized state, as shown in Figure 12-42.
Figure 12-42 Viewing Metro Mirror relationships
658
Implementing the IBM System Storage SAN Volume Controller V4.3
Starting a Metro Mirror consistency group To start the Metro Mirror consistency group CG_W2K3_MM, we select Metro Mirror Consistency Groups, as shown in Figure 12-43.
Figure 12-43 Selecting Metro Mirror consistency groups
In Figure 12-44, we select the Metro Mirror consistency group CG_W2K3_MM, and from the drop-down menu, we select Start Copy Process and click Go.
Figure 12-44 Starting copy process for the consistency group
As shown in Figure 12-45, we click OK to start the copy process. We cannot select the Forced start, Mark as clean, or Copy Direction options, as our consistency group is currently in the Inconsistent stopped state.
Figure 12-45 Selecting options and starting the copy process
Chapter 12. Copy Services: Metro Mirror
659
As shown in Figure 12-46, we are returned to the Metro Mirror consistency group list and the consistency group CG_W2K3_MM has transitioned to the Inconsistent copying state.
Figure 12-46 Viewing Metro Mirror consistency groups
Since the consistency group was in the Inconsistent stopped state, it enters the Inconsistent copying state until the background copy has completed for all the relationships in the consistency group. Upon completion of the background copy for all the relationships in the consistency group, it enters the Consistent synchronized state.
Monitoring background copy progress The status of the background copy progress can either be shown in the Viewing Metro Mirror Relationships window, last column, or by opening it under the My Work, Manage progress view, and clicking View progress. This will allow you to view the Metro Mirror progress, as shown in Figure 12-47.
Figure 12-47 Viewing background copy progress for Metro Mirror relationships
Note: Setting up SNMP traps for the SVC enables automatic notification when the Metro Mirror consistency group or relationships change state.
12.4.3 Stopping and restarting Metro Mirror Now that the Metro Mirror consistency group and relationships are running, in this and the following sections, we describe how to stop, restart, and change the direction of the stand-alone Metro Mirror relationships, as well as the consistency group. In this section, we show how to stop and restart the stand-alone Metro Mirror relationships and the consistency group. 660
Implementing the IBM System Storage SAN Volume Controller V4.3
Stopping a stand-alone Metro Mirror relationship To stop a Metro Mirror relationship, while enabling access (write I/O) to both the primary and secondary VDisk, we select the relationship and select Stop Copy Process from the drop-down menu and click Go, as shown in Figure 12-48.
Figure 12-48 Stopping a stand-alone Metro MIrror relationship
As shown in Figure 12-49, we check the Enable write access... option and click OK to stop the Metro Mirror relationship.
Figure 12-49 Enable access to the secondary VDisk while stopping relationship
As shown in Figure 12-50, the Metro Mirror relationship transits to the Idling state when stopped while enabling access to the secondary VDisk.
Figure 12-50 Viewing the Metro Mirror relationships
Chapter 12. Copy Services: Metro Mirror
661
Stopping a Metro Mirror consistency group As shown in Figure 12-51, we select the Metro Mirror consistency group and Stop Copy Process from the drop-down menu and click Go.
Figure 12-51 Selecting the Metro Mirror consistency group to be stopped
As shown in Figure 12-52, we click OK without specifying Enable write access... to the secondary VDisks.
Figure 12-52 Stopping consistency group without enabling access to secondary VDisks
As shown in Figure 12-53, the consistency group enters the Consistent stopped state, when stopped without enabling access to the secondary.
Figure 12-53 Viewing Metro Mirror consistency groups
If, afterwards, we want to enable write access (write I/O) to the secondary VDisks, we can reissue the Stop Copy Process and this time specify that we want to enable write access to the secondary VDisks.
662
Implementing the IBM System Storage SAN Volume Controller V4.3
In Figure 12-54, we select the Metro Mirror relationship and select Stop Copy Process from drop-down menu and click Go.
Figure 12-54 Stopping the Metro Mirror consistency group
As shown in Figure 12-55, we check the Enable write access... check box and click OK.
Figure 12-55 Enabling access to secondary VDisks
When applying the Enable write access... option, the consistency group transits to the Idling state, as shown in Figure 12-56.
Figure 12-56 Viewing Metro Mirror consistency group in the Idling state
Restarting a Metro Mirror relationship in the Idling state When restarting a Metro Mirror relationship in the Idling state, we must specify the copy direction. If any updates have been performed on either the master or auxiliary VDisks in the Metro Mirror relationship, then consistency will have been compromised. In this situation, we must check the Force option to start the copy process, otherwise the command will fail.
Chapter 12. Copy Services: Metro Mirror
663
As shown in Figure 12-57, we select the Metro Mirror relationship and Start Copy Process from the drop-down menu and click Go.
Figure 12-57 Starting a stand-alone Metro Mirror relationship in the Idling state
As shown in Figure 12-58, we check the Force option, since write I/O has been performed while in the Idling state, and we select the copy direction by defining the master VDisk as the primary, and click OK.
Figure 12-58 Specifying options while starting copy process
The Metro Mirror relationship enters the Consistent copying state. When background copy is complete, the relationship transits to the Consistent synchronized state, as shown in Figure 12-59.
Figure 12-59 Viewing Metro Mirror relationship
664
Implementing the IBM System Storage SAN Volume Controller V4.3
Restarting a Metro Mirror consistency group in the Idling state When restarting a Metro Mirror consistency group in the Idling state, we must specify the copy direction. If any updates have been performed on either the master or auxiliary VDisk in any of the Metro Mirror relationships in the consistency group, then consistency will have been compromised. In this situation, we must check the Force option to start the copy process, otherwise the command will fail. As shown in Figure 12-60, we select the Metro Mirror consistency group and Start Copy Process from the drop-down menu and click Go.
Figure 12-60 Starting the copy process for the consistency group
As shown in Figure 12-61, we check the Force option and set the copy direction by selecting the primary as the master.
Figure 12-61 Specifying the options while starting the copy process in the consistency group
When the background copy completes, the Metro Mirror consistency group enters the Consistent synchronized state shown in Figure 12-62.
Figure 12-62 Viewing Metro Mirror consistency groups
Chapter 12. Copy Services: Metro Mirror
665
12.4.4 Changing copy direction for Metro Mirror In this section, we show how to change the copy direction of the stand-alone Metro Mirror relationships and the consistency group.
Switching copy direction for a Metro Mirror consistency group When a Metro Mirror consistency group is in the Consistent synchronized state, we can change the copy direction for the Metro Mirror consistency group. In Figure 12-63, we select the consistency group CG_W2K3_MM and Switch Copy Direction from the drop-down menu and click Go. Note: When the copy direction is switched, it is crucial that there is no outstanding I/O to the VDisks that will change from primary to secondary, since all I/O will be inhibited when the Vdisks become secondary. Therefore, careful planning is required prior to switching the copy direction.
Figure 12-63 Selecting the consistency group for which the copy direction is to be changed
In Figure 12-64, we see that the currently primary VDisks are the master. So, to change the copy direction for the Metro Mirror consistency group, we specify the auxiliary VDisks to become the primary, and click OK.
Figure 12-64 Selecting primary VDisk, as auxiliary, to switch the copy direction
666
Implementing the IBM System Storage SAN Volume Controller V4.3
The copy direction is now switched and we are returned to the Metro Mirror consistency group list, where we see that the copy direction has been switched, as shown in Figure 12-65.
Figure 12-65 Viewing Metro Mirror consistency group after changing the copy direction
In Figure 12-66, we show the new copy direction for individual relationships within that consistency group.
Figure 12-66 Viewing Metro Mirror relationship after changing the copy direction
Switching the copy direction for a Metro Mirror relationship When a Metro Mirror relationship is in the Consistent synchronized state, we can change the copy direction for relationship. In Figure 12-67 on page 668, we select the relationship MMREL3 and Switch Copy Direction from the drop-down menu and click Go. Note: When the copy direction is switched, it is crucial that there is no outstanding I/O to the VDisk that transits from primary to secondary, since all I/O will be inhibited to that VDisk when it becomes the secondary. Therefore, careful planning is required prior to switching the copy direction for a Metro Mirror relationship.
Chapter 12. Copy Services: Metro Mirror
667
Figure 12-67 Selecting relationship whose copy direction needs to be changed
In Figure 12-68, we see that the current primary VDisk is the master, so to change the copy direction for the stand-alone Metro Mirror relationship, we specify the auxiliary VDisk to be the primary, and click OK.
Figure 12-68 Selecting primary VDisk, as auxiliary, to switch copy direction
The copy direction is now switched and we are returned to the Metro Mirror relationship list, where we see that the copy direction has been switched and the auxiliary VDisk has become the primary, as shown in Figure 12-69.
Figure 12-69 Viewing Metro Mirror relationships
668
Implementing the IBM System Storage SAN Volume Controller V4.3
13
Chapter 13.
Copy Services: Global Mirror In this chapter, we describe the Global Mirror (GM) copy service, which is an asynchronous remote copy service. It provides and maintains a consistent mirrored copy of a source VDisk to a target VDisk. Data is written from the source VDisk to the target VDisk asynchronously. This method was previously known as Asynchronous Peer-to-Peer Remote Copy.
© Copyright IBM Corp. 2003-2008. All rights reserved.
669
13.1 Global Mirror overview Global Mirror works by defining a Global Mirror relationship between two VDisks of equal size and maintains the data consistency in an asynchronous manner. Therefore, when a host writes to a source VDisk, the data is copied from the source VDisk cache to the target VDisk cache. At the initiation of that data copy, confirmation of I/O completion is transmitted back to the host. Note: The minimum firmware requirement for GM functionality is V4.1.1. Any cluster or partner cluster not running this minimum level will not have GM functionality available. Even if you have a Global Mirror relationship running on a downlevel partner cluster and you only wish to use intracluster GM, the functionality will not be available to you. SVC provides both intracluster and intercluster Global Mirror.
13.1.1 Intracluster Global Mirror Although Global Mirror is available for intracluster, it has no functional value for production use. Intracluster Metro Mirror provides the same capability with less overhead. However, leaving this functionality in place simplifies testing and does allow for customer experimentation and testing (for example, to validate server failover on a single test cluster).
13.1.2 Intercluster Global Mirror Intercluster Global Mirror operations require a pair of SVC clusters that are commonly separated by a number of moderately high bandwidth links. The two SVC clusters must be defined in an SVC cluster partnership to establish a fully functional Global Mirror relationship. Note: When a local and a remote fabric are connected together for Global Mirror purposes, the ISL hop count between a local node and a remote node should not exceed seven hops.
13.2 Remote copy techniques Global Mirror is an asynchronous remote copy that is briefly explained below. To illustrate the differences between synchronous and asynchronous remote copy, synchronous remote copy is also explained below.
13.2.1 Asynchronous remote copy Global Mirror is an asynchronous remote copy technique. In asynchronous remote copy, write operations are completed on the primary site and the write acknowledgement is sent to the host before it is received at the secondary site. An update of this write operation is sent to the secondary site at a later stage. This provides the capability of performing remote copy over distances exceeding the limitations of synchronous remote copy. Figure 13-1 on page 671 shows that a write operation to the master VDisk is acknowledged back to the host issuing the write before it is mirrored to the cache for the auxiliary VDisk.
670
Implementing the IBM System Storage SAN Volume Controller V4.3
Host
1 W r it e
2 Ack
C ache
3 W r ite to r e m o te
M a s te r V D is k
C ache
A u x ilia r y V D is k G lo b a l M ir r o r R e la tio n s h ip
Figure 13-1 Write on VDisk in Global Mirror relationship
In a failover scenario, where the secondary site needs to become the primary source of data, some updates might be missing at the secondary site. Therefore, any applications that will use this data must have some external mechanism for recovery the missing updates and reapplying them, for example, transaction log replay.
13.2.2 Synchronous remote copy The synchronous remote copy technique ensures that updates are committed at both primary and secondary VDisks before the application completes an update. Figure 13-2 on page 672 illustrates how a write operation to the master VDisk is mirrored to the cache for the auxiliary VDisk before an acknowledge of the write is sent back to the host issuing the write. This ensures that the secondary is real-time synchronized, in case it is needed in a failover situation. However, this also means that the application is fully exposed to the latency and bandwidth limitations of the communication link to the secondary site. This might lead to unacceptable application performance, particularly when placed under peak load. This is the reason for the distance limitations when using Metro Mirror.
Chapter 13. Copy Services: Global Mirror
671
Host
1 W r ite
4 Ack 3 A c k n o w le d g e W r ite
C ache
C ache 2 M ir r o r w r ite
M a s te r V D is k
A u x ilia r y V D is k M e tr o M ir r o r R e la tio n s h ip
Figure 13-2 Write on VDisk in Metro Mirror relationship
13.2.3 SVC Global Mirror features SVC Global Mirror supports the following features: Asynchronous remote copy of VDisks dispersed over metropolitan scale distances is supported. SVC implements the Global Mirror relationship between a VDisk pair, with each VDisk in the pair being managed by an SVC cluster. SVC supports intracluster Global Mirror, where both VDisks belong to the same cluster (and IO group). Although, as stated earlier, this functionality is better suited to Metro Mirror. SVC supports intercluster Global Mirror, where each VDisk belongs to their separate SVC cluster. A given SVC cluster can be configured for partnership with another cluster. A given SVC cluster can only communicate with one other cluster. All intercluster Global Mirror takes place between the two SVC clusters in the configured partnership. Intercluster and intracluster Global Mirror can be used concurrently within a cluster for different relationships. SVC does not require a control network or fabric to be installed to manage Global Mirror. For intercluster Global Mirror, the SVC maintains a control link between the two clusters. This control link is used to control state and coordinate updates at either end. The control link is implemented on top of the same FC fabric connection as the SVC uses for Global Mirror I/O. SVC implements a configuration model that maintains the Global Mirror configuration and state through major events, such as failover, recovery, and resynchronization, to minimize user configuration action through these events. SVC maintains and polices a strong concept of consistency and makes this available to guide configuration activity.
672
Implementing the IBM System Storage SAN Volume Controller V4.3
SVC implements flexible resynchronization support, enabling it to re-synchronize VDisk pairs that have suffered write I/O to both disks and to resynchronize only those regions that are known to have changed.
13.2.4 Global Mirror relationships Global Mirror relationships are similar to FlashCopy mappings. They can be stand-alone or combined in consistency groups. The start and stop commands can be issued either against the stand-alone relationship or the consistency group. Figure 13-3 illustrates the Global Mirror relationship.
Figure 13-3 Global Mirror relationship
A Global Mirror relationship is composed of two VDisks that are equal in size. The master VDisk and the auxiliary VDisk can be in the same I/O group, within the same SVC cluster (intracluster Global Mirror), or can be on separate SVC clusters that are defined as SVC partners (intercluster Global Mirror). Note: Be aware that: A VDisk can only be part of one Global Mirror relationship at a time. A VDisk that is a FlashCopy target cannot be part of a Global Mirror relationship.
Global Mirror relationship between primary and secondary VDisk When creating a Global Mirror relationship, the master VDisk is initially assigned as the primary, and the auxiliary VDisk as the secondary. This implies that the initial copy direction is mirroring the master VDisk to the auxiliary VDisk. After the initial synchronization is complete, the copy direction can be changed, if appropriate. In most common applications of Global Mirror, the master VDisk contains the production copy of the data, and is used by the host application, while the auxiliary VDisk contains the mirrored copy of the data and is used for failover in disaster recovery scenarios. The terms master and auxiliary help support this use. If Global Mirror is applied differently, the terms master and auxiliary VDisks need to be interpreted appropriately.
Importance of write ordering Many applications that uses block storage have a requirement to survive failures such as loss of power or a software crash, and not to lose data that existed prior to the failure. Since many applications need to perform large numbers of update operations in parallel to that storage, maintaining write ordering is key to ensuring the correct operation of applications following a disruption.
Chapter 13. Copy Services: Global Mirror
673
An application, for example, databases, that is performing a large set of updates is usually designed with the concept of dependent writes. These are the writes where it is important to ensure that an earlier write has completed before a later write is started. Reversing the order of dependent writes can undermine the applications algorithms and can lead to problems, such as detected or undetected data corruption.
Dependent writes that span multiple VDisks The following scenario illustrates a simple example of a sequence of dependent writes, and in particular what can happen if they span multiple VDisks. Consider the following typical sequence of writes for a database update transaction: 1. A write is executed to update the database log, indicating that a database update is to be performed. 2. A second write is executed to update the database. 3. A third write is executed to update the database log, indicating that the database update has completed successfully. The write sequence is illustrated in Figure 13-4. Time Log
Step 1
DB file
Log
Step 2 DB file
Log
Step 3
Log: Update record xyz ... started
DB file
Log: Update record xyz ... started
DB: record xyz ...
Log: Update record xyz ... started Log: Update record xyz ... completed
DB: record xyz ...
Figure 13-4 Dependent writes for a database
The database ensures the correct ordering of these writes by waiting for each step to complete before starting the next. Note: All databases have logs associated with them. These logs keep records of database changes. If a database needs to be restored to a point beyond the last full, offline backup, logs are required to roll the data forward to the point of failure.
674
Implementing the IBM System Storage SAN Volume Controller V4.3
But imagine if the database log and the database itself are on different VDisks and a Global Mirror relationship is stopped during this update. In this case, you need to consider the possibility that the Global Mirror relationship for the VDisk with the database file is stopped slightly before the VDisk containing the database log. If this were the case, then it could be possible that the secondary VDisks see writes (1) and (3) but not (2). Then, if the database was restarted using the data available from the secondary disks, the database log would indicate that the transaction had completed successfully, when this is not the case. In this scenario, the integrity of the database is in question.
Global Mirror consistency groups Global Mirror consistency groups address the issue of dependent writes across VDisks, where the objective is to preserve data consistency across multiple Global Mirrored VDisks. Consistency groups ensure a consistent data set, because applications have relational data spanning across multiple VDisks. A Global Mirror consistency group can contain an arbitrary number of relationships up to the maximum number of Global Mirror relationships supported by the SVC Cluster. Global Mirror commands can be issued to a Global Mirror consistency group, and thereby simultaneously for all Global Mirror relationships defined within that consistency group, or to a single Metro Mirror relationship, if not part of a Global Mirror consistency group. For example, when issuing a Global Mirror start command to the consistency group, all of the Global Mirror relationships in the consistency group are started at the same time.
Chapter 13. Copy Services: Global Mirror
675
In Figure 13-5, the concept of Global Mirror consistency groups is illustrated. Since the GM_Relationship 1 and GM_Relationship 2 are part of the consistency group, they can be handled as one entity, while the stand-alone GM_Relationship 3 is handled separately.
Consistency Group GM1
VDisk1M GM_Master
GM_Relationship 1
VDisk1A GM_Auxiliary
VDisk2M GM_Master
GM_Relationship 2
VDisk2A GM_Auxiliary
VDisk3M GM_Master
GM_Relationshi 3
VDisk3A GM_Auxiliary
Figure 13-5 Global Mirror consistency group
Certain uses of Global Mirror require manipulation of more than one relationship. Global Mirror consistency groups can provide the ability to group relationships so that they are manipulated in unison. Global Mirror relationships within a consistency group can be in any form: Global Mirror relationships can be part of a consistency group, or be stand-alone and therefore handled as single instances. A consistency group can contain zero or more relationships. An empty consistency group, with zero relationships in it, has little purpose until it is assigned its first relationship, except that it has a name. All the relationships in a consistency group must have matching master and auxiliary SVC clusters. Although it is possible to use consistency groups to manipulate sets of relationships that do not need to satisfy these strict rules, such manipulation can lead to some undesired side effects. The rules behind a consistency group mean that certain configuration commands are prohibited. Where this would not be the case was if the relationship was not part of a consistency group. For example, consider the case of two applications that are completely independent, yet they are placed into a single consistency group. In the event of an error there is a loss of synchronization, and a background copy process is required to recover synchronization.
676
Implementing the IBM System Storage SAN Volume Controller V4.3
While this process is in progress, Global Mirror rejects attempts to enable access to secondary VDisks of either application. If one application finishes its background copy much more quickly than the other, Global Mirror still refuses to grant access to its secondary, even though it is safe in this case, because Global Mirror policy is to refuse access to the entire consistency group if any part of it is inconsistent. Stand-alone relationships and consistency groups share a common configuration and state model. All the relationships in a non-empty consistency group have the same state as the consistency group.
13.2.5 How Global Mirror works This section discusses how Global Mirror works.
Intercluster communication and zoning All intercluster communication is performed through the SAN. Prior to creating intercluster Global Mirror relationships, you must create a partnership between the two clusters. All SVC node ports on each SVC cluster must be able to access each other to facilitate the partnership creation. Therefore, a zone in each fabric must be defined for intercluster communication; see Chapter 3, “Planning and configuration” on page 25.
SVC Cluster partnership Each SVC cluster can only be in a partnership with one other SVC cluster. When the SVC cluster partnership has been defined on both clusters, further communication facilities between the nodes in each of the cluster are established. This comprises: A single control channel, which is used to exchange and coordinate configuration information I/O channels between each of the nodes in the clusters These channels are maintained and updated as nodes appear and disappear and as links fail, and are repaired to maintain operation where possible. If communication between the SVC clusters is interrupted or lost, an error is logged (and consequently Global Mirror relationships will stop). To handle error conditions, the SVC can be configured to raise SNMP traps to the enterprise monitoring system.
Maintenance of the intercluster link All SVC nodes maintain a database of the other devices that are visible on the fabric. This is updated as devices appear and disappear. Devices that advertise themselves as SVC nodes are categorized according to the SVC cluster to which they belong. SVC nodes that belong to the same cluster establish communication channels between themselves and begin to exchange messages to implement the clustering and functional protocols of SVC. Nodes that are in different clusters do not exchange messages after the initial discovery is complete unless they have been configured together to perform Global Mirror.
Chapter 13. Copy Services: Global Mirror
677
The intercluster link carries the control traffic to coordinate activity between the two clusters. It is formed between one node in each cluster, which is termed the focal point. The traffic between the focal point nodes is distributed among the logins that exist between those nodes. If the focal point node should fail (or all its logins to the remote cluster fail), then a new focal point is chosen to carry the control traffic. Changing the focal point causes I/O to pause but does not cause relationships to become Consistent Stopped.
13.2.6 Global Mirror process There are several steps in the Global Mirror process: 1. An SVC cluster partnership is created between two SVC clusters (for intercluster Global Mirror). 2. A Global Mirror relationship is created between two VDisks of the same size. 3. To manage multiple Global Mirror relationships as one entity, the relationships can be made part of a Global Mirror consistency group. This is to ensure data consistency across multiple Global Mirror relationships, or simply for ease of management. 4. The Global Mirror relationship is started, and when the background copy has completed, the relationship is consistent and synchronized. 5. Once synchronized, the secondary VDisk holds a copy of the production data at the primary that can be used for disaster recovery. 6. To access the auxiliary VDisk, the Global Mirror relationship must be stopped with the access option enabled, before write I/O is submitted to the secondary. 7. The remote host server is mapped to the auxiliary VDisk and the disk is available for I/O.
13.2.7 Methods of synchronization This section describes three methods that can be used to establish a relationship.
Full synchronization after creation This is the default method. It is the simplest and it requires no administrative activity apart from issuing the necessary commands. However, in some environments, the bandwidth available will make this method unsuitable. The sequence for a single relationship is as follows: A new relationship is created (mkrcrelationship is issued) without specifying the -sync flag. A new relationship is started (startrcrelationship is issued) without the -clean flag.
Synchronized before creation In this method, the administrator must ensure that the master and auxiliary virtual disks contain identical data before creating the relationship. There are two ways in which this might be done: Both disks are created with the security delete (-fmtdisk) feature so as to make all data zero. A complete tape image (or other method of moving data) is copied from one disk to the other. In either technique, no write I/O must take place either on Master or Auxiliary before the relationship is established. 678
Implementing the IBM System Storage SAN Volume Controller V4.3
Then, the administrator must ensure that: A new relationship is created (mkrcrelationship is issued) with the -sync flag. A new relationship is started (startrcrelationship is issued) without the -clean flag. If these steps are not performed correctly, the relationship will be reported as being consistent, when it is not. This is likely to make any secondary disk useless. This method has an advantage over full synchronization: It does not require all the data to be copied over a constrained link. However, if the data needs to be copied, the master and auxiliary disks cannot be used until the copy is complete, which might be unacceptable.
Quick synchronization after creation In this method, the administrator must still copy data from master to auxiliary, but it can be used without stopping the application at the master. The administrator must ensure that: A new relationship is created (mkrcrelationship is issued) with the -sync flag. A new relationship is stopped (mkrcrelationship is issued) with the -access flag. A tape image (or other method of transferring data) is used to copy the entire master disk to the auxiliary disk. Once the copy is complete, the administrator must ensure that a new relationship is started (startrcrelationship is issued) with the -clean flag. With this technique, only the data that has changed since the relationship was created, including all regions that were incorrect in the tape image, are copied from master and auxiliary. As with “Synchronized before creation” on page 678, the copy step must be performed correctly, or else the auxiliary will be useless, although the copy will report it as being synchronized.
Chapter 13. Copy Services: Global Mirror
679
Global Mirror states and events In this section, we explain the different states of a Global Mirror relationship, and the series of events that modify these states. In Figure 13-6, the Global Mirror relationship state diagram shows an overview of the states that apply to a Global Mirror relationship in the connected state.
1a (in
c) syn
4b
Stop, Enable access
(ou t of
For ced Star t (out of s y nc) Stop
Start
or Error
3 Consistent Synchronized
Background copy complete
5a 4a
1b syn c)
Inconsistent Stopped
Consistent Stopped
2a
Create Global Mirror Relationship
Stop, Enable access
Start (in sync)
2b Start
Stop or Error
Inconsistent Copying
5b rt Sta d e c For c) syn f o t (ou
Idle
Figure 13-6 Global Mirror state diagram
When creating the Global Mirror relationship, you can specify if the auxiliary VDisk is already in sync with the master VDisk, and the background copy process is then skipped. This is especially useful when creating Global Mirror relationships for VDisks that have been created with the format option. The following steps explain the Global Mirror state diagram: 1. Step 1 is done as follows: a. The Global Mirror relationship is created with the -sync option and the Global Mirror relationship enters the Consistent stopped state. b. The Global Mirror relationship is created without specifying that the master and auxiliary VDisks are in sync, and the Global Mirror relationship enters the Inconsistent stopped state. 2. Step 2 is done as follows: a. When starting a Global Mirror relationship in the Consistent stopped state, it enters the Consistent synchronized state. This implies that no updates (write I/O) have been performed on the primary VDisk while in the Consistent stopped state, otherwise the
680
Implementing the IBM System Storage SAN Volume Controller V4.3
-force option must be specified, and the Global Mirror relationship then enters the Inconsistent copying state, while the background copy is started. b. When starting a Global Mirror relationship in the Inconsistent stopped state, it enters the Inconsistent copying state, while the background copy is started. 3. Step 3 is done as follows: a. When the background copy completes, the Global Mirror relationship transits from the Inconsistent copying state to the Consistent synchronized state. 4. Step 4 is done as follows: a. When stopping a Global Mirror relationship in the Consistent synchronized state, where specifying the -access option enables write I/O on the secondary VDisk, the Global Mirror relationship enters the Idling state. b. To enable write I/O on the secondary VDisk, when the Global Mirror relationship is in the Consistent stopped state, issue the command svctask stoprcrelationship, specifying the -access option, and the Global Mirror relationship enters the Idling state. 5. Step 5 is done as follows: a. When starting a Global Mirror relationship that is in the Idling state, you must specify the -primary argument to set the copy direction. Given that no write I/O has been performed (to either the master or auxiliary VDisk) while in the Idling state, the Global Mirror relationship enters the Consistent synchronized state. b. In case write I/O has been performed to either the master or the auxiliary VDisk, then the -force option must be specified, and the Global Mirror relationship then enters the Inconsistent copying state, while the background copy is started. Stop or Error: When a Global Mirror relationship is stopped (either intentionally or due to an error), a state transition is applied. For example, this means that Global Mirror relationships in the Consistent synchronized state enter the Consistent stopped state and Global Mirror relationships in the Inconsistent copying state enter the Inconsistent stopped state. In a case where the connection is broken between the SVC clusters in a partnership, then all (intercluster) Global Mirror relationships enter a disconnected state. For further information, refer to “Connected versus disconnected” on page 681. Note: Stand-alone relationships and consistency groups share a common configuration and state model. This means that all the Global Mirror relationships in a non-empty consistency group have the same state as the consistency group.
13.2.8 State overview The SVC defined concepts of state are key to understanding the configuration concepts and are therefore explained in more detail below.
Connected versus disconnected This distinction can arise when a Global Mirror relationship is created with the two virtual disks in different clusters. Under certain error scenarios, communications between the two clusters might be lost. For example, power might fail causing one complete cluster to disappear. Alternatively, the fabric connection between the two clusters might fail, leaving the two clusters running but unable to communicate with each other.
Chapter 13. Copy Services: Global Mirror
681
When the two clusters can communicate, the clusters and the relationships spanning them are described as connected. When they cannot communicate, the clusters and the relationships spanning them are described as disconnected. In this scenario, each cluster is left with half the relationship and has only a portion of the information that was available to it before. Some limited configuration activity is possible, and is a subset of what was possible before. The disconnected relationships are portrayed as having a changed state. The new states describe what is known about the relationship, and what configuration commands are permitted. When the clusters can communicate again, the relationships become connected once again. Global Mirror automatically reconciles the two state fragments, taking into account any configuration or other event that took place while the relationship was disconnected. As a result, the relationship can either return to the state it was in when it became disconnected or it can enter a different connected state. Relationships that are configured between virtual disks in the same SVC cluster (intracluster) will never be described as being in a disconnected state.
Consistent versus inconsistent Relationships or consistency groups that contains relationships can be described as being consistent or inconsistent. The consistent or inconsistent property describes the state of the data on the secondary in relation to that on the primary VDisk. It can be considered a property of the secondary VDisk itself. A secondary is described as consistent if it contains data that could have been read by a host system from the primary if power had failed at some imaginary point in time while I/O was in progress and power was later restored. This imaginary point in time is defined as the recovery point. The requirements for consistency are expressed with respect to activity at the primary up to the recovery point: The secondary VDisk contains the data from all writes to the primary, for which the host had received good completion and that data had not been overwritten by a subsequent write (before the recovery point). The writes, for which the host did not receive good completion (that is, the host received bad completion or no completion at all) and the host subsequently performed a read from the primary of that data. If that read returned good completion and no later write was sent (before the recovery point), then the secondary contains the same data as that returned by the read from the primary. From the point of view of an application, consistency means that a secondary VDisk contains the same data as the primary VDisk at the recovery point (the time at which the imaginary power failure occurred). If an application is designed to cope with an unexpected power failure, this guarantee of consistency means that the application will be able to use the secondary and begin operation just as though it had been restarted after the hypothetical power failure. Again, the application is dependent on the key properties of consistency: Write ordering Read stability for correct operation at the secondary
682
Implementing the IBM System Storage SAN Volume Controller V4.3
If a relationship, or a set of relationships, is inconsistent and an attempt is made to start an application using the data in the secondaries, a number of outcomes are possible: The application might decide that the data is corrupt and crash or exit with an error code. The application might fail to detect that the data is corrupt and return erroneous data. The application might work without a problem. Because of the risk of data corruption, and in particular undetected data corruption, Global Mirror strongly enforces the concept of consistency and prohibits access to inconsistent data. Consistency as a concept can be applied to a single relationship or a set of relationships in a consistency group. Write ordering is a concept that an application can maintain across a number of disks accessed through multiple systems and therefore consistency must operate across all those disks. When deciding how to use consistency groups, the administrator must consider the scope of an application’s data, taking into account all the interdependent systems that communicate and exchange information. If two programs or systems communicate and store details as a result of the information exchanged, then either of the following actions might occur: All the data accessed by the group of systems must be placed into a single consistency group. The systems must be recovered independently (each within its own consistency group). Then, each system must perform recovery with the other applications to become consistent with them.
Consistent versus synchronized A copy that is consistent and up-to-date is described as synchronized. In a synchronized relationship, the primary and secondary virtual disks are only different in regions where writes are outstanding from the host. Consistency does not mean that the data is up-to-date. A copy can be consistent and yet contain data that was frozen at some point in time in the past. Write I/O might have continued to a primary and not have been copied to the secondary. This state arises when it becomes impossible to keep up-to-date and maintain consistency. An example is a loss of communication between clusters when writing to the secondary. When communication is lost for an extended period of time, Global Mirror tracks the changes that happen at the primary, but not the order of such changes, or the details of such changes (write data). When communication is restored, it is impossible to make the secondary synchronized without sending write data to the secondary out-of-order, and therefore losing consistency. Two policies can be used to cope with this: Make a point-in-time copy of the consistent secondary before allowing the secondary to become inconsistent. In the event of a disaster, before consistency is achieved again, the point-in-time copy target provides a consistent, though out-of-date, image. Accept the loss of consistency, and loss of useful secondary, while making it synchronized.
Chapter 13. Copy Services: Global Mirror
683
13.2.9 Detailed states The following sections detail the states that are portrayed to the user, for either consistency groups or relationships. It also details the extra information available in each state. The different major states are constructed to provide guidance as to the configuration commands that are available.
InconsistentStopped This is a connected state. In this state, the primary is accessible for read and write I/O, but the secondary is not accessible for either. A copy process needs to be started to make the secondary consistent. This state is entered when the relationship or consistency group was InconsistentCopying and has either suffered a persistent error or received a stop command that has caused the copy process to stop. A start command causes the relationship or consistency group to move to the InconsistentCopying state. A stop command is accepted, but has no effect. If the relationship or consistency group becomes disconnected, the secondary side transits to InconsistentDisconnected. The primary side transits to IdlingDisconnected.
InconsistentCopying This is a connected state. In this state, the primary is accessible for read and write I/O, but the secondary is not accessible for either read or write I/O. This state is entered after a start command is issued to an InconsistentStopped relationship or consistency group. It is also entered when a forced start is issued to an Idling or ConsistentStopped relationship or consistency group. In this state, a background copy process runs that copies data from the primary to the secondary virtual disk. In the absence of errors, an InconsistentCopying relationship is active, and the Copy Progress increases until the copy process completes. In some error situations, the copy progress might freeze or even regress. A persistent error or Stop command places the relationship or consistency group into the InconsistentStopped state. A start command is accepted, but has no effect. If the background copy process completes on a stand-alone relationship, or on all relationships for a consistency group, the relationship or consistency group transits to the ConsistentSynchronized state. If the relationship or consistency group becomes disconnected, then the secondary side transits to InconsistentDisconnected. The primary side transitions to IdlingDisconnected.
ConsistentStopped This is a connected state. In this state, the secondary contains a consistent image, but it might be out-of-date with respect to the primary. This state can arise when a relationship was in Consistent Synchronized state and suffers an error that forces a Consistency Freeze. It can also arise when a relationship is created with a CreateConsistentFlag set to TRUE.
684
Implementing the IBM System Storage SAN Volume Controller V4.3
Normally, following an I/O error, subsequent write activity cause updates to the primary and the secondary is no longer synchronized (set to FALSE). In this case, to re-establish synchronization, consistency must be given up for a period. A start command with the -force option must be used to acknowledge this, and the relationship or consistency group transits to InconsistentCopying. Do this only after all outstanding errors are repaired. In the unusual case where the primary and secondary are still synchronized (perhaps following a user stop, and no further write I/O was received), a start command takes the relationship to ConsistentSynchronized. No -force option is required. Also, in this unusual case, a switch command is permitted that moves the relationship or consistency group to ConsistentSynchronized and reverses the roles of the primary and secondary. If the relationship or consistency group becomes disconnected, then the secondary side transits to ConsistentDisconnected. The primary side transitions to IdlingDisconnected. An informational status log is generated every time a relationship or consistency group enters the ConsistentStopped with a status of Online state. This can be configured to enable an SNMP trap and provide a trigger to automation software to consider issuing a start command following a loss of synchronization.
ConsistentSynchronized This is a connected state. In this state, the primary VDisk is accessible for read and write I/O. The secondary VDisk is accessible for read-only I/O. Writes that are sent to the primary VDisk are sent to both primary and secondary VDisks. Either good completion must be received for both writes, the write must be failed to the host, or a state must transit out of ConsistentSynchronized state before a write is completed to the host. A stop command takes the relationship to the ConsistentStopped state. A stop command with the -access parameter takes the relationship to the Idling state. A switch command leaves the relationship in the ConsistentSynchronized state, but reverses the primary and secondary roles. A start command is accepted, but has no effect. If the relationship or consistency group becomes disconnected, the same transitions are made as for ConsistentStopped.
Idling This is a connected state. Both master and auxiliary disks are operating in the primary role. Consequently, both are accessible for write I/O. In this state, the relationship or consistency group accepts a start command. Global Mirror maintains a record of regions on each disk that received write I/O while Idling. This is used to determine what areas need to be copied following a start command. The start command must specify the new copy direction. A start command can cause a loss of consistency if either VDisk in any relationship has received write I/O. This is indicated by the synchronized status. If the start command leads to loss of consistency, then a -force parameter must be specified. Following a start command, the relationship or consistency group transits to ConsistentSynchronized if there is no loss of consistency, or to InconsistentCopying if there is such a loss.
Chapter 13. Copy Services: Global Mirror
685
Also, while in this state, the relationship or consistency group accepts a -clean option on the start command. If the relationship or consistency group becomes disconnected, then both sides change their state to IdlingDisconnected.
IdlingDisconnected This is a disconnected state. The VDisk or disks in this half of the relationship or consistency group are all in the primary role and accept read or write I/O. The main priority in this state is to recover the link and make the relationship or consistency group connected once more. No configuration activity is possible (except for deletes or stops) until the relationship becomes connected again. At that point, the relationship transits to a connected state. The exact connected state that is entered depends on the state of the other half of the relationship or consistency group, which depends on: The state when it became disconnected The write activity since it was disconnected The configuration activity since it was disconnected If both halves are IdlingDisconnected, then the relationship becomes Idling when reconnected. While IdlingDisconnected, if a write I/O is received that causes loss of synchronization (synchronized attribute transits from TRUE to FALSE) and the relationship was not already stopped (either through user stop or a persistent error), then an error log is raised to notify this. This error log is the same as that raised when the same situation arises when ConsistentSynchronized.
InconsistentDisconnected This is a disconnected state. The virtual disks in this half of the relationship or consistency group are all in the secondary role and do not accept read or write I/O. No configuration activity except for deletes is permitted until the relationship becomes connected again. When the relationship or consistency group becomes connected again, the relationship becomes InconsistentCopying automatically unless either: The relationship was InconsistentStopped when it became disconnected. The user issued a stop while disconnected. In either case, the relationship or consistency group becomes InconsistentStopped.
ConsistentDisconnected This is a disconnected state. The VDisks in this half of the relationship or consistency group are all in the secondary role and accept read I/O but not write I/O. This state is entered from ConsistentSynchronized or ConsistentStopped when the secondary side of a relationship becomes disconnected. In this state, the relationship or consistency group displays an attribute of FreezeTime, which is the point in time that Consistency was frozen. When entered from ConsistentStopped, it retains the time it had in that state. When entered from ConsistentSynchronized, the FreezeTime shows the last time at which the relationship or consistency group was known to be consistent. This corresponds to the time of the last successful heartbeat to the other cluster. 686
Implementing the IBM System Storage SAN Volume Controller V4.3
A stop command with the -access flag set to TRUE transits the relationship or consistency group to the IdlingDisconnected state. This allows write I/O to be performed to the secondary VDisk and is used as part of a disaster recovery scenario. When the relationship or consistency group becomes connected again, the relationship or consistency group becomes ConsistentSynchronized only if this does not lead to a loss of Consistency. This is the case provided: The relationship was ConsistentSynchronized when it became disconnected. No writes received successful completion at the primary while disconnected. Otherwise, the relationship become ConsistentStopped. The FreezeTime setting is retained.
Empty This state only applies to consistency groups. It is the state of a consistency group that has no relationships and no other state information to show. It is entered when a consistency group is first created. It is exited when the first relationship is added to the consistency group, at which point the state of the relationship becomes the state of the consistency group.
Background copy Global Mirror paces the rate at which background copy is performed by the appropriate relationships. Background copy takes place on relationships that are in the InconsistentCopying state with a Status of Online. The quota of background copy (configured on the intercluster link) is divided evenly between all the nodes that are performing background copy for one of the eligible relationships. This allocation is made irrespective of the number of disks that node is responsible for. Each node in turn divides its allocation evenly between the multiple relationships performing a background copy. For intracluster relationships, each node is assigned a static quota of 25 MBps.
13.2.10 Practical use of Global Mirror To use Global Mirror, a relationship must be defined between two VDisks. When creating the Global Mirror relationship, one VDisk is defined as the master, and the other as the auxiliary. The relationship between the two copies is asymmetric. When the Global Mirror relationship is created, the master VDisk is initially considered the primary copy (often referred to as the source), and the auxiliary VDisk is considered the secondary copy (often referred to as the target). The master VDisk is the production VDisk, and updates to this copy are real time mirrored to the auxiliary VDisk. The contents of the auxiliary VDisk that existed when the relationship was created are destroyed. Note: The copy direction for a Global Mirror relationship can be switched so the auxiliary VDisk becomes the primary and the master VDisk becomes the secondary.
Chapter 13. Copy Services: Global Mirror
687
While the Global Mirror relationship is active, the secondary copy (VDisk) is not accessible for host application write I/O at any time. The SVC allows read-only access to the secondary VDisk when it contains a “consistent” image. This is only intended to allow boot time operating system discovery to complete without error, so that any hosts at the secondary site can be ready to start up the applications with minimal delay if required. For example, many operating systems need to read Logical Block Address (LBA) 0 (zero) to configure a logical unit. Although read access is allowed at the secondary in practice, the data on the secondary volumes cannot be read by a host. The reason for this is that most operating systems write a “dirty bit” to the file system when it is mounted. Because this write operation is not allowed on the secondary volume, the volume cannot be mounted. This access is only provided where consistency can be guaranteed. However, there is no way in which coherency can be maintained between reads performed at the secondary and later write I/Os performed at the primary. To enable access to the secondary VDisk for host operations, the Global Mirror relationship must be stopped by specifying the -access parameter. While access to the secondary VDisk for host operations is enabled, the host must be instructed to mount the VDisk and other related tasks, before the application can be started or instructed to perform a recovery process. The Global Mirror requirement to enable the secondary copy for access differentiates it from, for example, third-party mirroring software on the host, which aims to emulate a single, reliable disk regardless of which system is accessing it. Global Mirror retains the property that there are two volumes in existence, but suppresses one while the copy is being maintained. Using a secondary copy demands a conscious policy decision by the administrator that a failover is required, and the tasks to be performed on the host involved in establishing operation on the secondary copy are substantial. The goal is to make this rapid (much faster when compared to recovering from a backup copy) but not seamless. The failover process can be automated through failover management software. The SVC provides Simple Network Management Protocol (SNMP) traps and programming (or scripting) for the command-line interface (CLI) to enable this automation.
13.2.11 Global Mirror configuration limits Table 13-1 lists the Global Mirror configuration limits. Table 13-1 Global Mirror configuration limits
688
Parameter
Value
Number of Global Mirror consistency groups
255 per SVC cluster.
Number of Global Mirror relationships
1024 per SVC cluster.
Total VDisk size per I/O group
1024 TB is the per I/O group limit on the quantity of primary and secondary VDisk address space that can participate in Global Mirror relationships.
Implementing the IBM System Storage SAN Volume Controller V4.3
13.3 Global Mirror commands Here we summarize some of the most important Global Mirror commands. For complete details about all the Global Mirror commands, see IBM System Storage SAN Volume Controller: Command-Line Interface User's Guide, SC26-7903. The command set for Global Mirror contains two broad groups: Commands to create, delete, and manipulate relationships and consistency groups Commands that cause state changes Where a configuration command affects more than one cluster, Global Mirror performs the work to coordinate configuration activity between the clusters. Some configuration commands can only be performed when the clusters are connected and fail with no effect when they are disconnected. Other configuration commands are permitted even though the clusters are disconnected. The state is reconciled automatically by Global Mirror when the clusters become connected once more. For any given command, with one exception, a single cluster actually receives the command from the administrator. This is significant for defining the context for a CreateRelationShip (mkrcrelationship) or CreateConsistencyGroup (mkrcconsistgrp) command, in which case, the cluster receiving the command is called the local cluster. This exception, as mentioned previously, is the command that sets clusters into a Global Mirror partnership. The mkpartnership command must be issued to both the local and to the remote cluster. The commands are described here as an abstract command set. These are implemented as: A command-line interface (CLI), which can be used for scripting and automation A graphical user interface (GUI), which can be used for one-off tasks
13.3.1 Listing the available SVC cluster partners To create an SVC cluster partnership, we use the command svcinfo lsclustercandidate.
svcinfo lsclustercandidate The svcinfo lsclustercandidate command is used to list the clusters that are available for setting up a two-cluster partnership. This is a prerequisite for creating Global Mirror relationships. To display the characteristics of the cluster, use the command svcinfo lscluster, specifying the name of the cluster.
svctask chcluster There are three parameters for Global Mirror in the command output: -gmlinktolerance link_tolerance Specifies the maximum period of time that the system will tolerate delay before stopping GM relationships. Specify values between 60 and 86400 seconds in increments of 10 seconds. The default value is 300. Do not change this value except under direction of IBM suPport.
Chapter 13. Copy Services: Global Mirror
689
-gminterdelaysimulation link_tolerance Specifies the number of milliseconds that I/O activity (intercluster copying to a secondary VDisk) is delayed. This permits you to test performance implications before deploying Global Mirror and obtaining a long distance link. Specify a value from 0 to 100 milliseconds in 1 millisecond increments. The default value is 0. Use this argument to test each intercluster Global Mirror relationship separately. -gmintradelaysimulation link_tolerance Specifies the number of milliseconds that I/O activity (intracluster copying to a secondary VDisk) is delayed. This permits you to test performance implications before deploying Global Mirror and obtaining a long distance link. Specify a value from 0 to 100 milliseconds in 1 millisecond increments. The default value is 0. Use this argument to test each intracluster Global Mirror relationship separately. Using svctask chcluster to adjust these values should be done as follows: svctask chcluster -gmlinktolerance 300 You can view all the above parameter values with the svcinfo lscluster command.
13.3.2 Creating an SVC cluster partnership To create an SVC cluster partnership, use the command svctask mkpartnership.
svctask mkpartnership The svctask mkpartnership command is used to establish a one-way Global Mirror partnership between the local cluster and a remote cluster. To establish a fully functional Global Mirror partnership, you must issue this command on both clusters. This step is a prerequisite for creating Global Mirror relationships between VDisks on the SVC clusters. When creating the partnership, you can specify the bandwidth to be used by the background copy process between the local and the remote SVC cluster, and if it is not specified, the bandwidth defaults to 50 MBps. The bandwidth should be set to a value that is less than or equal to the bandwidth that can be sustained by the intercluster link.
Background copy bandwidth impact on foreground I/O latency The background copy bandwidth determines the rate at which the background copy will be attempted for the SAN Volume Controller Global Mirror. The background copy bandwidth can affect foreground I/O latency in one of three ways: The following result can occur if the background copy bandwidth is set too high compared to the Global Mirror intercluster link capacity: – The background copy I/Os can back up on the Global Mirror intercluster link. – There is a delay in the synchronous secondary writes of foreground I/Os. – The foreground I/O latency will increase as perceived by applications. If the background copy bandwidth is set too high for the storage at the primary site, background copy read I/Os overload the primary storage and delay foreground I/Os. If the background copy bandwidth is set too high for the storage at the secondary site, background copy writes at the secondary overload the secondary storage and again delay the synchronous secondary writes of foreground I/Os.
690
Implementing the IBM System Storage SAN Volume Controller V4.3
In order to set the background copy bandwidth optimally, make sure that you consider all three resources (the primary storage, the intercluster link bandwidth, and the secondary storage). Provision the most restrictive of these three resources between the background copy bandwidth and the peak foreground I/O workload. This provisioning can be done by calculation as above or alternatively by determining experimentally how much background copy can be allowed before the foreground I/O latency becomes unacceptable and then backing off to accommodate peaks in workload and some additional safety margin.
svctask chpartnership To change the bandwidth available for background copy in an SVC cluster partnership, the command svctask chpartnership can be used to specify the new bandwidth.
13.3.3 Creating a Global Mirror consistency group To create a Global Mirror consistency group, use the command svctask mkrcconsistgrp.
svctask mkrcconsistgrp The svctask mkrcconsistgrp command is used to create a new, empty Global Mirror consistency group. The Global Mirror consistency group name must be unique across all consistency groups known to the clusters owning this consistency group. If the consistency group involves two clusters, the clusters must be in communication throughout the creation process. The new consistency group does not contain any relationships and will be in the empty state. Global Mirror relationships can be added to the group, either upon creation or afterwards, using the svctask chrelationship command.
13.3.4 Creating a Global Mirror relationship To create a Global Mirror relationship, use the command svctask mkrcrelationship. Note: If you do not use the -global optional parameter, a Metro Mirror relationship will be made instead of a Global Mirror relationship.
svctask mkrcrelationship The svctask mkrcrelationship command is used to create a new Global Mirror relationship. This relationship persists until it is deleted. The auxiliary virtual disk must be equal in size to the master virtual disk or the command will fail, and if both VDisks are in the same cluster, they must both be in the same I/O group. The master and auxiliary VDisk cannot be in an existing relationship, and they cannot be the target of a FlashCopy mapping. This command returns the new relationship (relationship_id) when successful. When creating the Global Mirror relationship, it can be added to an already existing consistency group, or be a stand-alone Global Mirror relationship if no consistency group is specified. To check whether the master or auxiliary VDisks comply with the prerequisites to participate in a Global Mirror relationship, use the command svcinfo lsrcrelationshipcandidate, as shown in “svcinfo lsrcrelationshipcandidate” on page 692.
Chapter 13. Copy Services: Global Mirror
691
svcinfo lsrcrelationshipcandidate The svcinfo lsrcrelationshipcandidate command is used to list the available VDisks eligible to form a Global Mirror relationship. When issuing the command, you can specify the master VDisk name and auxiliary cluster to list candidates that comply with the prerequisites to create a Global Mirror relationship. If the command is issued with no parameters, all VDisks that are not disallowed by some other configuration state, such as being a FlashCopy target, are listed.
13.3.5 Changing a Global Mirror relationship To modify the properties of a Global Mirror relationship, use the command svctask chrcrelationship.
svctask chrcrelationship The svctask chrcrelationship command is used to modify the following properties of a Global Mirror relationship: Change the name of a Global Mirror relationship. Add a relationship to a group. Remove a relationship from a group using the -force flag. Note: When adding a Global Mirror relationship to a consistency group that is not empty, the relationship must have the same state and copy direction as the group in order to be added to it.
13.3.6 Changing a Global Mirror consistency group To change the name of a Global Mirror consistency group, we use the command svctask chrcconsistgrp.
svctask chrcconsistgrp The svctask chrcconsistgrp command is used to change the name of a Global Mirror consistency group.
13.3.7 Starting a Global Mirror relationship To start a stand-alone Global Mirror relationship, use the command svctask startrcrelationship.
svctask startrcrelationship The svctask startrcrelationship command is used to start the copy process of a Global Mirror relationship. When issuing the command, the copy direction can be set if undefined, and, optionally, mark the secondary VDisk of the relationship as clean. The command fails if it is used as an attempt to start a relationship that is already a part of a consistency group. This command can only be issued to a relationship that is connected. For a relationship that is idling, this command assigns a copy direction (primary and secondary roles) and begins the copy process. Otherwise, this command restarts a previous copy process that was stopped either by a stop command or by some I/O error.
692
Implementing the IBM System Storage SAN Volume Controller V4.3
If the resumption of the copy process leads to a period when the relationship is not consistent, then you must specify the -force parameter when restarting the relationship. This situation can arise if, for example, the relationship was stopped, and then further writes were performed on the original primary of the relationship. The use of the -force parameter here is a reminder that the data on the secondary will become inconsistent while resynchronization (background copying) takes place, and therefore is not usable for disaster recovery purposes before the background copy has completed. In the idling state, you must specify the primary VDisk to indicate the copy direction. In other connected states, you can provide the primary argument, but it must match the existing setting.
13.3.8 Stopping a Global Mirror relationship To stop a stand-alone Global Mirror relationship, use the command svctask stoprcrelationship.
svctask stoprcrelationship The svctask stoprcrelationship command is used to stop the copy process for a relationship. It can also be used to enable write access to a consistent secondary VDisk by specifying the -access parameter. This command applies to a stand-alone relationship. It is rejected if it is addressed to a relationship that is part of a consistency group. You can issue this command to stop a relationship that is copying from primary to secondary. If the relationship is in an inconsistent state, any copy operation stops and does not resume until you issue an svctask startrcrelationship command. Write activity is no longer copied from the primary to the secondary VDisk. For a relationship in the ConsistentSynchronized state, this command causes a Consistency Freeze. When a relationship is in a consistent state (that is, in the ConsistentStopped, ConsistentSynchronized, or ConsistentDisconnected state), then the -access parameter can be used with the svctask stoprcrelationship command to enable write access to the secondary virtual disk.
13.3.9 Starting a Global Mirror consistency group To start a Global Mirror consistency group, use the command svctask startrcconsistgrp.
svctask startrcconsistgrp The svctask startrcconsistgrp command is used to start a Global Mirror consistency group. This command can only be issued to a consistency group that is connected. For a consistency group that is idling, this command assigns a copy direction (primary and secondary roles) and begins the copy process. Otherwise, this command restarts a previous copy process that was stopped either by a stop command or by some I/O error.
Chapter 13. Copy Services: Global Mirror
693
13.3.10 Stopping a Global Mirror consistency group To stop a Global Mirror consistency group, use the command svctask stoprcconsistgrp.
svctask stoprcconsistgrp The svctask startrcconsistgrp command is used to stop the copy process for a Global Mirror consistency group. It can also be used to enable write access to the secondary VDisks in the group if the group is in a consistent state. If the consistency group is in an inconsistent state, any copy operation stops and does not resume until you issue the svctask startrcconsistgrp command. Write activity is no longer copied from the primary to the secondary VDisks, which belong to the relationships in the group. For a consistency group in the ConsistentSynchronized state, this command causes a Consistency Freeze. When a consistency group is in a consistent state (for example, in the ConsistentStopped, ConsistentSynchronized, or ConsistentDisconnected state), then the -access parameter can be used with the svctask stoprcconsistgrp command to enable write access to the secondary VDisks within that group.
13.3.11 Deleting a Global Mirror relationship To delete a Global Mirror relationship, use the command svctask rmrcrelationship.
svctask rmrcrelationship The svctask rmrcrelationship command is used to delete the relationship that is specified. Deleting a relationship only deletes the logical relationship between the two virtual disks. It does not affect the virtual disks themselves. If the relationship is disconnected at the time that the command is issued, then the relationship is only deleted on the cluster on which the command is being run. When the clusters reconnect, then the relationship is automatically deleted on the other cluster. Alternatively, if the clusters are disconnected, and you still wish to remove the relationship on both clusters, you can issue the rmrcrelationship command independently on both of the clusters. A relationship cannot be deleted if it is part of a consistency group. You must first remove the relationship from the consistency group. If you delete an inconsistent relationship, the secondary virtual disk becomes accessible even though it is still inconsistent. This is the one case in which Global Mirror does not inhibit access to inconsistent data.
13.3.12 Deleting a Global Mirror consistency group To delete a Global Mirror consistency group, use the command svctask rmrcconsistgrp.
svctask rmrcconsistgrp The svctask rmrcconsistgrp command is used to delete a Global Mirror consistency group. This command deletes the specified consistency group. You can issue this command for any existing consistency group.
694
Implementing the IBM System Storage SAN Volume Controller V4.3
If the consistency group is disconnected at the time that the command is issued, then the consistency group is only deleted on the cluster on which the command is being run. When the clusters reconnect, the consistency group is automatically deleted on the other cluster. Alternatively, if the clusters are disconnected, and you still want to remove the consistency group on both clusters, you can issue the svctask rmrcconsistgrp command separately on both of the clusters. If the consistency group is not empty, then the relationships within it are removed from the consistency group before the group is deleted. These relationships then become stand-alone relationships. The state of these relationships is not changed by the action of removing them from the consistency group.
13.3.13 Reversing a Global Mirror relationship To reverse a Global Mirror relationship, use the svctask switchrcrelationship command.
svctask switchrcrelationship The svctask switchrcrelationship command is used to reverse the roles of primary and secondary VDisk when a stand-alone relationship is in a consistent state; when issuing the command, the desired primary needs to be specified.
13.3.14 Reversing a Global Mirror consistency group To reverse a Global Mirror consistency group, use the command svctask switchrcconsistgrp.
svctask switchrcconsistgrp The svctask switchrcconsistgrp command is used to reverse the roles of primary and secondary VDisk when a consistency group is in a consistent state. This change is applied to all the relationships in the consistency group, and when issuing the command, the desired primary needs to be specified.
13.4 Global Mirror scenario using the CLI In the following scenario, we will be setting up an intercluster Global Mirror relationship between the SVC cluster ITSO-CLS1 at the primary site and SVC cluster ITSO-CLS2 at the secondary site. Note: This example is for intercluster only. In case you wish to set up intracluster, we highlight those parts in the following procedure that you do not need to perform. The details of the VDisks are shown in Table 13-2. Table 13-2 Details of VDisks for Global Mirror relationship scenario Content of Vdisk
VDisks at primary site
VDisks at secondary site
Database Files
GM_DB_Pri
GM_DB_Sec
Database Log Files
GM_DBLog_Pri
GM_DBLog_Sec
Application Files
GM_App_Pri
GM_App_Sec
Chapter 13. Copy Services: Global Mirror
695
Since data consistency is needed across GM_DB_Pri and GM_DBLog_Pri, we create a consistency group to handle Global Mirror relationships for them. While in this scenario the application files are independent of the database, we create a stand-alone Global Mirror relationship for GM_App_Pri. The Global Mirror relationship setup is illustrated in Figure 13-7. Primary Site
Secondary Site
SVC Cluster – ITSO-CLS1
SVC Cluster – ITSO-CLS2
Consistency Group CG_W2K3_GM
GM_DB_Pri
GM Relationship 1
GM_DB_Sec
GM_DBLog_Pri
GM Relationship 2
GM_DBLog_Sec
GM_App_Pri
GM Relationship 3
GM_App_Sec
Figure 13-7 Global Mirror Scenario using the CLI
13.4.1 Setting up Global Mirror In the following section, we assume that the source and target VDisks have already been created and that the ISLs and zoning are in place, enabling the SVC clusters to communicate. To set up the Global Mirror, the following steps must be performed: 1. Create a SVC partnership between ITSO_CLS1 and ITSO_CLS2, on both SVC clusters: – Bandwidth 10 MBps 2. Create a Global Mirror consistency group: – Name CG_W2K3_GM 3. Create the Global Mirror relationship for GM_DB_Pri: – Master GM_DB_Pri – Auxiliary GM_DB_Sec – Auxiliary SVC cluster ITSO-CLS2 – Name GMREL1 – Consistency group CG_W2K3_GM 4. Create the Global Mirror relationship for GM_DBLog_Pri: – Master GM_DBLog_Pri – Auxiliary GM_DBLog_Sec – Auxiliary SVC cluster ITSO-CLS2 – Name GMREL2 – Consistency group CG_W2K3_GM 5. Create the Global Mirror relationship for GM_App_Pri: – Master GM_App_Pri – Auxiliary GM_App_Sec
696
Implementing the IBM System Storage SAN Volume Controller V4.3
– Auxiliary SVC cluster ITSO-CLS2 – Name GMREL3 In the following sections, each step is carried out using the CLI.
Creating an SVC partnership between ITSO-CLS1 and ITSO-CLS2 We create an SVC partnership between both clusters. Note: If you are creating an intracluster Global Mirror, do not perform the next step; instead, go to “Changing link tolerance and cluster delay simulation” on page 698.
Pre-verification To verify that both clusters can communicate with each other, use the svcinfo lsclustercandidate command. Example 13-1 confirms that our clusters are communicating, as ITSO-CLS2 is an eligible SVC cluster candidate, at ITSO-CLS1, for the SVC cluster partnership and vice versa. This confirms that both clusters are communicating with each other. Example 13-1 Listing the available SVC clusters for partnership
IBM_2145:ITSO-CLS1:admin>svcinfo lsclustercandidate id configured 0000020068603A42 no
cluster_name ITSO-CLS2
IBM_2145:ITSO-CLS2:admin>svcinfo lsclustercandidate id configured 0000020060C06FCA no
cluster_name ITSO-CLS1
In Example 13-2, we show the output of svcinfo lscluster, before setting up the SVC clusters’ partnership for Global Mirror. It is shown for comparison after we have set up the SVC partnership. Example 13-2 Pre-verification of cluster configuration
IBM_2145:ITSO-CLS1:admin>svcinfo lscluster -delim : id:name:location:partnership:bandwidth:cluster_IP_address:cluster_service_IP_addre ss:cluster_IP_address_6:cluster_service_IP_address_6:id_alias 0000020060C06FCA:ITSO-CLS1:local:::9.43.86.117:9.43.86.118:::0000020060C06FCA IBM_2145:ITSO-CLS2:admin>svcinfo lscluster -delim : id:name:location:partnership:bandwidth:cluster_IP_address:cluster_service_IP_addre ss:cluster_IP_address_6:cluster_service_IP_address_6:id_alias 0000020068603A42:ITSO-CLS2:local:::9.43.86.119:9.43.68.120:::0000020068603A42
Chapter 13. Copy Services: Global Mirror
697
Partnership between clusters In Example 13-3, we create the partnership from ITSO-CLS1 to ITSO-CLS2, specifying 10 MBps bandwidth to be used for the background copy. To verify the status of the newly created partnership, we issue the command svcinfo lscluster. Notice that the new partnership is only partially configured. It will remain partially configured until we run mkpartnership on the other cluster. Example 13-3 Creating the partnership from ITSO-CLS1 to ITSO-CLS2 and verifying the partnership
IBM_2145:ITSO-CLS1:admin>svctask mkpartnership -bandwidth 10 ITSO-CLS2 IBM_2145:ITSO-CLS1:admin>svcinfo lscluster -delim : id:name:location:partnership:bandwidth:cluster_IP_address:cluster_service_IP_addre ss:cluster_IP_address_6:cluster_service_IP_address_6:id_alias 0000020060C06FCA:ITSO-CLS1:local:::9.43.86.117:9.43.86.118:::0000020060C06FCA 0000020068603A42:ITSO-CLS2:remote:partially_configured_local:10:9.43.86.119:9.43.6 8.120:::0000020068603A42 In Example 13-4, we create the partnership from ITSO-CLS2 back to ITSO-CLS1, specifying 10 MBps bandwidth to be used for the background copy. After creating the partnership, verify that the partnership is fully configured by re-issuing the svcinfo lscluster command. Example 13-4 Creating the partnership from ITSO-CLS2 to ITSO-CLS1 and verify partnership
IBM_2145:ITSO-CLS2:admin>svctask mkpartnership -bandwidth 10 ITSO-CLS1 IBM_2145:ITSO-CLS2:admin>svcinfo lscluster -delim : id:name:location:partnership:bandwidth:cluster_IP_address:cluster_service_IP_addre ss:cluster_IP_address_6:cluster_service_IP_address_6:id_alias 0000020068603A42:ITSO-CLS2:local:::9.43.86.119:9.43.68.120:::0000020068603A42 0000020060C06FCA:ITSO-CLS1:remote:fully_configured:10:9.43.86.117:9.43.86.118:::00 00020060C06FCA IBM_2145:ITSO-CLS1:admin>svcinfo lscluster -delim : id:name:location:partnership:bandwidth:cluster_IP_address:cluster_service_IP_addre ss:cluster_IP_address_6:cluster_service_IP_address_6:id_alias 0000020060C06FCA:ITSO-CLS1:local:::9.43.86.117:9.43.86.118:::0000020060C06FCA 0000020068603A42:ITSO-CLS2:remote:fully_configured:10:9.43.86.119:9.43.68.120:::00 00020068603A42
Changing link tolerance and cluster delay simulation The gm_link_tolerance defines how sensitive the SVC is to inter-link overload conditions. The value is the number of seconds of continuous link difficulties that will be tolerated before the SVC will stop the remote copy relationships in order to prevent impacting host I/O at the primary site. In order to change the value, use the following command: svctask chcluster -gmlinktolerance link_tolerance The link_tolerance value is between 60 and 86400 seconds in increments of 10 seconds. The default value for the link tolerance is 300 seconds. A value of 0 causes link tolerance to be disabled.
698
Implementing the IBM System Storage SAN Volume Controller V4.3
Recommendation: We strongly recommend that you use the default value. If the link is overloaded for a period, which would impact host I/O at the primary site, the relationships will be stopped to protect those hosts.
Intercluster and intracluster delay simulation This Global Mirror feature permits a simulation of a delayed write to a remote VDisk. This feature allows testing to be performed that detects colliding writes, and so can be used to test an application before full deployment of the Global Mirror feature. The delay simulation can be enabled separately for each of intracluster or intercluster Global Mirror. To enable this feature, you need to run the following command either for the intracluster or intercluster simulation: For intercluster: svctask chcluster -gminterdelaysimulation For intracluster: svctask chcluster -gmintradelaysimulation inter_cluster_delay_simulation and intra_cluster_delay_simulation express the amount of time (in milliseconds) secondary I/Os are delayed respectively for intercluster and intracluster relationships. These values specify the number of milliseconds that I/O activity, that is, copying a primary VDisk to a secondary VDisk, is delayed. A value from 0 to 100 milliseconds in 1 millisecond increments can be set for the cluster_delay_simulation in the commands above. A value of zero disables the feature. To check the current settings for the delay simulation, use the following command: svcinfo lscluster In Example 13-5, we show the modification of the delay simulation value and a change of the Global Mirror link tolerance parameters. We also show the changed values of the Global Mirror link tolerance and delay simulation parameters. Example 13-5 Delay simulation and link tolerance modification
IBM_2145:ITSO-CLS1:admin>svctask chcluster -gminterdelaysimulation 20 IBM_2145:ITSO-CLS1:admin>svctask chcluster -gmintradelaysimulation 40 IBM_2145:ITSO-CLS1:admin>svctask chcluster -gmlinktolerance 200 IBM_2145:ITSO-CLS1:admin>svcinfo lscluster ITSO-CLS1 id 0000020060C06FCA name ITSO-CLS1 location local partnership bandwidth cluster_IP_address 9.43.86.117 cluster_service_IP_address 9.43.86.118 total_mdisk_capacity 756.0GB space_in_mdisk_grps 756.0GB space_allocated_to_vdisks 339.02GB total_free_space 417.0GB statistics_status off statistics_frequency 15 required_memory 8192 cluster_locale en_US Chapter 13. Copy Services: Global Mirror
699
SNMP_setting all SNMP_community SVC SNMP_server_IP_address 9.43.86.160 subnet_mask 255.255.252.0 default_gateway 9.43.85.1 time_zone 520 US/Pacific email_setting email_id code_level 4.3.0.0 (build 8.15.0806110000) FC_port_speed 2Gb console_IP 9.43.86.115:9080 id_alias 0000020060C06FCA gm_link_tolerance 200 gm_inter_cluster_delay_simulation 20 gm_intra_cluster_delay_simulation 40
Creating a Global Mirror consistency group In Example 13-6, we create the Global Mirror consistency group using the svctask mkrcconsistgrp command. This consistency group will be used for the Global Mirror relationships for the database VDisks and is named CG_W2K3_GM. Example 13-6 Creating the Global Mirror consistency group CG_W2K3_GM
IBM_2145:ITSO-CLS1:admin>svctask mkrcconsistgrp -cluster ITSO-CLS2 -name CG_W2K3_GM RC Consistency Group, id [255], successfully created IBM_2145:ITSO-CLS1:admin>svcinfo lsrcconsistgrp -delim : id:name:master_cluster_id:master_cluster_name:aux_cluster_id:aux_cluster_name:prim ary:state:relationship_count:copy_type 255:CG_W2K3_GM:0000020060C06FCA:ITSO-CLS1:0000020068603A42:ITSO-CLS2::empty:0:empt y_group
Creating Global Mirror relationship for GM_DB_Pri and GM_DBLog_Pri In Example 13-8 on page 701, we create the Global Mirror relationships GMREL1 and GMREL2 for VDisks GM_DB_Pri and GM_DBLog_Pri, respectively. We also make them members of the Global Mirror consistency group CG_W2K3_GM. We use svcinfo lsvdisk to list all the VDisks in the ITSO-CLS1 cluster, and then use the svcinfo lsrcrelationshipcandidate command to show the possible VDisks candidates for GM_DB_Pri in ITSO-CLS2. After checking all of the above conditions, use the command svctask mkrcrelationship to create the Global Mirror relationship. To verify, the newly created Global Mirror relationships, list them with the command svcinfo lsrcrelationship. Example 13-7 Creating Global Mirror GMREL1and GMREL2
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk -delim : id:name:IO_group_id:IO_group_name:status:mdisk_grp_id:mdisk_grp_name:capacity:type:FC_id:FC_name :RC_id:RC_name:vdisk_UID:fc_map_count:copy_count 9:GM_DB_Pri:0:io_grp0:online:0:MDG_DS45:10.0GB:striped:::::60050768018301BF280000000000000D:0:1
700
Implementing the IBM System Storage SAN Volume Controller V4.3
10:GM_DBLog_Pri:0:io_grp0:online:0:MDG_DS45:10.0GB:striped:::::60050768018301BF280000000000000E: 0:1 11:GM_App_Pri:1:io_grp1:online:1:MDG_DS47:10.0GB:striped:::::60050768018301BF280000000000000F:0: 1 IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationshipcandidate -aux ITSO-CLS2 -master GM_DB_Pri id vdisk_name 16 GM_DB_Sec IBM_2145:ITSO-CLS1:admin>svctask mkrcrelationship -master GM_DB_Pri -aux GM_DB_Sec -cluster ITSO-CLS2 -consistgrp CG_W2K3_GM -name GMREL1 -global RC Relationship, id [9], successfully created IBM_2145:ITSO-CLS1:admin>svctask mkrcrelationship -master GM_DBLog_Pri -aux GM_DBLog_Sec -cluster ITSO-CLS2 -consistgrp CG_W2K3_GM -name GMREL2 -global RC Relationship, id [10], successfully created IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationship -delim : id:name:master_cluster_id:master_cluster_name:master_vdisk_id:master_vdisk_name:aux_cluster_id:a ux_cluster_name:aux_vdisk_id:aux_vdisk_name:primary:consistency_group_id:consistency_group_name: state:bg_copy_priority:progress:copy_type 9:GMREL1:0000020060C06FCA:ITSO-CLS1:9:GM_DB_Pri:0000020068603A42:ITSO-CLS2:16:GM_DB_Sec:master:2 55:CG_W2K3_GM:inconsistent_stopped:50:0:global 10:GMREL2:0000020060C06FCA:ITSO-CLS1:10:GM_DBLog_Pri:0000020068603A42:ITSO-CLS2:17:GM_DBLog_Sec: master:255:CG_W2K3_GM:inconsistent_stopped:50:0:global
Creating the Stand-alone Global Mirror relationship for GM_App_Pri In Example 13-8, we create the stand-alone Global Mirror relationship GMREL3 for GM_App_Pri. Once it is created, we will check the status of each of our Global Mirror relationships. Notice that the status of GMREL3 is consistent_stopped, and this is because it was created with the -sync option. The -sync option indicates that the secondary (auxiliary) virtual disk is already synchronized with the primary (master) virtual disk. The initial background synchronization is skipped when this option is used. GMREL1 and GMREL2 are in the inconsistent_stopped state, because they were not created with the -sync option, so their auxiliary VDisks needs to be synchronized with their primary VDisks. Example 13-8 Creating a stand-alone Global Mirror relationship and verifying it
IBM_2145:ITSO-CLS1:admin>svctask mkrcrelationship -master GM_App_Pri -aux GM_App_Sec -cluster ITSO-CLS2 -sync -name GMREL3 -global RC Relationship, id [11], successfully created IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationship -delim : id:name:master_cluster_id:master_cluster_name:master_vdisk_id:master_vdisk_name:aux_cluster_id:a ux_cluster_name:aux_vdisk_id:aux_vdisk_name:primary:consistency_group_id:consistency_group_name: state:bg_copy_priority:progress:copy_type 9:GMREL1:0000020060C06FCA:ITSO-CLS1:9:GM_DB_Pri:0000020068603A42:ITSO-CLS2:16:GM_DB_Sec:master:2 55:CG_W2K3_GM:inconsistent_stopped:50:0:global 10:GMREL2:0000020060C06FCA:ITSO-CLS1:10:GM_DBLog_Pri:0000020068603A42:ITSO-CLS2:17:GM_DBLog_Sec: master:255:CG_W2K3_GM:inconsistent_stopped:50:0:global
Chapter 13. Copy Services: Global Mirror
701
11:GMREL3:0000020060C06FCA:ITSO-CLS1:11:GM_App_Pri:0000020068603A42:ITSO-CLS2:18:GM_App_Sec:mast er:::consistent_stopped:50::global
13.4.2 Starting Global Mirror Now that we have created the Global Mirror consistency group and relationships, we are ready to use the Global Mirror relationships in our environment. When implementing Global Mirror, the goal is to reach a consistent and synchronized state that can provide redundancy in case a hardware failure occurs that affects the SAN at the production site. In this section, we show how to start the stand-alone Global Mirror relationships and the consistency group.
Starting a stand-alone Global Mirror relationship In Example 13-9, we start the stand-alone Global Mirror relationship GMREL3. Because the Global Mirror relationship was in the Consistent stopped state and no updates have been made to the primary VDisk, the relationship quickly enters the Consistent synchronized state. Example 13-9 Starting the stand-alone Global Mirror relationship
IBM_2145:ITSO-CLS1:admin>svctask startrcrelationship GMREL3 IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationship GMREL3 id 11 name GMREL3 master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 master_vdisk_id 11 master_vdisk_name GM_App_Pri aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 aux_vdisk_id 18 aux_vdisk_name GM_App_Sec primary master consistency_group_id consistency_group_name state consistent_synchronized bg_copy_priority 50 progress freeze_time status online sync copy_type global
Starting a Global Mirror consistency group In Example 13-10 on page 703, we start the Global Mirror consistency group CG_W2K3_GM. Because the consistency group was in the Inconsistent stopped state, it enters the Inconsistent copying state until the background copy has completed for all relationships in the consistency group. Upon completion of the background copy, it enters the Consistent synchronized state (see Figure 13-6 on page 680).
702
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 13-10 Starting the Global Mirror consistency group
IBM_2145:ITSO-CLS1:admin>svctask startrcconsistgrp CG_W2K3_GM IBM_2145:ITSO-CLS1:admin>svcinfo lsrcconsistgrp CG_W2K3_GM id 255 name CG_W2K3_GM master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 primary master state inconsistent_copying relationship_count 2 freeze_time status online sync copy_type global RC_rel_id 9 RC_rel_name GMREL1 RC_rel_id 10 RC_rel_name GMREL2
Monitoring background copy progress To monitor the background copy progress, use the svcinfo lsrcrelationship command. This command will show us all the defined Global Mirror relationships if used without any parameters. In the command output, progress indicates the current background copy progress. Our Global Mirror relationships are shown in Example 13-11. Note: Setting up SNMP traps for the SVC enables automatic notification when Global Mirror consistency groups or relationships change state. Example 13-11 Monitoring background copy progress example
IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationship GMREL1 id 9 name GMREL1 master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 master_vdisk_id 9 master_vdisk_name GM_DB_Pri aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 aux_vdisk_id 16 aux_vdisk_name GM_DB_Sec primary master consistency_group_id 255 consistency_group_name CG_W2K3_GM state inconsistent_copying bg_copy_priority 50 progress 21 freeze_time status online Chapter 13. Copy Services: Global Mirror
703
sync copy_type global IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationship GMREL2 id 10 name GMREL2 master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 master_vdisk_id 10 master_vdisk_name GM_DBLog_Pri aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 aux_vdisk_id 17 aux_vdisk_name GM_DBLog_Sec primary master consistency_group_id 255 consistency_group_name CG_W2K3_GM state inconsistent_copying bg_copy_priority 50 progress 54 freeze_time status online sync copy_type global When all the Global Mirror relationships complete the background copy, the consistency group enters the consistent synchronized state, as shown in Example 13-12. Example 13-12 Listing the Global Mirror consistency group
IBM_2145:ITSO-CLS1:admin>svcinfo lsrcconsistgrp CG_W2K3_GM id 255 name CG_W2K3_GM master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 primary master state consistent_synchronized relationship_count 2 freeze_time status online sync copy_type global RC_rel_id 9 RC_rel_name GMREL1 RC_rel_id 10 RC_rel_name GMREL2
13.4.3 Stopping and restarting Global Mirror Now that the Global Mirror consistency group and relationships are running, we now describe how to stop, restart and also change the direction of the stand-alone Global Mirror relationships as well as the consistency group.
704
Implementing the IBM System Storage SAN Volume Controller V4.3
First, we show how to stop and restart the stand-alone Global Mirror relationships and the consistency group.
Stopping a stand-alone Global Mirror relationship In Example 13-13, we stop the stand-alone Global Mirror relationship, while enabling access (write I/O) to both the primary and the secondary VDisk, and as a result, the relationship enters the Idling state. Example 13-13 Stopping the stand-alone Global Mirror relationship
IBM_2145:ITSO-CLS1:admin>svctask stoprcrelationship -access GMREL3 IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationship GMREL3 id 11 name GMREL3 master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 master_vdisk_id 11 master_vdisk_name GM_App_Pri aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 aux_vdisk_id 18 aux_vdisk_name GM_App_Sec primary consistency_group_id consistency_group_name state idling bg_copy_priority 50 progress freeze_time status sync in_sync copy_type global
Stopping a Global Mirror consistency group In Example 13-14, we stop the Global Mirror consistency group without specifying the -access parameter. This means that the consistency group enters the Consistent stopped state. Example 13-14 Stopping a Global Mirror consistency group without -access
IBM_2145:ITSO-CLS1:admin>svctask stoprcconsistgrp CG_W2K3_GM IBM_2145:ITSO-CLS1:admin>svcinfo lsrcconsistgrp CG_W2K3_GM id 255 name CG_W2K3_GM master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 primary master state consistent_stopped relationship_count 2 freeze_time 2008/06/24/18/34/00 status online
Chapter 13. Copy Services: Global Mirror
705
sync in_sync copy_type global RC_rel_id 9 RC_rel_name GMREL1 RC_rel_id 10 RC_rel_name GMREL2 If, afterwards, we want to enable access (write I/O) for the secondary VDisk, we can re-issue the svctask stoprcconsistgrp command, specifying the -access parameter, and the consistency group transits to the Idling state, as shown in Example 13-15. Example 13-15 Stopping a Global Mirror consistency group
IBM_2145:ITSO-CLS1:admin>svctask stoprcconsistgrp -access CG_W2K3_GM IBM_2145:ITSO-CLS1:admin>svcinfo lsrcconsistgrp CG_W2K3_GM id 255 name CG_W2K3_GM master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 primary state idling relationship_count 2 freeze_time status sync in_sync copy_type global RC_rel_id 9 RC_rel_name GMREL1 RC_rel_id 10 RC_rel_name GMREL2
Restarting a Global Mirror relationship in the Idling state When restarting a Global Mirror relationship in the Idling state, we must specify the copy direction. If any updates have been performed on either the master or the auxiliary VDisk, consistency will be compromised. Therefore, we must issue the -force parameter to re-start the relationship. If the -force parameter is not used, then the command will fail. This is shown in Example 13-16. Example 13-16 Restarting a Global Mirror relationship after updates in the Idling state
IBM_2145:ITSO-CLS1:admin>svctask startrcrelationship -primary master -force GMREL3 IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationship GMREL3 id 11 name GMREL3 master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 master_vdisk_id 11 master_vdisk_name GM_App_Pri aux_cluster_id 0000020068603A42
706
Implementing the IBM System Storage SAN Volume Controller V4.3
aux_cluster_name ITSO-CLS2 aux_vdisk_id 18 aux_vdisk_name GM_App_Sec primary master consistency_group_id consistency_group_name state consistent_synchronized bg_copy_priority 50 progress freeze_time status online sync copy_type global
Restarting a Global Mirror consistency group in the Idling state When restarting a Global Mirror consistency group in the Idling state, we must specify the copy direction. If any updates have been performed on either the master or the auxiliary VDisk in any of the Global Mirror relationships in the consistency group, then consistency will be compromised. Therefore, we must issue the -force parameter to start the relationship. If the -force parameter is not used, then the command will fail. In Example 13-17, we restart the consistency group and change the copy direction by specifying the auxiliary VDisks to be the primaries. Example 13-17 Restarting a Global Mirror relationship while changing the copy direction
IBM_2145:ITSO-CLS1:admin>svctask startrcconsistgrp -primary aux CG_W2K3_GM IBM_2145:ITSO-CLS1:admin>svcinfo lsrcconsistgrp CG_W2K3_GM id 255 name CG_W2K3_GM master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 primary aux state consistent_synchronized relationship_count 2 freeze_time status online sync copy_type global RC_rel_id 9 RC_rel_name GMREL1 RC_rel_id 10 RC_rel_name GMREL2
Chapter 13. Copy Services: Global Mirror
707
13.4.4 Changing direction for Global Mirror In this section, we show how to change the copy direction of the stand-alone Global Mirror relationships and the consistency group.
Switching copy direction for a Global Mirror relationship When a Global Mirror relationship is in the consistent synchronized state, we can change the copy direction for the relationship, using the command svctask switchrcrelationship, specifying the primary VDisk. If the VDisk specified as the primary when issuing this command is already a primary, then the command has no effect. In Example 13-18, we change the copy direction for the stand-alone Global Mirror relationship, specifying the auxiliary VDisk to be the primary. Note: When the copy direction is switched, it is crucial that there is no outstanding I/O to the VDisk that transits from primary to secondary, since all I/O will be inhibited to that VDisk when it becomes the secondary. Therefore, careful planning is required prior to using the svctask switchrcrelationship command. Example 13-18 Switching the copy direction for a Global Mirror relationship
IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationship GMREL3 id 11 name GMREL3 master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 master_vdisk_id 11 master_vdisk_name GM_App_Pri aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 aux_vdisk_id 18 aux_vdisk_name GM_App_Sec primary master consistency_group_id consistency_group_name state consistent_synchronized bg_copy_priority 50 progress freeze_time status online sync copy_type global IBM_2145:ITSO-CLS1:admin>svctask switchrcrelationship -primary aux GMREL3 IBM_2145:ITSO-CLS1:admin>svcinfo lsrcrelationship GMREL3 id 11 name GMREL3 master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 master_vdisk_id 11 master_vdisk_name GM_App_Pri aux_cluster_id 0000020068603A42
708
Implementing the IBM System Storage SAN Volume Controller V4.3
aux_cluster_name ITSO-CLS2 aux_vdisk_id 18 aux_vdisk_name GM_App_Sec primary aux consistency_group_id consistency_group_name state consistent_synchronized bg_copy_priority 50 progress freeze_time status online sync copy_type global
Switching copy direction for a Global Mirror consistency group When a Global Mirror consistency group is in the consistent synchronized state, we can change the copy direction for the relationship using the command svctask switchrcconsistgrp, specifying the primary VDisk. If the VDisk specified as the primary when issuing this command is already a primary, then the command has no effect. In Example 13-19, we change the copy direction for the Global Mirror consistency group, specifying the auxiliary to become the primary. Note: When the copy direction is switched, it is crucial that there is no outstanding I/O to the VDisks that transits from primary to secondary, since all I/O will be inhibited when they become the secondary. Therefore, careful planning is required prior to using the svctask switchrcconsistgrp command. Example 13-19 Switching the copy direction for a Global Mirror consistency group
IBM_2145:ITSO-CLS1:admin>svcinfo lsrcconsistgrp CG_W2K3_GM id 255 name CG_W2K3_GM master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 primary master state consistent_synchronized relationship_count 2 freeze_time status online sync copy_type global RC_rel_id 9 RC_rel_name GMREL1 RC_rel_id 10 RC_rel_name GMREL2 IBM_2145:ITSO-CLS1:admin>svctask switchrcconsistgrp -primary aux CG_W2K3_GM IBM_2145:ITSO-CLS1:admin>svcinfo lsrcconsistgrp CG_W2K3_GM
Chapter 13. Copy Services: Global Mirror
709
id 255 name CG_W2K3_GM master_cluster_id 0000020060C06FCA master_cluster_name ITSO-CLS1 aux_cluster_id 0000020068603A42 aux_cluster_name ITSO-CLS2 primary aux state consistent_synchronized relationship_count 2 freeze_time status online sync copy_type global RC_rel_id 9 RC_rel_name GMREL1 RC_rel_id 10 RC_rel_name GMREL2
13.5 Global Mirror scenario using the GUI Next, we show how to set up Global Mirror using the GUI. Note: This example is for intercluster only. In case you wish to set up intracluster, we highlight those parts of the following procedure that you do not need to perform. In the following scenario, we will be setting up an intercluster Global Mirror relationship between the SVC cluster ITSO-CLS1 at primary site and SVC cluster ITSO-CLS2 at the secondary site. Details of the VDisks are shown in Table 13-3. Table 13-3 Details of VDisks for Global Mirror relationship Content of Vdisk
VDisks at primary site
VDisks at secondary site
Database Files
GM_DB_Pri
GM_DB_Sec
Database Log Files
GM_DBLog_Pri
GM_DBLog_Sec
Application Files
GM_App_Pri
GM_App_Sec
Since data consistency is needed across GM_DB_Pri and GM_DBLog_Pri, we create a consistency group to handle Global Mirror relationships for them. While in this scenario the application files are independent of the database, we create a stand-alone Global Mirror relationship for GM_App_Pri. The Global Mirror setup is illustrated in Figure 13-8 on page 711.
710
Implementing the IBM System Storage SAN Volume Controller V4.3
Primary Site
Secondary Site
SVC Cluster – ITSO-CLS1
SVC Cluster – ITSO-CLS2
Consistency Group CG_W2K3_MM
GM_DB_Pri
GM_Relationship 1
GM_DB_Sec
GM_DBLog_Pri
GM_Relationship 2
GM_DBLog_Sec
GM_App_Pri
GM_Relationship 3
GM_App_Sec
Figure 13-8 Global Mirror scenario using the GUI
13.5.1 Setting up Global Mirror In the following section, we assume that the source and target VDisks have already been created and that the ISLs and zoning are in place, enabling the SVC clusters to communicate. To set up the Global Mirror, you must perform the following steps: 1. Create SVC partnership between ITSO-CLS1 and ITSO-CLS2, on both SVC clusters: – Bandwidth 10 MBps 2. Create a Global Mirror consistency group: – Name CG_W2K3_GM 3. Create the Global Mirror relationship for GM_DB_Pri. – Master GM_DB_Pri – Auxiliary GM_DB_Sec – Auxiliary SVC cluster ITSO-CLS2 – Name GMREL1 – Consistency group CG_W2K3_GM 4. Create the Global Mirror relationship for GM_DBLog_Pri: – Master GM_DBLog_Pri – Auxiliary GM_DBLog_Sec – Auxiliary SVC cluster ITSO-CLS2 – Name GMREL2 – Consistency group CG_W2K3_GM 5. Create the Global Mirror relationship for GM_App_Pri: – Master GM_App_Pri – Auxiliary GM_App_Sec – Auxiliary SVC cluster ITSO-CLS2 – Name GMREL3
Chapter 13. Copy Services: Global Mirror
711
Creating an SVC partnership between ITSO-CLS1 and ITSO-CLS2 In this section, we create the SVC partnership on both clusters. Note: If you are creating an intracluster Global Mirror, do not perform the next step; instead, go to “Creating a Global Mirror consistency group” on page 716. To create a Global Mirror partnership between the SVC clusters using the GUI, we launch the SVC GUI for ITSO-CLS1. Then we select Manage Copy Services and click Metro & Global Mirror Cluster Partnership, as shown in Figure 13-9.
Figure 13-9 Selecting Global Mirror Cluster Partnership on ITSO-CLS1
Figure 13-10 shows the cluster partnership defined for this cluster. Since there is no existing partnership, nothing can be listed. It also gives a warning stating that for any type of copy relationship between VDisks across two different clusters, the partnership must exist between them. To create a new relationship, click Create.
Figure 13-10 Creating a new partnership
712
Implementing the IBM System Storage SAN Volume Controller V4.3
In Figure 13-11 the available SVC cluster candidates are listed, which in our case is only ITSO-CLS2. We select ITSO-CLS2 and specify the available bandwidth for the background copy; in this case, we select 10 MBps and then click OK.
Figure 13-11 Selecting SVC cluster partner and specifying bandwidth for background copy
In the resulting window, shown in Figure 13-12, the newly created Global Mirror cluster partnership is shown as Partially Configured.
Figure 13-12 Viewing newly created Global Mirror partnership
To fully configure the Global Mirror cluster partnership, we must carry out the same steps on ITSO-CLS2 as we did on ITSO-CLS1. For simplicity, in the following figures, only the last two windows are shown.
Chapter 13. Copy Services: Global Mirror
713
Launching the SVC GUI for ITSO-CLS2, we select ITSO-CLS1 for the Global Mirror cluster partnership and specify the available bandwidth for background copy, again 10 MBps, and then click OK, as shown in Figure 13-13.
Figure 13-13 Selecting SVC cluster partner and specifying bandwidth for background copy
Now that both sides of the SVC Cluster Partnership are defined, the resulting window shown in Figure 13-14 confirms that our Global Mirror cluster partnership is Fully Configured.
Figure 13-14 Global Mirror cluster partnership is fully configured
Note: Link tolerance, intercluster delay simulation, and intracluster delay simulation are introduced with the use of the Global Mirror feature.
Global Mirror link tolerance and delay simulations We overview link tolerance and delay simulations.
Global Mirror link tolerance The gm_link_tolerance parameter defines how sensitive the SVC is to interlinking overload conditions. The value is the number of seconds of continuous link difficulties that will be tolerated before the SVC will stop the remote copy relationships in order to prevent impacting host I/O at the primary site. In order to change the value, refer to “Changing link tolerance and delay simulation values for Global Mirror” on page 715. The link tolerance values are between 60 and 86400 seconds in increments of 10 seconds. The default value for the link tolerance is 300 seconds. 714
Implementing the IBM System Storage SAN Volume Controller V4.3
Recommendation: We strongly recommend using the default value. If the link is overloaded for a period that would impact host I/O at the primary site, the relationships will be stopped to protect those hosts.
Global Mirror intercluster and intracluster delay simulation This Global Mirror feature permits a simulation of a delayed write to a remote VDisk. This feature allows testing to be performed that detects colliding writes and so can be used to test an application before full deployment of the Global Mirror feature. The delay simulation can be enabled separately for either intracluster or intercluster Global Mirror. To enable and change to the appropriate value, refer to “Changing link tolerance and delay simulation values for Global Mirror” on page 715. inter_cluster_delay_simulation and intra_cluster_delay_simulation express the amount of time secondary I/Os are delayed respectively for intercluster and intracluster relationships. These values specifies the number of milliseconds that I/O activity, that is, copying the primary VDisk to a secondary VDisk, is delayed. A value from 0 to 100 milliseconds in 1 millisecond increments can be set and a value of zero disables this feature. To check the current settings for the delay simulation, refer to “Changing link tolerance and delay simulation values for Global Mirror” on page 715.
Changing link tolerance and delay simulation values for Global Mirror Here, we show the modification of the delay simulations and the Global Mirror link tolerance values. We also show the changed values for the Global Mirror link tolerance and delay simulation parameters. Launching the SVC GUI for ITSO-CLS1, we select the Global Mirror Cluster Partnership option to view and to modify the parameters, as shown in Figure 13-15 and Figure 13-16 on page 716, respectively.
Figure 13-15 View and modify Global Mirror link tolerance and delay simulation parameters
Chapter 13. Copy Services: Global Mirror
715
Figure 13-16 Set Global Mirror link tolerance and delay simulations parameters
After performing the steps, the GUI returns to the Global Mirror Partnership window and lists the new parameter settings, as shown in Figure 13-17.
Figure 13-17 View modified parameters
Creating a Global Mirror consistency group To create the consistency group to be used by the Global Mirror relationships for the VDisks with the database and database log files, we select Manage Copy Services and click Global Mirror Consistency Groups, as shown in Figure 13-18.
Figure 13-18 Selecting Global Mirror consistency groups
716
Implementing the IBM System Storage SAN Volume Controller V4.3
To start the creation process, we select Create Consistency Group from the drop-down menu and click Go, as shown in Figure 13-19.
Figure 13-19 Create a consistency group
We are presented with a wizard that helps us create the Global Mirror consistency group. The first step in this wizard gives an introduction to the steps involved in the creation of the Global Mirror consistency group, as shown in Figure 13-20. Click Next to proceed.
Figure 13-20 Introduction to Global Mirror consistency group creation wizard
As shown in Figure 13-21, we specify the consistency group name and whether it is to be used for intercluster or intracluster relationships. In our scenario, we select Create an inter-cluster consistency group and click Next.
Figure 13-21 Specifying consistency group name and type
Chapter 13. Copy Services: Global Mirror
717
Figure 13-22 would show any existing Global Mirror relationships that could be included in the Global Mirror consistency group. As we do not have any existing relationships at this time, we will create an empty group by clicking Next to proceed.
Figure 13-22 Select the existing Global Mirror relationship
Verify the settings for the consistency group and click Finish to create the Global Mirror Consistency Group, as shown in Figure 13-23.
Figure 13-23 Verifying the settings for the Global Mirror consistency group
When the Global Mirror consistency group is created, we are returned to the Viewing Global Mirror Consistency Groups window. It shows our newly created Global Mirror consistency group, as shown in Figure 13-24.
Figure 13-24 Viewing Global Mirror consistency groups
718
Implementing the IBM System Storage SAN Volume Controller V4.3
Creating Global Mirror relationships for GM_DB_Pri and GM_DBLog_Pri To create the Global Mirror Relationships for GM_DB_Pri and GM_DBLog_Pri, we select Manage Copy Services and click Global Mirror Cluster Relationships, as shown in Figure 13-25.
Figure 13-25 Selecting Global Mirror relationships
To start the creation process, we select Create a Relationship from the drop-down menu and click Go, as shown in Figure 13-26.
Figure 13-26 Create a relationship
We are presented with a wizard that helps us create Global Mirror relationships. The first step in the wizard gives an introduction to the steps involved in the creation of the Global Mirror relationship, as shown in Figure 13-27. Click Next to proceed.
Figure 13-27 Introduction to Global Mirror relationship creation wizard
Chapter 13. Copy Services: Global Mirror
719
As shown in Figure 13-28, we name our first Global Mirror relationship GMREL1, click Global Mirror Relationship, and select the relationship for the cluster. In this case, it is an intercluster relationship, as shown in Figure 13-8 on page 711.
Figure 13-28 Naming the Global Mirror relationship and selecting the type of the cluster relationship
The next step will enable us to select a master VDisk. As this list could potentially be large, the Filtering Master VDisks Candidates window appears, which will enable us to reduce the list of eligible VDisks based on a defined filter. In Figure 13-29, use the filter for GM* (use * to list all VDisks) and click Next.
Figure 13-29 Defining the filter for master VDisk candidates
720
Implementing the IBM System Storage SAN Volume Controller V4.3
As shown in Figure 13-30, we select GM_DB_Pri to be the master VDisk of the relationship, and click Next to proceed.
Figure 13-30 Selecting the master VDisk
The next step will require us to select an auxiliary VDisk. The Global Mirror relationship wizard will automatically filter this list so that only eligible VDisks are shown. Eligible VDisks are those that have the same size as the master VDisk and are not already part of a Global Mirror relationship. As shown in Figure 13-31, we select GM_DB_Sec as the auxiliary VDisk for this relationship, and click Next to proceed.
Figure 13-31 Selecting the auxiliary VDisk
Chapter 13. Copy Services: Global Mirror
721
As shown in Figure 13-32, select the relationship to be part of the consistency group that we have created and click Next to proceed.
Figure 13-32 Selecting the relationship to be part of a consistency group
Info: It is not mandatory to make the relationship part of a consistency group at this stage. It can be also be done after the creation of the relationship at a later stage. The relationship can be added to the consistency group by modifying that relationship. Finally, in Figure 13-33, we verify the Global Mirror Relationship attributes and click Finish to create it.
Figure 13-33 Verifying the Global Mirror relationship
722
Implementing the IBM System Storage SAN Volume Controller V4.3
After successful creation of the relationship, the GUI returns to the Viewing Global Mirror Relationships window, as shown in Figure 13-34. This window will list the newly created relationship.
Figure 13-34 Viewing Global Mirror relationships
Using the same process, the second Global Mirror relationship, GMREL2, is also created. Both relationships are shown in Figure 13-35. .
Figure 13-35 Viewing the Global Mirror relationships after creating GMREL2
Creating the stand-alone Global Mirror relationship for GM_App_Pri To create the stand-alone Global Mirror relationship, we start the creation process by selecting Create a Relationship from the drop-down menu and click Go, as shown in Figure 13-36.
Figure 13-36 Create a Global Mirror relationship
Chapter 13. Copy Services: Global Mirror
723
Next, we are presented with the wizard that shows the steps involved in the process of creating a Global Mirror relationship, as shown in Figure 13-37. Click Next to proceed.
Figure 13-37 Introduction to Global Mirror relationship creation wizard
In Figure 13-38, we name the Global Mirror relationship GMREL3, specify that it is an intercluster relationship, and click Next.
Figure 13-38 Naming the Global Mirror relationship and selecting the type of cluster relationship
As shown in Figure 13-39 on page 725, we are prompted for a filter prior to presenting the master VDisk candidates. We use * to list all candidates and click Next.
724
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 13-39 Filtering master VDisk candidates
As shown in Figure 13-40, we select GM_App_Pri to be the master VDisk for the relationship, and click Next to proceed.
Figure 13-40 Selecting the master VDisk
Chapter 13. Copy Services: Global Mirror
725
As shown in Figure 13-41, we select GM_App_Sec as the auxiliary VDisk for the relationship, and click Next to proceed.
Figure 13-41 Selecting auxiliary VDisk
As shown in Figure 13-42, we did not select a consistency group, as we are creating a stand-alone Global Mirror relationship.
Figure 13-42 Selecting options for the Global Mirror relationship
We also specify that the master and auxiliary VDisk are already synchronized; for the purpose of this example, we can assume that they are pristine. This is shown in Figure 13-43 on page 727.
726
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 13-43 Selecting the synchronized option for Global Mirror relationship
Note: To add a Global Mirror relationship to a consistency group, it must be in the same state as the consistency group. Even if we intend to make the Global Mirror relationship GMREL3 part of the consistency group CG_W2K3_GM, we are not offered the option, as shown in Figure 13-43. This is because the state of the relationship GMREL3 is Consistent Stopped, because we selected the synchronized option. The state of the consistency group CG_W2K3_GM is currently Inconsistent Stopped. Finally, Figure 13-44 shows the actions that will be performed. We click Finish to create this new relationship.
Figure 13-44 Verifying Global Mirror relationship
Chapter 13. Copy Services: Global Mirror
727
After successful creation, we are returned to the Viewing Global Mirror Relationship window. Figure 13-45 now shows all our defined Global Mirror relationships.
Figure 13-45 Viewing Global Mirror relationships
13.5.2 Starting Global Mirror Now that we have created the Global Mirror consistency group and relationships, we are ready to use the Global Mirror relationships in our environment. When performing Global Mirror, the goal is to reach a consistent and synchronized state that can provide redundancy in case a hardware failure occurs that affects the SAN at the production site. In this section, we show how to start the stand-alone Global Mirror relationship and the consistency group.
Starting a stand-alone Global Mirror relationship In Figure 13-46, we select the stand-alone Global Mirror relationship GMREL3, and from the drop-down menu, we select Start Copy Process and click Go.
Figure 13-46 Starting the stand-alone Global Mirror relationship
728
Implementing the IBM System Storage SAN Volume Controller V4.3
In Figure 13-47, we do not need to change the parameters Forced start, Mark as clean, or Copy Direction, as this is the first time we are invoking this Global Mirror relationship (and we already defined the relationship as being synchronized in Figure 13-43 on page 727). We click OK to start the stand-alone Global Mirror relationship GMREL3.
Figure 13-47 Selecting options and starting the copy process
Since the Global Mirror relationship was in the Consistent Stopped state and no updates have been made on the primary VDisk, the relationship quickly enters the Consistent Synchronized state, as shown in Figure 13-48.
Figure 13-48 Viewing Global Mirror relationship
Starting a Global Mirror consistency group To start the Global Mirror consistency group CG_W2K3_GM, select Global Mirror Consistency Groups, as shown in Figure 13-49.
Figure 13-49 Selecting Global Mirror consistency groups
Chapter 13. Copy Services: Global Mirror
729
In Figure 13-50, we select the Global Mirror consistency group CG_W2K3_GM, and from the drop-down menu, we select Start Copy Process and click Go.
Figure 13-50 Selecting Global Mirror consistency group and starting the copy process
As shown in Figure 13-51, we click OK to start the copy process. We cannot select the options Forced start, Mark as clean, or Copy Direction, as this is the first time we are invoking this Global Mirror relationship.
Figure 13-51 Selecting options and starting the copy process
As shown in Figure 13-52, we are returned to the Viewing Global Mirror Consistency Groups window and the consistency group CG_W2K3_GM has transitioned to the Inconsistent copying state.
Figure 13-52 Viewing Global Mirror consistency groups
Since the consistency group was in the Inconsistent stopped state, it enters the Inconsistent copying state until the background copy has completed for all relationships in the consistency group. Upon completion of the background copy for all relationships in the consistency group, it enters the Consistent Synchronized state.
730
Implementing the IBM System Storage SAN Volume Controller V4.3
Monitoring background copy progress The status of the background copy progress can be seen in the Viewing Global Mirror Relationships window, as shown in Figure 13-53, or alternatively, use the Manage Progress section under My Work and select Viewing Global Mirror Progress, as shown in Figure 13-54.
Figure 13-53 Monitoring background copy process for Global Mirror relationships
Figure 13-54 Monitoring background copy process for Global Mirror relationships
Note: Setting up SNMP traps for the SVC enables automatic notification when Global Mirror consistency groups or relationships change state.
13.5.3 Stopping and restarting Global Mirror Now that the Global Mirror consistency group and relationships are running, we now describe how to stop, restart, and change the direction of the stand-alone Global Mirror relationships, as well as the consistency group. In this section, we show how to stop and restart the stand-alone Global Mirror relationships and the consistency group.
Chapter 13. Copy Services: Global Mirror
731
Stopping a stand-alone Global Mirror relationship To stop a Global Mirror relationship while enabling access (write I/O) to the secondary VDisk, we select the relationship and Stop Copy Process from the drop-down menu and click Go, as shown in Figure 13-55.
Figure 13-55 Stopping a stand-alone Global Mirror relationship
As shown in Figure 13-56, we check the Enable write access... option and click OK to stop the Global Mirror relationship.
Figure 13-56 Enable access to the secondary VDisk while stopping the relationship
As shown in Figure 13-57, the Global Mirror relationship transits to the Idling state when stopped, while enabling write access to the secondary VDisk.
Figure 13-57 Viewing Global Mirror relationships
732
Implementing the IBM System Storage SAN Volume Controller V4.3
Stopping a Global Mirror consistency group As shown in Figure 13-58, we select the Global Mirror consistency group and Stop Copy Process from the drop-down menu and click Go.
Figure 13-58 Selecting the Global Mirror consistency group to be stopped
As shown in Figure 13-59, we click OK without specifying the Enable write access... option to the secondary VDisk.
Figure 13-59 Stopping the consistency group without enabling access to the secondary VDisk
As shown in Figure 13-60, the consistency group enters the Consistent stopped state when stopped.
Figure 13-60 Viewing Global Mirror consistency groups
Chapter 13. Copy Services: Global Mirror
733
If, afterwards, we want to enable access (write I/O) to the secondary VDisks, we can reissue the Stop Copy Process, specifying access to be enabled to the secondary VDisks. In Figure 13-61, we select the Global Mirror relationship and Stop Copy Process from the drop-down menu and click Go.
Figure 13-61 Selecting the Global Mirror consistency group
As shown in Figure 13-62, we check the Enable write access... check box and click OK. .
Figure 13-62 Enabling access to the secondary VDisks
When applying the Enable write access... option, the consistency group transits to the Idling state, as shown in Figure 13-63.
Figure 13-63 Viewing the Global Mirror consistency group after write access to the secondary VDisk
Restarting a Global Mirror Relationship in the Idling state When restarting a Global Mirror relationship in the Idling state, we must specify the copy direction. If any updates have been performed on either the master or the auxiliary VDisk in any of the Global Mirror relationships in the consistency group, then consistency will have been compromised. In this situation, we must check the Force option to start the copy process or the command will fail.
734
Implementing the IBM System Storage SAN Volume Controller V4.3
As shown in Figure 13-64, we select the Global Mirror relationship and Start Copy Process from the drop-down menu and click Go.
Figure 13-64 Starting stand-alone Global Mirror relationship in the Idling state
As shown in Figure 13-65, we check the Force option, since write I/O has been performed while in the Idling state, and we select the copy direction by defining the master VDisk as the primary, and click OK.
Figure 13-65 Restarting the copy process
The Global Mirror relationship enters the Consistent copying state. When the background copy is complete, the relationship transits to the Consistent synchronized state, as shown in Figure 13-66.
Figure 13-66 Viewing the Global Mirror relationship
Chapter 13. Copy Services: Global Mirror
735
Restarting a Global Mirror consistency group in the Idling state When restarting a Global Mirror consistency group in the Idling state, we must specify the copy direction. If any updates have been performed on either the master or the auxiliary VDisk in any of the Global Mirror relationships in the consistency group, then consistency will have been compromised. In this situation, we must check the Force option to start the copy process or the command will fail. As shown in Figure 13-67, we select the Global Mirror consistency group and Start Copy Process from the drop-down menu and click Go.
Figure 13-67 Starting the copy process for Global Mirror consistency group
As shown in Figure 13-68, we check the Force option and set the copy direction by selecting the auxiliary as the master.
Figure 13-68 Restarting the copy process for the consistency group
When the background copy completes, the Global Mirror consistency group enters the Consistent synchronized state, as shown in Figure 13-69.
Figure 13-69 Viewing Global Mirror consistency groups
736
Implementing the IBM System Storage SAN Volume Controller V4.3
Also shown in Figure 13-70 are the individual relationships within that consistency group.
Figure 13-70 Viewing Global Mirror relationships
13.5.4 Changing copy direction for Global Mirror In this section, we show how to change the copy direction of the stand-alone Global Mirror relationships and the consistency group.
Switching copy direction for a stand-alone Global Mirror relationship When a Global Mirror relationship is in the Consistent synchronized state, we can change the copy direction for the relationship. In Figure 13-71, we select the relationship GMREL3 and Switch Copy Direction from the drop-down menu and click Go. Note: When the copy direction is switched, it is crucial that there is no outstanding I/O to the VDisk that transits from primary to secondary, because all I/O will be inhibited to that VDisk when it becomes the secondary. Therefore, careful planning is required prior to switching the copy direction for a Global Mirror relationship.
Figure 13-71 Selecting the relationship for which copy direction is to be changed
Chapter 13. Copy Services: Global Mirror
737
In Figure 13-72, we see that the current primary VDisk is the master, so to change the copy direction for the stand-alone Global Mirror relationship, we specify the auxiliary VDisk to be the primary, and click OK.
Figure 13-72 Selecting the primary VDisk as auxiliary to switch the copy direction
The copy direction is now switched and we are returned to the Viewing Global Mirror Relationship window, where we see that the copy direction has been switched, as shown in Figure 13-73.
Figure 13-73 Viewing Global Mirror relationship after changing the copy direction
Switching copy direction for a Global Mirror consistency group When a Global Mirror consistency group is in the Consistent synchronized state, we can change the copy direction for the Global Mirror consistency group. In Figure 13-74 on page 739, we select the consistency group CG_W2K3_GM and Switch Copy Direction from the drop-down menu and click Go. Note: When the copy direction is switched, it is crucial that there is no outstanding I/O to the VDisks that transits from primary to secondary, because all I/O will be inhibited when they become the secondary. Therefore, careful planning is required prior to switching the copy direction.
738
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 13-74 Selecting the consistency group for which the copy direction is to be changed
In Figure 13-75, we see that currently the primary VDisks are also the master. So, to change the copy direction for the Global Mirror consistency group, we specify the auxiliary VDisks to become the primary, and click OK.
Figure 13-75 Selecting the primary VDisk as auxiliary to switch the copy direction
The copy direction is now switched and we are returned to the Viewing Global Mirror Consistency Group window, where we see that the copy direction has been switched. Figure 13-76 shows that the auxiliary is now the primary. .
Figure 13-76 Viewing Global Mirror consistency groups after changing the copy direction
Chapter 13. Copy Services: Global Mirror
739
Figure 13-77 shows the new copy direction for individual relationships within that consistency group.
Figure 13-77 Viewing Global Mirror Relationships, after changing copy direction for Consistency Group
As everything has been completed to our expectations, we are now finished with Global Mirror.
740
Implementing the IBM System Storage SAN Volume Controller V4.3
14
Chapter 14.
Migration to and from the SAN Volume Controller In this chapter, we explain how to migrate from a conventional storage infrastructure to a virtualized storage infrastructure applying the SVC. We also explain how the SVC can be phased out of a virtualized storage infrastructure, for example, after a trial period.
© Copyright IBM Corp. 2003-2008. All rights reserved.
741
14.1 Migration overview The SVC allows the mapping of Virtual Disk (VDisk) extents to Managed Disk (MDisk) extents to be changed, without interrupting host access to the VDisk. This functionality is utilized when performing VDisk migrations, and can be performed for any VDisk defined on the SVC. This functionality can be used for: Redistribution of VDisks, and thereby the workload within an SVC cluster across back-end storage: – Moving workload onto newly installed storage – Moving workload off old/failing storage, ahead of decommissioning it – Moving workload to re-balance a changed workload Migrating data from older back-end storage to SVC managed storage Migrating data from one back-end controller to another using the SVC as a data block mover and afterwards removing the SVC from the SAN Migrating data from managed mode back into image mode prior to removing the SVC from a SAN
14.2 Migration operations Migration can be performed at either the VDisk or the extent level depending on the purpose of the migration. The different supported migration activities are: Migrating extents within a Managed Disk Group (MDG), redistributing the extents of a given VDisk on the MDisks in the MDG Migrating extents off an MDisk, which is removed from the MDG (to other MDisks in the MDG) Migrating a VDisk from one MDG to another MDG Migrating a VDisk to change the virtualization type of the VDisk to Image Migrating a VDisk between I/O Groups
14.2.1 Migrating multiple extents (within an MDG) A number of VDisk extents can be migrated at once using the migrateexts command. Extents are allocated on the destination MDisk using the algorithm described in 3.6.3, “Extents” on page 48. When executed, this command migrates a given number of extents from the source MDisk, where the extents of the VDisk specified resides, to a defined target MDisk that must be part of the same MDG. The number of migration threads that will be used in parallel can be specified from 1 to 4. If the type of the VDisk is image, the VDisk type transitions to striped when the first extent is migrated, while the MDisk access mode transitions from image to managed.
742
Implementing the IBM System Storage SAN Volume Controller V4.3
The syntax of the CLI command is: svctask migrateexts -source src_mdisk_id | src_mdisk_name -exts num_extents -target target_mdisk_id | target_mdisk_name [-threads number_of_threads] -vdisk vdisk_id | vdisk_name The parameters for the CLI command are: -vdisk: Specifies the VDisk ID or name to which the extents belong. -source: Specifies the source Managed Disk ID or name on which the extents currently reside. -exts: Specifies the number of extents to migrate. -target: Specifies the target MDisk ID or name onto which the extents are to be migrated. -threads: Optional parameter that specifies the number of threads to use while migrating these extents, from 1 to 4.
14.2.2 Migrating extents off an MDisk that is being deleted When an MDisk is deleted from an MDG using the rmmdisk -force command, any occupied extents on the MDisk are migrated off the MDisk (to other MDisks in the MDG) prior to its deletion. In this case, the extents that need to be migrated are moved onto the set of MDisks that are not being deleted, and the extents are distributed according to the algorithm described in 3.6.3, “Extents” on page 48. This statement holds true if multiple MDisks are being removed from the MDG at the same time, and MDisks that are being removed are not candidates for supplying free extents to the allocation of the free extents algorithm. If a VDisk uses one or more extents that need to be moved as a result of a delete mdisk command, then the virtualization type for that VDisk is set to striped (if it was previously sequential or image). If the MDisk is operating in image mode, the MDisk transitions to manage mode while the extents are being migrated, and upon deletion it transitions to unmanaged mode. The syntax of the CLI command is: svctask rmmdisk -mdisk mdisk_id_list | mdisk_name_list [-force] mdisk_group_id | mdisk_group_name The parameters for the CLI command are: -mdisk: Specifies one or more MDisk IDs or names to delete from the group. -force: Migrates any data that belongs to other VDisks before removing the MDisk. Note: If the -force flag is not supplied, and VDisk(s) occupy extents on one or more of the MDisks specified, the command will fail. When the -force flag is supplied, and VDisks exist that are made from extents on one or more of the MDisks specified, all extents on the MDisk(s) will be migrated to the other MDisks in the MDG, if there are enough free extents in the MDG. The deletion of the MDisk(s) is postponed until all extents are migrated, which can take some time. In case there are not enough free extents in the MDG, the command will fail. When the -force flag is supplied, the command will complete asynchronously.
Chapter 14. Migration to and from the SAN Volume Controller
743
14.2.3 Migrating a VDisk between MDGs An entire VDisk can be migrated from one MDG to another MDG using the migratevdisk command. A VDisk can be migrated between MDGs regardless of the virtualization type (image, striped, or sequential), though it will transition to the virtualization type of striped.The command will vary depending on the type of migration, as shown in Table 14-1. Table 14-1 Migration type MDG to MDG type
Command
Managed to managed
migratevdisk
Image to managed
migratevdisk
Managed to image
migratetoimage
Image to image
migratetoimage
The syntax of the CLI command is: svctask migratevdisk -mdiskgrp mdisk_group_id | mdisk_group_name [-threads number_of_threads -copy_id] -vdisk vdisk_id | vdisk_name The parameters for the CLI command are: -vdisk: Specifies the VDisk ID or name to migrate into another MDG. -mdiskgrp: Specifies the target MDG ID or name. -threads: An optional parameter that specifies the number of threads to use while migrating these extents, from 1 to 4. -copy_id: Required if the specified VDisk has more than one copy. The syntax of the CLI command is: svctask migratetoimage -copy_id -vdisk source_vdisk_id | name -mdisk unmanaged_target_mdisk_id | name -mdiskgrp managed_disk_group_id | name [-threads number_of_threads] The parameters for the CLI command are: -vdisk: Specifies the name or ID of the source VDisk to be migrated. -copy_id: Required if the specified VDisk has more than one copy. -mdisk: Specifies the name of the MDisk to which the data must be migrated. (This MDisk must be unmanaged and large enough to contain the data of the disk being migrated.) -mdiskgrp: Specifies the MDG into which the MDisk must be placed once the migration has completed. -threads: Optional parameter that specifies the number of threads to use while migrating these extents, from 1 to 4.
744
Implementing the IBM System Storage SAN Volume Controller V4.3
In Figure 14-1 we illustrate how the VDisk V3 is being migrated from MDG1 to MDG2. Important: In order for the migration to be “legal”, the source and destination MDisk must have the same extent size.
I/O G r o u p 0 S V C 1 IO -G rp 0 Node 1 S V C 1 IO -G r p 0 Node 2
V2
V1
V4
V3
V6
V3
V5
MDG 1
M1
M2
MDG 2
M3
M4
R A ID C o n tr o lle r A
MDG 3
M5
M6
M7
R A ID C o n tr o lle r B
Figure 14-1 Managed VDisk migration to another MDG
Extents are allocated to the migrating VDisk, from the set of MDisks in the target MDG, using the extent allocation algorithm described in 3.6.3, “Extents” on page 48. The process can be prioritized by specifying the number of threads to use while migrating; using only one thread will put the least background load on the system. If a large number of extents are being migrated, you can specify the number of threads that will be used in parallel (from 1 to 4). For the duration of the move, the offline rules described in “I/O handling and offline conditions” on page 57 apply to both MDGs. Therefore, referring back to Figure 14-1, if any of the MDisks M4, M5, M6, or M7 go offline, then VDisk V3 goes offline. If MDisk M4 goes offline, then V3 and V5 goes offline, but V1, V2, V4, and V6 remain online. If the type of the VDisk is image, the VDisk type transitions to striped when the first extent is migrated while the MDisk access mode transitions from image to managed. For the duration of the move, the VDisk is listed as being a member of the original MDG. For the purposes of configuration, the VDisk moves to the new MDG instantaneously at the end of the migration.
Chapter 14. Migration to and from the SAN Volume Controller
745
14.2.4 Migrating the VDisk to image mode The facility to migrate a VDisk to an image mode VDisk can be combined with the ability to migrate between MDGs. The source for the migration can be a managed mode or an image mode VDisk. This leads to four possibilities:
Migrate image mode to image mode within an MDG. Migrate managed mode to image mode within an MDG. Migrate image mode to image mode between MDGs. Migrate managed mode to image mode between MDGs.
To be able to migrate: The destination MDisk must be greater than or equal to the size of the VDisk. The MDisk specified as the target must be in an unmanaged state at the time the command is run. If the migration is interrupted by a cluster recovery, then the migration will resume after the recovery completes. If the migration involves moving between Managed Disk groups, then the VDisk behaves as described in 14.2.3, “Migrating a VDisk between MDGs” on page 744. The syntax of the CLI command is: svctask migratetoimage -copy_id -vdisk source_vdisk_id | name -mdisk unmanaged_target_mdisk_id | name -mdiskgrp managed_disk_group_id | name [-threads number_of_threads] The parameters for the CLI command are: -copy_id: Required if the specified VDisk has more than one copy. -vdisk: Specifies the name or ID of the source VDisk to be migrated. -mdisk: Specifies the name of the MDisk to which the data must be migrated. (This MDisk must be unmanaged and large enough to contain the data of the disk being migrated.) -mdiskgrp: Specifies the MDG into which the MDisk must be placed once the migration has completed. -threads: An optional parameter that specifies the number of threads to use while migrating these extents, from 1 to 4. Regardless of the mode that the VDisk starts in, it is reported as a managed mode during the migration. Also, both of the MDisks involved are reported as being in image mode during the migration. Upon completion of the command, the VDisk is classified as an image mode VDisk.
14.2.5 Migrating a VDisk between I/O groups A VDisk can be migrated between I/O groups using the svctask chvdisk command. This is only supported if the VDisk is not in a FlashCopy Mapping or Remote Copy relationship. In order to move a VDisk between I/O groups, the cache must be flushed. The SVC will attempt to destage all write data for the VDisk from the cache during the I/O group move. This flush will fail if data has been pinned in the cache for any reason (such as an MDG being offline). By default, this will cause the migration between I/O groups to fail, but this behavior can be overridden using the -force flag. If the -force flag is used and if the SVC is unable to destage all write data from the cache, then the result is that the contents of the VDisk are
746
Implementing the IBM System Storage SAN Volume Controller V4.3
corrupted by the loss of the cached data. During the flush, the VDisk operates in cache write-through mode. Attention: Do not move a VDisk to an offline I/O group under any circumstance. You must ensure that the I/O group is online before you move the VDisks to avoid any data loss. You must quiesce host I/O before the migration for two reasons: If there is significant data in cache that takes a long time to destage, then the command line will time out. SDD vpaths associated with the VDisk are deleted before the VDisk move takes place in order to avoid data corruption. So, data corruption could occur if I/O is still ongoing at a particular LUN ID when it is re-used for another VDisk. When migrating a VDisk between I/O Groups, you do not have the ability to specify the preferred node. The preferred node is assigned by the SVC. The syntax of the CLI command is: svctask chvdisk [-name -new_name_arg][-iogrp -io_group_id | - io_group_name [-force]] [-node -node_id | - node_name [-rate -throttle_rate]] [-unitmb -udid -vdisk_udid] [-warning -disk_size | -disk_size_percentage] [-autoexpand -on | -off [ -copy -id]] [-primary -copy_id][-syncrate -percentage_arg] [vdisk_name | vdisk_id [-unit [-b | -kb | -mb | -gb | -tb | -pb]]] The parameters for the CLI command are: -name new_name_arg (Optional): Specifies a new name to assign to the virtual disk. You cannot use this parameter with the -iogrp, -rate, -node, or -udid parameters. This parameter is required if you do not use the -iogrp, -rate, or -udid parameters. -iogrp io_group_id | io_group_name (Optional): Specifies a new I/O group to move the virtual disk to, by IO group ID or IO group name. You can use the -node parameter with the -iogrp parameter to specify a preferred node for the specified VDisk. Note: If the VDisk has a mapping to any hosts, it is not possible to move the VDisk to an I/O group that does not include any of those hosts. This parameter can fail if there is not enough space to allocate bitmaps for a mirrored VDisk in the target IO group. This parameter can fail if any copy is not synchronized. The -force parameter can be used to force the move, but this resynchronizes the VDisk. -force (Optional): Forces the VDisk to be removed from an I/O group. This parameter can only be used with the -iogrp parameter. Note: If the -force parameter is used and the cluster is unable to destage all write data from the cache, the contents of the VDisk are corrupted by the loss of the cached data. If the -force parameter is used to move a VDisk that has out-of-sync copies, a full resynchronization is required.
Chapter 14. Migration to and from the SAN Volume Controller
747
-rate throttle_rate [-unitmb] (Optional): Specifies the I/O governing rate for the VDisk, which caps the amount of I/O that is accepted. The default throttle_rate units are I/Os. To change the throttle_rate units to megabytes per second (MBps), specify the -unitmb parameter. The governing rate for a virtual disk can be specified by I/Os or by MBps, but not both. However, you can set the rate to I/Os for some virtual disks and to MBps for others. You cannot use this parameter with the -name, -iogrp, -node, or -udid parameters. -udid vdisk_udid (Optional): Specifies the unit number (udid) for the disk. The vdisk_udid is an identifier that is required to support OpenVMS hosts; no other systems use this parameter. Valid options are a decimal number from 0 to 32 767 or a hexadecimal number from 0 to 0x7FFF. A hexadecimal number must be preceded by 0x (for example, 0x1234). If you do not use the -udid parameter, the default udid is 0. You cannot use this parameter with the -name, -iogrp, -node, or -rate parameters. -warning disk_size | disk_size_percentage% (Optional): Generates a warning when the used disk capacity on the space-efficient copy first exceeds the specified threshold. You can specify a disk_size integer, which defaults to MBs unless the -unit parameter is specified, or you can specify a disk_size%, which is a percentage of the virtual disk size. To disable warnings, specify 0 or 0%. -unit b | kb | mb | gb | tb | pb (Optional): Specifies the data units to use for the -warning disk_size parameter. -autoexpand on | off (Optional): Specifies whether space-efficient VDisk copies automatically expand their real capacities by allocating new extents from their managed disk group. To use this parameter, the VDisk must be space-efficient. -copy id (Optional): Specifies the copy to apply the changes to. You must specify this parameter with the -autoexpand or -warning parameter. The -copy parameter is required if the specified VDisk is mirrored and only one VDisk copy is space-efficient. If both copies are space-efficient and the -copy parameter is not specified, the specified -autoexpand or -warning parameter is set on both copies. -primary copy_id (Optional): Specifies the primary copy. Changing the primary copy only takes effect when the new primary copy is online and synchronized. If the new primary is online and synchronized when the command is issued, the change takes effect immediately. -syncrate percentage (Optional): Specifies the copy synchronization rate, as a percentage of the peak synchronization rate. A value of zero (0) prevents synchronization. -node node_id | node_name (Optional): Specifies a preferred node for the specified VDisk. When using this parameter, you must also specify the -iogrp parameter. You cannot use this parameter with the -name, -rate, or -udid parameters. -vdisk_name | vdisk_id (Required): Specifies the virtual disk to modify, either by ID or by name. The chvdisk command modifies a single property of a virtual disk (VDisk). To change the VDisk name and modify the I/O group, for example, you must issue the command twice. A VDisk that is a member of a FlashCopy or Remote Copy relationship cannot be moved to another I/O Group, and this cannot be overridden by using the -force flag.
14.2.6 Monitoring the migration progress To monitor the progress of ongoing migrations, use the CLI command: svcinfo lsmigrate
748
Implementing the IBM System Storage SAN Volume Controller V4.3
To determine the extent allocation of MDisks and VDisks, use the following commands: To list the VDisk IDs and the corresponding number of extents the VDisks are occupying on the queried MDisk, use the following CLI command: svcinfo lsmdiskextent <mdiskname | mdisk_id> To list the MDisk IDs and the corresponding number of extents the queried VDisks are occupying on the listed MDisks, use the following CLI command: svcinfo lsvdiskextent To list the number of free extents available on an MDisk, use the following CLI command: svcinfo lsfreeextents <mdiskname | mdisk_id> Important: After a migration has been started, there is no way for you to stop the migration. The migration runs to completion unless it is stopped or suspended by an error condition, or if the VDisk being migrated is deleted.
14.3 Functional overview of migration This section describes the functional view of data migration.
14.3.1 Parallelism Some of the activities described below can be carried out in parallel.
Per cluster An SVC cluster supports up to 32 active concurrent instances of members of the set of migration activities:
Migrate multiple extents Migrate between MDGs Migrate off deleted MDisk Migrate to image mode
These high-level migration tasks operate by scheduling single extent migrations, as follows: Up to 256 single extent migrations can run concurrently. This number is made up of single extent migrates, which result from the operations listed above. The Migrate Multiple Extents and Migrate Between MDGs commands support a flag that allows you to specify the number of “threads” to use between 1 and 4. This parameter affects the number of extents that will be concurrently migrated for that migration operation. Thus, if the thread value is set to 4, up to four extents can be migrated concurrently for that operation, subject to other resource constraints.
Per MDisk The SVC supports up to four concurrent single extent migrates per MDisk. This limit does not take into account whether the MDisk is the source or the destination. If more than four single extent migrates are scheduled for a particular MDisk, further migrations are queued pending the completion of one of the currently running migrations.
Chapter 14. Migration to and from the SAN Volume Controller
749
14.3.2 Error handling If a medium error occurs on a read from the source, and the destinations medium error table is full, if an I/O error occurs on a read from the source repeatedly, or if the MDisk(s) go offline repeatedly, the migration is suspended or stopped. The migration will be suspended if any of the following conditions exist, otherwise it will be stopped: The migration is between Managed Disk Groups and has progressed beyond the first extent. These migrations are always suspended rather than being stopped because stopping a migration in progress would leave a VDisk spanning MDGs, which is not a valid configuration other than during a migration. The migration is a Migrate to Image Mode (even if it is processing the first extent). These migrations are always suspended rather than being stopped because stopping a migration in progress would leave the VDisk in an inconsistent state. A migration is waiting for a metadata checkpoint that has failed. If a migration is stopped, then if any migrations are queued awaiting the use of the MDisk for migration, these migrations are now considered. If, however, a migration is suspended, then the migration continues to use resources, and so another migration is not started. The SVC will attempt to resume the migration if the error log entry is marked as fixed using the CLI or the GUI. If the error condition no longer exists, then the migration will proceed. The migration might resume on a different node than the one that started the migration.
14.3.3 Migration algorithm This section describes the effect of the migration algorithm.
Chunks Regardless of the extent size for the MDG, data is migrated in units of 16 MB. In this description, this unit is referred to as a chunk. The algorithm used to migrate an extent is as follows: 1. Pause (this means to queue all new I/O requests in the virtualization layer in SVC and wait for all outstanding requests to complete) all I/O on the source MDisk on all nodes in the SVC cluster. The I/O to other extents is unaffected. 2. Unpause I/O on all of the source MDisk extent apart from writes to the specific chunk that is being migrated. Writes to the extent are mirrored to the source and destination. 3. On the node performing the migrate, for each 256 K section of the chunk: – Synchronously read 256 K from the source. – Synchronously write 256 K to the target. 4. Once the entire chunk has been copied to the destination, repeat the process for the next chunk within the extent. 5. Once the entire extent has been migrated, pause all I/O to the extent being migrated, checkpoint the extent move to on-disk metadata, redirect all further reads to the destination, and stop mirroring writes (writes only to destination). 6. If the checkpoint fails, then the I/O is unpaused.
750
Implementing the IBM System Storage SAN Volume Controller V4.3
During the migration, the extent can be divided into three regions, as shown in Figure 14-2. Region B is the chunk that is being copied. Writes to region B are queued (paused) in the virtualization layer waiting for the chunk to be copied. Reads to Region A are directed to the destination because this data has already been copied. Writes to Region A are written to both the source and the destination extent in order to maintain the integrity of the source extent. Reads and writes to Region C are directed to the source because this region has yet to be migrated. The migration of a chunk requires 64 synchronous reads and 64 synchronous writes. During this time, all writes to the chunk from higher layers in the software stack (such as cache destages) are held back. If the back-end storage is operating with significant latency, then it is possible that this operation might take some time (minutes) to complete. This can have an adverse affect on the overall performance of the SVC. To avoid this situation, if the migration of a particular chunk is still active after one minute, then the migration is paused for 30 seconds. During this time, writes to the chunk are allowed to proceed. After 30 seconds, the migration of the chunk is resumed. This algorithm is repeated as many times as necessary to complete the migration of the chunk.
Managed Disk Extents Extent N-1
Extent N
Region A (already copied)
Region B (copying) reads/writ
Extent N+1
Region C (yet to be copied) reads/writes go
16 MB
Not to scale
Figure 14-2 Migrating an extent
SVC guarantees read stability during data migrations even if the data migration is stopped by a node reset or a cluster shutdown. This is possible because SVC disallows writes on all nodes to the area being copied, and upon a failure the extent migration is restarted from the beginning.
14.4 Migrating data from an image mode VDisk This section describes how to migrate data from an image mode VDisk to a VDisk.
14.4.1 Image mode VDisk migration concept First, we describe the concepts associated with this operation.
Chapter 14. Migration to and from the SAN Volume Controller
751
MDisk modes There are three different MDisk modes: 1. Unmanaged MDisk: An MDisk is reported as unmanaged when it is not a member of any MDG. An unmanaged MDisk is not associated with any VDisks and has no metadata stored on it. The SVC will not write to an MDisk that is in unmanaged mode except when it attempts to change the mode of the MDisk to one of the other modes. 2. Image Mode MDisk: Image Mode provides a direct block-for-block translation from the MDisk to the VDisk with no virtualization. Image Mode VDisks have a minimum size of one block (512 bytes) and always occupy at least one extent. An Image Mode MDisk is associated with exactly one VDisk. 3. Managed Mode MDisk: Managed Mode Mdisks contribute extents to the pool of extents available in the MDG. Zero or more Managed Mode VDisks might use these extents.
Transitions between the different modes The following state transitions can occur to an MDisk (see Figure 14-3 on page 753): 1. Unmanaged mode to managed mode. This occurs when an MDisk is added to an MDisk group. This makes the MDisk eligible for the allocation of data and metadata extents. 2. Managed mode to unmanaged mode. This occurs when an MDisk is removed from an MDisk group. 3. Unmanaged mode to image mode. This occurs when an image mode MDisk is created on an MDisk that was previously unmanaged. It also occurs when an MDisk is used as the target for a Migrate to Image Mode. 4. Image mode to unmanaged mode. There are two distinct ways in which this can happen: – When an image mode VDisk is deleted. The MDisk that supported the VDisk becomes unmanaged. – When an image mode VDisk is migrated in image mode to another MDisk, the MDisk that is being migrated from remains in image mode until all data has been moved off it. It then transitions to unmanaged mode. 5. Image mode to managed mode. This occurs when the image mode VDisk that is using the MDisk is migrated into managed mode. 6. Managed mode to image mode is not possible. There is no operation that will take an MDisk directly from managed mode to image mode. This can be achieved by performing operations that convert the MDisk to unmanaged mode and then to image mode.
752
Implementing the IBM System Storage SAN Volume Controller V4.3
add to group
Managed mode
Not in group remove from group
delete image mode vdisk start migrate to managed mode
complete migrate
create image mode vdisk
Migrating to image mode
start migrate to image mode
Image mode
Figure 14-3 Different states of a VDisk
Image mode VDisks have the special property that the last extent in the VDisk can be a partial extent. Managed mode disks do not have this property. To perform any type of migration activity on an image mode VDisk, the image mode disk must first be converted into a managed mode disk. If the image mode disk has a partial last extent, then this last extent in the image mode VDisk must be the first to be migrated. This migration is handled as a special case. After this special migration operation has occurred, the VDisk becomes a managed mode VDisk and is treated in the same way as any other managed mode VDisk. If the image mode disk does not have a partial last extent, then no special processing is performed, the image mode VDisk is simply changed into a managed mode VDisk, and is treated in the same way as any other managed mode VDisk. After data is migrated off a partial extent, there is no way to migrate data back onto the partial extent.
Chapter 14. Migration to and from the SAN Volume Controller
753
14.4.2 Migration tips You have several methods to migrate an image mode VDisk into a managed mode VDisk: If your image mode VDisk is in the same MDG as the MDisks on which you want to migrate the extents, you can: – Migrate a single extent. You have to migrate the last extent of the image mode VDisk (number N-1). – Migrate multiple extents. – Migrate all the in-use extents from an MDisk. Migrate extents off of an MDisk that is being deleted. If you have two MDGs, one for the image mode VDisk, and one for the managed mode VDisks, you can migrate a VDisk from one MDG to another. The recommended method is to have one MDG for all the image mode VDisks, and other MDGs for the managed mode VDisks, and to use the migrate VDisk facility. Do not forget to check that enough extents are available in the target MDG.
14.5 Data migration for Windows using the SVC GUI In this section, we move the two LUNs from a Windows 2008 server that is currently attached to a DS4700 storage subsystem over to the SVC. We then manage those LUNs with SVC, migrate them from an image mode VDisk to a VDisk, migrate one of them back to an image mode Vdisk, and then finally move it to another image mode VDisk on another storage subsystem, so that those LUNs can then be masked/mapped back to the host directly. This would of course also work if we move the LUN back to the same storage subsystem. Using this example will help you perform any one of the following activities in your environment: Move a Microsoft server’s SAN LUNs from a storage subsystem and virtualize those same LUNs through the SVC. This would be the first activity that you would do when introducing the SVC into your environment. This section shows that your host downtime is only a few minutes while you remap/remask disks using your storage subsystem LUN management tool. This step is detailed in 14.5.1, “SVC added between the host system and the DS4700” on page 758. Migrate your image mode VDisk to a VDisk while your host is still running and servicing your business application. You might perform this activity if you were removing a storage subsystem from your SAN environment, or wanting to move the data onto LUNs that are more appropriate for the type of data stored on those LUNs taking into account availability, performance, and redundancy. This step is covered in 14.5.3, “Migrating the VDisk from image mode to managed mode” on page 768. Migrate your VDisk to an image mode Vdisk. You might perform this activity if you were removing the SVC from your SAN environment after a trial period. This step is detailed in 14.5.4, “Migrating the VDisk from managed mode to image mode” on page 771. Move an image mode Vdisk to another image mode VDisk. This procedure can be used to migrate data from one storage subsystem to the other. This step is detailed in 14.6.6, “Migrate the VDisks to image mode VDisks” on page 799.
754
Implementing the IBM System Storage SAN Volume Controller V4.3
These activities can be used individually, or together, enabling you to migrate your server’s LUNs from one storage subsystem to another storage subsystem using SVC as your migration tool. The only downtime required for these activities will be the time it takes you to remask/remap the LUNs between the storage subsystems and your SVC.
Windows 2008 host system connected directly to the DS4700 In our example configuration, we use a Windows 2008 host, a DS4700, and a DS4500. The host does have two LUNs (drive X and Z). The two LUNs are part of one DS4700 array. Example 14-1 shows the configuration of our example array named Database_W2k8 and that the LUNs are directly mapped to W2k8_host. Example 14-1 DS4700 configuration
show Array [Database_W2k8]; Executing script... Name: Status: Capacity RAID level: Drive type: Enclosure loss protection: Current owner:
Database_W2k8 Optimal 836.684 GB 5 Fibre Channel No Controller in slot A
Associated logical drives and free capacity Logical Drive Capacity Log_Files 250.000 GB Data 560.000 GB Free Capacity
26.684 GB
Associated drives - present (in piece order) Enclosure Slot 0 9 0 10 0 11 85 8 show storageSubsystem lunMappings host ["W2k8_host"]; Executing script... MAPPINGS (Storage Partitioning - Enabled (6 of 8 used))------------------Logical Drive Name LUN Controller Accessible by Logical Drive status Data 1 A Host W2k8_host Optimal Log_Files 0 A Host W2k8_host Optimal Script execution complete.
Chapter 14. Migration to and from the SAN Volume Controller
755
Before the migration, LUN masking is defined in the DS4700 to give access to the Windows 2008 host system for the volume from DS4700 label X and Z (see Figure 14-4).
Figure 14-4 Windows host directly attached
The following actions occur for the migration to the SVC: 1. The Windows 2008 host system is shut down before changing LUN masking in the DS4700. 2. The volume is first discovered as an “MDisk unmanaged” by the SVC. 3. A VDisk is created in image mode using this MDisk. 4. This new VDisk is mapped to the host system W2K8. 5. The Windows 2008 host system is restarted. 6. The VDisk is again available for the host system Win2K8.
756
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 14-5 shows the Windows 2008 Disk Management. The drive letters X and Z are assigned to the DS4700 LUNs.
Figure 14-5 Disk management: two volumes from DS4700 with label X and Z
Figure 14-6 shows the properties of one of the DS4700 disks using the Subsystem Device Driver DSM (SDDDSM). The disk appears as an IBM 1814 Fast Multipath Device.
Figure 14-6 Disk properties
Chapter 14. Migration to and from the SAN Volume Controller
757
14.5.1 SVC added between the host system and the DS4700 Figure 14-7 shows the new environment with the SVC and a second storage subsystem attached to the SAN. The second storage subsystem would not be required to migrate to SVC, but in the following examples, we will show that it is possible to move data across storage subsystems without any host downtime.
Figure 14-7 Add SVC and second storage
To add the SVC between the host system and the DS4700 storage subsystem, perform the following steps: 1. Check that you have installed supported device drivers on your host system. Support information is covered in Chapter 8, “Host configuration” on page 209. 2. Check that your SAN environment fulfills the supported zoning configurations. More information about zoning recommendations are covered in 8.1, “SVC setup” on page 210. 3. Shut down the host. 4. Change the LUN masking in the DS4700. Mask the LUNs to the SVC and remove the masking for the host. Example 14-2 shows that the mapping has changed from W2k8 to SVC_2_Node_Cluster. Example 14-2 Changes storage LUN masking
show storageSubsystem lunMappings host ["W2k8_host"]; Executing script... NO MAPPINGS (Storage Partitioning - Enabled (5 of 8 used))------------------Script execution complete. show storageSubsystem lunMappings hostGroup ["SVC_2_Node_Cluster"]; 758
Implementing the IBM System Storage SAN Volume Controller V4.3
Executing script... MAPPINGS (Storage Partitioning - Enabled (5 of 8 used))-------------------
Logical Drive Name LUN Data 9 Optimal Log_Files 8 Optimal Script execution complete.
Controller A
Accessible by Logical Drive status Host Group SVC_2_Node_Cluster_Tate
A
Host Group SVC_2_Node_Cluster_Tate
5. Log on to your SVC Console, open Work with Managed Disks and Managed Disks, select Discover Managed Disks in the drop-down field and click Go (Figure 14-8).
Figure 14-8 Discover managed disks
6. Now the new MDisks are discovered in the SVC (Figure 14-9). In our example configuration, they are mdisk24 and mdisk25.
Figure 14-9 New discovered MDisks in SVC
Chapter 14. Migration to and from the SAN Volume Controller
759
7. Now we create new VDisks named W2k8_Log and W2k8_Data using the two new discovered Mdisks in the MDisk group MDG0 as follows: a. Open the Work with Virtual Disks and Virtual Disks views (Figure 14-10).
Figure 14-10 Virtual Disk View
b. As shown in Figure 14-11, select Create an Imagemode VDisk from the list and click Go.
Figure 14-11 Create a image mode Vdisk
c. The Create Image Mode Virtual Disk window (Figure 14-12 on page 761) displayed. Click Next.
760
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 14-12 Create Image Mode Virtual Disk window
d. Type the name that you would like to use for the VDisk and select the attributes, in our case, the name is W2k8_Log. Click Next (Figure 14-13).
Figure 14-13 Set the attributes for the image mode Virtual Disk
Chapter 14. Migration to and from the SAN Volume Controller
761
e. Select the MDisk to create the image mode virtual disk and click Next (Figure 14-14).
Figure 14-14 Select the MDisk to use for your image disk
f. Select an I/O group, the Preferred Node, and the MDisk group. Optionally, you can let this system choose these settings (Figure 14-15). Click Next.
Figure 14-15 Select I/O Group and MDisk Group
Note: If you have more than two nodes in the cluster, select the I/O group of the nodes to evenly share the load. g. Review the summary and click Finish to create the Image Mode VDisk (Figure g).
762
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 14-16 Verify Attributes
8. Repeat steps a through g for each LUN you want to migrate to the SVC. 9. In the Viewing Virtual Disk view, we see the two newly created VDisks, as shown in Figure 14-17. In our example, they are named W2k8_Log and W2k8_Data.
Figure 14-17 Viewing Virtual DIsks
Chapter 14. Migration to and from the SAN Volume Controller
763
10.In the MDisk view (Figure 14-18), we see the two new MDisks are now shown as “Image” Mode Disk. In our example, they are named mdisk24 and mdisk25.
Figure 14-18 Viewing Managed Disks
11.Map the VDisks again to the Windows 2008 host system: a. Open the Work with Virtual Disks and Virtual Disks view, mark the VDisks and select Map Virtual Disk to a Host, and click Go (Figure 14-19).
Figure 14-19 Map virtual disk to a host
b. Choose the host and enter the SCSI LUN IDs. Click OK (Figure 14-20 on page 765).
764
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 14-20 Creating Virtual DIsk to host mappings
14.5.2 Put the migrated disks on a Windows 2008 host online 1. Start the Windows 2008 host system again and open disk management. The two disks appear as “offline” in the disk management view (Figure 14-21).
Figure 14-21 Disk Management
Chapter 14. Migration to and from the SAN Volume Controller
765
2. Figure 14-22 shows the new disk properties. The Type has changed (Figure 14-6 on page 757) to a 2145 SVC device.
Figure 14-22 LUNfromDS4k is now a 2145 SDD Disk device
3. Right-click the disk in the disk management window and select Online (Figure 14-23). Repeat this also on the second disk.
Figure 14-23 Place disk online
766
Implementing the IBM System Storage SAN Volume Controller V4.3
4. Wait until the “online” command completes. Afterwards, the disks will be available with the same data as before the migration, the assigned drive letter, and they are now ready for use, as shown in Figure 14-24.
Figure 14-24 Migrated disks are available
5. Select Start → All Programs → Subsystem Device Driver DSM → Subsystem Device Driver DSM to open the SDDDSM command-line utility (Figure 14-25).
Figure 14-25 Subsystem Device Driver DSM CLI
Chapter 14. Migration to and from the SAN Volume Controller
767
6. Enter the command datapath query device to check if all paths are available, as planned in your SAN environment (Example 14-3). Example 14-3 datapath query device
C:\Program Files\IBM\SDDDSM>datapath query device Total Devices : 2
DEV#: 0 DEVICE NAME: Disk1 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 6005076801A180E9080000000000003A ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port2 Bus0/Disk1 Part0 OPEN NORMAL 0 0 1 Scsi Port3 Bus0/Disk1 Part0 OPEN NORMAL 1106 0 2 Scsi Port2 Bus0/Disk1 Part0 OPEN NORMAL 1092 0 3 Scsi Port3 Bus0/Disk1 Part0 OPEN NORMAL 0 0 DEV#: 1 DEVICE NAME: Disk2 Part0 TYPE: 2145 POLICY: OPTIMIZED SERIAL: 6005076801A180E9080000000000003B ============================================================================ Path# Adapter/Hard Disk State Mode Select Errors 0 Scsi Port2 Bus0/Disk2 Part0 OPEN NORMAL 0 0 1 Scsi Port3 Bus0/Disk2 Part0 OPEN NORMAL 2319 0 2 Scsi Port2 Bus0/Disk2 Part0 OPEN NORMAL 2356 0 3 Scsi Port3 Bus0/Disk2 Part0 OPEN NORMAL 0 0 C:\Program Files\IBM\SDDDSM>
14.5.3 Migrating the VDisk from image mode to managed mode The VDisk is migrated to managed mode by migrating the completed VDisk as follows: 1. As shown in Figure 14-26 on page 769, select the VDisk. Then select Migrate a VDisk from the list and click Go.
768
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 14-26 Migrate a VDisk
2. Select the MDG to which to migrate the disk and the number of used threads, as shown in Figure 14-27. Click OK.
Figure 14-27 Migrating virtual disks
Note: If you migrate the VDisks to another MDisk group, the extent size of the source and target managed disk group has to be equal.
Chapter 14. Migration to and from the SAN Volume Controller
769
3. The Migration Progress view will appear and enable you to monitor the migration progress (Figure 14-28).
Figure 14-28 View Progress
4. Click the percentage to show more detailed information about this VDisk (Figure 14-29).
Figure 14-29 View Progress details
5. During the migration process, the VDisks are still in the old MDisk group. In our example, the old group is FC_MDG (Figure 14-30).
Figure 14-30 VDisks in the old MDisk group
6. After the migration is complete, the VDisk is now in the new MDisk group. Figure 14-31 on page 771 shows after the migration for the disk W2k8_log has completed, and the migration for W2k8_data is still in progress.
770
Implementing the IBM System Storage SAN Volume Controller V4.3
At the migration for VDisk W2k8_Log completes, the MDisk group has changed to the new MDisk group, which is also named W2k8_Log. The second VDisk will also change the MDisk group once the migration is completed.
Figure 14-31 VDisk W2k8_Log in new MDisk group
14.5.4 Migrating the VDisk from managed mode to image mode The VDisk in managed mode can be migrated to image mode. One reason for doing this would be after an SVC virtualization trial period has expired, and you are returning the volume to its original state. In this example, we migrate a managed VDisk to an image mode VDisk. 1. Check the VDisk you want to migrate and select Migrate to an image mode VDisk from the drop-down menu (Figure 14-32). Click Go.
Figure 14-32 Migrate to an image mode VDisk
Chapter 14. Migration to and from the SAN Volume Controller
771
2. The Introduction window appears. Click Next (Figure 14-33).
Figure 14-33 Migrate to an image mode VDisk
3. Select the source VDisk copy and click Next (Figure 14-34).
Figure 14-34 Migrate to a image mode VDisk
772
Implementing the IBM System Storage SAN Volume Controller V4.3
4. Select a target MDisk by clicking the radio button for it (Figure 14-35). Click Next.
Figure 14-35 Select the Target MDisk
5. Select an MDG by clicking the radio button for it (Figure 14-36). Click Next.
Figure 14-36 Select target MDisk group
Note: If you migrate the VDisks to another MDisk group, the extent size of the source and target managed disk group has to be equal.
Chapter 14. Migration to and from the SAN Volume Controller
773
6. Select the number of threads (1 to 4). The higher the number, the higher the priority (Figure 14-37). Click Next.
Figure 14-37 Select the number of threads
7. Verify the migration attributes (Figure 14-38) and click Finish.
Figure 14-38 Verify Migration Attributes
8. The progress window will appear (Figure 14-39).
Figure 14-39 Viewing Image Mode Migration Progress
774
Implementing the IBM System Storage SAN Volume Controller V4.3
9. If you open the Managed Disk view, you can see that the mdisk28 selected in Figure 14-35 on page 773 is in image disk mode, as shown in Figure 14-40.
Figure 14-40 Viewing Managed Disks
10.Repeat these steps for every VDisk you want to migrate to an Image Mode VDisk. 11.Free the data from the SVC by using the procedure in 14.5.6, “Free the data from the SVC” on page 779.
14.5.5 Migrating the VDisk from image mode to image mode The migrating a VDisk from image mode to image mode process is used to move image mode VDisks from one storage subsystem to another storage subsystem. The data stays available for the applications during this migration, so this is a zero downtime data move from one disk subsystem to another. This procedure is nearly the same as in 14.5.4, “Migrating the VDisk from managed mode to image mode” on page 771. In this section, we describe how to migrate an image mode VDisk to another image mode VDisk. In our example, we migrate the VDisk W2k8_Log to another disk subsystem as an image mode VDisk. The second storage subsystem is a DS4500; a new LUN is configured on the storage and mapped to the SVC Cluster. The LUN is available in SVC as unmanaged disk29, as shown in Figure .
Figure 14-41 Unmanaged disk on a DS4500 storage subsystem
Chapter 14. Migration to and from the SAN Volume Controller
775
To migrate the image mode VDisk to another image mode VDisk, perform the following steps: 1. Check the VDisk to migrate and select Migrate to an image mode VDisk from the drop-down menu (Figure 1). Click Go.
Figure 14-42 Migrate to an image mode VDisk
2. The Introduction window appears. Click Next (Figure 14-43).
Figure 14-43 Migrate to an image mode VDisk
3. Select the VDisk source copy and click Next (Figure 14-44 on page 777).
776
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 14-44 Select copy
4. Select a target MDisk by clicking the radio button for it (Figure 14-45). Click Next.
Figure 14-45 Select Target MDisk
5. Select a target managed disk group by clicking the radio button for it (Figure 5). Click Next.
Figure 14-46 Select MDisk Group Chapter 14. Migration to and from the SAN Volume Controller
777
6. Select the number of threads (1 to 4). The higher the number, the higher the priority (Figure 14-47). Click Next.
Figure 14-47 Select the Threads
7. Verify the migration attributes (Figure 14-48) and click Finish.
Figure 14-48 Verify Migration Attributes
778
Implementing the IBM System Storage SAN Volume Controller V4.3
8. Check the progress window (Figure 14-49) and click Close.
Figure 14-49 Viewing Image Mode Migration Progress
9. Repeat these steps for all image mode VDisks you want to migrate. 10.If you want to free the data from the SVC, use the procedure in 14.5.6, “Free the data from the SVC” on page 779.
14.5.6 Free the data from the SVC If your data resides in an image mode VDisk inside the SVC, it is possible to free the data from the SVC. The sections listed below show how to migrate data to an image mode VDisk. Depending on your environment you might have to follow these procedures before freeing the data of the SVC: 14.5.4, “Migrating the VDisk from managed mode to image mode” on page 771 14.5.5, “Migrating the VDisk from image mode to image mode” on page 775 To free data from the SVC, we use the delete vdisk command.
Chapter 14. Migration to and from the SAN Volume Controller
779
If the command succeeds on an image mode VDisk, then the underlying back-end storage controller will be consistent with the data that a host could previously have read from the image mode VDisk, that is, all fast write data will have been flushed to the underlying LUN. Deleting an image mode VDisk causes the MDisk associated with the VDisk to be ejected from the MDG. The mode of the MDisk will be returned to Unmanaged. Note: This only applies to image mode VDisks. If you delete a normal VDisk, all data will also be deleted. As shown in Figure 14-22 on page 766, the SAN disks currently reside on the SVC 2145 device. Check that you have installed supported device drivers on your host system. Support information is covered in Chapter 8, “Host configuration” on page 209. To switch back to the storage subsystem, perform the following steps: 1. Shut down your host system. 2. Edit the LUN masking on your storage subsystem. Remove the SVC from the LUN masking and add the host to the masking. 3. Open the Virtual Disk to Host mappings view in the SVC Console, mark your host, select Delete a Mapping, and click Go (Figure 14-50).
Figure 14-50 Delete a mapping
4. Confirm the task by clicking Delete (Figure 14-51).
Figure 14-51 Delete a mapping
5. Open the virtual disk view in the SVC console, check the disk, select Delete a VDisk, and click Go (Figure 14-52 on page 781). 780
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 14-52 Delete a VDisk
6. Confirm the deletion task by clicking OK (Figure 14-53).
Figure 14-53 Delete a VDisk
7. The VDisk is removed from the SVC. 8. Repeat steps 2 to 7 for every disk you want to free from the SVC. 9. Power on your host system.
14.5.7 Put the disks online in Windows 2008 that have been freed from SVC 1. Open your Disk Management window; the disk that has been free up from SVC appears as offline (Figure 14-54).
Figure 14-54 W2k8 Disk Management
Chapter 14. Migration to and from the SAN Volume Controller
781
2. Right-click the offline disk and select Online (Figure 14-55).
Figure 14-55 Disk Management
3. After the disk is set to online, the assigned drive letter and disk label appears, and it is ready for use (Figure 14-56).
Figure 14-56 DIsk Management
4. In the Device Manager window, we see that the disk has successfully been moved to the storage subsystem. The device type of the disk has changed to 1722 (DS4500), as shown in Figure 14-57 on page 783.
782
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 14-57 W2k8 Device Manager
14.6 Migrating Linux SAN disks to SVC disks In this section, we move the two LUNs from a Linux server that is currently booting directly off of our DS4000 storage subsystem over to the SVC. We then manage those LUNs with SVC, move them between other managed disks, and then finally move them back to image mode disks, so that those LUNs can then be masked/mapped back to the Linux server directly. Using this example will help you perform any one of the following activities in your environment: Move a Linux server’s SAN LUNs from a storage subsystem and virtualize those same LUNs through the SVC. This would be the first activity that you would do when introducing SVC into your environment. This section shows that your host downtime is only a few minutes while you remap/remask disks using your storage subsystem LUN management tool. This step is detailed in 14.6.2, “Prepare your SVC to virtualize disks” on page 786. Move data between storage subsystems while your Linux server is still running and servicing your business application. You might perform this activity if you were removing a storage subsystem from your SAN environment, or wanting to move the data onto LUNs that are more appropriate for the type of data stored on those LUNs, taking availability, performance and redundancy into account. This step is covered in 14.6.4, “Migrate the image mode VDisks to managed MDisks” on page 793. Move your Linux server’s LUNs back to image mode VDisks so that they can be remapped/remasked directly back to the Linux server. This step is detailed in 14.6.5, “Preparing to migrate from the SVC” on page 796. These three activities can be used individually, or together, enabling you to migrate your Linux server’s LUNs from one storage subsystem to another storage subsystem using SVC as your migration tool. If you do not use all three activities, it will enable you to introduce or remove the SVC from your environment. The only downtime required for these activities will be the time it takes you to remask/remap the LUNs between the storage subsystems and your SVC.
Chapter 14. Migration to and from the SAN Volume Controller
783
In Figure 14-58, we show our Linux environment.
Zoning for migration scenarios LINUX Host
SAN
Green Zone
IBM or OEM Storage Subsystem
Figure 14-58 Linux SAN environment
Figure 14-58 shows our Linux server connected to our SAN infrastructure. It has two LUNs that are masked directly to it from our storage subsystem: The LUN with SCSI ID 0 has the host operating system (our host is Red Hat Enterprise Linux V5.1) and this LUN is used to boot the system directly from the storage subsystem. The operating system identifies it as /dev/mapper/VolGroup00-LogVol00. Note: To successfully boot a host off of the SAN, the LUN needs to have been assigned as SCSI LUN ID 0. This LUN is seen by Linux as our /dev/sda disk. We have also mapped a second disk (SCSI ID 1) to the host. It is 5 GB in size, and is mounted in the folder / data on disk /dev/dm-2 Example 14-4 shows our directly attached disks to the Linux hosts. Example 14-4 Directly attached disks
[root@Palau data]# df Filesystem 1K-blocks /dev/mapper/VolGroup00-LogVol00 10093752 /dev/sda1 101086 tmpfs 1033496 /dev/dm-2 5160576
784
Used Available Use% Mounted on 1971344 12054 0 158160
7601400 83813 1033496 4740272
Implementing the IBM System Storage SAN Volume Controller V4.3
21% 13% 0% 4%
/ /boot /dev/shm /data
[root@Palau data]# Our Linux server represents a typical SAN environment with a host directly using LUNs created on a SAN storage subsystem, as shown in Figure 14-58 on page 784: The Linux server’s HBA cards are zoned so that they are in the Green zone with our storage subsystem. The two LUNs that have been defined on the storage subsystem, using LUN masking, are directly available to our Linux server.
14.6.1 Connecting the SVC to your SAN fabric This section covers the basic steps that you would take to introduce the SVC into your SAN environment. While this section only summarizes these activities, you should be able to accomplish this without any downtime to any host or application that is also using your storage area network. If you have an SVC already connected, then you can safely go to 14.6.2, “Prepare your SVC to virtualize disks” on page 786. Be very careful connecting the SVC into your storage area network, as it will require you to connect cables to your SAN switches, and therefore alter your switch zone configuration. Doing these activities incorrectly could render your SAN inoperable, so make sure you fully understand the impact of everything you are doing. Connecting the SVC to your SAN fabric will require you to: Assemble your SVC components (nodes, UPS, and master console), cable it correctly, power it on, and verify that it is visible on your storage area network. This is covered in much greater detail in Chapter 3, “Planning and configuration” on page 25. Create and configure your SVC cluster. This is covered in greater detail in Chapter 5, “SVC Console” on page 93 and Chapter 6, “Quickstart configuration using the command-line interface” on page 157. Create these additional zones: – An SVC node zone (our Black zone in Figure 14-59 on page 786). This zone should just contain all the ports (or WWN) for each of the SVC nodes in your cluster. Our SVC is made up of a two node cluster, where each node has four ports. So our Black zone has eight WWNs defined. – A storage zone (our Red zone). This zone should also have all the ports/WWN from the SVC node zone as well as the ports/WWN for all the storage subsystems that SVC will virtualize. – A host zone (our Blue zone). This zone should contain the ports/WWNs for each host that will access VDisk, together with the ports defined in the SVC node zone. Attention: Do not put your storage subsystems in the host (Blue) zone. This is an unsupported configuration and could lead to data loss!
Chapter 14. Migration to and from the SAN Volume Controller
785
Our environment has been set up as described above and can be seen in Figure 14-59.
Zoning per Migration Scenarios LINUX Host
SAN
IBM or OEM Storage Subsystem
IBM or OEM Storage Subsystem
SVC I/O grp0 SVC SVC
Green Zone Red Zone Blue Zone Black Zone
By Pinocchio 12-09-2005
Figure 14-59 SAN environment with SVC attached
14.6.2 Prepare your SVC to virtualize disks This section covers the preparation tasks that we can perform before taking our Linux server offline. These are all nondisruptive activities and should not affect your SAN fabric or your existing SVC configuration (if you already have a production SVC in place).
Create a managed disk group When we move the two Linux LUNs to the SVC, they will first be used in image mode, and as such, we need a managed disk group to hold those disks. First, we need to create an empty manage disk group for each of the disks, using the commands in Example 14-5. Our managed disk group will be called Palau-MDG0 and Palau-MDG1 and will hold our boot LUN and data LUN, respectively. Example 14-5 Create empty mdiskgroup
IBM_2145:ITSO-CLS1:admin>svctask mkmdiskgrp -name Palau_Data -ext 512 MDisk Group, id [7], successfully created IBM_2145:ITSO-CLS1:admin>svcinfo lsmdiskgrp id name status mdisk_count vdisk_count capacity extent_size free_capacity virtual_capacity used_capacity real_capacity overallocation warning
786
Implementing the IBM System Storage SAN Volume Controller V4.3
6 Palau_SANB online 512 0 0.00MB 0 7 Palau_Data online 512 0 0.00MB 0 IBM_2145:ITSO-CLS1:admin>
0 0.00MB
0 0.00MB
0 0
0 0.00MB
0 0.00MB
0 0
Create your host definition If your zone preparation has been performed correctly, the SVC should be able to see the Linux server’s HBA adapters on the fabric (our host only had one HBA). The svcinfo lshbaportcandidate command on the SVC will list all the WWNs that the SVC can see on the SAN fabric that has not yet been allocated to a host. Example 14-6 shows the output of the nodes it found on our SAN fabric. (If the port did not show up, it would indicate that we have a zone configuration problem.) Example 14-6 Display HBA port candidates
IBM_2145:ITSO-CLS1:admin>svcinfo lshbaportcandidate id 210000E08B89C1CD 210000E08B054CAA 210000E08B0548BC 210000E08B0541BC 210000E08B89CCC2 IBM_2145:ITSO-CLS1:admin> If you do not know the WWN of your Linux server, you can look at which WWNs are currently configured on your storage subsystem for this host. Figure 14-60 shows our configured ports on an IBM DS4700 storage subsystem.
Figure 14-60 Display port WWNs
Chapter 14. Migration to and from the SAN Volume Controller
787
After verifying that the SVC can see our host (linux2), we create the host entry and assign the WWN to this entry. These commands can be seen in Example 14-7. Example 14-7 Create the host entry
IBM_2145:ITSO-CLS1:admin>svctask mkhost -name Palau -hbawwpn 210000E08B054CAA:210000E08B89C1CD Host, id [0], successfully created IBM_2145:ITSO-CLS1:admin>svcinfo lshost Palau id 0 name Palau port_count 2 type generic mask 1111 iogrp_count 4 WWPN 210000E08B89C1CD node_logged_in_count 4 state inactive WWPN 210000E08B054CAA node_logged_in_count 4 state inactive IBM_2145:ITSO-CLS1:admin>
Verify that we can see our storage subsystem If our zoning has been performed correctly, the SVC should also be able to see the storage subsystem with the svcinfo lscontroller command (Example 14-8). Example 14-8 Discover storage controller
IBM_2145:ITSO-CLS1:admin>svcinfo lscontroller id controller_name ctrl_s/n vendor_id product_id_low product_id_high 0 DS4500 IBM 1742-900 1 DS4700 IBM 1814 FAStT IBM_2145:ITSO-CLS1:admin> The storage subsystem can be renamed to something more meaningful (if we had many storage subsystems connected to our SAN fabric, then renaming them makes it considerably easier to identify them) with the svctask chcontroller -name command.
Get the disk serial numbers To help avoid the possibility of creating the wrong VDisks from all the available, unmanaged MDisks (in case there are many seen by the SVC), we get the LUN serial numbers from our storage subsystem administration tool (Storage Manager). When we discover these MDisks, we confirm that we have the right serial numbers before we create the image mode VDisks. If you are also using a DS4000 family storage subsystem, Storage Manager will provide the LUN serial numbers. Right-click your logical drive and choose Properties. Our serial numbers are shown in Figure 14-61 on page 789 and Figure 14-62 on page 789.
788
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 14-61 Obtaining the disk serial number
Figure 14-62 Obtaining the disk serial number
Chapter 14. Migration to and from the SAN Volume Controller
789
Before we move the LUNs to the SVC, we have to configure the host multipath configuration for the SVC. To do this, add the following entry to your multipath.conf file, as shown in Example 14-9, and add the content of Example 14-10 to the file. Example 14-9 Edit the multipath.conf file
[root@Palau ~]# vi /etc/multipath.conf [root@Palau ~]# service multipathd stop Stopping multipathd daemon: [root@Palau ~]# service multipathd start Starting multipathd daemon: [root@Palau ~]#
[
OK
]
[
OK
]
Example 14-10 Data to add to file
# SVC device { vendor "IBM" product "2105800" path_grouping_policy group_by_serial } We are now ready to move the ownership of the disks to the SVC, discover them as MDisks, and give them back to the host as VDisks.
14.6.3 Move the LUNs to the SVC In this step, we move the LUNs assigned to the Linux server and reassign them to the SVC. Our Linux server has two LUNs: One LUN is for our boot disk and operating system file systems, and the other LUN holds our application and data files. Moving both LUNs at once will require the host to be shut down. If we only wanted to move the LUN that holds our application and data files, then we could do that without rebooting the host. The only requirement would be that we unmount the file system, and vary off the volume group to ensure data integrity between the re-assignment. As we intend to move both LUNs at the same time, these are the required steps: 1. Confirm that the multipath.conf file is configured for SVC. 2. Shut down the host. If you were just moving the LUNs that contained the application and data, then you could follow this procedure instead: a. Stop the applications that are using the LUNs. b. Unmount those file systems with the umount MOUNT_POINT command. c. If the file systems are an LVM volume, then deactivate that volume group with the vgchange -a n VOLUMEGROUP_NAME. d. If you can, also unload your HBA driver using rmmod DRIVER_MODULE. This will remove the SCSI definitions from the kernel (we will reload this module and rediscover the disks later). It is possible to tell the Linux SCSI subsystem to rescan for new disks without requiring you to unload the HBA driver; however, these details are not provided here.
790
Implementing the IBM System Storage SAN Volume Controller V4.3
3. Using Storage Manager (our storage subsystem management tool), we can unmap/unmask the disks from the Linux server and remap/remask the disks to the SVC. Note: Even though we are using Boot from SAN, you can also map the boot disk with any LUN number to the SVC. It does not have to be 0. This is only important afterwards when we configure the mapping in the SVC to the host. 4. From the SVC, discover the new disks with the svctask detectmdisk command. The disks will be discovered and named mdiskN, where N is the next available MDisk number (starting from 0). Example 14-11 shows the commands we used to discover our MDisks and verify that we have the correct ones. Example 14-11 Discover the new MDisks
IBM_2145:ITSO-CLS1:admin>svctask detectmdisk IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID 26 mdisk26 online unmanaged 12.0GB 0000000000000008 DS4700 600a0b800026b2820000428f48739bca00000000000000000000000000000000 27 mdisk27 online unmanaged 5.0GB 0000000000000009 DS4700 600a0b800026b282000042f84873c7e100000000000000000000000000000000 IBM_2145:ITSO-CLS1:admin> Important: Match your discovered MDisk serial numbers (UID on the svcinfo lsmdisk task display) with the serial number you took earlier (in Figure 14-61 and Figure 14-62 on page 789). 5. Once we have verified that we have the correct MDisks, we rename them to avoid confusion in the future when we perform other MDisk related tasks (Example 14-12). Example 14-12 Rename the MDisks
IBM_2145:ITSO-CLS1:admin>svctask chmdisk -name md_palauS mdisk26 IBM_2145:ITSO-CLS1:admin>svctask chmdisk -name md_palauD mdisk27 IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID 26 md_palauS online unmanaged 12.0GB 0000000000000008 DS4700 600a0b800026b2820000428f48739bca00000000000000000000000000000000 27 md_palauD online unmanaged 5.0GB 0000000000000009 DS4700 600a0b800026b282000042f84873c7e100000000000000000000000000000000 IBM_2145:ITSO-CLS1:admin>
Chapter 14. Migration to and from the SAN Volume Controller
791
6. We create our image mode VDisks with the svctask mkvdisk command and the -vtype image option (Example 14-13). This command will virtualize the disks in the exact same layout as though they were not virtualized. Example 14-13 Create the image mode VDisks
IBM_2145:ITSO-CLS1:admin>svctask mkvdisk -mdiskgrp Palau_SANB -iogrp 0 -vtype image -mdisk md_palauS -name palau_SANB Virtual Disk, id [29], successfully created IBM_2145:ITSO-CLS1:admin> IBM_2145:ITSO-CLS1:admin> IBM_2145:ITSO-CLS1:admin> IBM_2145:ITSO-CLS1:admin>svctask mkvdisk -mdiskgrp Palau_Data -iogrp 0 -vtype image -mdisk md_palauD -name palau_Data Virtual Disk, id [30], successfully created IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID 26 md_palauS online image 6 Palau_SANB 12.0GB 0000000000000008 DS4700 600a0b800026b2820000428f48739bca00000000000000000000000000000000 27 md_palauD online image 7 Palau_Data 5.0GB 0000000000000009 DS4700 600a0b800026b282000042f84873c7e100000000000000000000000000000000 IBM_2145:ITSO-CLS1:admin> 7. Map the new image mode VDisks to the host (Example 14-14). Attention: Make sure that you map the boot VDisk with SCSI ID 0 to your host. The host must be able to identify the boot volume during the boot process. Example 14-14 Map the VDisks to the host
IBM_2145:ITSO-CLS1:admin>svctask mkvdiskhostmap -host Palau -scsi 0 palau_SANB Virtual Disk to Host map, id [0], successfully created IBM_2145:ITSO-CLS1:admin>svctask mkvdiskhostmap -host Palau -scsi 1 palau_Data Virtual Disk to Host map, id [1], successfully created IBM_2145:ITSO-CLS1:admin>svcinfo lshostvdiskmap Palau id name SCSI_id vdisk_id vdisk_name wwpn vdisk_UID 0 Palau 0 29 palau_SANB 210000E08B89C1CD 60050768018301BF280000000000002B 0 Palau 1 30 palau_Data 210000E08B89C1CD 60050768018301BF280000000000002C IBM_2145:ITSO-CLS1:admin> Note: While the application is in a quiescent state, you could choose to FlashCopy the new image VDisks onto other VDisks. You do not need to wait until the FlashCopy has completed before starting your application.
792
Implementing the IBM System Storage SAN Volume Controller V4.3
8. Power on your host server and enter your FC HBA adapter BIOS before booting the OS, and make sure that you change the boot configuration so that it points to the SVC. In our example, we performed the following steps on a QLogic HBA: a. Press Ctrl + Q to enter the HBA BIOS. b. Open Configuration Settings. c. Open Selectable Boot Settings. d. Change the entry from your storage subsystem to the SVC 2145 LUN with SCSI ID 0. e. Exit the menu and save your changes. 9. Boot up your Linux operating system. If you only moved the application LUN to the SVC and left your Linux server running, then you would need to follow these steps to see the new VDisk: a. Load your HBA driver with the modprobe DRIVER_NAME command. If you did not (and cannot) unload your HBA driver, then you can issue commands to the kernel to rescan the SCSI bus to see the new VDisks (these details are beyond the scope of this book). b. Check your syslog and verify that the kernel found the new VDisks. On Red Hat Enterprise Linux, the syslog is stored in /var/log/messages. c. If your application and data is on an LVM volume, rediscover the volume group, then run vgchange -a y VOLUME_GROUP to activate the volume group. 10.Mount your file systems with the mount /MOUNT_POINT command (Example 14-15). The df output shows us that all disks are available again. Example 14-15 Mount data disk
[root@Palau data]# mount /dev/dm-2 /data [root@Palau data]# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/VolGroup00-LogVol00 10093752 1938056 7634688 21% / /dev/sda1 101086 12054 83813 13% /boot tmpfs 1033496 0 1033496 0% /dev/shm /dev/dm-2 5160576 158160 4740272 4% /data [root@Palau data]# 11.You are now ready to start your application.
14.6.4 Migrate the image mode VDisks to managed MDisks While the Linux server is still running, and our file systems are in use, we now migrate the image mode VDisks onto striped VDisks, with the extents being spread over the other three MDisks. In our example, the three new LUNs are located on an DS4500 storage subsystem, so we will also move to another storage subsystem in this example.
Preparing MDisks for striped mode VDisks From our second storage subsystem, we have:
Created and allocated three LUNs to the SVC. Discovered them as MDisks. Renamed these LUNs to something more meaningful. Created a new MDisk group. Put all these MDisks into this group.
Chapter 14. Migration to and from the SAN Volume Controller
793
You can see the output of our commands in Example 14-16. Example 14-16 Create a new MDisk group
IBM_2145:ITSO-CLS1:admin>svctask detectmdisk IBM_2145:ITSO-CLS1:admin>svctask mkmdiskgrp -name MD_palauVD -ext 512 MDisk Group, id [8], successfully created IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID 600a0b8000174233000000b5486d255b00000000000000000000000000000000 26 md_palauS online image 6 Palau_SANB 12.0GB 0000000000000008 DS4700 600a0b800026b2820000428f48739bca00000000000000000000000000000000 27 md_palauD online image 7 Palau_Data 5.0GB 0000000000000009 DS4700 600a0b800026b282000042f84873c7e100000000000000000000000000000000 28 mdisk28 online unmanaged 8.0GB 0000000000000010 DS4500 600a0b8000174233000000b9487778ab00000000000000000000000000000000 29 mdisk29 online unmanaged 8.0GB 0000000000000011 DS4500 600a0b80001744310000010f48776bae00000000000000000000000000000000 30 mdisk30 online unmanaged 8.0GB 0000000000000012 DS4500 600a0b8000174233000000bb487778d900000000000000000000000000000000 IBM_2145:ITSO-CLS1:admin>svctask chmdisk -name palau-md1 mdisk28 IBM_2145:ITSO-CLS1:admin>svctask chmdisk -name palau-md2 mdisk29 IBM_2145:ITSO-CLS1:admin>svctask chmdisk -name palau-md3 mdisk30 IBM_2145:ITSO-CLS1:admin>svctask addmdisk -mdisk palau-md1 MD_palauVD IBM_2145:ITSO-CLS1:admin>svctask addmdisk -mdisk palau-md2 MD_palauVD IBM_2145:ITSO-CLS1:admin>svctask addmdisk -mdisk palau-md3 MD_palauVD IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID 26 md_palauS online image 6 Palau_SANB 12.0GB 0000000000000008 DS4700 600a0b800026b2820000428f48739bca00000000000000000000000000000000 27 md_palauD online image 7 Palau_Data 5.0GB 0000000000000009 DS4700 600a0b800026b282000042f84873c7e100000000000000000000000000000000 28 palau-md1 online managed 8 MD_palauVD 8.0GB 0000000000000010 DS4500 600a0b8000174233000000b9487778ab00000000000000000000000000000000 29 palau-md2 online managed 8 MD_palauVD 8.0GB 0000000000000011 DS4500 600a0b80001744310000010f48776bae00000000000000000000000000000000 30 palau-md3 online managed 8 MD_palauVD 8.0GB 0000000000000012 DS4500 600a0b8000174233000000bb487778d900000000000000000000000000000000 IBM_2145:ITSO-CLS1:admin>
794
Implementing the IBM System Storage SAN Volume Controller V4.3
Migrate the VDisks We are now ready to migrate the image mode VDisks onto striped VDisks in the MD_palauVD mdiskgroup with the svctask migratevdisk command (Example 14-17). While the migration is running, our Linux server is still running. To check the overall progress of the migration, we use the svcinfo lsmigrate command, as shown in Example 14-17. Listing the MDisk group with the svcinfo lsmdiskgrp command shows that the free capacity on the old MDisk groups is slowly increasing as those extents are moved to the new MDisk group. Example 14-17 Migrating image mode VDisks to striped VDisks
IBM_2145:ITSO-CLS1:admin>svctask migratevdisk -vdisk palau_SANB -mdiskgrp MD_palauVD IBM_2145:ITSO-CLS1:admin>svctask migratevdisk -vdisk palau_Data -mdiskgrp MD_palauVD IBM_2145:ITSO-CLS1:admin>svcinfo lsmigrate migrate_type MDisk_Group_Migration progress 25 migrate_source_vdisk_index 29 migrate_target_mdisk_grp 8 max_thread_count 4 migrate_source_vdisk_copy_id 0 migrate_type MDisk_Group_Migration progress 70 migrate_source_vdisk_index 30 migrate_target_mdisk_grp 8 max_thread_count 4 migrate_source_vdisk_copy_id 0 IBM_2145:ITSO-CLS1:admin> Once this task has completed, Example 14-18 shows that the VDisks are now spread over three MDisks. Example 14-18 Migration complete
IBM_2145:ITSO-CLS1:admin>svcinfo lsmdiskgrp MD_palauVD id 8 name MD_palauVD status online mdisk_count 3 vdisk_count 2 capacity 24.0GB extent_size 512 free_capacity 7.0GB virtual_capacity 17.00GB used_capacity 17.00GB real_capacity 17.00GB overallocation 70 warning 0 IBM_2145:ITSO-CLS1:admin>svcinfo lsvdiskmember palau_SANB id 28 29 30
Chapter 14. Migration to and from the SAN Volume Controller
795
IBM_2145:ITSO-CLS1:admin>svcinfo lsvdiskmember palau_Data id 28 29 30 IBM_2145:ITSO-CLS1:admin> Our migration to striped VDisks on another Storage Subsystem (DS4500) is now complete. The original MDisks (Palau-MDG0 and Palau-MD1) can now be removed from the SVC, and these LUNs removed from the storage subsystem. If these LUNs were the last used LUNs on our DS4700 storage subsystem, then we could remove it from our SAN fabric.
14.6.5 Preparing to migrate from the SVC Before we move the Linux servers LUNs from being accessed by the SVC as virtual disks to become directly accessed from the storage subsystem, we need to convert the VDisks into image mode VDisks. You might want to perform this activity for any one of these reasons: You purchased a new storage subsystem, and you were using SVC as a tool to migrate from your old storage subsystem to this new storage subsystem. You used the SVC to FlashCopy or Metro Mirror a VDisk onto another VDisk, and you no longer need that host connected to the SVC. You want to ship a host and its data that is currently connected to the SVC to a site where there is no SVC. Changes to your environment no longer require this host to use the SVC. There are also some other preparation activities that we can do before we have to shut down the host, and reconfigure the LUN masking/mapping. This section covers those activities. If you are moving the data to a new storage subsystem, it is assumed that storage subsystem is connected to your SAN fabric, powered on, and visible from your SAN switches. Your environment should look similar to ours, shown in Figure 14-63 on page 797.
796
Implementing the IBM System Storage SAN Volume Controller V4.3
Zoning for migration scenarios LINUX Host
SAN
IBM or OEM Storage Subsystem
IBM or OEM Storage Subsystem
SVC I/O grp0 SVC SVC
Green Zone Red Zone Blue Zone Black Zone
Figure 14-63 Environment with SVC
Make fabric zone changes The first step is to set up the SAN configuration so that all the zones are created. The new storage subsystem should be added to the Red zone so that the SVC can talk to it directly. We also need a Green zone for our host to use when we are ready for it to directly access the disk after it has been removed from the SVC. It is assumed that you have created the necessary zones. Once your zone configuration is set up correctly, the SVC should see the new storage subsystems controller using the svcinfo lscontroller command, as shown in Figure 14-8 on page 759. It is also a good idea to rename it to something more useful, which can be done with the svctask chcontroller -name command.
Chapter 14. Migration to and from the SAN Volume Controller
797
Create new LUNs On our storage subsystem, we created two LUNs, and masked the LUNs so that the SVC can see them. These two LUNs will eventually be given directly to the host, removing the VDisks that it currently has. To check that the SVC can use them, issue the svctask detectmdisk command, as shown in Example 14-19. Example 14-19 Discover the new MDisks
IBM_2145:ITSO-CLS1:admin>svctask detectmdisk IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID 0 mdisk0 online managed 600a0b800026b282000042f84873c7e100000000000000000000000000000000 28 palau-md1 online managed 8 MD_palauVD 8.0GB 0000000000000010 DS4500 600a0b8000174233000000b9487778ab00000000000000000000000000000000 29 palau-md2 online managed 8 MD_palauVD 8.0GB 0000000000000011 DS4500 600a0b80001744310000010f48776bae00000000000000000000000000000000 30 palau-md3 online managed 8 MD_palauVD 8.0GB 0000000000000012 DS4500 600a0b8000174233000000bb487778d900000000000000000000000000000000 31 mdisk31 online unmanaged 6.0GB 0000000000000013 DS4500 600a0b8000174233000000bd4877890f00000000000000000000000000000000 32 mdisk32 online unmanaged 12.5GB 0000000000000014 DS4500 600a0b80001744310000011048777bda00000000000000000000000000000000 IBM_2145:ITSO-CLS1:admin> Even though the MDisks will not stay in the SVC for long, we still recommend that you rename them to something more meaningful, just so that they do not get confused with other MDisks being used by other activities. Also, we create the MDisk groups to hold our new MDisks. This is shown in Example 14-20. Example 14-20 Rename the MDisks
IBM_2145:ITSO-CLS1:admin>svctask chmdisk -name mdpalau_ivd mdisk32 IBM_2145:ITSO-CLS1:admin>svctask mkmdiskgrp -name MDG_Palauivd -ext 512 MDisk Group, id [9], successfully created IBM_2145:ITSO-CLS1:admin>svctask mkmdiskgrp -name MDG_Palauivd -ext 512 CMMVC5758E Object name already exists. IBM_2145:ITSO-CLS1:admin>svcinfo lsmdiskgrp id name status mdisk_count vdisk_count capacity extent_size free_capacity virtual_capacity used_capacity real_capacity overallocation warning 8 MD_palauVD online 3 2 24.0GB 512 7.0GB 17.00GB 17.00GB 17.00GB 70 0 9 MDG_Palauivd online 0 0 0 512 0 0.00MB 0.00MB 0.00MB 0 0 IBM_2145:ITSO-CLS1:admin>
798
Implementing the IBM System Storage SAN Volume Controller V4.3
Our SVC environment is now ready for the VDisk migration to image mode VDisks.
14.6.6 Migrate the VDisks to image mode VDisks While our Linux server is still running, we migrate the managed VDisks onto the new MDisks using image mode VDisks. The command to perform this action is svctask migratetoimage and is shown in Example 14-21. Example 14-21 Migrate the VDisks to image mode VDisks
IBM_2145:ITSO-CLS1:admin>svctask migratetoimage -vdisk palau_SANB -mdisk mdpalau_ivd -mdiskgrp MD_palauVD IBM_2145:ITSO-CLS1:admin>svctask migratetoimage -vdisk palau_Data -mdisk mdpalau_ivd1 -mdiskgrp MD_palauVD IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID 28 palau-md1 online managed 8 MD_palauVD 8.0GB 0000000000000010 DS4500 600a0b8000174233000000b9487778ab00000000000000000000000000000000 29 palau-md2 online managed 8 MD_palauVD 8.0GB 0000000000000011 DS4500 600a0b80001744310000010f48776bae00000000000000000000000000000000 30 palau-md3 online managed 8 MD_palauVD 8.0GB 0000000000000012 DS4500 600a0b8000174233000000bb487778d900000000000000000000000000000000 31 mdpalau_ivd1 online image 8 MD_palauVD 6.0GB 0000000000000013 DS4500 600a0b8000174233000000bd4877890f00000000000000000000000000000000 32 mdpalau_ivd online image 8 MD_palauVD 12.5GB 0000000000000014 DS4500 600a0b80001744310000011048777bda00000000000000000000000000000000 IBM_2145:ITSO-CLS1:admin>svcinfo lsmigrate migrate_type Migrate_to_Image progress 4 migrate_source_vdisk_index 29 migrate_target_mdisk_index 32 migrate_target_mdisk_grp 8 max_thread_count 4 migrate_source_vdisk_copy_id 0 migrate_type Migrate_to_Image progress 30 migrate_source_vdisk_index 30 migrate_target_mdisk_index 31 migrate_target_mdisk_grp 8 max_thread_count 4 migrate_source_vdisk_copy_id 0 IBM_2145:ITSO-CLS1:admin> During the migration, our Linux server will not be aware that its data is being physically moved between storage subsystems.
Chapter 14. Migration to and from the SAN Volume Controller
799
Once the migration has completed, the image mode VDisks will be ready to be removed from the Linux server, and the real LUNs can be mapped/masked directly to the host using the storage subsystems tool.
14.6.7 Remove the LUNs from the SVC The next step requires downtime on the Linux server, as we will remap/remask the disks so that the host sees them directly through the Green zone, as shown in Figure 14-63 on page 797. Our Linux server has two LUNs, one LUN being our boot disk and operating system file systems, and the other LUN holds our application and data files. Moving both LUNs at once will require the host to be shut down. If we only wanted to move the LUN that holds our application and data files, then we could do that without rebooting the host. The only requirement would be that we unmount the file system, and vary off the volume group to ensure data integrity during the reassignment. Before you start: Moving LUNs to another storage subsystem might need an additional entry in the multipath.conf file. Check with the storage subsystem vendor to see which content you have to add to the file. You might be able to install and modify it ahead of time. As we intend to move both LUNs at the same time, here are the required steps: 1. Confirm that your operating system is configured for the new storage. 2. Shut down the host. If you were just moving the LUNs that contained the application and data, then you could follow this procedure instead: a. Stop the applications that are using the LUNs. b. Unmount those file systems with the umount MOUNT_POINT command. c. If the file systems are an LVM volume, then deactivate that volume group with the vgchange -a n VOLUMEGROUP_NAME command. d. If you can, unload your HBA driver using rmmod DRIVER_MODULE. This will remove the SCSI definitions from the kernel (we will reload this module and rediscover the disks later). It is possible to tell the Linux SCSI subsystem to rescan for new disks without requiring you to unload the HBA driver; however, these details are not provided here. 3. Remove the VDisks from the host by using the svctask rmvdiskhostmap command (Example 14-22). To double-check that you have removed the VDisks, use the svcinfo lshostvdiskmap command, which should show that these disks are no longer mapped to the Linux server. Example 14-22 Remove the VDisks from the host
IBM_2145:ITSO-CLS1:admin>svctask rmvdiskhostmap -host Palau palau_SANB IBM_2145:ITSO-CLS1:admin>svctask rmvdiskhostmap -host Palau palau_Data IBM_2145:ITSO-CLS1:admin>svcinfo lshostvdiskmap Palau IBM_2145:ITSO-CLS1:admin> 4. Remove the VDisks from the SVC by using the svctask rmvdisk command. This will make them unmanaged, as seen in Example 14-23 on page 801.
800
Implementing the IBM System Storage SAN Volume Controller V4.3
Note: When you run the svctask rmvdisk command, the SVC will first double-check that there is no outstanding dirty cache data for the VDisk being removed. If there is still uncommitted cached data, then the command will fail with the following error message: CMMVC6212E The command failed because data in the cache has not been committed to disk You will have to wait for this cached data to be committed to the underlying storage subsystem before you can remove the VDisk. The SVC will automatically de-stage uncommitted cached data two minutes after the last write activity for the VDisk. How much data there is to destage, and how busy the I/O subsystem is, will determine how long this command takes to complete. You can check if the VDisk has uncommitted data in the cache by using the command svcinfo lsvdisk and checking the fast_write_state attribute. This attribute has the following meanings: empty not_empty corrupt
No modified data exists in the cache. Some modified data might exist in the cache. Some modified data might have existed in the cache, but any such data has been lost.
Example 14-23 Remove the VDisks from the SVC
IBM_2145:ITSO-CLS1:admin>svctask rmvdisk palau_SANB IBM_2145:ITSO-CLS1:admin>svctask rmvdisk palau_Data IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID 31 mdpalau_ivd1 online unmanaged 6.0GB 0000000000000013 DS4500 600a0b8000174233000000bd4877890f00000000000000000000000000000000 32 mdpalau_ivd online unmanaged 12.5GB 0000000000000014 DS4500 600a0b80001744310000011048777bda00000000000000000000000000000000 IBM_2145:ITSO-CLS1:admin> 5. Using Storage Manager (our storage subsystem management tool), unmap/unmask the disks from the SVC back to the Linux server. Attention: If one of the disks is used to boot your Linux server, than you need to make sure that it is presented back to the host as SCSI ID 0, so that the FC adapter BIOS finds it during its initialization. 6. Power on your host server and enter your FC HBA Adapter BIOS before booting the OS and make sure that you change the boot configuration so that it points to the SVC. In our example, we have performed the following steps on a QLogic HBA: a. Press Ctrl + Q to enter the HBA BIOS. b. Open Configuration Settings. c. Open Selectable Boot Settings. d. Change the entry from the SVC to your storage subsystem LUN with SCSI ID 0. Chapter 14. Migration to and from the SAN Volume Controller
801
e. Exit the menu and save your changes. Important: This is the last step that you can perform and still safely back out everything you have done so far. Up to this point, you can reverse all the actions that you have performed so far to get the server back online without data loss, that is: Remap/remask the LUNs back to the SVC. Run the svctask detectmdisk to rediscover the MDisks. Recreate the VDisks with svctask mkvdisk. Remap the VDisks back to the server with svctask mkvdiskhostmap. Once you start the next step, you might not be able to turn back without the risk of data loss. We are now ready to restart the Linux server. If all the zoning and LUN masking/mapping was done successfully, our Linux server should boot as though nothing has happened. If you only moved the application LUN to the SVC, and left your Linux server running, then you would need to follow these steps to see the new VDisk: a. Load your HBA driver with the modprobe DRIVER_NAME command. If you did not (and cannot) unload your HBA driver, then you can issue commands to the kernel to rescan the SCSI bus to see the new VDisks (these details are beyond the scope of this book). b. Check your syslog and verify that the kernel found the new VDisks. On Red Hat Enterprise Linux, the syslog is stored in /var/log/messages. c. If your application and data is on an LVM volume, run vgscan to rediscover the volume group, then run the vgchange -a y VOLUME_GROUP to activate the volume group. 7. Mount your file systems with the mount /MOUNT_POINT command (Example 14-24). The df output shows us that all disks are available again. Example 14-24 File system after migration
[root@Palau ~]# mount /dev/dm-2 /data [root@Palau ~]# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/VolGroup00-LogVol00 10093752 1938124 7634620 21% / /dev/sda1 101086 12054 83813 13% /boot tmpfs 1033496 0 1033496 0% /dev/shm /dev/dm-2 5160576 158160 4740272 4% /data [root@Palau ~]# 8. You should be ready to start your application. And finally, to make sure that the MDisks are removed from the SVC, run the svctask detectmdisk command. The MDisks will first be discovered as offline, and then will automatically be removed once the SVC determines that there are no VDisks associated with these MDisks.
802
Implementing the IBM System Storage SAN Volume Controller V4.3
14.7 Migrating ESX SAN disks to SVC disks In this section we move the two LUNs from our VMware ESX server to the SVC. The ESX operating system itself is installed locally on the host, but two SAN disks are connected. The virtual machines stored there. We then manage those LUNs with the SVC, move them between other managed disks, and then finally move them back to image mode disks, so that those LUNs can then be masked/mapped back to the VMware ESX server directly. This example should help you perform any one of the following activities in your environment: Move your ESX server’s data LUNs (that are your VMware vmfs file systems where you might have your virtual machines stored), which are directly accessed from a storage subsystem, to virtualized disks under the control of the SVC. Move LUNs between storage subsystems while your VMware virtual machines are still running. You might perform this activity to move the data onto LUNs that are more appropriate for the type of data stored on those LUNs, taking into account availability, performance and redundancy. This step is covered in 14.7.4, “Migrate the image mode VDisks” on page 813. Move your VMware ESX server’s LUNs back to image mode VDisks so that they can be remapped/remasked directly back to the server. This step starts in 14.7.5, “Preparing to migrate from the SVC” on page 816. These activities can be used individually, or together, enabling you to migrate your VMware ESX server’s LUNs from one storage subsystem to another storage subsystem, using SVC as your migration tool. If you do not use all three activities, it will enable you to introduce the SVC in your environment, or to move the data between your storage subsystems. The only downtime required for these activities will be the time it takes you to remask/remap the LUNs between the storage subsystems and your SVC.
Chapter 14. Migration to and from the SAN Volume Controller
803
In Figure 14-64, we show our starting SAN environment.
Figure 14-64 ESX environment before migration
Figure 14-64 shows our ESX server connected to the SAN infrastructure. It has two LUNs that are masked directly to it from our storage subsystem. Our ESX server represents a typical SAN environment with a host directly using LUNs created on a SAN storage subsystem, as shown in Figure 14-64: The ESX Server’s HBA cards are zoned so that they are in the Green zone with our storage subsystem. The two LUNs that have been defined on the storage subsystem, and using LUN masking, are directly available to our ESX server.
14.7.1 Connecting the SVC to your SAN fabric This section covers the basic steps to take to introduce the SVC into your SAN environment. While we only summarize these activities here, you should be able to accomplish this without any downtime to any host or application that is also using your storage area network. If you have an SVC already connected, then you can safely jump to the instructions given in 14.7.2, “Prepare your SVC to virtualize disks” on page 806.
804
Implementing the IBM System Storage SAN Volume Controller V4.3
Be very careful connecting the SVC into your storage area network, as it will require you to connect cables to your SAN switches, and alter your switch zone configuration. Doing these activities incorrectly could render your SAN inoperable, so make sure you fully understand the impact of everything you are doing. Connecting the SVC to your SAN fabric will require you to: Assemble your SVC components (nodes, UPS, and master console), cable it correctly, power it on, and verify that it is visible on your storage area network. Create and configure your SVC cluster. Create these additional zones: – An SVC node zone (the Black zone in our picture on Example 14-47 on page 828). This zone should just contain all the ports (or WWN) for each of the SVC nodes in your cluster. Our SVC is made up of a two node cluster where each node has four ports. So our Black zone has eight WWNs defined. – A storage zone (our Red zone). This zone should also have all the ports/WWN from the SVC node zone as well as the ports/WWN for all the storage subsystems that SVC will virtualize. – A host zone (our Blue zone). This zone should contain the ports/WWNs for each host that will access VDisks, together with the ports defined in the SVC node zone. Attention: Do not put your storage subsystems in the host (Blue) zone. This is an unsupported configuration and could lead to data loss!
Chapter 14. Migration to and from the SAN Volume Controller
805
Our environment has been set up as described above and can be seen in Figure 14-65. More information about the zoning recommendations are covered in 8.9, “VMware configuration information” on page 287.
Figure 14-65 SAN environment with SVC attached
14.7.2 Prepare your SVC to virtualize disks This section covers the preparatory tasks we perform before taking our ESX server or virtual machines offline. These are all nondisruptive activities, and should not affect your SAN fabric or your existing SVC configuration (if you already have a production SVC in place).
Create a managed disk group When we move the two ESX LUNs to the SVC, they will first be used in image mode, and as such, we need a managed disk group to hold those disks. First, we need to create an empty managed disk group for each of the disks, using the commands in Example 14-25. Our managed disk group will be called ESX-BOOT-MDG and ESX-DATA-MDG to hold our boot LUN and data LUN, respectively. Example 14-25 Create empty MDisk group
IBM_2145:ITSO-CLS1:admin>svctask mkmdiskgrp -name MDG_Nile_VM -ext 512 MDisk Group, id [3], successfully created
Create the host definition If your zone preparation above has been performed correctly, the SVC should be able to see the ESX server’s HBA adapters on the fabric (our host only had one HBA).
806
Implementing the IBM System Storage SAN Volume Controller V4.3
First, we get the WWN for our ESX server’s HBA, as we have many hosts connected to our SAN fabric and in the Blue zone. We want to make sure we have the correct WWN to reduce our ESX servers downtime. Log into your VMware management console as root, navigate to Configuration, and then select Storage Adapter. The Storage Adapters are shown to the right of this window and display all the necessary information. Figure 14-66 shows our WWNs, which are 210000E08B89B8C0 and 210000E08B892BCD.
Figure 14-66 Obtain your WWN using the VMware Management Console
The svcinfo lshbaportcandidate command on the SVC will list all the WWNs that the SVC can see on the SAN fabric that have not yet been allocated to a host. Example 14-26 shows the output of the nodes it found on our SAN fabric. (If the port did not show up, it would indicate that we have a zone configuration problem.) Example 14-26 Add the host to the SVC
IBM_2145:ITSO-CLS1:admin>svcinfo lshbaportcandidate id 210000E08B89B8C0 210000E08B892BCD 210000E08B0548BC 210000E08B0541BC 210000E08B89CCC2 IBM_2145:ITSO-CLS1:admin>
Chapter 14. Migration to and from the SAN Volume Controller
807
After verifying that the SVC can see our host, we create the host entry and assign the WWN to this entry. These commands can be seen in Example 14-27. Example 14-27 Create the host entry
IBM_2145:ITSO-CLS1:admin>svctask mkhost -name Nile -hbawwpn 210000E08B89B8C0:210000E08B892BCD Host, id [1], successfully created IBM_2145:ITSO-CLS1:admin>svcinfo lshost Nile id 1 name Nile port_count 2 type generic mask 1111 iogrp_count 4 WWPN 210000E08B892BCD node_logged_in_count 4 state active WWPN 210000E08B89B8C0 node_logged_in_count 4 state active IBM_2145:ITSO-CLS1:admin>
Verify that you can see your storage subsystem If our zoning has been performed correctly, the SVC should also be able to see the storage subsystem with the svcinfo lscontroller command (Example 14-28). Example 14-28 Available storage controllers
IBM_2145:ITSO-CLS1:admin>svcinfo lscontroller id controller_name ctrl_s/n product_id_low product_id_high 0 DS4500 1742-900 1 DS4700 1814 FAStT
vendor_id IBM IBM
Get your disk serial numbers To help avoid the possibility of creating the wrong VDisks from all the available unmanaged MDisks (in case there are many seen by the SVC), we get the LUN serial numbers from our storage subsystem administration tool (Storage Manager). When we discover these MDisks, we confirm that we have the right serial numbers before we create the image mode VDisks. If you are also using a DS4000 family storage subsystem, Storage Manager will provide the LUN serial numbers. Right-click your logical drive and choose Properties. Our serial numbers are shown in Figure 14-68 on page 809 and Figure 14-67 on page 809.
808
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 14-67 Obtaining the disk serial number
Figure 14-68 Obtaining the disk serial number
We are now ready to move the ownership of the disks to the SVC, discover them as MDisks, and give them back to the host as VDisks.
Chapter 14. Migration to and from the SAN Volume Controller
809
14.7.3 Move the LUNs to the SVC In this step, we move the LUNs assigned to the ESX server and reassign them to the SVC. Our ESX server has two LUNs, as shown in Figure 14-69.
Figure 14-69 VMWare LUNs
The virtual machines are located on these LUNs. So, in order to move this LUN under the control of the SVC, we do not need to reboot the whole ESX server, but we have to stop/suspend all VMware guests that are using this LUN.
Move VMware guest LUNs To move the VMware LUNs to the SVC, perform the following steps: 1. Using Storage Manager, we have identified the LUN number that has been presented to the ESX Server. Make sure to remember which LUN had which LUN number (Figure 14-70).
Figure 14-70 Identify LUN numbers in IBM DS4000 Storage Manager
2. Next, identify all the VMware guests that are using this LUN and shut them down. One way to identify them is to highlight the virtual machine and open the Summary Tab. The datapool used is displayed under Datastore. Figure 14-71 on page 811 shows a Linux virtual machine using the datastore named SLES_Costa_Rica.
810
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 14-71 Identify LUNs used by virtual machines
3. If you have several ESX hosts, also check the other ESX hosts to make sure that there is no guest operating system that is running and using this datastore. 4. Repeat steps 1 to 3 for every datastore you want to migrate. 5. Once the guests are suspended, we use Storage Manager (our storage subsystem management tool) to unmap/unmask the disks from the ESX server and remap/remask the disks to the SVC. 6. From the SVC, discover the new disks with the svctask detectmdisk command. The disks will be discovered and named as mdiskN, where N is the next available MDisk number (starting from 0). Example 14-29 shows the commands we used to discover our MDisks and verify that we have the correct ones. Example 14-29 Discover the new MDisks
IBM_2145:ITSO-CLS1:admin>svctask detectmdisk IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID 21 mdisk21 online unmanaged 60.0GB 0000000000000008 DS4700 600a0b800026b282000041ca486d14a500000000000000000000000000000000 22 mdisk22 online unmanaged 70.0GB 0000000000000009 DS4700 600a0b80002904de0000447a486d14cd00000000000000000000000000000000 IBM_2145:ITSO-CLS1:admin>
Chapter 14. Migration to and from the SAN Volume Controller
811
Important: Match your discovered MDisk serial numbers (UID on the svcinfo lsmdisk task display) with the serial number you obtained earlier (in Figure 14-67 and Figure 14-68 on page 809). 7. Once we have verified that we have the correct MDisks, we rename them to avoid confusion in the future when we perform other MDisk related tasks (Example 14-30). Example 14-30 Rename the MDisks
IBM_2145:ITSO-CLS1:admin>svctask chmdisk -name ESX_W2k3 mdisk22 IBM_2145:ITSO-CLS1:admin>svctask chmdisk -name ESX_SLES mdisk21 IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk 21 ESX_SLES online unmanaged 60.0GB 0000000000000008 DS4700 600a0b800026b282000041ca486d14a500000000000000000000000000000000 22 ESX_W2k3 online unmanaged 70.0GB 0000000000000009 DS4700 600a0b80002904de0000447a486d14cd00000000000000000000000000000000 IBM_2145:ITSO-CLS1:admin> 8. We create our image mode VDisks with the svctask mkvdisk command (Example 14-31). The parameter -vtype image makes sure that it will create image mode VDisks, which means the virtualized disks will have the exact same layout as though they were not virtualized. Example 14-31 Create the image mode VDisks
IBM_2145:ITSO-CLS1:admin>svctask mkvdisk -mdiskgrp MDG_Nile_VM -iogrp 0 -vtype image -mdisk ESX_W2k3 -name ESX_W2k3_IVD Virtual Disk, id [29], successfully created IBM_2145:ITSO-CLS1:admin>svctask mkvdisk -mdiskgrp MDG_Nile_VM -iogrp 0 -vtype image -mdisk ESX_SLES -name ESX_SLES_IVD Virtual Disk, id [30], successfully created IBM_2145:ITSO-CLS1:admin> 9. Finally, we can map the new image mode VDisks to the host. Use the same SCSI LUNs ID as on the storage subsystem for the mapping (Example 14-32). Example 14-32 Map the VDisks to the host
IBM_2145:ITSO-CLS1:admin>svctask mkvdiskhostmap -host Nile -scsi 0 ESX_SLES_IVD Virtual Disk to Host map, id [0], successfully created IBM_2145:ITSO-CLS1:admin>svctask mkvdiskhostmap -host Nile -scsi 1 ESX_W2k3_IVD Virtual Disk to Host map, id [1], successfully created IBM_2145:ITSO-CLS1:admin>svcinfo lshostvdiskmap id name SCSI_id vdisk_id vdisk_name wwpn vdisk_UID 1 Nile 0 30 ESX_SLES_IVD 210000E08B892BCD 60050768018301BF280000000000002A 1 Nile 1 29 ESX_W2k3_IVD 210000E08B892BCD 60050768018301BF2800000000000029
812
Implementing the IBM System Storage SAN Volume Controller V4.3
10.Now, using the VMware management console, rescan to discover the new VDisk. Open the configuration tab, select Storage Adapters, and click Rescan. During the rescan, you might receive geometry errors, as ESX discovers that the old disk has disappeared. Your VDisk will appear with new vmhba devices. 11.We are now ready to restart the VMware guests again. The VMware LUNs have now been successfully migrated to the SVC.
14.7.4 Migrate the image mode VDisks While the VMware server and its virtual machines are still running, we now migrate the image mode VDisks onto striped VDisks, with the extents being spread over three other MDisks.
Preparing MDisks for striped mode VDisks In this example, we migrate the image mode VDisks to VDisks and we move the data to another storage subsystem in one step.
Adding a new storage subsystem to SVC If you are moving the data to a new storage subsystem, it is assumed that this storage subsystem is connected to your SAN fabric, powered on, and visible from your SAN switches. Your environment should look similar to ours, as shown in Figure 14-72.
Figure 14-72 ESX SVC SAN environment
Make fabric zone changes The first step is to set up the SAN configuration so that all the zones are created. The new storage subsystem should be added to the Red zone so that the SVC can talk to it directly.
Chapter 14. Migration to and from the SAN Volume Controller
813
We also need a Green zone for our host to use when we are ready for it to directly access the disk, after it has been removed from the SVC. We assume that you have created the necessary zones. In our environment, we have:
Created three LUNs on another storage subsystem and mapped it to the SVC. Discovered them as MDisks. Created a new MDisk group. Renamed these LUNs to something more meaningful. Put all these MDisks into this group.
You can see the output of our commands in Example 14-33. Example 14-33 Create a new MDisk group IBM_2145:ITSO-CLS1:admin>svctask detectmdisk IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID 21 ESX_SLES online image 3 MDG_Nile_VM 60.0GB 0000000000000008 DS4700 600a0b800026b282000041ca486d14a500000000000000000000000000000000 22 ESX_W2k3 online image 3 MDG_Nile_VM 70.0GB 0000000000000009 DS4700 600a0b80002904de0000447a486d14cd00000000000000000000000000000000 23 mdisk23 online unmanaged 55.0GB 000000000000000D DS4500 600a0b8000174233000000b4486d250300000000000000000000000000000000 24 mdisk24 online unmanaged 55.0GB 000000000000000E DS4500 600a0b800017443100000108486d182c00000000000000000000000000000000 25 mdisk25 online unmanaged 55.0GB 000000000000000F DS4500 600a0b8000174233000000b5486d255b00000000000000000000000000000000 IBM_2145:ITSO-CLS1:admin>svctask mkmdiskgrp -name MDG_ESX_VD -ext 512 IBM_2145:ITSO-CLS1:admin>svctask chmdisk -name IBMESX-MD1 mdisk23 IBM_2145:ITSO-CLS1:admin>svctask chmdisk -name IBMESX-MD2 mdisk24 IBM_2145:ITSO-CLS1:admin>svctask chmdisk -name IBMESX-MD3 mdisk25 IBM_2145:ITSO-CLS1:admin>svctask addmdisk -mdisk IBMESX-MD1 MDG_ESX_VD IBM_2145:ITSO-CLS1:admin>svctask addmdisk -mdisk IBMESX-MD2 MDG_ESX_VD IBM_2145:ITSO-CLS1:admin>svctask addmdisk -mdisk IBMESX-MD3 MDG_ESX_VD IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID 21 ESX_SLES online image 3 MDG_Nile_VM 60.0GB 0000000000000008 DS4700 600a0b800026b282000041ca486d14a500000000000000000000000000000000 22 ESX_W2k3 online image 3 MDG_Nile_VM 70.0GB 0000000000000009 DS4700 600a0b80002904de0000447a486d14cd00000000000000000000000000000000 23 IBMESX-MD1 online managed 4 MDG_ESX_VD 55.0GB 000000000000000D DS4500 600a0b8000174233000000b4486d250300000000000000000000000000000000 814
Implementing the IBM System Storage SAN Volume Controller V4.3
24 IBMESX-MD2 online managed MDG_ESX_VD 55.0GB 000000000000000E DS4500 600a0b800017443100000108486d182c00000000000000000000000000000000 25 IBMESX-MD3 online managed MDG_ESX_VD 55.0GB 000000000000000F DS4500 600a0b8000174233000000b5486d255b00000000000000000000000000000000 IBM_2145:ITSO-CLS1:admin>
4
4
Migrate the VDisks We are now ready to migrate the image mode VDisks onto striped VDisks in the new managed disk group (MDG_ESX_VD) with the svctask migratevdisk command (Example 14-34). While the migration is running, our VMware ESX server will remain running, as will our VMware guests. To check the overall progress of the migration, we use the svcinfo lsmigrate command, as shown in Example 14-34. Listing the MDisk group with the svcinfo lsmdiskgrp command shows that the free capacity on the old MDisk group is slowly increasing as those extents are moved to the new MDisk group. Example 14-34 Migrating image mode VDisks to striped VDisks
IBM_2145:ITSO-CLS1:admin>svctask migratevdisk -vdisk ESX_SLES_IVD -mdiskgrp MDG_ESX_VD IBM_2145:ITSO-CLS1:admin>svctask migratevdisk -vdisk ESX_W2k3_IVD -mdiskgrp MDG_ESX_VD IBM_2145:ITSO-CLS1:admin>svcinfo lsmigrate migrate_type MDisk_Group_Migration progress 0 migrate_source_vdisk_index 30 migrate_target_mdisk_grp 4 max_thread_count 4 migrate_source_vdisk_copy_id 0 migrate_type MDisk_Group_Migration progress 0 migrate_source_vdisk_index 29 migrate_target_mdisk_grp 4 max_thread_count 4 migrate_source_vdisk_copy_id 0 IBM_2145:ITSO-CLS1:admin>svcinfo lsmigrate migrate_type MDisk_Group_Migration progress 1 migrate_source_vdisk_index 30 migrate_target_mdisk_grp 4 max_thread_count 4 migrate_source_vdisk_copy_id 0 migrate_type MDisk_Group_Migration progress 0 migrate_source_vdisk_index 29 migrate_target_mdisk_grp 4 max_thread_count 4 migrate_source_vdisk_copy_id 0 IBM_2145:ITSO-CLS1:admin>svcinfo lsmdiskgrp
Chapter 14. Migration to and from the SAN Volume Controller
815
id name capacity extent_size real_capacity overallocation 3 MDG_Nile_VM 130.0GB 512 130.00GB 100 4 MDG_ESX_VD 165.0GB 512 0.00MB 0 IBM_2145:ITSO-CLS1:admin>
status free_capacity warning online 1.0GB 0 online 35.0GB 0
mdisk_count vdisk_count virtual_capacity used_capacity 2 130.00GB
2 130.00GB
3 0.00MB
0 0.00MB
If you compare the svcinfo lsmdiskgrp output after the migration, as shown in Example 14-35, you can see that all the virtual capacity has now been moved from the old MDisk group (MDG_Nile_VM) to the new MDisk group (MDG_ESX_VD). The mdisk_count column shows that the capacity is now spread over three MDisks. Example 14-35 List MDisk group
IBM_2145:ITSO-CLS1:admin>svcinfo lsmdiskgrp id name status capacity extent_size free_capacity real_capacity overallocation warning 3 MDG_Nile_VM online 130.0GB 512 130.0GB 0.00MB 0 0 4 MDG_ESX_VD online 165.0GB 512 35.0GB 130.00GB 78 0 IBM_2145:ITSO-CLS1:admin>
mdisk_count vdisk_count virtual_capacity used_capacity 2 0.00MB
0 0.00MB
3 130.00GB
2 130.00GB
Our migration to the SVC is now complete. The original MDisks can now be removed from the SVC, and these LUNs removed from the storage subsystem. If these LUNs were the last used LUNs on our storage subsystem, then we could remove them from our SAN fabric.
14.7.5 Preparing to migrate from the SVC Before we move the ESX servers LUNs from being accessible by the SVC as virtual disks to becoming directly accessed from the storage subsystem, we need to convert the VDisks into image mode VDisks. You might want to perform this activity for any one of these reasons: You purchased a new storage subsystem, and you were using SVC as a tool to migrate from your old storage subsystem to this new storage subsystem. You used SVC to FlashCopy or Metro Mirror a VDisk onto another VDisk, and you no longer need that host connected to the SVC. You want to ship a host and its data that currently is connected to the SVC, to a site where there is no SVC. Changes to your environment no longer require this host to use the SVC.
816
Implementing the IBM System Storage SAN Volume Controller V4.3
There are also some other preparatory activities that we can do before we need to shut down the host and reconfigure the LUN masking/mapping. This section covers those activities. In our example, we will move VDisks located on a DS4500 to image mode VDisks located on a DS4700. If you are moving the data to a new storage subsystem, it is assumed that this storage subsystem is connected to your SAN fabric, powered on, and visible from your SAN switches. Your environment should look similar to ours, as described in “Adding a new storage subsystem to SVC” on page 813 and “Make fabric zone changes” on page 813.
Create new LUNs On our storage subsystem, we create two LUNs and mask the LUNs so that the SVC can see them. These two LUNs will eventually be given directly to the host, removing the VDisks that it currently has. To check that the SVC can use them, issue the svctask detectmdisk command, as shown in Example 14-36. Example 14-36 Discover the new MDisks
IBM_2145:ITSO-CLS1:admin>svctask detectmdisk IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID 23 IBMESX-MD1 online managed MDG_ESX_VD 55.0GB 000000000000000D DS4500 600a0b8000174233000000b4486d250300000000000000000000000000000000 24 IBMESX-MD2 online managed MDG_ESX_VD 55.0GB 000000000000000E DS4500 600a0b800017443100000108486d182c00000000000000000000000000000000 25 IBMESX-MD3 online managed MDG_ESX_VD 55.0GB 000000000000000F DS4500 600a0b8000174233000000b5486d255b00000000000000000000000000000000 26 mdisk26 online unmanaged 120.0GB 000000000000000A DS4700 600a0b800026b282000041f0486e210100000000000000000000000000000000 27 mdisk27 online unmanaged 100.0GB 000000000000000B DS4700 600a0b800026b282000041e3486e20cf00000000000000000000000000000000
4
4
4
Even though the MDisks will not stay in the SVC for long, we still recommend that you rename them to something more meaningful, just so that they do not get confused with other MDisks being used by other activities. Also, we create the MDisk groups to hold our new MDisks. This is all shown in Example 14-37. Example 14-37 Rename the MDisks
IBM_2145:ITSO-CLS1:admin>svctask chmdisk -name ESX_IVD_SLES mdisk26 IBM_2145:ITSO-CLS1:admin>svctask chmdisk -name ESX_IVD_W2K3 mdisk27 IBM_2145:ITSO-CLS1:admin>svctask mkmdiskgrp -name MDG_IVD_ESX -ext 512 MDisk Group, id [5], successfully created IBM_2145:ITSO-CLS1:admin>svcinfo lsmdiskgrp id name status mdisk_count vdisk_count capacity extent_size free_capacity virtual_capacity used_capacity real_capacity overallocation warning
Chapter 14. Migration to and from the SAN Volume Controller
817
4 MDG_ESX_VD online 3 165.0GB 512 35.0GB 130.00GB 130.00GB 78 0 5 MDG_IVD_ESX online 0 512 0 0.00MB 0.00MB 0 IBM_2145:ITSO-CLS1:admin>
2 130.00GB 0 0.00MB
0 0
Our SVC environment is now ready for the VDisk migration to image mode VDisks.
14.7.6 Migrate the managed VDisks to image mode VDisks While our ESX server is still running, we migrate the managed VDisks onto the new MDisks using image mode VDisks. The command to perform this action is svctask migratetoimage, and is shown in Example 14-38. Example 14-38 Migrate the VDisks to image mode VDisks
IBM_2145:ITSO-CLS1:admin>svctask migratetoimage -vdisk ESX_SLES_IVD -mdisk ESX_IVD_SLES -mdiskgrp MDG_IVD_ESX IBM_2145:ITSO-CLS1:admin>svctask migratetoimage -vdisk ESX_W2k3_IVD -mdisk ESX_IVD_W2K3 -mdiskgrp MDG_IVD_ESX IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID 23 IBMESX-MD1 online managed 4 MDG_ESX_VD 55.0GB 000000000000000D DS4500 600a0b8000174233000000b4486d250300000000000000000000000000000000 24 IBMESX-MD2 online managed 4 MDG_ESX_VD 55.0GB 000000000000000E DS4500 600a0b800017443100000108486d182c00000000000000000000000000000000 25 IBMESX-MD3 online managed 4 MDG_ESX_VD 55.0GB 000000000000000F DS4500 600a0b8000174233000000b5486d255b00000000000000000000000000000000 26 ESX_IVD_SLES online image 5 MDG_IVD_ESX 120.0GB 000000000000000A DS4700 600a0b800026b282000041f0486e210100000000000000000000000000000000 27 ESX_IVD_W2K3 online image 5 MDG_IVD_ESX 100.0GB 000000000000000B DS4700 600a0b800026b282000041e3486e20cf00000000000000000000000000000000 IBM_2145:ITSO-CLS1:admin>
During the migration, our ESX server will not be aware that its data is being physically moved between storage subsystems. We can continue to run and use the virtual machines running on the server. You can check the migration status with the command svcinfo lsmigrate, as shown in Example 14-39. Example 14-39 The svcinfo lsmigrate command and output
IBM_2145:ITSO-CLS1:admin>svcinfo lsmigrate migrate_type Migrate_to_Image progress 2 818
Implementing the IBM System Storage SAN Volume Controller V4.3
migrate_source_vdisk_index 29 migrate_target_mdisk_index 27 migrate_target_mdisk_grp 5 max_thread_count 4 migrate_source_vdisk_copy_id 0 migrate_type Migrate_to_Image progress 12 migrate_source_vdisk_index 30 migrate_target_mdisk_index 26 migrate_target_mdisk_grp 5 max_thread_count 4 migrate_source_vdisk_copy_id 0 IBM_2145:ITSO-CLS1:admin> Once the migration has completed, the image mode VDisks will be ready to be removed from the ESX server, and the real LUNs can be mapped/masked directly to the host using the storage subsystem’s tool.
14.7.7 Remove the LUNs from the SVC Your ESX server’s configuration determines in what order your LUNs are removed from the control of the SVC, and whether you need to reboot the ESX server as well as suspending the VMware guests. In our example, we have moved the virtual machine disks, so in order to remove these LUNs from the control of the SVC, we have to stop/suspend all VMware guests that are using this LUN. Perform the following steps: 1. Check which SCSI LUN IDs are assigned to the migrated disks. This can be achieved with the svcinfo lshostvdiskmap command, as shown in Example 14-40. Compare the VDisk UID and sort out the information. Example 14-40 Note SCSI LUN IDs
IBM_2145:ITSO-CLS1:admin>svcinfo lshostvdiskmap id name SCSI_id vdisk_id wwpn vdisk_UID 1 Nile 0 30 210000E08B892BCD 60050768018301BF280000000000002A 1 Nile 1 29 210000E08B892BCD 60050768018301BF2800000000000029 IBM_2145:ITSO-CLS1:admin>svcinfo lsvdisk id name IO_group_id mdisk_grp_id mdisk_grp_name capacity FC_name RC_id RC_name copy_count 0 vdisk_A 0 2 MDG_Image 36.0GB 29 ESX_W2k3_IVD 0 4 MDG_ESX_VD 70.0GB striped 60050768018301BF2800000000000029 0 30 ESX_SLES_IVD 0 4 MDG_ESX_VD 60.0GB striped 60050768018301BF280000000000002A 0
vdisk_name ESX_SLES_IVD ESX_W2k3_IVD
IO_group_name status type FC_id vdisk_UID fc_map_count io_grp0 image io_grp0
online online
1 io_grp0
online
1
Chapter 14. Migration to and from the SAN Volume Controller
819
IBM_2145:ITSO-CLS1:admin> 2. Shut down/suspend all our guests using the LUNs. You can use the same method used in “Move VMware guest LUNs” on page 810 to identify the guests using this LUN. 3. Remove the VDisks from the host by using the svctask rmvdiskhostmap command (Example 14-41). To double-check that you have removed the VDisks, use the svcinfo lshostvdiskmap command, which should show that these VDisks are no longer mapped to the ESX server. Example 14-41 Remove the VDisks from the host
IBM_2145:ITSO-CLS1:admin>svctask rmvdiskhostmap -host Nile ESX_W2k3_IVD IBM_2145:ITSO-CLS1:admin>svctask rmvdiskhostmap -host Nile ESX_SLES_IVD 4. Remove the VDisks from the SVC by using the svctask rmvdisk command. This will make the MDisks unmanaged, as shown in Example 14-42. Note: When you run the svctask rmvdisk command, the SVC will first double-check that there is no outstanding dirty cache data for the VDisk being removed. If there is still uncommitted cached data, then the command will fail with the error message: CMMVC6212E The command failed because data in the cache has not been committed to disk You have to wait for this cached data to be committed to the underlying storage subsystem before you can remove the VDisk. The SVC will automatically de-stage uncommitted cached data two minutes after the last write activity for the VDisk. Depending on how much data there is to destage, and how busy the I/O subsystem is will determine how long this command takes to complete. You can check if the VDisk has uncommitted data in the cache by using the command svcinfo lsvdisk and checking the fast_write_state attribute. This attribute has the following meanings: empty
No modified data exists in the cache.
not_empty
Some modified data might exist in the cache.
corrupt
Some modified data might have existed in the cache, but any such data has been lost.
Example 14-42 Remove the VDisks from the SVC
IBM_2145:ITSO-CLS1:admin>svctask rmvdisk ESX_W2k3_IVD IBM_2145:ITSO-CLS1:admin>svctask rmvdisk ESX_SLES_IVD IBM_2145:ITSO-CLS1:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID 26 ESX_IVD_SLES online unmanaged 120.0GB 000000000000000A DS4700 600a0b800026b282000041f0486e210100000000000000000000000000000000 27 ESX_IVD_W2K3 online unmanaged 100.0GB 000000000000000B DS4700 600a0b800026b282000041e3486e20cf00000000000000000000000000000000
820
Implementing the IBM System Storage SAN Volume Controller V4.3
IBM_2145:ITSO-CLS1:admin> 5. Using Storage Manager (our storage subsystem management tool), unmap/unmask the disks from the SVC back to the ESX server. Remember in Example 14-40 on page 819 that we have noted the SCSI LUNs ID. To map your LUNs on the storage subsystem, use the same SCSI LUN IDs as before in the SVC. Important: This is the last step that you can perform, and still safely back out of everything you have done so far. Up to this point, you can reverse all the actions that you have performed so far to get the server back online without data loss, that is: Remap/remask the LUNs back to the SVC. Run svctask detectmdisk to rediscover the MDisks. Recreate the VDisks with svctask mkvdisk. Remap the VDisks back to the server with svctask mkvdiskhostmap. Once you start the next step, you might not be able to turn back without the risk of data loss. 6. Now, using the VMware management console, rescan to discover the new VDisk. Figure 14-73 shows the view before the rescan. Figure 14-74 on page 822 shows the view after the rescan. Note that the size of the LUN has changed because we have moved to another LUN on another storage subsystem.
Figure 14-73 Before adapter rescan
Chapter 14. Migration to and from the SAN Volume Controller
821
Figure 14-74 After adapter rescan
During the rescan, you might receive geometry errors as ESX discovers that the old disk has disappeared. Your VDisk will appear with a new vmhba address, and VMware will recognize it as our VMWARE-GUESTS disk. 7. We are now ready to restart the VMware guests. 8. Finally, to make sure that the MDisks are removed from the SVC, run the svctask detectmdisk command. The MDisks will first be discovered as offline, and then will automatically be removed, once the SVC determines that there are no VDisks associated with these MDisks.
14.8 Migrating AIX SAN disks to SVC disks In this section, we move the two LUNs from an AIX server, which is directly off of our DS4000 storage subsystem, over to the SVC. We then manage those LUNs with the SVC, move them between other managed disks, and then finally move them back to image mode disks, so that those LUNs can then be masked/mapped back to the AIX server directly. If you use this example, this should help you perform any one of the following activities in your environment: Move an AIX server’s SAN LUNs from a storage subsystem and virtualize those same LUNs through the SVC. This would be the first activity that you would do when introducing the SVC into your environment. This section shows that your host downtime is only a few minutes while you remap/remask disks using your storage subsystem LUN management tool. This step starts in 14.8.2, “Prepare your SVC to virtualize disks” on page 825. Move data between storage subsystems while your AIX server is still running and servicing your business application. You might perform this activity if you were removing a storage subsystem from your SAN environment and want to move the data onto LUNs that are more appropriate for the type of data stored on those LUNs, taking into account
822
Implementing the IBM System Storage SAN Volume Controller V4.3
availability, performance, and redundancy. This step is covered in 14.8.4, “Migrate image mode VDisks to VDisks” on page 832. Move your AIX server’s LUNs back to image mode VDisks, so that they can be remapped/remasked directly back to the AIX server. This step starts in 14.8.5, “Preparing to migrate from the SVC” on page 834. These three activities can be used individually or together, enabling you to migrate your AIX server’s LUNs from one storage subsystem to another storage subsystem, using the SVC as your migration tool. If you do not use all three activities, it will enable you to introduce or remove the SVC from your environment. The only downtime required for these activities will be the time it takes you to remask/remap the LUNs between the storage subsystems and your SVC. We show our AIX environment in Figure 14-75.
Zoning for migration scenarios AIX Host
SAN
Green Zone
IBM or OEM Storage Subsystem
Figure 14-75 AIX SAN environment
Figure 14-75 shows our AIX server connected to our SAN infrastructure. It has two LUNs (hdisk3 and hdisk4) that are masked directly to it from our storage subsystem. The disk, hdisk3, makes up the LVM group itsoaixvg, and disk hdisk4 makes up the LVM group itsoaixvg1, as shown in Example 14-43 on page 824.
Chapter 14. Migration to and from the SAN Volume Controller
823
Example 14-43 AIX SAN configuration
#lsdev hdisk0 hdisk1 hdisk2 hdisk3 hdisk4 #lspv hdisk0 hdisk1 hdisk2 hdisk3 hdisk4 #
-Cc disk Available Available Available Available Available
1S-08-00-8,0 1S-08-00-9,0 1S-08-00-10,0 1D-08-02 1D-08-02
16 Bit LVD SCSI Disk Drive 16 Bit LVD SCSI Disk Drive 16 Bit LVD SCSI Disk Drive 1814 DS4700 Disk Array Device 1814 DS4700 Disk Array Device
0009cddaea97bf61 0009cdda43c9dfd5 0009cddabaef1d99 0009cdda0a4c0dd5 0009cdda0a4d1a64
rootvg rootvg rootvg itsoaixvg itsoaixvg1
active active active active active
Our AIX server represents a typical SAN environment with a host directly using LUNs created on a SAN storage subsystem, as shown in Figure 14-75 on page 823. The AIX server’s HBA cards are zoned so that they are in the Green (Dotted line) zone, with our storage subsystem. The two LUNs, hdisk3 and hdisk4, have been defined on the storage subsystem, and using LUN masking, are directly available to our AIX server.
14.8.1 Connecting the SVC to your SAN fabric This section covers the basic steps that you would take to introduce the SVC into your SAN environment. While this section only summarizes these activities, you should be able to accomplish this without any downtime to any host or application that is also using your storage area network. If you have an SVC already connected, then you can go to 14.8.2, “Prepare your SVC to virtualize disks” on page 825. Be very careful, as connecting the SVC into your storage area network will require you to connect cables to your SAN switches and alter your switch zone configuration. Doing these activities incorrectly could render your SAN inoperable, so make sure you fully understand the impact of everything you are doing. Connecting the SVC to your SAN fabric will require you to: Assemble your SVC components (nodes, UPS, and master console), cable it correctly, power it on, and verify that it is visible on your storage area network. Create and configure your SVC cluster. Create these additional zones: – An SVC node zone (our Black zone in Example 14-56 on page 834). This zone should just contain all the ports (or WWN) for each of the SVC nodes in your cluster. Our SVC is made up of a two node cluster, where each node has four ports. So our Black zone has eight WWNs defined. – A storage zone (our Red zone). This zone should also have all the ports/WWN from the SVC node zone as well as the ports/WWN for all the storage subsystems that SVC will virtualize. – A host zone (our Blue zone). This zone should contain the ports/WWNs for each host that will access the VDisk, together with the ports defined in the SVC node zone.
824
Implementing the IBM System Storage SAN Volume Controller V4.3
Attention: Do not put your storage subsystems in the host (Blue) zone. This is an unsupported configuration and could lead to data loss! Our environment has been set up as described above and can be seen in Figure 14-76.
Zoning for migration scenarios AIX Host
SAN
IBM or OEM Storage Subsystem
IBM or OEM Storage Subsystem
SVC I/O grp0 SVC SVC
Green Zone Red Zone Blue Zone Black Zone
Figure 14-76 SAN environment with SVC attached
14.8.2 Prepare your SVC to virtualize disks This section covers the preparatory tasks that we perform before taking our AIX server offline. These are all nondisruptive activities and should not affect your SAN fabric or your existing SVC configuration (if you already have a production SVC in place).
Create a managed disk group When we move the two AIX LUNs to the SVC, they will first be used in image mode, and as such we need a managed disk group to hold those disks. First, we need to create an empty managed disk group for each of the disks, using the commands in Example 14-44 on page 826. Our managed disk group to hold our LUNs will be called KANAGA_MDG_0 and KANAGA_MDG_1.
Chapter 14. Migration to and from the SAN Volume Controller
825
Example 14-44 Create empty mdiskgroup
IBM_2145:ITSO-CLS2:admin>svctask mkmdiskgrp -name aix_imgmdg -ext 512 MDisk Group, id [7], successfully created IBM_2145:ITSO-CLS2:admin>svcinfo lsmdiskgrp id name status mdisk_count vdisk_count capacity extent_size free_capacity virtual_capacity used_capacity real_capacity overallocation warning 7 aix_imgmdg online 512 0 0.00MB 0 IBM_2145:ITSO-CLS2:admin>
0 0.00MB
0 0.00MB
0 0
Create our host definition If your zone preparation above has been performed correctly, the SVC should be able to see the AIX server’s HBA adapters on the fabric (our host only had one HBA). First, we get the WWN for our AIX server’s HBA, as we have many hosts connected to our SAN fabric and in the Blue zone. We want to make sure we have the correct WWN to reduce our AIX servers’ downtime. Example 14-45 shows the commands to get the WWN; our host has a WWN of 10000000C932A7FB. Example 14-45 Discover your WWN
#lsdev -Ccadapter|grep fcs fcs0 Available 1Z-08 FC Adapter fcs1 Available 1D-08 FC Adapter #lscfg -vpl fcs0 fcs0 U0.1-P2-I4/Q1 FC Adapter Part Number.................00P4494 EC Level....................A Serial Number...............1E3120A68D Manufacturer................001E Device Specific.(CC)........2765 FRU Number.................. 00P4495 Network Address.............10000000C932A7FB ROS Level and ID............02C03951 Device Specific.(Z0)........2002606D Device Specific.(Z1)........00000000 Device Specific.(Z2)........00000000 Device Specific.(Z3)........03000909 Device Specific.(Z4)........FF401210 Device Specific.(Z5)........02C03951 Device Specific.(Z6)........06433951 Device Specific.(Z7)........07433951 Device Specific.(Z8)........20000000C932A7FB Device Specific.(Z9)........CS3.91A1 Device Specific.(ZA)........C1D3.91A1 Device Specific.(ZB)........C2D3.91A1 Device Specific.(YL)........U0.1-P2-I4/Q1
PLATFORM SPECIFIC
826
Implementing the IBM System Storage SAN Volume Controller V4.3
Name: fibre-channel Model: LP9002 Node: fibre-channel@1 Device Type: fcp Physical Location: U0.1-P2-I4/Q1 #lscfg -vpl fcs1 fcs1 U0.1-P2-I5/Q1 FC Adapter Part Number.................00P4494 EC Level....................A Serial Number...............1E3120A67B Manufacturer................001E Device Specific.(CC)........2765 FRU Number.................. 00P4495 Network Address.............10000000C932A800 ROS Level and ID............02C03891 Device Specific.(Z0)........2002606D Device Specific.(Z1)........00000000 Device Specific.(Z2)........00000000 Device Specific.(Z3)........02000909 Device Specific.(Z4)........FF401050 Device Specific.(Z5)........02C03891 Device Specific.(Z6)........06433891 Device Specific.(Z7)........07433891 Device Specific.(Z8)........20000000C932A800 Device Specific.(Z9)........CS3.82A1 Device Specific.(ZA)........C1D3.82A1 Device Specific.(ZB)........C2D3.82A1 Device Specific.(YL)........U0.1-P2-I5/Q1
PLATFORM SPECIFIC Name: fibre-channel Model: LP9000 Node: fibre-channel@1 Device Type: fcp Physical Location: U0.1-P2-I5/Q1 ## The svcinfo lshbaportcandidate command on the SVC will list all the WWNs that the SVC can see on the SAN fabric that have not yet been allocated to a host. Example 14-46 shows the output of the nodes it found in our SAN fabric. (If the port did not show up, it would indicate that we have a zone configuration problem.) Example 14-46 Add the host to the SVC
IBM_2145:ITSO-CLS2:admin>svcinfo lshbaportcandidate id 10000000C932A7FB 10000000C932A800 210000E08B89B8C0 IBM_2145:ITSO-CLS2:admin>
Chapter 14. Migration to and from the SAN Volume Controller
827
After verifying that the SVC can see our host (Kanaga), we create the host entry and assign the WWN to this entry. These commands can be seen in Example 14-47. Example 14-47 Create the host entry
IBM_2145:ITSO-CLS2:admin>svctask mkhost -name Kanaga -hbawwpn 10000000C932A7FB:10000000C932A800 Host, id [5], successfully created IBM_2145:ITSO-CLS2:admin>svcinfo lshost Kanaga id 5 name Kanaga port_count 2 type generic mask 1111 iogrp_count 4 WWPN 10000000C932A800 node_logged_in_count 2 state inactive WWPN 10000000C932A7FB node_logged_in_count 2 state inactive IBM_2145:ITSO-CLS2:admin>
Verify that we can see our storage subsystem If our zoning has been performed correctly, the SVC should also be able to see the storage subsystem with the svcinfo lscontroller command (Example 14-48). Example 14-48 Discover the storage controller
IBM_2145:ITSO-CLS2:admin>svcinfo lscontroller id controller_name ctrl_s/n vendor_id product_id_low product_id_high 0 DS4500 IBM 1742-900 1 DS4700 IBM 1814 IBM_2145:ITSO-CLS2:admin> Note: The svctask chcontroller command enables you to change the discovered storage subsystem name in SVC. In complex SANs, it might be a good idea to rename your storage subsystem to something more meaningful.
Get the disk serial numbers To help avoid the possibility of creating the wrong VDisks from all the available unmanaged MDisks (in case there are many seen by the SVC), we obtain the LUN serial numbers from our storage subsystem administration tool (Storage Manager). When we discover these MDisks, we confirm that we have the correct serial numbers before we create the image mode VDisks. If you are also using a DS4000 family storage subsystem, Storage Manager will provide the LUN serial numbers. Right-click your logical drive and choose Properties. Our serial numbers are shown in Figure 14-77 on page 829 and Figure 14-78 on page 829.
828
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure 14-77 Obtaining disk serial number
Figure 14-78 Obtaining disk serial number
We are now ready to move the ownership of the disks to the SVC, discover them as MDisks and give them back to the host as VDisks.
Chapter 14. Migration to and from the SAN Volume Controller
829
14.8.3 Move the LUNs to the SVC In this step, we move the LUNs assigned to the AIX server and reassign them to the SVC. As we only wanted to move the LUN that holds our application and data files, we can do that without rebooting the host. The only requirement would be that we unmount the file system, and vary off the volume group to ensure data integrity after the reassignment. Before you start: Moving LUNs to the SVC will require that the SDD device driver is installed on the AIX server. This could also be installed ahead of time; however, it might require an outage of your host to do so. As we intend to move both LUNs at the same time, here are the required steps: 1. Confirm that the SDD device driver is installed. 2. Unmount and vary off the volume groups: a. Stop the applications that are using the LUNs. b. Unmount those file systems with the umount MOUNT_POINT command. c. If the file systems are an LVM volume, then deactivate that volume group with the varyoffvg VOLUMEGROUP_NAME. Example 14-49 shows our commands that we ran on Kanaga. Example 14-49 AIX command sequence
#varyoffvg itsoaixvg #varyoffvg itsoaixvg1 #lsvg rootvg itsoaixvg itsoaixvg1 #lsvg -o rootvg 3. Using Storage Manager (our storage subsystem management tool), we can unmap/unmask the disks from the AIX server and remap/remask the disks to the SVC. 4. From the SVC, discover the new disks with the svctask detectmdisk command. The disks will be discovered and named mdiskN, where N is the next available mdisk number (starting from 0). Example 14-50 shows the commands we used to discover our MDisks and verify that we have the correct ones. Example 14-50 Discover the new MDisks
IBM_2145:ITSO-CLS2:admin>svctask detectmdisk IBM_2145:ITSO-CLS2:admin>svcinfo lsmdisk id name mdisk_grp_id mdisk_grp_name controller_name UID
status capacity
24 mdisk24 online unmanaged 5.0GB 0000000000000008 DS4700 600a0b800026b282000043224874f41900000000000000000000000000000000 25 mdisk25 online unmanaged 8.0GB 0000000000000009 DS4700 600a0b800026b2820000432f4874f57c00000000000000000000000000000000
mode ctrl_LUN_#
830
Implementing the IBM System Storage SAN Volume Controller V4.3
IBM_2145:ITSO-CLS2:admin> Important: Match your discovered MDisk serial numbers (UID on the svcinfo lsmdisk task display) with the serial number you discovered earlier (in Figure 14-77 and Figure 14-78 on page 829). 5. Once we have verified that we have the correct MDisks, we rename them to avoid confusion in the future when we perform other MDisk related tasks (Example 14-51). Example 14-51 Rename the MDisks
IBM_2145:ITSO-CLS2:admin>svctask chmdisk -name Kanaga_AIX mdisk24 IBM_2145:ITSO-CLS2:admin>svctask chmdisk -name Kanaga_AIX1 mdisk25 IBM_2145:ITSO-CLS2:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID 5.0GB 0000000000000008 DS4700 600a0b800026b282000043224874f41900000000000000000000000000000000 25 Kanaga_AIX1 online unmanaged 8.0GB 0000000000000009 DS4700 600a0b800026b2820000432f4874f57c00000000000000000000000000000000 IBM_2145:ITSO-CLS2:admin> 6. We create our image mode VDisks with the svctask mkvdisk command and the option -vtype image (Example 14-52). This command will virtualize the disks in the exact same layout as though they were not virtualized. Example 14-52 Create the image mode VDisks
IBM_2145:ITSO-CLS2:admin>svctask mkvdisk -mdiskgrp aix_imgmdg -iogrp 0 -vtype image -mdisk Kanaga_AIX -name IVD_Kanaga Virtual Disk, id [8], successfully created IBM_2145:ITSO-CLS2:admin>svctask mkvdisk -mdiskgrp aix_imgmdg -iogrp 0 -vtype image -mdisk Kanaga_AIX1 -name IVD_Kanaga1 Virtual Disk, id [9], successfully created IBM_2145:ITSO-CLS2:admin> 7. Finally, we can map the new image mode VDisks to the host (Example 14-53). Example 14-53 Map the VDisks to the host
IBM_2145:ITSO-CLS2:admin>svctask mkvdiskhostmap -host Kanaga IVD_Kanaga Virtual Disk to Host map, id [0], successfully created IBM_2145:ITSO-CLS2:admin>svctask mkvdiskhostmap -host Kanaga IVD_Kanaga1 Virtual Disk to Host map, id [1], successfully created IBM_2145:ITSO-CLS2:admin> Note: While the application is in a quiescent state, you could choose to FlashCopy the new image VDisks onto other VDisks. You do not need to wait until the FlashCopy has completed before starting your application. Now we are ready to perform the following steps to put the image mode VDisks online: 1. Remove the old disk definitions, if you have not done so already. 2. Run cfgmgr -vs to rediscover the available LUNs. Chapter 14. Migration to and from the SAN Volume Controller
831
3. If your application and data is on an LVM volume, rediscover the volume group, then run the varyonvg VOLUME_GROUP to activate the volume group. 4. Mount your file systems with the mount /MOUNT_POINT command. 5. You should be ready to start your application.
14.8.4 Migrate image mode VDisks to VDisks While the AIX server still running, and our file systems are in use, we now migrate the image mode VDisks onto striped VDisks, with the extents being spread over three other MDisks.
Preparing MDisks for stripped mode VDisks From our storage subsystem, we have:
Created and allocated three LUNs to the SVC. Discovered them as MDisks. Renamed these LUNs to something more meaningful. Created a new MDisk group. Put all these MDisks into this group.
You can see the output of our commands in Example 14-54. Example 14-54 Create a new MDisk group
IBM_2145:ITSO-CLS2:admin>svctask mkmdiskgrp -name aix_vd -ext 512 IBM_2145:ITSO-CLS2:admin>svctask detectmdisk IBM_2145:ITSO-CLS2:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID 24 Kanaga_AIX online image 7 aix_imgmdg 5.0GB 0000000000000008 DS4700 600a0b800026b282000043224874f41900000000000000000000000000000000 25 Kanaga_AIX1 online image 7 aix_imgmdg 8.0GB 0000000000000009 DS4700 600a0b800026b2820000432f4874f57c00000000000000000000000000000000 26 mdisk26 online unmanaged 6.0GB 000000000000000A DS4700 600a0b800026b2820000439c48751ddc00000000000000000000000000000000 27 mdisk27 online unmanaged 6.0GB 000000000000000B DS4700 600a0b800026b2820000438448751da900000000000000000000000000000000 28 mdisk28 online unmanaged 6.0GB 000000000000000C DS4700 600a0b800026b2820000439048751dc200000000000000000000000000000000 IBM_2145:ITSO-CLS2:admin>svctask chmdisk -name aix_vd0 mdisk26 IBM_2145:ITSO-CLS2:admin>svctask chmdisk -name aix_vd1 mdisk27 IBM_2145:ITSO-CLS2:admin>svctask chmdisk -name aix_vd2 mdisk28 IBM_2145:ITSO-CLS2:admin> IBM_2145:ITSO-CLS2:admin>svctask addmdisk -mdisk aix_vd0 aix_vd IBM_2145:ITSO-CLS2:admin>svctask addmdisk -mdisk aix_vd1 aix_vd IBM_2145:ITSO-CLS2:admin>svctask addmdisk -mdisk aix_vd2 aix_vd IBM_2145:ITSO-CLS2:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID
832
Implementing the IBM System Storage SAN Volume Controller V4.3
24 Kanaga_AIX online image aix_imgmdg 5.0GB 0000000000000008 DS4700 600a0b800026b282000043224874f41900000000000000000000000000000000 25 Kanaga_AIX1 online image aix_imgmdg 8.0GB 0000000000000009 DS4700 600a0b800026b2820000432f4874f57c00000000000000000000000000000000 26 aix_vd0 online managed aix_vd 6.0GB 000000000000000A DS4700 600a0b800026b2820000439c48751ddc00000000000000000000000000000000 27 aix_vd1 online managed aix_vd 6.0GB 000000000000000B DS4700 600a0b800026b2820000438448751da900000000000000000000000000000000 28 aix_vd2 online managed aix_vd 6.0GB 000000000000000C DS4700 600a0b800026b2820000439048751dc200000000000000000000000000000000 IBM_2145:ITSO-CLS2:admin>
7
7
6
6
6
Migrate the VDisks We are now ready to migrate the image mode VDisks onto striped VDisks with the svctask migratevdisk command (Example 14-17 on page 795). While the migration is running, our AIX server is still running, and we can continue accessing the files. To check the overall progress of the migration, we use the svcinfo lsmigrate command, as shown in Example 14-55. Listing the MDisk group with the svcinfo lsmdiskgrp command shows that the that the free capacity on the old MDisk group is slowly increasing as those extents are moved to the new MDisk group. Example 14-55 Migrating image mode VDisks to striped VDisks
IBM_2145:ITSO-CLS2:admin>svctask migratevdisk -vdisk IVD_Kanaga -mdiskgrp aix_vd IBM_2145:ITSO-CLS2:admin>svctask migratevdisk -vdisk IVD_Kanaga1 -mdiskgrp aix_vd IBM_2145:ITSO-CLS2:admin>svcinfo lsmigrate migrate_type MDisk_Group_Migration progress 10 migrate_source_vdisk_index 8 migrate_target_mdisk_grp 6 max_thread_count 4 migrate_source_vdisk_copy_id 0 migrate_type MDisk_Group_Migration progress 0 migrate_source_vdisk_index 9 migrate_target_mdisk_grp 6 max_thread_count 4 migrate_source_vdisk_copy_id 0 IBM_2145:ITSO-CLS2:admin>
Chapter 14. Migration to and from the SAN Volume Controller
833
Once this task has completed, Example 14-56 shows that the VDisks are now spread over three MDisks in the managed disk group aix_vd. The old mdiskgrp is empty now. Example 14-56 Migration complete
IBM_2145:ITSO-CLS2:admin>svcinfo lsmdiskgrp aix_vd id 6 name aix_vd status online mdisk_count 3 vdisk_count 2 capacity 18.0GB extent_size 512 free_capacity 5.0GB virtual_capacity 13.00GB used_capacity 13.00GB real_capacity 13.00GB overallocation 72 warning 0 IBM_2145:ITSO-CLS2:admin>svcinfo lsmdiskgrp aix_imgmdg id 7 name aix_imgmdg status online mdisk_count 2 vdisk_count 0 capacity 13.0GB extent_size 512 free_capacity 13.0GB virtual_capacity 0.00MB used_capacity 0.00MB real_capacity 0.00MB overallocation 0 warning 0 IBM_2145:ITSO-CLS2:admin> Our migration to the SVC is now complete. The original MDisks can now be removed from the SVC, and these LUNs removed from the storage subsystem. If these LUNs were the last used LUNs on our storage subsystem, then we could remove it from our SAN fabric.
14.8.5 Preparing to migrate from the SVC Before we change the AIX servers’ LUNs from being accessed by the SVC as virtual disks to being directly accessed from the storage subsystem, we need to convert the VDisks into image mode VDisks. You might want to perform this activity for any one of these reasons: You purchased a new storage subsystem, and you were using the SVC as a tool to migrate from your old storage subsystem to this new storage subsystem. You used the SVC to FlashCopy or Metro Mirror a VDisk onto another VDisk and you no longer need that host connected to the SVC. You want to ship a host and its data that is currently connected to the SVC to a site where there is no SVC.
834
Implementing the IBM System Storage SAN Volume Controller V4.3
Changes to your environment no longer require this host to use the SVC. There are also some other preparatory activities that we can do before we need to shut down the host and reconfigure the LUN masking/mapping. This section covers those activities. If you are moving the data to a new storage subsystem, it is assumed that this storage subsystem is connected to your SAN fabric, powered on, and visible from your SAN switches. Your environment should look similar to ours, as shown in Figure 14-79.
Zoning for migration scenarios AIX Host
SAN
IBM or OEM Storage Subsystem
IBM or OEM Storage Subsystem
SVC I/O grp0 SVC SVC
Green Zone Red Zone Blue Zone Black Zone
Figure 14-79 Environment with SVC
Make fabric zone changes The first step is to set up the SAN configuration so that all the zones are created. The new storage subsystem should be added to the Red zone so that the SVC can talk to it directly. We also need a Green zone for our host to use when we are ready for it to directly access the disk, after it has been removed from the SVC. It is assumed that you have created the necessary zones. Once your zone configuration is set up correctly, the SVC should see the new storage subsystems controller using the svcinfo lscontroller command, as shown in Example 14-57 on page 836. It is also a good idea to rename it to something more meaningful. This can be performed with the svctask chcontroller -name command.
Chapter 14. Migration to and from the SAN Volume Controller
835
Example 14-57 Discovering the new storage subsystem
IBM_2145:ITSO-CLS2:admin>svcinfo lscontroller id controller_name ctrl_s/n vendor_id product_id_low product_id_high 0 DS4500 IBM 1742-900 1 DS4700 IBM 1814 FAStT IBM_2145:ITSO-CLS2:admin>
Create new LUNs On our storage subsystem, we created two LUNs and masked them so that the SVC can see them. These two LUNs will eventually be given directly to the host, removing the VDisks that it currently has. To check that the SVC can use them, issue the svctask detectmdisk command, as shown in Example 14-58. In our example, we will use the two 10 GB LUNs located on the DS4500 subsystem, so in this step we migrate back to image mode VDisks and to another subsystem in one step. We have already deleted the old LUNs on the DS4700 storage subsystem, which is the reason why they appear offline here. Example 14-58 Discover the new MDisks
IBM_2145:ITSO-CLS2:admin>svctask detectmdisk IBM_2145:ITSO-CLS2:admin>svcinfo lsmdiks CMMVC5987E lsmdiks is not a valid command line option. IBM_2145:ITSO-CLS2:admin>svcinfo lsmdik CMMVC5987E lsmdik is not a valid command line option. IBM_2145:ITSO-CLS2:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID 24 Kanaga_AIX offline managed 7 aix_imgmdg 5.0GB 0000000000000008 DS4700 600a0b800026b282000043224874f41900000000000000000000000000000000 25 Kanaga_AIX1 offline managed 7 aix_imgmdg 8.0GB 0000000000000009 DS4700 600a0b800026b2820000432f4874f57c00000000000000000000000000000000 26 aix_vd0 online managed 6 aix_vd 6.0GB 000000000000000A DS4700 600a0b800026b2820000439c48751ddc00000000000000000000000000000000 27 aix_vd1 online managed 6 aix_vd 6.0GB 000000000000000B DS4700 600a0b800026b2820000438448751da900000000000000000000000000000000 28 aix_vd2 online managed 6 aix_vd 6.0GB 000000000000000C DS4700 600a0b800026b2820000439048751dc200000000000000000000000000000000 29 mdisk29 online unmanaged 10.0GB 0000000000000010 DS4500 600a0b8000174233000000b84876512f00000000000000000000000000000000 30 mdisk30 online unmanaged 10.0GB 0000000000000011 DS4500 600a0b80001744310000010e4876444600000000000000000000000000000000 IBM_2145:ITSO-CLS2:admin> Even though the MDisks will not stay in the SVC for long, we still recommend that you rename them to something more meaningful just so that they do not get confused with other MDisks being used by other activities. Also, we create the MDisk groups to hold our new MDisks. This is shown in Example 14-59 on page 837.
836
Implementing the IBM System Storage SAN Volume Controller V4.3
Example 14-59 Rename the MDisks
IBM_2145:ITSO-CLS2:admin>svctask chmdisk -name AIX_MIG mdisk29 IBM_2145:ITSO-CLS2:admin>svctask chmdisk -name AIX_MIG1 mdisk30 IBM_2145:ITSO-CLS2:admin>svctask mkmdiskgrp -name KANAGA_AIXMIG -ext 512 MDisk Group, id [3], successfully created IBM_2145:ITSO-CLS2:admin>svcinfo lsmdiskgrp id name status mdisk_count vdisk_count capacity extent_size free_capacity virtual_capacity used_capacity real_capacity overallocation warning 3 KANAGA_AIXMIG online 0 0 0 512 0 0.00MB 0.00MB 0.00MB 0 0 6 aix_vd online 3 2 18.0GB 512 5.0GB 13.00GB 13.00GB 13.00GB 72 0 7 aix_imgmdg offline 2 0 13.0GB 512 13.0GB 0.00MB 0.00MB 0.00MB 0 0 IBM_2145:ITSO-CLS2:admin>
Our SVC environment is now ready for the VDisk migration to image mode VDisks.
14.8.6 Migrate the managed VDisks While our AIX server is still running, we migrate the managed VDisks onto the new MDisks using image mode VDisks. The command to perform this action is svctask migratetoimage and is shown in Example 14-60. Example 14-60 Migrate the VDisks to image mode VDisks
IBM_2145:ITSO-CLS2:admin>svctask migratetoimage -vdisk IVD_Kanaga -mdisk AIX_MIG -mdiskgrp KANAGA_AIXMIG IBM_2145:ITSO-CLS2:admin>svctask migratetoimage -vdisk IVD_Kanaga1 -mdisk AIX_MIG1 -mdiskgrp KANAGA_AIXMIG IBM_2145:ITSO-CLS2:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID 24 Kanaga_AIX offline managed 7 aix_imgmdg 5.0GB 0000000000000008 DS4700 600a0b800026b282000043224874f41900000000000000000000000000000000 25 Kanaga_AIX1 offline managed 7 aix_imgmdg 8.0GB 0000000000000009 DS4700 600a0b800026b2820000432f4874f57c00000000000000000000000000000000 26 aix_vd0 online managed 6 aix_vd 6.0GB 000000000000000A DS4700 600a0b800026b2820000439c48751ddc00000000000000000000000000000000 27 aix_vd1 online managed 6 aix_vd 6.0GB 000000000000000B DS4700 600a0b800026b2820000438448751da900000000000000000000000000000000 28 aix_vd2 online managed 6 aix_vd 6.0GB 000000000000000C DS4700 600a0b800026b2820000439048751dc200000000000000000000000000000000
Chapter 14. Migration to and from the SAN Volume Controller
837
29 AIX_MIG online image KANAGA_AIXMIG 10.0GB 0000000000000010 DS4500 600a0b8000174233000000b84876512f00000000000000000000000000000000 30 AIX_MIG1 online image KANAGA_AIXMIG 10.0GB 0000000000000011 DS4500 600a0b80001744310000010e4876444600000000000000000000000000000000 IBM_2145:ITSO-CLS2:admin>svcinfo lsmigrate migrate_type Migrate_to_Image progress 50 migrate_source_vdisk_index 9 migrate_target_mdisk_index 30 migrate_target_mdisk_grp 3 max_thread_count 4 migrate_source_vdisk_copy_id 0 migrate_type Migrate_to_Image progress 50 migrate_source_vdisk_index 8 migrate_target_mdisk_index 29 migrate_target_mdisk_grp 3 max_thread_count 4 migrate_source_vdisk_copy_id 0 IBM_2145:ITSO-CLS2:admin>
3
3
During the migration, our AIX server will not be aware that its data is being physically moved between storage subsystems. Once the migration has completed, the image mode VDisks will be ready to be removed from the AIX server, and the real LUNs can be mapped/masked directly to the host using the storage subsystems tool.
14.8.7 Remove the LUNs from the SVC The next step will require downtime, as we remap/remask the disks so that the host sees them directly through the Green zone. As our LUNs only hold data files, and a unique volume group is used, we could do that without rebooting the host. The only requirement is that we unmount the file system and vary off the volume group to ensure data integrity after the reassignment. Before you start: Moving LUNs to another storage system might need a different driver than SDD. Check with the storage subsystems vendor to see what driver you will need. You might be able to install this driver ahead of time. Here are the required steps to remove the SVC: 1. Confirm that the correct device driver for the new storage subsystem is loaded. As we are moving to a DS4500, we can continue to use the SDD device driver. 2. Shut down any applications and unmount the file systems: a. Stop the applications that are using the LUNs. b. Unmount those file systems with the umount MOUNT_POINT command. c. If the file systems are an LVM volume, then deactivate that volume group with the varyoffvg VOLUMEGROUP_NAME command.
838
Implementing the IBM System Storage SAN Volume Controller V4.3
3. Remove the VDisks from the host by using the svctask rmvdiskhostmap command (Example 14-61). To double-check that you have removed the VDisks, use the svcinfo lshostvdiskmap command, which should show that these disks are no longer mapped to the AIX server. Example 14-61 Remove the VDisks from the host
IBM_2145:ITSO-CLS2:admin>svctask rmvdiskhostmap -host Kanaga IVD_Kanaga IBM_2145:ITSO-CLS2:admin>svctask rmvdiskhostmap -host Kanaga IVD_Kanaga1 IBM_2145:ITSO-CLS2:admin>svcinfo lshostvdiskmap Kanaga IBM_2145:ITSO-CLS2:admin> 4. Remove the VDisks from the SVC by using the svctask rmvdisk command. This will make the MDisks unmanaged, as shown in Example 14-62. Note: When you run the svctask rmvdisk command, the SVC will first double-check that there is no outstanding dirty cache data for the VDisk being removed. If there is still uncommitted cached data, then the command will fail with the following error message: CMMVC6212E The command failed because data in the cache has not been committed to disk You will have to wait for this cached data to be committed to the underlying storage subsystem before you can remove the VDisk. The SVC will automatically destage uncommitted cached data two minutes after the last write activity for the VDisk. How much data there is to destage and how busy the I/O subsystem is will determine how long this command takes to complete. You can check if the VDisk has uncommitted data in the cache by using the command svcinfo lsvdisk and checking the fast_write_state attribute. This attribute has the following meanings: empty
No modified data exists in the cache.
not_empty
Some modified data might exist in the cache.
corrupt
Some modified data might have existed in the cache, but any such data has been lost.
Example 14-62 Remove the VDisks from the SVC
IBM_2145:ITSO-CLS2:admin>svctask rmvdisk IVD_Kanaga IBM_2145:ITSO-CLS2:admin>svctask rmvdisk IVD_Kanaga1 IBM_2145:ITSO-CLS2:admin>svcinfo lsmdisk id name status mode mdisk_grp_id mdisk_grp_name capacity ctrl_LUN_# controller_name UID 29 AIX_MIG online unmanaged 10.0GB 0000000000000010 DS4500 600a0b8000174233000000b84876512f00000000000000000000000000000000 30 AIX_MIG1 online unmanaged 10.0GB 0000000000000011 DS4500 600a0b80001744310000010e4876444600000000000000000000000000000000 IBM_2145:ITSO-CLS2:admin>
Chapter 14. Migration to and from the SAN Volume Controller
839
5. Using Storage Manager (our storage subsystem management tool), unmap/unmask the disks from the SVC back to the AIX server. Important: This is the last step that you can perform and still safely back out of everything you have done so far. Up to this point, you can reverse all the actions that you have performed so far to get the server back online without data loss: Remap/remask the LUNs back to the SVC. Run the svctask detectmdisk command to rediscover the MDisks. Recreate the VDisks with svctask mkvdisk. Remap the VDisks back to the server with svctask mkvdiskhostmap. Once you start the next step, you might not be able to turn back without the risk of data loss. We are now ready to access the LUNs from the AIX server. If all the zoning and LUN masking/mapping was done successfully, our AIX server should boot as though nothing happened. 1. Run cfgmgr -S to discover the storage subsystem. 2. Use lsdev -Ccdisk to verify that you discovered your new disk. 3. Remove the references to all the old disks. Example 14-63 shows the removal using SDD and Example 14-64 on page 841 shows the removal using SDDPCM. Example 14-63 Remove references to old paths using SDD
#lsdev -Cc disk hdisk0 Available 1S-08-00-8,0 16 Bit LVD SCSI Disk Drive hdisk1 Available 1S-08-00-9,0 16 Bit LVD SCSI Disk Drive hdisk2 Available 1S-08-00-10,0 16 Bit LVD SCSI Disk Drive hdisk3 Available 1Z-08-02 1742-900 (900) Disk Array Device hdisk4 Available 1Z-08-02 1742-900 (900) Disk Array Device hdisk5 Defined 1Z-08-02 SAN Volume Controller Device hdisk6 Defined 1Z-08-02 SAN Volume Controller Device hdisk7 Defined 1D-08-02 SAN Volume Controller Device hdisk8 Defined 1D-08-02 SAN Volume Controller Device hdisk10 Defined 1Z-08-02 SAN Volume Controller Device hdisk11 Defined 1Z-08-02 SAN Volume Controller Device hdisk12 Defined 1D-08-02 SAN Volume Controller Device hdisk13 Defined 1D-08-02 SAN Volume Controller Device vpath0 Defined Data Path Optimizer Pseudo Device Driver vpath1 Defined Data Path Optimizer Pseudo Device Driver vpath2 Defined Data Path Optimizer Pseudo Device Driver # for i in 5 6 7 8 10 11 12 13; do rmdev -dl hdisk$i -R;done hdisk5 deleted hdisk6 deleted hdisk7 deleted hdisk8 deleted hdisk10 deleted hdisk11 deleted hdisk12 deleted hdisk13 deleted #for i in 0 1 2; do rmdev -dl vpath$i -R;done 840
Implementing the IBM System Storage SAN Volume Controller V4.3
vpath0 vpath1 vpath2 #lsdev hdisk0 hdisk1 hdisk2 hdisk3 hdisk4 #
deleted deleted deleted -Cc disk Available Available Available Available Available
1S-08-00-8,0 1S-08-00-9,0 1S-08-00-10,0 1Z-08-02 1Z-08-02
16 Bit LVD SCSI Disk Drive 16 Bit LVD SCSI Disk Drive 16 Bit LVD SCSI Disk Drive 1742-900 (900) Disk Array Device 1742-900 (900) Disk Array Device
Example 14-64 Remove references to old paths using SDDPCM
# lsdev -Cc disk hdisk0 Available 1S-08-00-8,0 16 Bit LVD SCSI hdisk1 Available 1S-08-00-9,0 16 Bit LVD SCSI hdisk2 Available 1S-08-00-10,0 16 Bit LVD SCSI hdisk3 Defined 1D-08-02 MPIO FC 2145 hdisk4 Defined 1D-08-02 MPIO FC 2145 hdisk5 Available 1D-08-02 MPIO FC 2145 # for i in 3 4; do rmdev -dl hdisk$i -R;done hdisk3 deleted hdisk4 deleted # lsdev -Cc disk hdisk0 Available 1S-08-00-8,0 16 Bit LVD SCSI hdisk1 Available 1S-08-00-9,0 16 Bit LVD SCSI hdisk2 Available 1S-08-00-10,0 16 Bit LVD SCSI hdisk5 Available 1D-08-02 MPIO FC 2145
Disk Drive Disk Drive Disk Drive
Disk Drive Disk Drive Disk Drive
4. If your application and data is on an LVM volume, rediscover the volume group, then run the varyonvg VOLUME_GROUP to activate the volume group. 5. Mount your file systems with the mount /MOUNT_POINT command. 6. You should be ready to start your application. Finally, to make sure that the MDisks are removed from the SVC, run the svctask detectmdisk command. The MDisks will first be discovered as offline, and then will automatically be removed once the SVC determines that there are no VDisks associated with these MDisks.
14.9 Using SVC for storage migration The primary use of the SVC is not as a storage migration tool. However, the advanced capabilities of the SVC enable us to use the SVC as a “storage migration tool”. This means that the SVC will be temporarily added to your SAN environment to copy the data from one storage subsystem to the other. The SVC enables you to copy image mode VDisks directly from one subsystem to the other while host I/O is running. The only downtime required is when SVC is added and removed to your SAN environment. To use SVC for migration purposes only, perform the following steps: 1. Add the SVC to your SAN environment. 2. Prepare the SVC.
Chapter 14. Migration to and from the SAN Volume Controller
841
3. Depending on your operating system, unmount the selected LUNs or shut down the host. 4. Add the SVC between your storage and the host. 5. Mount the LUNs or start the host again. 6. Start the migration. 7. After the migration process is complete, unmount the selected LUNs or shut down the host. 8. Remove the SVC from your SAN. 9. Mount the LUNs or start the host again. 10.The migration is complete. As you can see, very little downtime is required. If you prepare everything correctly, you are able to reduce your downtime to a few minutes. The copy process is handled by the SVC, so the host does not hinder performance while the migration progresses. To use the SVC for storage migrations, only perform the steps described in the following sections: 1. 14.5.1, “SVC added between the host system and the DS4700” on page 758 2. 14.5.5, “Migrating the VDisk from image mode to image mode” on page 775 3. 14.5.6, “Free the data from the SVC” on page 779
842
Implementing the IBM System Storage SAN Volume Controller V4.3
A
Appendix A.
Copy Services and open systems In this appendix, we describe the basic tasks that you need to perform on the individual host systems when using IBM System Storage SAN Volume Controller (SVC) Copy Services. We explain how to bring FlashCopy target vpaths online on the same host as well as on another host. This appendix covers AIX and Windows.
© Copyright IBM Corp. 2003-2008. All rights reserved.
843
AIX specifics In this section, we show how to use SVC FlashCopy, Metro Mirror, and Global Mirror features in an AIX environment.
AIX and FlashCopy The FlashCopy functionality for SVC makes the entire point in time contents of a source VDisk available to one or more target VDisks. If the source VDisk is defined to the AIX Logical Volume Manager (LVM), all of its data structures and identifiers are copied to the target VDisk as well. This includes the Volume Group Descriptor Area (VGDA), which contains the physical volume identifier (PVID) and volume group identifier (VGID). For AIX LVM, it is currently not possible to activate a volume group with a disk that contains a VGID and a PVID that is already used in an active volume group on the same server, even if the PVID is cleared and reassigned with the following two commands: chdev -l -a pv=clear chdev -l -a pv=yes Therefore, it is necessary to redefine the volume group information on the FlashCopy target using the recreatevg command. Refer to “AIX recreatevg command” on page 847 for further details. This alters the VGID of the FlashCopy target volume so that there are no conflicts with existing VGIDs on active volume groups. If you do not redefine the volume group information prior to importing the volume group, then the importvg command fails.
Accessing FlashCopy target on another AIX host Accessing a FlashCopy target on another AIX host is recommended. Important: When using FlashCopy, make sure to FlashCopy all VDisks comprising the volume group, and all at the same time. This can be done by using FlashCopy consistency groups. The following procedure makes the data of the FlashCopy target VDisks available to another AIX system that has no prior definitions in its ODM: 1. The target VDisk is new to AIX. Therefore, Configuration Manager should be run on all Fibre Channel adapters: cfgmgr -l 2. When using Subsystem Device Driver (SDD), the cfallvpath command should be run to discover all new vpath devices: cfallvpath Note: If you just execute cfgmgr or cfgmgr -vS, the SDD will discover the new disks and make the vpaths automatically. Sometimes the cfgmgr command can take a while and will not affect the server configuration at all, therefore the cfgmgr -l command can be the fastest way to discover the new disks. 3. Check which new MPIO volume or vpath is on the host; this is your FlashCopy target: lsdev -c disk | grep vpath (For vpath devices) lsdev -c disk | grep 2145 (For mpio devices)
844
Implementing the IBM System Storage SAN Volume Controller V4.3
4. Verify that all the paths are working fine by using the following commands: datapath query device (For vpath devices) pcmpath query device (For MPIO devices) 5. Import the volume group: importvg -y 6. The importvg command should vary on the volume group; if not, use the varyonvg command: varyonvg 7. Verify the access to the volume group using: lqueryvg -Atp 8. To list all file systems in the volume group, run: lsvg -l 9. Verify consistency of all file systems on the FlashCopy target: fsck -y 10.Mount all the target file systems: mount The data is now available. You could now perform a tape backup on this FlashCopy image. You can also use the data for other purposes, such as testing. This procedure can be run as soon as the relationship between the FlashCopy source and the target is established, even though data is still being copied from the source to the target in the background. It might be the case that the disks containing the target VDisks were previously defined to the target AIX system. For example, if you periodically do backups using the same VDisks, then in this case, perform the following actions on the target system before recreating the new image: 1. Unmount all file systems in the target volume group: umount 2. Vary off the volume group: varyoffvg 3. Export the volume group: exportvg 4. Delete the hdisks and vpaths or MPIO devices using the command rmdev: rmdev -Rdl dpo rmdev -dl Tip: rmdev -Rdl dpo will delete all the vpath devices. If there are too many hdisks, and you do not want to delete them one by one, you can also use rmdev -Rdl fcsx commands to delete all HBA devices and all child devices.
Appendix A. Copy Services and open systems
845
5. Perform FlashCopy with the SVC management GUI or SSH client (for example, OpenSSL). You can also integrate the SVC command into a host script by installing OpenSSL, which has the correct private key in the .ssh directory: ssh -l admin -i keyfile_name svc svctask mkfcmap -source VDisk_name -target VDisk_name -name FlashCopy_map_name -consistgrp FlashCopy_consistgroup_name -copyrate copyrate 6. The target VDisk is new to AIX. Therefore, Configuration Manager should be run on all Fibre Channel adapters: cfgmgr -l 7. When using the Subsystem Device Driver (SDD), the cfallvpath command should be run to discover new vpath devices: cfallvpath 8. Check which new MPIO volume or vpath is on the host; this is your FlashCopy target: lsdev -c disk | grep vpath (For vpath devices) lsdev -c disk | grep 2145 (For mpio devices) 9. Verify that all the paths are working fine by running the following commands: datapath query device (For vpath device) pcmpath query device (For MPIO devices) 10.Import the volume group: importvg -y 11.The importvg command should vary on the volume group; if not, use the varyonvg command: varyonvg 12.Verify the access to the volume group using the following command: lqueryvg -Atp 13.To list all file systems in a volume group, run: lsvg -l 14.Verify consistency of all file systems on the FlashCopy target using the following command: fsck -y 15.Mount all the target file systems using the following command: mount The data is now available as a point in time recreation of the image. You could now perform a tape backup on this FlashCopy image or also use the data for other purposes, such as testing.
Accessing FlashCopy source and target on the same AIX host This section describes a method to access the FlashCopy target VDisk on the same AIX host as the source VDisk. The procedure is intended to be used as a guide and might not cover all possible scenarios.
846
Implementing the IBM System Storage SAN Volume Controller V4.3
AIX recreatevg command The recreatevg command is packaged as a PTF for AIX V4.3.3 in APAR IY10456 and higher. It is officially available in: AIX V4.3.3 Recommended Maintenance Level 05 (RML05) or higher AIX 5L or any higher release The recreatevg command overcomes the problem of duplicated LVM data structures and identifiers that have resulted due to a disk copying process, such as FlashCopy, Metro Mirror, and Global Mirror. It is used to recreate an AIX volume group (VG) on a set of disks that are copied from another set of disks belonging to a specific volume group. The command allocates new PVIDs for the member disks and a new VGID to the volume group. The command also provides options to rename the volume group, the logical volumes with a prefix you specify, and options to rename “labels” to specify different mount points for file systems. Here is the AIX man page synopsis: recreatevg [-y VGname] [-p] [-f] [-Y lv_prefix | -l LvNameFile] [-L label_prefix] [-n] \ PVname... You can use this command to recreate a volume group on a set of disks that are mirrored from another set of disks belonging to a specific volume group. This command allocates new PVID for the member disks since the PVIDs are also duplicated by the disk mirroring. Similarly, other LVM logical members that are duplicated are also changed to new names with the specified prefixes. Here we describe the following flags for the recreatevg command: -y VGname specifies the volume group name rather than having the name generated automatically. Volume group names must be unique system wide and can range from one to 15 characters. The name cannot begin with a prefix already defined in the PdDv class in the device configuration database for other devices. The volume group name that is created is sent to standard output. -p disables the automatic generation of the new PVIDs. If a -p flag is used, you must ensure that there are no duplicated PVIDs on the system. All the disks that were hardware mirrored must have had their PVIDs changed to a unique value. -Y lv_prefix causes the logical volumes on the volume group being recreated to be renamed with this prefix. For the number of characters in the prefix, the total length of the prefix and the logical volume name must be less than or equal to 15 characters. If the length exceeds 15 characters, the logical volume is renamed with the default name. The name cannot begin with a prefix already defined in the PdDv class in the device configuration database for other devices, and it cannot be a name already used by another device. -L label_prefix causes the labels of logical volumes on the volume group being recreated to be changed with this prefix. The user must modify the /etc/file systems stanza manually if a simple modification of the mount point is not enough to define the stanza uniquely. -l LvNameFile entries in the LvNameFile must be in the format LV1:NEWLV1. After recreatevg runs, LV1 is renamed to NEWLV1. All the logical volumes that are not included in the LvNameFile will be recreated with the default system generated name. -f allows a volume group to be recreated that does not have all disks available. -n: After recreatevg runs, the volume group is imported but varied off. The default is imported and varied on.
Appendix A. Copy Services and open systems
847
Note the following points: To use this command, you must have root user authority. All the physical volumes (hdisk) of the volume group must be specified on the command line. The command fails if the input list does not match the list compiled from the VGDA. If you perform a Copy Services function on one half of a RAID 1 pair to reduce the capacity required for FlashCopy targets or Metro Mirror and Global Mirror secondary volumes, then use the -f option to force the creation of the volume group. Otherwise, the VGDA has PVIDs of volumes that make up the other half of the mirror at the source or primary site. In some situations, the volume group is imported or recreated on the hdisks from vpath. In this case, SDD has no effect on the volume group access. To switch your VG PCID from hdisk to vpath, use the command: dpovgfix or hd2vp
An example of accessing a FlashCopy target on the same host using recreatevg For example, we have a volume group that contains two physical volumes (vpaths), one each for a database and a database index. We want to perform a FlashCopy on the corresponding VDisks for the purpose of making a tape backup. To achieve this goal, we must put both source VDisks into a FlashCopy consistency group and trigger the FlashCopy. Also, we have an additional volume group that contains a single volume (vpath) for database logs, and to preserve consistency across the database, we must also put the database log in the same consistency group; then we will have all the FlashCopy target VDisks ready to map to the same AIX host. In Table A-1, details of the physical volume (vpath), AIX volume group, logical volume, mount points, and contents of the respective mount points that are used in this example have been listed. Table A-1 Scenario for FlashCopy target on the same AIX host FlashCopy volume type
Contents of file system
VDisks
Source
Database
FC_DB_Pri
vpath0
Database Index
FC_DBIndx_ Pri
vpath1
Database Logs
FC_DBLog_ Pri
vpath2
Database
FC_DB_Sec
vpath3
Database Index
FC_DBIndx_ Sec
vapth4
Database Logs
FC_DBLog_ Sec
vpath5
Target
848
Vpath / MPIO
AIX Volume group
Logical volume
Mount point
dbdatavg
db_lvm
/dbdata
dbindx_lvm
/dbindx
dblogvg
dblog_lvm
/dblog
fc_tgt_dbdata vg
backup_db_ lvm
/backup/ dbdata
bacup_dbindx _lvm
/backup/ dbindx
backup_dblog _lvm
/backup/dblog
fc_tgt_ dblogvg
Implementing the IBM System Storage SAN Volume Controller V4.3
Example A-1 shows FlashCopy relationships and the consistency group created for this example. Example: A-1 FlashCopy relationships and FlashCopy consistency group
IBM_2145:ITSO-CLS2:admin>svcinfo lsfcmap -delim : id:name:source_vdisk_id:source_vdisk_name:target_vdisk_id:target_vdisk_name:group_id:group_name: status:progress:copy_rate:clean_progress:incremental 0:FC_DB_Rel:15:FC_DB_Pri:19:FC_DB_Sec:1:CG_AIX_FC:stopped:0:0:100:off 1:FC_DBLog_Rel:20:FC_DBLog_Pri:21:FC_DBLog_Sec:1:CG_AIX_FC:stopped:0:0:100:off 2:Fc_DBIndx_Rel:2:FC_DBIndx_Pri:3:FC_DBIndx_Sec:1:CG_AIX_FC:stopped:0:0:100:off IBM_2145:ITSO-CLS2:admin>svcinfo lsfcconsistgrp id name status 1 CG_AIX_FC stopped Example A-2 shows the AIX physical volumes (vpath), volume groups, logical volumes, and the file systems that relate to Table A-1 on page 848. Example: A-2 AIX PhysicalVolumes (vpath), VolumeGroups, LogicalVolumes, and MountPoints
#lspv hdisk0 hdisk1 hdisk2 hdisk3 hdisk4 hdisk5 hdisk6 hdisk7 hdisk8 hdisk9 hdisk10 hdisk11 hdisk12 hdisk13 hdisk14 vpath0 vpath1 vpath2
0009cddaea97bf61 0009cdda43c9dfd5 0009cddabaef1d99 none none none none none none none none none none none none 0009cdda03a968ff 0009cdda03d8000e 0009cdda03a91e64
rootvg rootvg rootvg None None None None None None None None None None None None dbdatavg dbdatavg dblogvg
active active active
active active active
#datapath query device Total Devices : 3 DEV#: 0 DEVICE NAME: vpath0 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801A180E90800000000000047 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk3 OPEN NORMAL 0 0 1 fscsi0/hdisk6 OPEN NORMAL 3283 0 2 fscsi1/hdisk9 OPEN NORMAL 3387 0 3 fscsi1/hdisk12 OPEN NORMAL 0 0 DEV#:
1
DEVICE NAME: vpath1
TYPE: 2145
POLICY:
Optimized
Appendix A. Copy Services and open systems
849
SERIAL: 6005076801A180E90800000000000056 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk4 OPEN NORMAL 0 0 1 fscsi0/hdisk7 OPEN NORMAL 1458 0 2 fscsi1/hdisk10 OPEN NORMAL 1547 0 3 fscsi1/hdisk13 OPEN NORMAL 0 0 DEV#: 2 DEVICE NAME: vpath2 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801A180E90800000000000049 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk5 OPEN NORMAL 0 0 1 fscsi0/hdisk8 OPEN NORMAL 888 0 2 fscsi1/hdisk11 OPEN NORMAL 868 0 3 fscsi1/hdisk14 OPEN NORMAL 0 0 #lsvg -l dbdatavg dbdatavg: LV NAME db_lvm dbindx_lvm loglv01 #lsvg -l dblogvg dblogvg: LV NAME dblog_lvm loglv02
TYPE jfs2 jfs2 jfs2log
LPs 3168 1568 1
PPs 3168 1568 1
PVs 1 1 1
LV STATE open/syncd open/syncd open/syncd
MOUNT POINT /dbdata /dbindx N/A
TYPE jfs2 jfs2log
LPs 1248 1
PPs 1248 1
PVs 1 1
LV STATE open/syncd open/syncd
MOUNT POINT /dblog N/A
#df -g Filesystem GB blocks /dev/hd4 0.09 /dev/hd2 9.06 /dev/hd9var 0.03 /dev/hd3 0.19 /dev/hd1 0.03 /proc /dev/hd10opt 0.97 /dev/lv00 0.41 /dev/db_lvm 99.00 /dev/dbindx_lvm 49.00 /dev/dblog_lvm 39.00
Free %Used Iused %Iused Mounted on 0.05 45% 1579 12% / 4.30 53% 18285 2% /usr 0.01 58% 179 6% /var 0.18 7% 62 1% /tmp 0.03 2% 11 1% /home - /proc 0.71 27% 2979 2% /opt 0.39 4% 19 1% /usr/sys/inst.images 23.78 76% 4 1% /dbdata 16.62 66% 5 1% /dbindx 10.19 74% 5 1% /dblog
Perform the following tasks to create the FlashCopy and make the target volumes available to AIX: 1. Stop all applications that access the FlashCopy source volumes. 2. Unmount all related file systems for the short period of time necessary for FlashCopy to start.
850
Implementing the IBM System Storage SAN Volume Controller V4.3
Tips: For database application, instead of shutting down the database application, you can also use a database suspend command. For example, in DB2®, to suspend I/O, you can use the db2 set write suspend command. After FlashCopy finishes, you can use the db2 set write resume command to resume the IO. You need to flush the OS buffer to disk by issuing a sync command after IO suspends. Example A-3 show the umount of the file systems for database logs, index, and data files Example: A-3 Umount related file systems
#umount /dbdata #umount /dbindx #umount /dblog 3. Prepare and start the FlashCopy consistency group. This will start all the relationships within that FlashCopy consistency group, resulting in a point in time image for all source VDisks on the corresponding target VDisks. Note: If you do not want all the data to be physically copied, create the FlashCopy relationship with the back ground copy parameter set to NOCOPY option. If you use the copy parameter set to zero, and you lose your source VDisk, you will not be able to restore data from the target VDisk, because no “real” data is copied from the source VDisk to the target VDisk, only the pointers. Example A-4 shows the preparing and starting of the FlashCopy consistency group. It also shows the FlashCopy relationships that are members of that consistency group as it transits to the copying state. Example: A-4 Preparing and starting FlashCopy consistency group
IBM_2145:ITSO-CLS2:admin>svctask prestartfcconsistgrp CG_AIX_FC IBM_2145:ITSO-CLS2:admin>svcinfo lsfcconsistgrp id 1
name CG_AIX_FC
status prepared
IBM_2145:ITSO-CLS2:admin>svctask startfcconsistgrp CG_AIX_FC IBM_2145:ITSO-CLS2:admin>svcinfo lsfcconsistgrp id name status 1 CG_AIX_FC copying IBM_2145:ITSO-CLS2:admin>svcinfo lsfcmap -delim : id:name:source_vdisk_id:source_vdisk_name:target_vdisk_id:target_vdisk_name:group_id:group_name: status:progress:copy_rate:clean_progress:incremental 0:FC_DB_Rel:15:FC_DB_Pri:19:FC_DB_Sec:1:CG_AIX_FC:copying:0:0:100:off 1:FC_DBLog_Rel:20:FC_DBLog_Pri:21:FC_DBLog_Sec:1:CG_AIX_FC:copying:0:0:100:off 2:Fc_DBIndx_Rel:2:FC_DBIndx_Pri:3:FC_DBIndx_Sec:1:CG_AIX_FC:copying:0:0:100:of 4. Remount all the related file systems on the source VDisks that were earlier unmounted.
Appendix A. Copy Services and open systems
851
In Example A-5, all file systems related to the database on the source VDisks are remounted. Example: A-5 Remounting related file systems on source VDisks
#mount /dbdata #mount /dbindx #mount /dblog #df -g Filesystem GB blocks /dev/hd4 0.09 /dev/hd2 9.06 /dev/hd9var 0.03 /dev/hd3 0.19 /dev/hd1 0.03 /proc /dev/hd10opt 0.97 /dev/lv00 0.41 /dev/db_lvm 99.00 /dev/dbindx_lvm 49.00 /dev/dblog_lvm 39.00
Free %Used Iused %Iused Mounted on 0.05 45% 1579 12% / 4.30 53% 18285 2% /usr 0.01 58% 179 6% /var 0.18 7% 62 1% /tmp 0.03 2% 11 1% /home - /proc 0.71 27% 2979 2% /opt 0.39 4% 19 1% /usr/sys/inst.images 23.78 76% 4 1% /dbdata 16.62 66% 5 1% /dbindx 10.19 74% 5 1% /dblog
5. Restart applications that access the FlashCopy source volumes. 6. Make the FlashCopy target volumes available to AIX. As shown in Example A-6, detect the new physical volumes and vpaths. Example: A-6 Detecting new physical volumes and vpaths
#cfgmgr -l fcs0 #cfgmgr -l fcs1 #cfallvpath #lspv hdisk0 hdisk1 hdisk2 hdisk3 hdisk4 hdisk5 hdisk6 hdisk7 hdisk8 hdisk9 hdisk10 hdisk11 hdisk12 hdisk13 hdisk14 vpath0 vpath1 vpath2 hdisk15 hdisk16 hdisk17 hdisk18 hdisk19
852
0009cddaea97bf61 0009cdda43c9dfd5 0009cddabaef1d99 none none none none none none none none none none none none 0009cdda03a968ff 0009cdda03d8000e 0009cdda03a91e64 none none none none none
Implementing the IBM System Storage SAN Volume Controller V4.3
rootvg rootvg rootvg None None None None None None None None None None None None dbdatavg dbdatavg dblogvg None None None None None
active active active
hdisk20 hdisk21 hdisk22 hdisk23 hdisk24 hdisk25 hdisk26 vpath3 vpath4 vpath5
none none none none none none none 0009cdda03a968ff 0009cdda03d8000e 0009cdda03a91e64
None None None None None None None dbdatavg dbdatavg dblogvg
#datapath query device Total Devices : 6 DEV#: 0 DEVICE NAME: vpath0 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801A180E90800000000000047 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk3 OPEN NORMAL 0 0 1 fscsi0/hdisk6 OPEN NORMAL 5145 0 2 fscsi1/hdisk9 OPEN NORMAL 5275 0 3 fscsi1/hdisk12 OPEN NORMAL 0 0 DEV#: 1 DEVICE NAME: vpath1 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801A180E90800000000000056 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk4 OPEN NORMAL 0 0 1 fscsi0/hdisk7 OPEN NORMAL 1997 0 2 fscsi1/hdisk10 OPEN NORMAL 2070 0 3 fscsi1/hdisk13 OPEN NORMAL 0 0 DEV#: 2 DEVICE NAME: vpath2 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801A180E90800000000000049 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk5 OPEN NORMAL 0 0 1 fscsi0/hdisk8 OPEN NORMAL 7384 0 2 fscsi1/hdisk11 OPEN NORMAL 7283 0 3 fscsi1/hdisk14 OPEN NORMAL 0 0 DEV#: 3 DEVICE NAME: vpath3 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801A180E90800000000000048 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk15 CLOSE NORMAL 0 0 1 fscsi0/hdisk18 CLOSE NORMAL 0 0 2 fscsi1/hdisk21 CLOSE NORMAL 0 0 3 fscsi1/hdisk24 CLOSE NORMAL 0 0 DEV#: 4 DEVICE NAME: vpath4 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801A180E90800000000000057 ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors
Appendix A. Copy Services and open systems
853
0 1 2 3
fscsi0/hdisk16 fscsi0/hdisk19 fscsi1/hdisk22 fscsi1/hdisk25
CLOSE CLOSE CLOSE CLOSE
NORMAL NORMAL NORMAL NORMAL
0 0 0 0
0 0 0 0
DEV#: 5 DEVICE NAME: vpath5 TYPE: 2145 POLICY: Optimized SERIAL: 6005076801A180E9080000000000004A ========================================================================== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk17 CLOSE NORMAL 0 0 1 fscsi0/hdisk20 CLOSE NORMAL 0 0 2 fscsi1/hdisk23 CLOSE NORMAL 0 0 3 fscsi1/hdisk26 CLOSE NORMAL 0 0 7. The targets vpath3, vpath4, and vpath5 now have the same volume group data structures as the sources vpath0, vpath1 and vpath2, respectively. Clear the PVIDs from the target vpaths to allow a new volume group to be made, as shown in Example A-7. Example: A-7 Removing old PVIDs on newly detected vpaths
#chdev vpath3 #chdev vpath4 #chdev vpath5 #lspv hdisk0 hdisk1 hdisk2 hdisk3 hdisk4 hdisk5 hdisk6 hdisk7 hdisk8 hdisk9 hdisk10 hdisk11 hdisk12 hdisk13 hdisk14 vpath0 vpath1 vpath2 hdisk15 hdisk16 hdisk17 hdisk18 hdisk19 hdisk20 hdisk21 hdisk22 hdisk23 hdisk24
-l vpath3 -a pv=clear changed -l vpath4 -a pv=clear changed -l vpath5 -a pv=clear changed
854
0009cddaea97bf61 0009cdda43c9dfd5 0009cddabaef1d99 none none none none none none none none none none none none 0009cdda03a968ff 0009cdda03d8000e 0009cdda03a91e64 none none none none none none none none none none
Implementing the IBM System Storage SAN Volume Controller V4.3
rootvg rootvg rootvg None None None None None None None None None None None None dbdatavg dbdatavg dblogvg None None None None None None None None None None
active active active
active active active
hdisk25 hdisk26 hdisk15 hdisk16 hdisk17 hdisk18 hdisk19 hdisk20 hdisk21 hdisk22 hdisk23 hdisk24 hdisk25 hdisk26 vpath3 vpath4 vpath5
none none none none none none none none none none none none none none none none none
None None None None None None None None None None None None None None None None None
8. Create the target volume groups, logical volumes, and mount points with prefix fc_tgt_, bkup_, and /backup, respectively, as shown in Example A-8. Notice the changes in the AIX Volume group, logic volumes, and file system names: Example: A-8 Create target volume groups
#recreatevg -y fc_tgt_dbdatavg -Y bkup_ -L /backup vpath3 vpath4 fc_tgt_dbdatavg #recreatevg -y fc_tgt_dblogvg -Y bkup_ -L /backup vpath5 fc_tgt_dblogvg #lspv hdisk0 hdisk1 hdisk2 hdisk3 hdisk4 hdisk5 hdisk6 hdisk7 hdisk8 hdisk9 hdisk10 hdisk11 hdisk12 hdisk13 hdisk14 vpath0 vpath1 vpath2 hdisk15 hdisk16 hdisk17 hdisk18 hdisk19 hdisk20 hdisk21
0009cddaea97bf61 0009cdda43c9dfd5 0009cddabaef1d99 none none none none none none none none none none none none 0009cdda03a968ff 0009cdda03d8000e 0009cdda03a91e64 none none none none none none none
rootvg rootvg rootvg None None None None None None None None None None None None dbdatavg dbdatavg dblogvg None None None None None None None
active active active
active active active
Appendix A. Copy Services and open systems
855
hdisk22 hdisk23 hdisk24 hdisk25 hdisk26 vpath3 vpath4 vpath5
none none none none none 0009cdda04f43fd2 0009cdda04f442c3 0009cdda04f52b93
None None None None None fc_tgt_dbdatavg active fc_tgt_dbdatavg active fc_tgt_dblogvg active
#lsvg -l fc_tgt_dbdatavg fc_tgt_dbdatavg: LV NAME bkup_db_lvm bkup_dbindx_lvm bkup_loglv01
TYPE jfs2 jfs2 jfs2log
LPs 3168 1568 1
PPs 3168 1568 1
PVs 1 1 1
LV STATE closed/syncd closed/syncd closed/syncd
MOUNT POINT /backup/dbdata /backup/dbindx N/A
LPs 1248 1
PPs 1248 1
PVs 1 1
LV STATE closed/syncd closed/syncd
MOUNT POINT /backup/dblog N/A
#lsvg -l fc_tgt_dblogvg fc_tgt_dblogvg: LV NAME bkup_dblog_lvm bkup_loglv02
TYPE jfs2 jfs2log
An extract from /etc/filesystems shows how recreatevg generates a new file system stanza, as shown in Example A-9. Example: A-9 Content of /etc/filesystems
856
/backup/dbdata: dev vfs log mount check options account
= = = = = = =
/dev/bkup_db_lvm jfs2 /dev/bkup_loglv01 true false rw false
/backup/dbindx: dev vfs log mount check options account
= = = = = = =
/dev/bkup_dbindx_lvm jfs2 /dev/bkup_loglv01 true false rw false
/backup/dblog: dev vfs log mount check options account
= = = = = = =
/dev/bkup_dblog_lvm jfs2 /dev/bkup_loglv02 true false rw false
Implementing the IBM System Storage SAN Volume Controller V4.3
9. Mount the new file systems that belong to the target volumes to make them accessible, as shown in Example A-10. Example: A-10 Mounting new file systems belonging to target VDisks
#fsck /backup/dbdata The current volume is: /dev/bkup_db_lvm Primary superblock is valid. J2_LOGREDO:log redo processing for /dev/bkup_db_lvm Primary superblock is valid. *** Phase 1 - Initial inode scan *** Phase 2 - Process remaining directories *** Phase 3 - Process remaining files *** Phase 4 - Check and repair inode allocation map *** Phase 5 - Check and repair block allocation map File system is clean. #fsck /backup/dbindx The current volume is: /dev/bkup_dbindx_lvm Primary superblock is valid. J2_LOGREDO:log redo processing for /dev/bkup_dbindx_lvm Primary superblock is valid. *** Phase 1 - Initial inode scan *** Phase 2 - Process remaining directories *** Phase 3 - Process remaining files *** Phase 4 - Check and repair inode allocation map *** Phase 5 - Check and repair block allocation map File system is clean. #fsck /backup/dblog The current volume is: /dev/bkup_dblog_lvm Primary superblock is valid. J2_LOGREDO:log redo processing for /dev/bkup_dblog_lvm Primary superblock is valid. *** Phase 1 - Initial inode scan *** Phase 2 - Process remaining directories *** Phase 3 - Process remaining files *** Phase 4 - Check and repair inode allocation map *** Phase 5 - Check and repair block allocation map File system is clean. #mount /backup/dbdata #mount /backup/dbindx #mount /backup/dblog #df -g Filesystem /dev/hd4 /dev/hd2 /dev/hd9var /dev/hd3 /dev/hd1 /proc
GB blocks 0.09 9.06 0.03 0.19 0.03 -
Free %Used 0.05 45% 4.30 53% 0.01 58% 0.18 7% 0.03 2% -
Iused %Iused Mounted on 1579 12% / 18285 2% /usr 179 6% /var 62 1% /tmp 11 1% /home - /proc
Appendix A. Copy Services and open systems
857
/dev/hd10opt 0.97 0.71 27% 2979 /dev/lv00 0.41 0.39 4% 19 /dev/db_lvm 99.00 23.78 76% 4 /dev/dbindx_lvm 49.00 16.62 66% 5 /dev/dblog_lvm 39.00 10.19 74% 5 /dev/bkup_db_lvm 99.00 23.78 76% /dev/bkup_dbindx_lvm 49.00 16.62 66% /dev/bkup_dblog_lvm 39.00 10.19 74%
2% /opt 1% /usr/sys/inst.images 1% /dbdata 1% /dbindx 1% /dblog 4 1% /backup/dbdata 5 1% /backup/dbindx 5 1% /backup/dblog
AIX and Metro Mirror and Global Mirror When you have Metro Mirror and Global Mirror primary and secondary VDisks in a mirror relationship, it is not possible to access the secondary VDisk from the hosts. Therefore, it is necessary to stop the mirror relationship and give hosts the access. When using Metro Mirror and Global Mirror, you have a primary (production) site where all the updates and changes are made. These changes are then mirrored, either synchronously or asynchronously, to the secondary (backup) site by SVC. In the same way as in FlashCopy, the target VDisk gets all the data structures and identifiers from the source VDisk. This includes the Volume Group Descriptor Area (VGDA), which contains the physical volume identifier (PVID) and the volume group identifier (VGID). Normally, both the primary site and the secondary site will have their own hosts. Metro Mirror and Global Mirror target VDisks will be mapped to the secondary site hosts to start the applications if the primary site has suffered a disaster. Generally speaking, this means that they will not be mapped to the same host at the same time with their source VDisks, so we will not have any PVID or VGID conflict problems, unless, that is, if in your test environment you have mapped the mirror target VDisks to the same host as the mirror source host at the same time. If that is the case, you must follow the procedure in “Accessing FlashCopy source and target on the same AIX host” on page 846. When the mirror relationship is stopped and access permission is given to the hosts, the mirror target VDisks can be mapped to the host. The command cfgmgr needs to be executed on the target system, or the procedure described in “Making updates to the LVM information” on page 859 has to be performed. Because these VDisks are new to the secondary AIX system, there is no conflict with existing PVIDs. The volume group on the secondary volumes containing the LV and file system information can now be imported into the Object Data Manager (ODM) and the /etc/filesystems file using the importvg command. If the Metro Mirror and Global Mirror secondary volumes were previously defined on the secondary AIX system as hdisks or vpaths, but the original volume group information was destroyed on the volumes, you must remove the old volume group and disk definitions (using exportvg and rmdev) and run cfgmgr again before running the importvg command to gather the new volume group definitions. If this is not done first, the importvg command imports the volume group improperly, and the file systems will not be accessible. Tip: We highly recommend cleaning all old SVC related devices from AIX ODM before mapping the new SVC VDisks. The following commands can be used before using cfgmgr on new VDisks: exportvg rmdev -Rdl dpo rmdev -Rdl fcsx
858
Implementing the IBM System Storage SAN Volume Controller V4.3
You can reactivate the volume group using varyonvg and mount its file systems.
Making updates to the LVM information When performing Metro Mirror and Global Mirror between primary and secondary VDisks, the primary AIX host might create, modify, or delete existing LVM information from a volume group. Therefore, when in a Metro Mirror and Global Mirror relationship, the secondary volume is not accessible, and the LVM information on the secondary AIX host is not current, so you have to get the secondary AIX host updated every time you make changes to the LVM information at the primary site. It might also be that you do not have scheduled periods where write I/Os to the primary Metro Mirror and Global Mirror volume can be quiesced and file systems unmounted so that the mirror relationship can be stopped, and the secondary AIX host can perform a learn command on the volume group (importvg -L). If that is the case, you can execute the varyonvg command on the primary AIX host for the volume group you have made changes to, which will remove the SCSI lock on the volumes in the SVC. The parameters necessary for the varyonvg command are -b -u, after you execute the importvg -L command on the secondary AIX host, and after the LVM changes are updated on the secondary AIX host. You have to execute the varyonvg command on the primary AIX host again to activate the SCSI lock on the volumes on the SVC. The import -L command takes a volume group and learns about possible changes performed to that volume group. Any new logical volumes created as a result of this command emulate the ownership, group identification, and permissions of the /dev special file for the volume group listed in the -y flag. The -L flag performs the functional equivalent of the -F and -n flags during execution. The following restrictions apply: The volume group must not be in an active state on the system executing the -L flag. The volume group’s disks must be unlocked on all systems that have the volume group varied on and operational. Volume groups and their disks might be unlocked, remain active, and used through the varyonvg -b -u command. The physical volume name provided must be of a good and known state; the disk named might not be in the missing or removed state. If an active node has both added and deleted logical volumes on the volume group, the -L flag might produce inconsistent results. The -L flag should be used after each addition or deletion, rather than being deferred until after a sequence of changes. If a logical volume name clash is detected, the command will fail. Unlike the basic importvg actions, clashing logical volume names will not be renamed. Here is an example of how to use the -L on a multi-tailed system:
Primary AIX node A has the volume group datavg varied on. Secondary AIX node B is aware of datavg, but it is not varied on. For primary AIX node A, run varyonvg -b -u datavg. For secondary AIX node B, run importvg -L datavg hdisk07. For primary AIX node A, run varyonvg datavg.
More detailed information about the varyonvg and importvg commands can be found in the AIX Commands Reference, Volume 1-6.
Appendix A. Copy Services and open systems
859
Windows NT and 2000/2003 specifics This section describes the tasks that are necessary when performing Copy Services operations on VDisks that are mapped to Microsoft Windows NT® and 2000/2003 hosts.
Windows NT and Copy Services This section explains the actions that you need to perform on Metro Mirror and Global Mirror and FlashCopy VDisks owned by Microsoft Windows NT operating systems. Windows NT handles disks in a way that is not similar to any other operating system covered in this book. The need to reboot a server to scan for new disks and the need to run a GUI-based Disk Administrator to manipulate the disks are the main factors that restrict the routine use of Metro Mirror, Global Mirror, and FlashCopy on Windows NT, making automation virtually impossible. It is possible to automate the actions of the GUI-based Disk Administrator using third-party software to remotely reboot the server. It is also possible to remotely assign the drive letter from the server that starts the Copy Services task. This was not tested during our project. If you are going to create an automated script with Windows NT, you need to be careful about data consistency. It could be that some part of the automation process might run a script on a source server, and subsequent actions might be taken by a script on a target server. Therefore, interprocess communication across servers might be required for timing. Otherwise, you might get inconsistent data. Not all applications allow this. You have two options on how to make the Metro Mirror and Global Mirror or FlashCopy target available to the server: with reboot or without reboot. We recommend that you reboot the server. It is safer because then it is guaranteed that all the registry entries are created. However, using Metro Mirror and Global Mirror or FlashCopy without rebooting is faster.
Copy Services with Windows NT Disk In the following sections, we describe copy services with Windows NT Disks.
Registering the Metro Mirror, Global Mirror, and FlashCopy volumes If you are going to reboot the server, you do not have to make the target disks known to Windows NT before you do the Metro Mirror and Global Mirror or FlashCopy. However, we recommend that you preassign and register them in the server. The “assign disk and run Metro Mirror and Global Mirror or FlashCopy” approach is useful for a non-routine Metro Mirror and Global Mirror or FlashCopy, for example, for testing or migration. For routine purposes, we recommend that you have target disks already present in Disk Administrator with partitions created and partition information saved. Select Start → Programs → Administrative Tools → Disk Administrator. Then follow these steps: 1. If the target disk was not previously seen by the system, Disk Administrator issues a pop-up message saying “No signature on Disk X. Should I write a signature?”, where X is the number assigned to the newly present disk. Click OK to save the signature on the target disk. 2. The Disk Administrator opens. Click the disk that is to be used as the Metro Mirror and Global Mirror or FlashCopy target (it should be gray and marked as free space) and select Create. 3. Confirm the partition parameters and click OK. The partition appears as Unknown. 4. Click the newly created partition and select Commit Changes Now. 860
Implementing the IBM System Storage SAN Volume Controller V4.3
5. Right-click the partition and select Assign Drive letter. 6. Assign a drive letter and click OK. 7. Exit Disk Administrator. After this procedure, the Metro Mirror and Global Mirror or FlashCopy target is properly registered in Windows NT.
Bringing down the target server Bring down the server that will use the target if you want to use the safest method. Also, keep in mind that if you assign the volume to the host just before you perform the Metro Mirror and Global Mirror or FlashCopy, you must use the volume serial number for the target.
Performing a Metro Mirror and Global Mirror or FlashCopy Stop all applications using the source volume. Now flush the data to the source volume. Select Start → Programs → Administrative Tools → Disk Administrator. Then follow these steps: 1. Right-click the disk that is to be used as the Metro Mirror and Global Mirror or FlashCopy source. It should have a drive letter assigned and be formatted. Then select Assign Drive letter. 2. From the pop-up window, select Do not assign a drive letter and click OK. 3. Now the data is flushed to the source. You can start the Metro Mirror and Global Mirror or FlashCopy task from the SVC Copy Services Web Interface or from any server CLI. 4. Observe the GUI, or enter the following command to see if the Metro Mirror and Global Mirror or FlashCopy task successfully started: svcinfo lsfcmapprogress and svcinfo lsrcrelationshipprogress 5. Reassign the drive letter to the source volume. Right-click the disk that is a Metro Mirror and Global Mirror or FlashCopy source and select Assign Drive Letter. 6. Assign a drive letter and click OK. 7. Exit Disk Administrator. You can resume using the source volume.
Bringing up the target server Next, you can boot up the target server. In this case, you just assign the target volumes to the host that will create the disk entry in the Windows NT registry. To verify that the registry entry is created, complete these tasks: 1. Select Start → Settings → Control Panel → Hardware → Device Manager. 2. In Control Panel, double-click Disk Drives. 3. Click the adapter that has the target volume attached. 4. A list of targets opens. Verify the list, including the target ID and LUN of the volume you just made available to the server. If you are using SDD, you see each disk entry several times [(# of vDisks) x (# of Nodes) x (4 Ports/Node) x (# of HBAs/host)], which is the number of paths to the volume that you have. You can also run the datapath query device command from the SDD command line to check whether the Metro Mirror and Global Mirror or FlashCopy targets are listed between the volumes. This command also enables you to check volume serial numbers and gives you a better overview of the volumes and their paths.
Appendix A. Copy Services and open systems
861
Making the Metro Mirror and Global Mirror or FlashCopy target available Log in, start the Windows NT Disk Administrator, write a signature if necessary (do not write a signature if data was already copied into this volume), and assign a drive letter. To begin, select Start → Programs → Administrative Tools → Disk Administrator. Then follow these steps: 1. If the disk was not previously seen by this system, Disk Administrator issues the “No signature on Disk X. Should I write a signature?” message, where X is the number assigned to the newly present disk. Click OK to save the signature on the target disk. 2. The Disk Administrator opens. Click the disk that is a Metro Mirror and Global Mirror or FlashCopy target. You should see a formatted partition on it. Select Assign Drive Letter. 3. If you cannot assign a drive letter, the target might be corrupt. Try repeating the whole process and then consider rebooting. 4. Assign a drive letter and click OK. Exit Disk Administrator. 5. From a Windows NT command prompt, run the following command, where x is the letter assigned to the Metro Mirror and Global Mirror or FlashCopy target: chkdsk x: /f /r An option is available to run the disk check from the Properties menu of a disk in Windows NT Explorer. After you complete this procedure, the Metro Mirror and Global Mirror or FlashCopy target is available to the Windows NT and can be handled like a normal disk.
Copy Services with Windows NT Volume Sets This section explains how to perform Copy Services functions with Windows NT Volume Sets.
Copy Services and Windows NT Volume Sets Metro Mirror, Global Mirror, and FlashCopy are supported when using normal disks and Volume Sets. When using Metro Mirror, Global Mirror, or FlashCopy with Volume Sets, because these outboard copy features do not copy the Volume Set information in the Windows Registry, certain limitations exist, and a special procedure is required, as described below. After SP6, it is possible to have the FlashCopy source and target volumes accessible by the same server. Prior to SP6, the FlashCopy source and target volumes must be attached to different servers. Metro Mirror and Global Mirror primary and secondary volumes must be attached to different servers.
Using Metro Mirror, Global Mirror, and FlashCopy with Volume Sets This special procedure is required to use Metro Mirror, Global Mirror, or FlashCopy with a Windows NT volume set. The procedure can also be applied to other Windows NT fault tolerant disk configurations, such as mirrored sets, striped sets, and striped sets with parity. Consider the case where the target disks are in the same order as the source disks, and the target disks are contiguous (that is, all the disks are next to each other as viewed by the target machine’s Disk Administrator). Then you simply create an identical volume set on the target machine and reboot prior to performing the FlashCopy. You do this before you perform FlashCopy or Metro Mirror or Global Mirror for the first time. Subsequent copies should work as expected, provided that the file system is unmounted (the drive letter is unassigned) on the target prior to performing a copy.
862
Implementing the IBM System Storage SAN Volume Controller V4.3
If the target disks do not appear contiguous to Windows NT or appear in a different order than on the source machine, then a different procedure must be used. Microsoft’s FTEDIT, available on the NT Resource Kit, is a Microsoft supported tool designed to write volume set information into the registry. Using FTEDIT is much safer than editing the registry directly. Important: Incorrect use of FTEDIT could result in loss of access to software RAID arrays. We recommend that you use Disk Administrator to save your disk configuration before using FTEDIT. In general, most errors made using FTEDIT are recoverable. For more information about how to recover from FTEDIT errors and on FTEDIT in general, see the Microsoft Knowledge Base article for Q131658: http://support.microsoft.com/default.aspx?scid=kb;en-us;131658 The following procedure explains how to use FlashCopy, Metro Mirror, and Global Mirror with FTEDIT.
Preparation On the target machine, complete the following tasks: 1. Back up the disk data using Disk Administrator, and registry information using regedit. 2. If the target disks were previously used, delete all of the target disks in Disk Administrator. Do not simply unmount them, but delete all of the partitions on the target disks. Commit the changes. 3. In the control panel, double-click Devices. Make sure that Ftdisk is started and set to start on boot. Ftdisk is the driver used by Windows NT to identify and access fault tolerant drives, such as volume sets. If there are any fault tolerant drives in use on the system, Ftdisk is started and set to start on boot. If it is not started, one way to start it is to create a fault tolerant drive on a couple of spare disks. This requires a reboot. On the source machine, obtain the order in which the disks were added to the volume set. One way to do this is to use a freeware utility called diskkey.exe, available from: http://www.sysinternals.com This utility is not supported by IBM and is known to report disk numbering and other information that is different than what Disk Administrator reports. However, the order in which the disks are included in the volume set is correct. Also, the correct ordering of the disks is the information required to create a duplicate volume set on the target server. Map the disks on the source machine to the disks on the target machine. For example, determine that Disk6 on the source is FlashCopy copied to Disk9 on the target.
Performing the Metro Mirror, Global Mirror, or FlashCopy On the target machine, follow these steps: 1. Run the FlashCopy establish or Metro Mirror or Global Mirror terminate tasks. 2. Start Disk Administrator. If it asks you to write a signature on any of the disks, click No (except in the special cases, see the following Important box). After Disk Administrator is up, commit the changes (this is very important), and close Disk Administrator.
Appendix A. Copy Services and open systems
863
Important: Disk Administrator asks you to write a signature when the FlashCopy is performed to the same machine because it detects a duplicate disk signature (the source and target volumes have the same disk signature) and it needs to write a new one. It is safe to do this, but be sure that you are writing the signature to the FlashCopy target disk. If a signature is written to the wrong disk, it could cause data corruption. When FlashCopying to a different machine, usually the disk signature on the target machine's disks are different than the FlashCopy source disks’ signature, so Disk Administrator does not need to write a new signature to the target disks to use it. It is unlikely, but possible, that by coincidence the disk signature of one of the source disks is the same as one of the disks on the target machine. In this case, you must write a signature on the target disk before you use it. Again, it is safe to do this, but be sure that you are writing the signature to the right disk. 3. Start FTEDIT by selecting Start → Resource Kit 4.0 → Disk Tools → Fault Tolerance Editor. 4. Read the warning and click OK. 5. There are two panes in the FTEDIT window. On the left pane is a list of the disks in the system. On the right pane is the list of partitions on that disk. You must add the disks to the volume set in the right order. Use the results of diskkey.exe to determine the order in which the disks were added on the source volume set. Note: If active Metro Mirror or Global Mirror target volumes are on the target, then the disk numbering used in FTEDIT might differ from the disk numbering used in the Disk Administrator. The Metro Mirror and Global Mirror target volumes are not seen by FTEDIT and are not included in the disk numbering scheme. Adjust your disk choices accordingly. 6. Click Make FT set in the lower left corner. 7. When it asks you what kind of set you want, choose Volume set and click OK. 8. Click the first target disk in the left pane. 9. The list of partitions on that disk should appear in the right pane. Choose the partition that contains the volume set on that disk (usually Partition 1). Double-click Partition 1 in the right pane. This adds this disk or partition to the volume set, in order. 10.Repeat Steps 8 and 9 for the rest of the disks. If you make a mistake, you can cancel and start from scratch. The disks must be added in the correct order. 11.After you add all of the disks, click Save FT set at the bottom. 12.Select Edit → Save Changes to System. 13.Close FTEDIT. 14.Reboot the system. 15.When Windows NT restarts, start Disk Administrator. The target disks should be yellow now, indicating that they are in a volume set. Assign a drive letter and commit the changes. If the drives are not usable at this point, then the disks were probably added in the wrong order. As long as the disk configuration does not change on the source or target, FlashCopy should work as expected. If the disk configuration is changed in anyway, such as adding an additional disk to the volume set or rearranging the disks, then you have to perform this procedure again. 864
Implementing the IBM System Storage SAN Volume Controller V4.3
Windows 2000/2003 and Copy Services Windows 2000/2003 handles its disks differently than Windows NT does. Windows 2000/2003 incorporates a stripped-down version of the Veritas Volume Manager, called the logical disk manager (LDM). With the LDM, you can create logical partitions, perform disk mounts, and create dynamic volumes. There are five types of dynamic volumes: simple, spanned, mirrored, striped, and RAID 5. On Windows NT, the information relating to the disks was stored in the Windows NT registry. With Windows 2000, this information is stored on the disk drive itself in a partition called the LDM database, which is kept on the last few tracks of the disk. Each volume has its own 128-bit Globally Unique Identifier (GUID). This is similar to the disk PVID in AIX. Since the LDM is stored on the physical drive itself, with Windows 2000, it is possible to move disk drives between different computers.
Copy Services limitations with Windows 2000 and Windows 2003 Having the drive information stored on the disk itself imposes some limitations when using Copy Services functionality on a Windows 2000/2003 system: The source and target volumes must be of the same physical size for two reasons: – The LDM database holds information relating to the size of the volume. Since this is copied from the source to the target, if the target volume is a different size from the source, then the database information is incorrect, and the host system returns an exception. – The LDM database is stored at the end of the volume. The copy process is a track-by-track copy, and unless the target is an identical size to the source, the database is not at the end of the target volume. It is not possible to have the source and target FlashCopy Volume on the same Windows 2000/2003 system when they were created as Windows 2000/2003 dynamic volumes. The reason is that each dynamic volume must have its own 128-bit GUID. As its name implies, the GUID must be unique on one system. When you perform FlashCopy, the GUID is copied as well. This means that if you tried to mount the source and target volume on the same host system, you would have two volumes with exactly the same GUID. This is not allowed, and you are not able to mount the target volume.
Copy Services on basic volumes Basic disks are the same as the Windows NT disks with the same restrictions. Dynamic disks are supported for Metro Mirror, Global Mirror, and FlashCopy, and the primary source and secondary target VDisks must be attached to different servers. On the other hand, for a basic disk, it is possible to attach the secondary target VDisk to the same server. In the following steps, we show how to make a FlashCopy on a basic disk, and mount it on the same server as the source disk is mounted. Tip: Since the source VDisk and target VDisk are basically the same, they can coexist in one Windows server when they are imported as basic disks. They cannot coexist in one Windows server when they are imported as dynamic disks, as Windows Logical Volume Manager cannot handle this unless a special procedure is performed. Consult the Microsoft Knowledge Base for more information about logical volume and disk management.
Appendix A. Copy Services and open systems
865
Before making any FlashCopy on the Windows server, we have two disks, the internal C: drive and an SVC disk, S:, as seen in Figure A-1.
Figure A-1 Windows server before adding FlashCopy target disk
To make a FlashCopy on Windows 2000/2003, we need to make a VDisk that we will use as the FlashCopy target and this VDisk needs to be exactly the same size as the source VDisk. To ensure that the size is exactly the same, first list the source VDisk on the SVC using the parameter -bytes; that gives the precise size. If we do not use the parameter -bytes, the size will be in GB, and, for example, 5 GB might not be the exact size, but will be the rounded-off size. We create a new VDisk with the same size and give it the name FC_Win_Sec, also using the size unit in bytes, as shown in Figure A-2 on page 867.
866
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure A-2 Create VDisk for FlashCopy target
After creating the target VDisk, we define the FlashCopy mapping on the SVC, as shown in Figure A-3.
Figure A-3 Creating FlashCopy relationship
Appendix A. Copy Services and open systems
867
We create the point in time image for the defined relationship by preparing and starting the relationship, as shown in Figure A-4.
Figure A-4 Preparing and starting the FlashCopy relationship
Now we have a point in time copy of the source VDisk from the Windows host. Next, we map the target VDisk to the same Windows host and perform a rescan at the server, as shown in Figure A-5.
Figure A-5 Assign FlashCopy target VDisk to host
868
Implementing the IBM System Storage SAN Volume Controller V4.3
After the rescan on the Windows server, it now shows the new volume and that it is already in NTFS format, but does not have a drive letter assigned yet, as shown in Figure A-6.
Figure A-6 Discovered FlashCopy target disk on Windows server
Appendix A. Copy Services and open systems
869
We then select the disk, assign a drive letter, and check that the data is on the target disk. This is shown in Figure A-7 and Figure A-8 on page 871.
Figure A-7 Chose disk and assign drive letter
870
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure A-8 The target disk is now ready on the Windows server with drive letter T
Enlarging an extended basic volume Initially, the Copy Services source could be a single simple volume. However, as requirements change on the application servers, you might want to extend the logical volume. However, you should not independently extend the target VDisks on the Windows host, but let Windows detect the correct sequence of the extended volumes during the import process of the extended SVC target VDisk. Attention: SVC FlashCopy, Metro Mirror, and Global Mirror require the source VDisk and target VDisk to be the same size, so do not change the capacity of the VDisk. Use the following procedure to increase the size of a volume at the OS level. You might have an extended volume, and in time the logical drive also might grow to include more of the initial volume (extended disk). When this occurs, it is necessary to perform a rescan of the disks and use the diskpart command to extend the disk, so the capacity can be available to an existing partition. On the initial FlashCopy, reboot the target server to configure the additional disks, or perform a rescan of disks. On subsequent FlashCopy copies to the target volume group, run only a chkdsk /F command on the target volume, to make Windows aware of the changes on the target volume. If you do not execute the chkdsk /F command, you cannot rely on the data. Using our previous example, the complete command would be chkdsk T: /F.
Appendix A. Copy Services and open systems
871
When expanding a basic disk that is subsequently being used as a FlashCopy source by using diskpart, the following process is necessary to keep the FlashCopy in a usable state on the target disk, either on the same server or on a different server. Follow these steps: 1. Remove the FlashCopy map between the source and target VDisk on the SVC. 2. Extend the VDisk on the SVC for the Windows volume that is going to be extended. 3. Rescan for disks on the server where the source volume is located. 4. Extend the Windows volume using the diskpart program. 5. Remove the target disk on the server. 6. Remove the mapping of the target VDisk. 7. Rescan for disks on the target server to remove the old target disk. 8. Extend the target VDisk to match the new size for the source VDisk on the SVC. 9. Make a “new” FlashCopy mapping on the SVC for the VDisks. 10.Make a new FlashCopy point in time image. 11.Rescan for disks on the server where the target volume is located. 12.Assign a drive letter to the new extended target volume.
Copy Services on dynamic volumes To see target dynamic volumes on a second Windows 2000/2003 host, you must complete these tasks: 1. Perform the Metro Mirror, Global Mirror, or FlashCopy function on the source VDisk. When using Metro Mirror or Global Mirror, ensure that the primary and secondary mirror relationship is in consistent mode. Also make sure that write I/O ceased prior to stopping the mirror relationship. 2. Map the target volume (VDisk) to the second Windows 2000/2003 host. 3. Select Computer Management → Disk Management. 4. Find the disk that is associated with your volume. There are two “panes” for each disk. The left pane should read Dynamic and Foreign. It is likely that no drive letter is associated with that volume. 5. Right-click that pane, and select Import Foreign Disks. Select OK, and then OK again. The volume now has a drive letter assigned to it. It is of Simple Layout and Dynamic Type. You can read and write to that volume. Tip: Disable the fast-indexing option on the source disk. Otherwise, operations to that volume are cached to speed up disk access. However, this means that data is not flushed from memory and the target disk might have copies of files or folders that were deleted from the source system. When performing subsequent Metro Mirror, Global Mirror, or FlashCopy copies to the target volume, to detect any changes to the contents of the target volume, it is necessary to run the following command on the target volume: chkdsk.exe /F
Example of a FlashCopy over two dynamic disks Next, we describe an example of how to make a FlashCopy on dynamic volumes with a dependency I/O relationship. We have two VDisks, FC DB Source and FC DBLog Source, mapped to the Windows servers, as shown in Figure A-9 on page 873.
872
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure A-9 Dynamic Disks on Windows server as the FlashCopy source VDisks
On the Windows source host using the SDD command datapath query device, find the VDisk that you will make a FlashCopy of. The resultant output will list the VDisk UID, and you should compare it with the VDisks information on the SVC to identify the VDisk. If this is the first time you are creating the FlashCopy, make sure that you have a target VDisk of the same size and, if not, create it. As to size, use the unit byte (b) so the VDisk will be exactly the same size. You can discover the exact size, in bytes, of the source VDisk using the following command: svcinfo lsvdisk -bytes <Source_VDisk> We make the new target VDisks, one each for FC DB Source and FC DBLog Source, using the unit byte, so the VDisk has the exact same size. After the creation of the target VDisk, we need to create the FlashCopy relationships between the source and target. Figure A-10 on page 874 shows the FlashCopy relationships between the source and target. In our example, FC_DB_Repl is the FlashCopy relationship between FC_DB_Pri as the source and FC_DB_Sec as the target, while FC_DBLog_Repl is the FlashCopy relationship between FC_DBLog_Pri as the primary and FC_DBLog_Sec as the secondary.
Appendix A. Copy Services and open systems
873
Figure A-10 FlashCopy Relationships for Windows Dynamic Disks FC DB Source and FC DBLog Source
Also, since these two dynamic disks have I/O dependencies, we have to make sure that the FlashCopy has been taken at the exact same time. To ensure this, make the two FlashCopy mappings members of a consistency group on the SVC. Figure A-11 and Figure A-12 shows the creation of the FlashCopy consistency group and making the FlashCopy relationship a member of this group.
Figure A-11 Creating FlashCopy consistency group and making FlashCopy relationships as the members
Figure A-12 Viewing FlashCopy relationships as a member FlashCopy consistency group
Before we can perform the FlashCopy, we need to make sure that there is no I/O to the disk on the Windows host.
874
Implementing the IBM System Storage SAN Volume Controller V4.3
Tip: You can use two options to make sure Windows and the local disks’ cache is already flushed to the SVC: Set the disk properties to Optimize for quick removal before preparing the FlashCopy mapping or consistency group, as shown in Figure A-13. Delete the drive letter in the Disk Management tool.
Figure A-13 Optimize for quick removal
After the host is ready for the FlashCopy image, we need to prepare the FlashCopy consistency group and start it. To prepare the FlashCopy Consistency Group, go to the Viewing FlashCopy Consistency Group window and, from the drop-down menu, select Prepare a Consistency Group. Once the FlashCopy Consistency Group is prepared, it can be started from the same menu by selecting Start a Consistency Group. Also, the Start a Consistency Group Wizard asks us to select Prepare Consistency Group if it is not already prepared. Figure A-14 shows how to start the FlashCopy consistency group to create a point in time image.
Figure A-14 Starting FlashCopy consistency group
Once the FlashCopy relationship is stared, the point in time image is created and the target VDisk is available for utilization. The FlashCopy relationship enters the Copying state; however, we do not need to wait until the copy is done before the target VDisk can be added to another Windows host.
Appendix A. Copy Services and open systems
875
The Background Copy Progress can be viewed in the View Progress window under Manage Progress. Select the FlashCopy option. Then, we need to map the target VDisks to a backup Windows server, as shown in Figure A-15. Note: Windows is using Logical Volume Manager for dynamic disks, so to avoid ID conflict with source VDisks dynamic disks, make sure you are mapping the target VDisks to a backup host.
Figure A-15 Mapping target VDisks to another Windows host
876
Implementing the IBM System Storage SAN Volume Controller V4.3
On the Windows host, we now need to scan for new disks, as shown in Figure A-16. After the scanning process is completed, all VDisks with corresponding virtual devices are listed under the Disk drives section.
Figure A-16 Scanning for FlashCopy target VDisks
Appendix A. Copy Services and open systems
877
The same virtual disk devices are also available as Disk 1 and Disk 2 in Disk Management. This is shown in Figure A-17. These disks are marked as Foreign, because this is the first time these disks have been discovered by the Windows host.
Figure A-17 Viewing target FlashCopy VDisks under Windows DIsk Management
To make the disks available to the Windows host, we need to import the foreign disks. To do this, we right-click one of the disks and select Import Foreign Disk, as shown in Figure A-18 on page 879.
878
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure A-18 Import Foreign Disk
While importing the new disks to the Windows host, if you had only chosen one of the foreign disks, it will tell you how many disks are going to be imported, as shown in Figure A-19. By clicking Disks, you can list all the members of foreign disks group that will be imported, as shown in Figure A-20 on page 880. Click OK to continue.
Figure A-19 Showing the foreign disk group
Appendix A. Copy Services and open systems
879
Figure A-20 Listing members of foreign disk group
Figure A-21 show the volumes that existed on these dynamic disks. This is what we had expected, so we accept this by clicking the OK button.
Figure A-21 Listing existing volumes on dynamic disks
880
Implementing the IBM System Storage SAN Volume Controller V4.3
When we have done this, the Windows Disk Manager shows the dynamic disks as online. The disks are now known to the host, and are no longer foreign disks, as shown in Figure A-22.
Figure A-22 The FlashCopied dynamic disks are online
Appendix A. Copy Services and open systems
881
Assign drive letters to these disks so they are ready to be utilized by applications. This is shown in Figure A-23.
Figure A-23 Dynamic disks ready to be utilized by applications
Metro Mirror, Global Mirror, and Windows dynamic volumes Follow this procedure when carrying out the Metro Mirror and Global Mirror of a Windows 2000 dynamic volumes set from Server (A) to Server (B): 1. On the source server (A), create a Windows dynamic volume set of multiple dynamic disks. 2. Reboot the target server (B), import multiple target disks, and write a disk signature on each as basic disks. 3. Once this is done, remove the target disks from the target server, because each copy relationship needs its target VDisk in read only mode for that relationship. 4. Establish Metro Mirror and Global Mirror between the source (A) and target volumes (B). 5. After the source and target volumes are synchronized, terminate Metro Mirror and Global Mirror. 6. Reboot the target host (B). 7. Start Disk Manager. The Metro Mirror and Global Mirror target volumes are seen as foreign dynamic disks. 8. The disks were imported into the target host and are seen as a dynamic volume.
882
Implementing the IBM System Storage SAN Volume Controller V4.3
To demonstrate failback to the original setup, carry out the following steps: 1. Remove the original paths and re-establish them in the reverse direction from (B) to (A). 2. Remove the dynamic volume drive letter from the original source, the dynamic volume on server (A). 3. Establish Metro Mirror and Global Mirror from (B) to (A) and write some data onto the dynamic volume. 4. Metro Mirror and Global Mirror stop. 5. Restore the drive letter to the dynamic volume on server (A). 6. The contents of the dynamic volume can now be read from server (A).
Appendix A. Copy Services and open systems
883
884
Implementing the IBM System Storage SAN Volume Controller V4.3
B
Appendix B.
DS4000 and DS8000 migration scenarios In this appendix, we present a high-level overview of migrating from “normal” SAN attached storage to virtualized storage. In all of the examples, we will use the IBM System Storage DS4000 Series, the IBM System Storage DS 8000, and IBM Shark as the storage system. The DS8000 recommendations also apply to the IBM Shark Storage systems.
© Copyright IBM Corp. 2003-2008. All rights reserved.
885
Initial considerations There are some basic factors that you must take into account in all situations before starting: Host device drivers must be changed, so all LUNs in a host partition must be moved to the SVC partition in one step. Each partition can only access a unique set of host bus adapter (HBA) ports, as defined by worldwide port names (WWPNs) of those adapters. Only one storage partition must be created that includes any IBM System Storage SVC ports of nodes that are in the same SVC cluster. The contents of an existing partition must be moved to the SVC partition at the same time. Some configurations might require backup, reconfigure, and restore. Some versions of the DS4000 and DS8000 firmware allow RAID arrays to be expanded, allowing their capacity to increase, which is not recommended, although it might be helpful for some configurations. DS 4000 considerations. – Depending upon the model and firmware version, a DS4000 can scale up to 2048 logical unit numbers (LUNs) across 64 host partitions. – Auto Volume Transfer (AVT): This is the required mode of operation for DS4000 controllers. Selecting the IBM TS SAN VCE Host type for all Logical Units attached to SVC will ensure that AVT mode is enabled. If you are accessing LUNs with a host type that has AVT disabled, an error log entry is generated (1625: Incorrect Controller Configuration). DS 8000 and Shark considerations – Select IBM SAN Volume Controller as the host type for the SVC. In some firmware versions if this host type is not available, then use RS/6000®. – The Use same ID / LUN in source target option should be enabled on your DS8000. Shark considerations (does not apply to DS8000). The SVC and Shark can scale up to 4000 logical unit numbers (LUNs) and all of them can be mapped to the SVC, but they can only map 250 LUNs to a port at a time. So, for example, in an SVC cluster with four ports on each node, mappings can be set up such that each Shark LUN group is mapped to one port on each SVC node. In this case, the maximum number of mapped LUNs from the Shark to the SVC would be 1000. The DS8000 does not have any LUN limitations that impact SVC configurations. Important: If you have more logical units than are supported in one partition, then some spare storage is required to allow for temporary migration of the data while the primary storage subsystem is re-configured to have fewer logical units.
886
Implementing the IBM System Storage SAN Volume Controller V4.3
Device recognition The IBM DS4000 storage subsystems will be recognized as shown in Table B-1 within the SVC. Table B-1 DS4000 recognition (* = wildcard) Bytes
Field name
Pattern(s)
Note
8-15
Vendor ID
“IBM”
16-31
Product ID
“3552” “3542” “1722-600” “1742” “1742-900” “********FAStT”
Model 500 Model 200 Model 600 / DS4300 Model 700 / DS4400 Model 900 / DS4500 Any FAStT / DS4000 Model
32-35
Product Revision Level
“****”
Firmware level
The IBM DS8000 storage subsystem will be recognized as displayed in Table B-2 within the SVC. Table B-2 DS8000 recognition (* = wildcard) Bytes
Field name
Pattern(s)
Note
8-15
Vendor ID
“IBM”
16-31
Product ID
“2105F20” “2105750” “2105-800” “2107***” “1750***”
2105-F20 2105-750 2105-800 2107-900 1750-500
32-35
Product Revision Level
“****”
firmware level
Appendix B. DS4000 and DS8000 migration scenarios
887
Scenario 1: DS4000 Total number of LUNs is less than maximum LUNs per partition In our initial configuration, the existing host or hosts attached to the DS4000 have fewer LUNs than is supported in one partition. Only one partition per host, or both partitions on a single host, have less than the maximum numbers of supported LUNs when combined. Figure B-1 shows the initial configuration. Note: Access LUN (A) is used by Host A for in-band configuration of the DS4000. This is deleted and the DS4000 is configured to use the SVC Console over Ethernet. The access LUN is not required by the SVC.
Figure B-1 Initial setup
Then we add the SVC and create an SVC partition on the DS4000. See Figure B-2 on page 889. The following steps are required for this task: 1. Modify the zoning so that only the SVC can “see” the DS4000. This allows partition 3 to be created, and access to partitions 0, 1, and 2 can continue. 2. The port or ports of the SVC Console must not be in any partition.
888
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure B-2 Add SVC and partition 3
We move the storage for host C from host partition 2 to partition 3 (SVC partition) to be managed by the SVC. See Figure B-3 on page 890. Note the following points: Concurrent access from host C to its logical units is not possible. Host C requires reconfiguration from the RDAC device drivers to SDD. Changes to the adapter configuration and microcode levels, settings, and so on, might also be required. Switch zoning changes are also required to prevent host C from “seeing” the DS4000 ports and instead “see” the SVC ports. LUNs from host partition 2, that are now in partition 3, must be configured as image mode VDisks in SVC and mapped to host C. Note: Partition 2 should now be deleted after all logical units are moved to partition 3.
Appendix B. DS4000 and DS8000 migration scenarios
889
Figure B-3 Storage moves from partition 2 to partition 3
The next step is to move the storage for host A and B from host partition 0 and 1 to partition 3 (SVC partition), to be managed by the SVC. This is shown in Figure B-4 on page 891. The following steps are required to do this: 1. Stop access from host A and B to its logical units. 2. Host A and B require reconfiguration from RDAC device drivers to SDD. Changes to adapter configuration and microcode levels, settings, and so on, might also be required. 3. Switch zoning changes are also required to prevent host A and B from “seeing” the DS4000 ports and instead “see” the SVC ports. 4. LUNs from host partition 0 and 1, which are now in partition 3, must be configured as image mode VDisks in the SVC and mapped to host A and B as they were before the migration, using their original logical unit numbers. 5. Partition 0 and 1 can be deleted if required, after all LUNs are moved to partition 3. Note that LUNs moved from partition 0 and 1 to partition 3 have different logical unit numbers on the DS4300, but the SVC will present the LUNs to the hosts with the same logical unit numbers as before the migration. Note: Access LUN (A) can no longer be used by host A for in-band configuration of the DS4000. This can therefore be deleted and the DS4000 configured to use the SVC Console over the Ethernet. The access LUN is not required by the SVC.
890
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure B-4 Partition 0 and 1 moves to partition 3
We must now move any remaining host storage moved from the host partitions to partition 3 (SVC partition). We use the previous steps to accomplish this. This gives us the configuration shown in Figure B-5. Image mode VDisks can now be converted to managed MDisks using data migration commands as required.
Figure B-5 All storage under SVC
Appendix B. DS4000 and DS8000 migration scenarios
891
Scenario 2: DS4000 total number of LUNs is more than the maximum LUNs per partition In this scenario, it is not possible to use the same solution as in “Scenario 1: DS4000 Total number of LUNs is less than maximum LUNs per partition” on page 888, because we will exceed the number of supported LUNs in one partition. An easy way to do the migration is shown here. However, it is also possible to solve this problem without the need of another DS4000 if there is free capacity, and the sum of the LUNs in the largest partition and the SVC partition do not exceed the maximum number of support LUNs in one partition. In this case, you have to follow the procedure in the previous scenario, but for one partition at a time. You have to move the LUNs from image mode to managed mode disk in the SVC, and the image mode MDisks are ejected from the group automatically. Thereafter, you can move the next partition into the SVC partition, but before you do this, you might have to expand the capacity for the SVC using the free capacity in the DS4000 that has now become free after removing the old LUNs. The initial configuration is shown in Figure B-6. Note the following points: There are more LUNs than the maximum supported in one partition on DS4000-1. The new DS4000 provides new storage larger than or equal to the capacity of DS4000-1. There is only one partition per host.
Figure B-6 Scenario 2 initial setup
We then add another DS4000 and carry out the following steps: 1. Create RAID arrays on the DS4000-2, one LUN per array, using equal numbers of arrays. 2. Rezone the switch to allow SVC ports to access DS4000-2 ports. 3. Create the partition, including all LUNs and SVC ports.
892
Implementing the IBM System Storage SAN Volume Controller V4.3
This is shown in Figure B-7.
Figure B-7 SVC and second DS4000 added
We then move the storage for host C under the control of the SVC. This is shown in Figure B-8.
Figure B-8 Partitions created on DS4000-2
Appendix B. DS4000 and DS8000 migration scenarios
893
We carry out the following steps: 1. Stop host C. 2. Rezone the switch so that host C port accesses the SVC ports as required, not the DS4000-1 ports. 3. Rezone the switch to allow the SVC ports to access DS4000-1 ports. 4. Change host C device drivers, settings, software, and so on, to support the SVC. 5. Change partition 2 to SVC host type and change the port names to SVC ports, removing the ports of host C. 6. Create SVC managed mode disks from storage in partition 0 on DS4000-2. 7. Create SVC image mode disks from storage in partition 2 on DS4000-1. 8. Migrate image mode VDisks for host C to managed disks on DS4000-2. 9. When migration completes, delete LUNs and partition 2 on DS4000-1. Figure B-9 shows the result of this procedure.
Figure B-9 Storage for Host C migrated
We repeat this procedure for the remaining host until all the storage is migrated to be under the control of the SVC. This is shown in Figure B-10 on page 895. DS4000-1 is now unused.
894
Implementing the IBM System Storage SAN Volume Controller V4.3
Note: Although we used a second DS4000 in this scenario, it is possible to carry out a similar procedure if there is enough spare capacity on DS4000-1.
Figure B-10 All storage under control of SVC
Appendix B. DS4000 and DS8000 migration scenarios
895
Scenario 3: Migrating DS8000 Storage to SVC A typical, non-virtualized DS8000 environment is displayed in Figure B-11. We have three hosts attached, and every host is in a separate volume group.
Figure B-11 Initial configuration
Then we add the SVC and create an SVC volume group on the DS8000. See Figure B-12 on page 897. The following steps are required for this task: 1. Modify the zoning so that only the SVC can “see” the DS8000. This allows volume group 4 to be created, and access to volume group 1, 2, and 3 can continue. 2. The port or ports of the SVC Console must not be in any volume group.
896
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure B-12 Add the SVC and SVC volume group
We move the storage for host C from volume group 3 to volume group 4 (SVC partition) to be managed by the SVC. See Figure B-13 on page 898. Note the following points: Concurrent access from host C to its logical units is not possible. Host C requires reconfiguration from storage device drivers to supported levels. Changes to the adapter configuration and microcode levels, settings, and so on, might also be required. Switch zoning changes are also required to prevent host C from “seeing” the DS8000 ports and instead “see” the SVC ports. LUNs from volume group 3 that are now in volume group 4 must be configured as image mode VDisks in the SVC and mapped to host C. Note: Volume group 3 should now be deleted after all logical units are moved to volume group 4.
Appendix B. DS4000 and DS8000 migration scenarios
897
Figure B-13 Move LUNs to the SVC volume group
The next step is to move the storage for host A and B from volume group 1 and 2 to volume group 4 (SVC volume group) to be managed by the SVC. This is shown in Figure B-14 on page 899. The following steps are required to do this: 1. Stop access from host A and B to its logical units. 2. Host A and B require reconfiguration from storage device drivers to supported levels. Changes to adapter configuration and microcode levels, settings, and so on, might also be required. 3. Switch zoning changes are also required to prevent host A and B from “seeing” the DS8000 ports and instead “see” the SVC ports. 4. LUNs from volume group 1and 2, which are now in volume group 4, must be configured as image mode VDisks in the SVC and mapped to host A and B as they were before the migration, using their original logical unit numbers. 5. Volume group 1 and 2 can be deleted if required, after all LUNs are moved to volume group 4. Note that LUNs moved from volume group 1 and 2 to volume group 4 have different logical unit numbers on the DS8000, but the SVC will present the LUNs to the hosts with the same logical unit numbers as before the migration.
898
Implementing the IBM System Storage SAN Volume Controller V4.3
Figure B-14 Remaining storage moves to SVC volume group
We must now move any remaining host storage moved from the host volume groups to volume group 4 (SVC volume group). We use the previous steps to accomplish this. This gives us the configuration shown in Figure B-15. Image mode VDisks can now be converted to managed MDisks using data migration commands as required.
Figure B-15 All storage managed by SVC
Appendix B. DS4000 and DS8000 migration scenarios
899
900
Implementing the IBM System Storage SAN Volume Controller V4.3
C
Appendix C.
Scripting In this appendix, we present a high-level overview of how to automate different tasks by creating scripts using the SVC command-line interface (CLI).
© Copyright IBM Corp. 2003-2008. All rights reserved.
901
Scripting structure When creating scripts to automate tasks on the SVC, use the structure illustrated in Figure C-1.
Create connection (SSH) to the SVC
Run the command(s) command
Scheduled activation or Manual activation
Perform logging
Figure C-1 Scripting structure for SVC task automation
Creating a connection (SSH) to the SVC When creating a connection to the SVC, if you are running the script, you must have access to a private key that corresponds to a public key previously uploaded to the SVC. The private key is used to establish the SSH connection needed to use the CLI on the SVC. If the SSH keypair is generated without a passphrase, you can connect without the need of special scripting to parse in the passphrase. On UNIX systems, the ssh command can be used to create an SSH connection with SVC. On Windows systems, a utility called plink.exe, which is provided with the PuTTY tool, can be used for this. In the following examples, we will use plink to create the SSH connection to the SVC.
Executing the command(s) When using the CLI, you can use the examples in Chapter 9, “SVC configuration and administration using the CLI” on page 303 for inspiration, or refer to the IBM System Storage SAN Volume Controller Command-Line Interface User’s Guide, which can be downloaded from the SVC documentation page for each SVC code-level at: http://www-304.ibm.com/systems/support/supportsite.wss/supportresources?brandind=5 000033&familyind=5329743&taskind=1
Performing logging When using the CLI, not all commands provide a usable response to determine the status of the invoked command. Therefore, we recommend that you always create checks that can be logged for monitoring and troubleshooting purposes.
902
Implementing the IBM System Storage SAN Volume Controller V4.3
Automated VDisk creation In the following example, we create a simple bat script to be used to automate VDisks creation to illustrate how scripts are created. Creating scripts to automate SVC administration tasks is not limited to bat scripting, and you can, in principle, encapsulate the CLI commands in scripts using any programming language you prefer, or even program applets to be used to perform routine tasks.
Connecting to the SVC using a predefined SSH connection The easiest way to create a SSH connection to the SVC is when plink can call a predefined PuTTY session, as shown in Figure C-2 on page 904. Define a session, including: The Auto-login user name, and setting that to your SVC admin user name (for example, admin). This parameter is set under the Connection → Data category. The Private key for authentication (for example, icat.ppk). This is the private key that you have already created. This parameter is set under the Connection → Session → Auth category. The IP address of the SVC cluster. This parameter is set under the Session category. A session name. Our example uses SVC:cluster1. Your version of PuTTY might have these parameters set in different categories.
Appendix C. Scripting
903
Figure C-2 Using a predefined SSH connection with plink
To use this predefined PuTTY session, the syntax is: plink SVC1:cluster1 If a predefined PuTTY session is not used, the syntax is: plink [email protected] -i "C:\DirectoryPath\KeyName.PPK"
Creating VDisks command using the CLI In our example, we decided the following parameters are variables when creating the VDisks: VDisk size (in GB): %1 VDisk name: %2 Managed Disk Group (MDG): %3 Use the following command: svctask mkvdisk -iogrp 0 -vtype striped -size %1 -unit gb -name %2 -mdiskgrp %3
904
Implementing the IBM System Storage SAN Volume Controller V4.3
Listing created VDisks To log the fact that our script created the VDisk we defined when executing the script, we use the -filtervalue parameter as follows: svcinfo lsvdisk -filtervalue 'name=%2' >> C:\DirectoryPath\VDiskScript.log
Invoking the sample script VDiskScript.bat Finally, putting it all together, our sample bat script for creating a VDisk is created, as shown in Figure C-3.
-------------------------------------VDiskScript.bat--------------------------plink SVC1 -l admin svctask mkvdisk -iogrp 0 -vtype striped -size %1 -unit gb -name %2 -mdiskgrp %3 plink SVC1 -l admin svcinfo lsvdisk -filtervalue 'name=%2' >> E:\SVC_Jobs\VDiskScript.log ------------------------------------------------------------------------------Figure C-3 VDiskScript.bat
Using the script, we now create a VDisk with the following parameters: VDisk size (in GB): 20 (%1) VDisk name: Host1_F_Drive (%2) Managed Disk Group (MDG): 1 (%3) This is illustrated in Example C-1. Example: C-1 Executing the script to create the VDisk
E:\SVC_Jobs>VDiskScript 4 Host1_E_Drive 1 E:\SVC_Jobs>plink SVC1:Cluster1 -l admin svctask mkvdisk -iogrp 0 -vtype striped -size 4 -unit gb -name Host1_E_Drive -mdiskgrp 1 Virtual Disk, id [32], successfully created E:\SVC_Jobs>plink SVC1:Cluster1 -l admin svcinfo lsvdisk -filtervalue 'name=Host 1_E_Drive' 1>>E:\SVC_Jobs\VDiskScript.log From the output of the log, as shown in Example C-2, we verify that the VDisk is created as intended. Example: C-2 Logfile output from VDiskScript.bat
id name IO_group_id IO_group_name status mdisk_grp_id mdisk_grp_name capacity type FC_id FC_name RC_id RC_name vdisk_UID fc_map_count copy_count 32 Host1_E_Drive 0 io_grp0 online 1 MDG_DS47 4.0GB striped 60050768018301BF280000000000002E 0 1
Appendix C. Scripting
905
SVC tree Here is another example of using scripting to talk to the SVC. This script displays a tree-like structure for the SVC, as shown in Example C-3. The script has been written in Perl, and should work without modification using Perl on UNIX systems (such as AIX or Linux), Perl for Windows, or Perl in a Windows Cygwin environment. Example: C-3 SVC Tree script output
$ ./svctree.pl 10.0.1.119 admin /cygdrive/c/Keys/icat.ssh + ITSO-CLS2 (10.0.1.119) + CONTROLLERS + DS4500 (0) + mdisk0 (ID: 0 CAP: 36.0GB MODE: managed) + mdisk1 (ID: 1 CAP: 36.0GB MODE: managed) + Kanaga_AIX (ID: 24 CAP: 5.0GB MODE: managed) + Kanaga_AIX1 (ID: 25 CAP: 8.0GB MODE: managed) + mdisk2 (ID: 2 CAP: 36.0GB MODE: managed) + mdisk_3 (ID: 3 CAP: 36.0GB MODE: managed) + DS4700 (1) + mdisk0 (ID: 0 CAP: 36.0GB MODE: managed) + mdisk1 (ID: 1 CAP: 36.0GB MODE: managed) + Kanaga_AIX (ID: 24 CAP: 5.0GB MODE: managed) + Kanaga_AIX1 (ID: 25 CAP: 8.0GB MODE: managed) + mdisk2 (ID: 2 CAP: 36.0GB MODE: managed) + mdisk_3 (ID: 3 CAP: 36.0GB MODE: managed) + MDISK GROUPS + MDG_0_DS45 (ID: 0 CAP: 144.0GB FREE: 120.0GB) + mdisk0 (ID: 0 CAP: 36.0GB MODE: managed) + mdisk1 (ID: 1 CAP: 36.0GB MODE: managed) + mdisk2 (ID: 2 CAP: 36.0GB MODE: managed) + mdisk_3 (ID: 3 CAP: 36.0GB MODE: managed) + aix_imgmdg (ID: 7 CAP: 13.0GB FREE: 3.0GB) + Kanaga_AIX (ID: 24 CAP: 5.0GB MODE: managed) + Kanaga_AIX1 (ID: 25 CAP: 8.0GB MODE: managed) + iogrp0 (0) + NODES + Node2 (5) + Node1 (2) + HOSTS + W2k8 (0) + Senegal (1) + VSS_FREE (2) + msvc0001 (ID: 10 CAP: 12.0GB TYPE: striped STAT: online) + msvc0002 (ID: 11 CAP: 12.0GB TYPE: striped STAT: online) + VSS_RESERVED (3) + Kanaga (5) + A_Kanaga_VD_IM1 (ID: 9 CAP: 10.0GB TYPE: many STAT: online) + VDISKS + MDG_SE_VDisk3 (ID: 0 CAP: 10.2GB TYPE: many) + mdisk2 (ID: 10 CAP: 36.0GB MODE: managed CONT: DS4500) + mdisk_3 (ID: 12 CAP: 36.0GB MODE: managed CONT: DS4500) + A_Kanaga_VD_IM1 (ID: 9 CAP: 10.0GB TYPE: many) + Kanaga_AIX (ID: 24 CAP: 5.0GB MODE: managed CONT: DS4700) + Kanaga_AIX1 (ID: 24 CAP: 8.0GB MODE: managed CONT: DS4700) 906
Implementing the IBM System Storage SAN Volume Controller V4.3
+
+
+
+
+
+ msvc0001 (ID: 10 CAP: 12.0GB TYPE: striped) + mdisk0 (ID: 8 CAP: 36.0GB MODE: managed CONT: + mdisk1 (ID: 9 CAP: 36.0GB MODE: managed CONT: + msvc0002 (ID: 11 CAP: 12.0GB TYPE: striped) + mdisk0 (ID: 8 CAP: 36.0GB MODE: managed CONT: + mdisk1 (ID: 9 CAP: 36.0GB MODE: managed CONT: iogrp1 (1) + NODES + HOSTS + VDISKS iogrp2 (2) + NODES + HOSTS + VDISKS iogrp3 (3) + NODES + HOSTS + VDISKS recovery_io_grp (4) + NODES + HOSTS + VDISKS recovery_io_grp (4) + NODES + HOSTS + itsosvc1 (2200642269468) + VDISKS
DS4500) DS4500) DS4500) DS4500)
Example C-4 shows the coding for our script. Example: C-4 svctree.pl
#!/usr/bin/perl $SSHCLIENT = “ssh”; # (plink or ssh) $HOST = $ARGV[0]; $USER = ($ARGV[1] ? $ARGV[1] : “admin”); $PRIVATEKEY = ($ARGV[2] ? $ARGV[2] : “/path/toprivatekey”); $DEBUG = 0; die(sprintf(“Please call script with cluster IP address. The syntax is: \n%s ipaddress loginname privatekey\n”,$0)) if (! $HOST); sub TalkToSVC() { my $COMMAND = shift; my $NODELIM = shift; my $ARGUMENT = shift; my @info; if ($SSHCLIENT eq “plink” || $SSHCLIENT eq “ssh”) { $SSH = sprintf(‘%s -i %s %s@%s ‘,$SSHCLIENT,$PRIVATEKEY,$USER,$HOST); } else { die (“ERROR: Unknown SSHCLIENT [$SSHCLIENT]\n”); Appendix C. Scripting
907
} if ($NODELIM) { $CMD = “$SSH svcinfo $COMMAND $ARGUMENT\n”; } else { $CMD = “$SSH svcinfo $COMMAND -delim : $ARGUMENT\n”; } print “Running $CMD” if ($DEBUG); open SVC,”$CMD|”; while (<SVC>) { print “Got [$_]\n” if ($DEBUG); chomp; push(@info,$_); } close SVC; return @info; } sub DelimToHash() { my $COMMAND = shift; my $MULTILINE = shift; my $NODELIM = shift; my $ARGUMENT = shift; my %hash; @details = &TalkToSVC($COMMAND,$NODELIM,$ARGUMENT); print “$COMMAND: Got [“,join(‘|’,@details).”]\n” if ($DEBUG); my $linenum = 0; foreach (@details) { print “$linenum, $_” if ($debug); if ($linenum == 0) { @heading = split(‘:’,$_); } else { @line = split(‘:’,$_); $counter = 0; foreach $id (@heading) { printf(“$COMMAND: ID [%s], value [%s]\n”,$id,$line[$counter]) if ($DEBUG); if ($MULTILINE) { $hash{$linenum,$id} = $line[$counter++]; } else { $hash{$id} = $line[$counter++]; } } } $linenum++; }
908
Implementing the IBM System Storage SAN Volume Controller V4.3
return %hash; } sub TreeLine() { my $indent = shift; my $line = shift; my $last = shift; for ($tab=1;$tab<=$indent;$tab++) { print “ “; } if (! $last) { print “+ $line\n”; } else { print “| $line\n”; } } sub TreeData() { my $indent = shift; my $printline = shift; *data = shift; *list = shift; *condition = shift; my $item; foreach $item (sort keys %data) { @show = (); ($numitem,$detail) = split($;,$item); next if ($numitem == $lastnumitem); $lastnumitem = $numitem; printf(“CONDITION:SRC [%s], DST [%s], DSTVAL [%s]\n”,$condition{“SRC”},$condition{“DST”},$data{$numitem,$condition{“DST”}}) if ($DEBUG); next if (($condition{“SRC”} && $condition{“DST”}) && ($condition{“SRC”} != $data{$numitem,$condition{“DST”}})); foreach (@list) { push(@show,$data{$numitem,$_}) } &TreeLine($indent,sprintf($printline,@show),0); } } # Gather our cluster information. %clusters = &DelimToHash(‘lscluster’,1); %iogrps = &DelimToHash(‘lsiogrp’,1); %nodes = &DelimToHash(‘lsnode’,1); %hosts = &DelimToHash(‘lshost’,1); %vdisks = &DelimToHash(‘lsvdisk’,1); %mdisks = &DelimToHash(‘lsmdisk’,1); %controllers = &DelimToHash(‘lscontroller’,1);
Appendix C. Scripting
909
%mdiskgrps = &DelimToHash(‘lsmdiskgrp’,1); # We are now ready to display it. # CLUSTER $indent = 0; foreach $cluster (sort keys %clusters) { ($numcluster,$detail) = split($;,$cluster); next if ($numcluster == $lastnumcluster); $lastnumcluster = $numcluster; next if ("$clusters{$numcluster,'location'}" =~ /remote/); &TreeLine($indent,sprintf(‘%s (%s)’,$clusters{$numcluster,’name’},$clusters{$numcluster,’cluster_IP_address’}),0 ); # CONTROLLERS &TreeLine($indentiogrp+1,’CONTROLLERS’,0); $lastnumcontroller = ““; foreach $controller (sort keys %controllers) { $indentcontroller = $indent+2; ($numcontroller,$detail) = split($;,$controller); next if ($numcontroller == $lastnumcontroller); $lastnumcontroller = $numcontroller; &TreeLine($indentcontroller, sprintf(‘%s (%s)’, $controllers{$numcontroller,’controller_name’}, $controllers{$numcontroller,’id’}) ,0); # MDISKS &TreeData($indentcontroller+1, ‘%s (ID: %s CAP: %s MODE: %s)’, *mdisks, [‘name’,’id’,’capacity’,’mode’], {“SRC”=>$controllers{$numcontroller,’controller_name’},”DST”=>”controller_name”}); } # MDISKGRPS &TreeLine($indentiogrp+1,’MDISK GROUPS’,0,[]); $lastnummdiskgrp = ““; foreach $mdiskgrp (sort keys %mdiskgrps) { $indentmdiskgrp = $indent+2; ($nummdiskgrp,$detail) = split($;,$mdiskgrp); next if ($nummdiskgrp == $lastnummdiskgrp); $lastnummdiskgrp = $nummdiskgrp; &TreeLine($indentmdiskgrp, sprintf(‘%s (ID: %s CAP: %s FREE: %s)’, $mdiskgrps{$nummdiskgrp,’name’}, $mdiskgrps{$nummdiskgrp,’id’}, $mdiskgrps{$nummdiskgrp,’capacity’},
910
Implementing the IBM System Storage SAN Volume Controller V4.3
$mdiskgrps{$nummdiskgrp,’free_capacity’}) ,0); # MDISKS &TreeData($indentcontroller+1, ‘%s (ID: %s CAP: %s MODE: %s)’, *mdisks, [‘name’,’id’,’capacity’,’mode’], {“SRC”=>$mdiskgrps{$nummdiskgrp,’id’},”DST”=>”mdisk_grp_id”}); } # IOGROUP $lastnumiogrp = ““; foreach $iogrp (sort keys %iogrps) { $indentiogrp = $indent+1; ($numiogrp,$detail) = split($;,$iogrp); next if ($numiogrp == $lastnumiogrp); $lastnumiogrp = $numiogrp; &TreeLine($indentiogrp,sprintf(‘%s (%s)’,$iogrps{$numiogrp,’name’},$iogrps{$numiogrp,’id’}),0); $indentiogrp++; # NODES &TreeLine($indentiogrp,’NODES’,0); &TreeData($indentiogrp+1, ‘%s (%s)’, *nodes, [‘name’,’id’], {“SRC”=>$iogrps{$numiogrp,’id’},”DST”=>”IO_group_id”}); # HOSTS &TreeLine($indentiogrp,’HOSTS’,0); $lastnumhost = ““; %iogrphosts = &DelimToHash(‘lsiogrphost’,1,0,$iogrps{$numiogrp,’id’}); foreach $host (sort keys %iogrphosts) { my $indenthost = $indentiogrp+1; ($numhost,$detail) = split($;,$host); next if ($numhost == $lastnumhost); $lastnumhost = $numhost; &TreeLine($indenthost, sprintf(‘%s (%s)’,$iogrphosts{$numhost,’name’},$iogrphosts{$numhost,’id’}), 0); # HOSTVDISKMAP %vdiskhostmap = &DelimToHash(‘lshostvdiskmap’,1,0,$hosts{$numhost,’id’}); $lastnumvdisk = ““; foreach $vdiskhost (sort keys %vdiskhostmap) { ($numvdisk,$detail) = split($;,$vdiskhost);
Appendix C. Scripting
911
next if ($numvdisk == $lastnumvdisk); $lastnumvdisk = $numvdisk; next if ($vdisks{$numvdisk,’IO_group_id’} != $iogrps{$numiogrp,’id’}); &TreeData($indenthost+1, ‘%s (ID: %s CAP: %s TYPE: %s STAT: %s)’, *vdisks, [‘name’,’id’,’capacity’,’type’,’status’], {“SRC”=>$vdiskhostmap{$numvdisk,’vdisk_id’},”DST”=>”id”}); } } # VDISKS &TreeLine($indentiogrp,’VDISKS’,0); $lastnumvdisk = ““; foreach $vdisk (sort keys %vdisks) { my $indentvdisk = $indentiogrp+1; ($numvdisk,$detail) = split($;,$vdisk); next if ($numvdisk == $lastnumvdisk); $lastnumvdisk = $numvdisk; &TreeLine($indentvdisk, sprintf(‘%s (ID: %s CAP: %s TYPE: %s)’, $vdisks{$numvdisk,’name’}, $vdisks{$numvdisk,’id’}, $vdisks{$numvdisk,’capacity’}, $vdisks{$numvdisk,’type’}), 0) if ($iogrps{$numiogrp,’id’} == $vdisks{$numvdisk,’IO_group_id’}); # VDISKMEMBERS if ($iogrps{$numiogrp,’id’} == $vdisks{$numvdisk,’IO_group_id’}) { %vdiskmembers = &DelimToHash(‘lsvdiskmember’,1,1,$vdisks{$numvdisk,’id’}); foreach $vdiskmember (sort keys %vdiskmembers) { &TreeData($indentvdisk+1, ‘%s (ID: %s CAP: %s MODE: %s CONT: %s)’, *mdisks, [‘name’,’id’,’capacity’,’mode’,’controller_name’], {“SRC”=>$vdiskmembers{$vdiskmember},”DST”=>”id”}); } } } } }
912
Implementing the IBM System Storage SAN Volume Controller V4.3
Scripting alternatives For an alternative to scripting, visit the Tivoli Storage Manager for Advanced Copy Services product page: http://www.ibm.com/software/tivoli/products/storage-mgr-advanced-copy-services/ Additionally, IBM provides a suite of scripting tools based on Perl. These can be downloaded from: http://www.alphaworks.ibm.com/tech/svctools
Appendix C. Scripting
913
914
Implementing the IBM System Storage SAN Volume Controller V4.3
D
Appendix D.
Node replacement In this appendix, we discuss the process to replace nodes. For the latest information about replacing a node, refer to the development page at one of the following sites: IBMers: http://w3-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/TD104437 Business Partners (login required): http://partners.boulder.ibm.com/src/atsmastr.nsf/WebIndex/TD104437 Clients: http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/TD104437
© Copyright IBM Corp. 2003-2008. All rights reserved.
915
Replacing nodes nondisruptively You can replace SAN Volume Controller 2145-4F2, SAN Volume Controller 2145-8F2, or SAN Volume Controller 2145-8F4 nodes with SAN Volume Controller 2145-8G4 nodes in an existing, active cluster without having an outage on the SVC or on your host applications. This procedure does not require that you change your SAN environment because the replacement (new) node uses the same worldwide node name (WWNN) as the node you are replacing. In fact, you can use this procedure to replace any model node with a different model node. This task assumes that the following conditions exist: The cluster software is at V4.2.0 or higher for older to newer model node replacements, the exception being the 2145-8G4 model node, which requires the cluster to be running V4.2.0 or higher. The new nodes that are configured are not powered on and not connected. All nodes that are configured in the cluster are present. All errors in the cluster error log are fixed. There are no VDisks, MDisks, or controllers with a status of degraded or offline. The SVC configuration has been backed up through the CLI or GUI and the file saved to the master console. Download, install, and run the latest “SVC Software Upgrade Test Utility” from http://www-1.ibm.com/support/docview.wss?rs=591&uid=ssg1S4000585 to verify there are no known issues with the current cluster environment before beginning the node upgrade procedure. You have a 2145 uninterruptible power supply-1U (2145 UPS-1U) unit for each new SAN Volume Controller 2145-8G4 node. Note: If you are planning to redeploy the old nodes in your environment to create a test cluster or to add to another cluster, you must ensure the WWNN of these old nodes are set to a unique number on your SAN. The recommendation is to document the factory WWNN of the new nodes you are using to replace the old nodes and in effect swap the WWNN so each node still has a unique number. Failure to do this could lead to a duplicate WWNN and WWPN causing unpredictable SAN problems. Perform the following steps to replace the nodes: 1. Perform the following steps to determine the node_name or node_id of the node you want to replace, the iogroup_id or iogroup_name it belongs to, and to determine which of the nodes is the configuration node. If the configuration node is to be replaced, it is recommended that it be upgraded last. If you already can identify which physical node equates to a node_name or node_id, the iogroup_id or iogroup_name it belongs to, and which node is the configuration node, then you can skip this step and proceed to step 2 below. a. Issue the following command from the command-line interface (CLI): svcinfo lsnode -delim : b. Under the column “config node”, look for the status of “yes” and record the node_name or node_id of this node for later use. c. Under the columns “id” and “name”, record the node_name or node_id of all the other nodes in the cluster. d. Under the columns “IO_group_id” and “IO_group_name”, record the iogroup_id or iogroup_name for all the nodes in the cluster. 916
Implementing the IBM System Storage SAN Volume Controller V4.3
e. Issue the following command from the CLI for each node_name or node_id to determine the front_panel_id for each node and record the ID. This front_panel_id is physically located on the front of every node (it is not the serial number) and you can use this to determine which physical node equates to the node_name or node_id you plan to replace: svcinfo lsnodevpd node_name or node_id 2. Perform the following steps to record the WWNN of the node that you want to replace: a. Issue the following command from the CLI, where node_name or node_id is the name or ID of the node for which you want to determine the WWNN: svcinfo lsnode -delim : node_name or node_id b. Record the WWNN of the node that you want to replace. 3. Verify that all VDisks, MDisks, and disk controllers are online and none are in a state of “Degraded”. If there are any in this state, then resolve this issue before going forward or loss of access to data may occur when you perform step 4 below. This is an especially important step if this is the second node in the I/O group to be replaced. a. Issue the following commands from the CLI, where object_id or object_name is the controller ID or controller name that you want to view. Verify that each disk controller shows its status as “degraded no”. svcinfo lsvdisk -filtervalue “status=degraded” svcinfo lsmdisk -filtervalue “status=degraded” svcinfo lscontroller object_id or object_name 4. Issue the following CLI command to shut down the node that will be replaced, where node_name or node_id is the name or ID of the node that you want to delete: svctask stopcluster -node node_name or node_id Attention: Do not power off the node through the front panel in lieu of using the above command. Be careful you do not issue the stopcluster command without the -node node_name or node_id parameters, as the entire cluster will be shut down if you do. Issue the following CLI command to ensure that the node is shut down and the status is “offline”, where node_name or node_id is the name or ID of the original node. The node status should be “offline”: svcinfo lsnode node_name or node_id 5. Issue the following CLI command to delete this node from the cluster and I/O group, where node_name or node_id is the name or ID of the node that you want to delete: svctask rmnode node_name or node_id 6. Issue the following CLI command to ensure that the node is no longer a member of the cluster, where node_name or node_id is the name or ID of the original node. The node should not be listed in the command output: svcinfo lsnode node_name or node_id
Appendix D. Node replacement
917
7. Perform the following steps to change the WWNN of the node that you just deleted to FFFFF: Attention: Record and mark the Fibre Channel cables with the SVC node port number (1-4) before removing them from the back of the node being replaced. You must reconnect the cables on the new node exactly as they were on the old node. Looking at the back of the node, the Fibre Channel ports on the SVC nodes are numbered 1-4 from left to right and must be reconnected in the same order or the port IDs will change, which could impact hosts’ access to VDisks or cause problems with adding the new node back into the cluster. The SVC Hardware Installation Guide shows the port numbering of the various node models. Failure to disconnect the fibre cables now will likely cause SAN devices and SAN management software to discover these new WWPNs generated when the WWNN is changed to FFFFF in the following steps. This may cause ghost records to be seen once the node is powered down. These do not necessarily cause a problem, but may require a reboot of a SAN device to clear out the record. In addition, it may cause problems with AIX dynamic tracking functioning correctly, assuming it is enabled, so we highly recommend disconnecting the node’s fibre cables as instructed in step a below before continuing on to any other steps. a. Disconnect the four Fibre Channel cables from this node before powering the node on in the next step. b. Power on this node using the power button on the front panel and wait for it to boot up before going to the next step. c. From the front panel of the node, press the down button until the “Node:” panel is displayed and then use the right and left navigation buttons to display the “Status:” panel. d. Press and hold the down button, press and release the select button, and then release the down button. The WWNN of the node is displayed. e. Press and hold the down button, press and release the select button, and then release the down button to enter the WWNN edit mode. The first character of the WWNN is highlighted. f. Press the up or down button to increment or decrement the character that is displayed. Note: The characters wrap F to 0 or 0 to F. g. Press the left navigation button to move to the next field or the right navigation button to return to the previous field and repeat step f for each field. At the end of this step, the characters that are displayed must be FFFFF. h. Press the select button to retain the characters that you have updated and return to the WWNN screen. i. Press the select button again to apply the characters as the new WWNN for the node. Note: You must press the select button twice as steps h and i instruct you to do. After step h, it may appear that the WWNN has been changed, but step i actually applies the change. 8. Power off this node using the power button on the front panel and remove the node from the rack if desired. 918
Implementing the IBM System Storage SAN Volume Controller V4.3
9. Install the replacement node and its UPS in the rack and connect the node to UPS cables according to the SVC Hardware Installation Guide available at: http://www.ibm.com/storage/support/2145 Note: Do not connect the Fibre Channel cables to the new node during this step. 10.Power on the replacement node from the front panel with the Fibre Channel cables disconnected. Once the node has booted, ensure the node displays “Cluster:” on the front panel and nothing else. If something other then this is displayed, contact IBM support for assistance before continuing. 11.Record the WWNN of this new node, as you will need it if you plan to redeploy the old nodes being replaced. Perform the following steps to change the WWNN of the replacement node to match the WWNN that you recorded in step 2 on page 917: a. From the front panel of the node, press the down button until the “Node:” panel is displayed and then use the right and left navigation buttons to display the “Status:” panel. b. Press and hold the down button, press and release the select button, and then release the down button. The WWNN of the node is displayed. Record this number for use in redeployment of the old nodes. c. Press and hold the down button, press and release the select button, and then release the down button to enter the WWNN edit mode. The first character of the WWNN is highlighted. d. Press the up or down button to increment or decrement the character that is displayed. e. Press the left navigation button to move to the next field or the right navigation button to return to the previous field and repeat step d for each field. At the end of this step, the characters that are displayed must be the same as the WWNN you recorded in step 2 on page 917. f. Press the select button to retain the characters that you have updated and return to the WWNN panel. g. Press the select button to apply the characters as the new WWNN for the node. Note: You must press the select button twice as steps f and g instruct you to do. After step f, it may appear that the WWNN has been changed, but step g actually applies the change. h. The node should display “Cluster:” on the front panel and is now ready to begin the process of adding the node to the cluster. If something other then this is displayed, contact IBM support for assistance before continuing. 12.Connect the Fibre Channel cables to the same port numbers on the new node as they were originally on the old node. See step 7 on page 918. Note: Do not connect the new nodes to different ports at the switch or director, as this will cause port IDs to change, which could impact hosts’ access to VDisks or cause problems with adding the new node back into the cluster. The new nodes have 4 Gbps HBAs in them, and the temptation is to move them to 4 Gbps switch/director ports at the same time, but this is not recommended while doing the hardware node upgrade. Moving the node cables to faster ports on the switch/director is a separate process that needs to be planned independently of upgrading the nodes in the cluster.
Appendix D. Node replacement
919
13.Issue the following CLI command to verify that the last five characters of the WWNN are correct: svcinfo lsnodecandidate Note: If the WWNN does not match the original node’s WWNN exactly as recorded in step 2 on page 917, you must repeat step 11 on page 919. 14.Add the node to the cluster and ensure it is added back to the same I/O group as the original node. Using the following command, where wwnn_arg and iogroup_name or iogroup_id are the items you recorded in steps 1 on page 916 and 2 on page 917. svctask addnode -wwnodename wwnn_arg -iogrp iogroup_name or iogroup_id 15.Verify that all the VDisks for this I/O group are back online and are no longer degraded. If the node replacement process is being done disruptively, such that no I/O is occurring to the I/O group, you still need to wait some period of time (we recommend 30 minutes in this case too) to make sure the new node is back online and available to take over before you do the next node in the I/O group. See step 3 on page 917. Both nodes in the I/O group cache data; however, the cache sizes are asymmetric if the remaining partner node in the I/O group is a SAN Volume Controller 2145-4F2 node. The replacement node is limited by the cache size of the partner node in the I/O group in this case. Therefore, the replacement node does not utilize the full 8 GB cache size until the other 2145-4F2 node in the I/O group is replaced. You do not have to reconfigure the host multipathing device drivers because the replacement node uses the same WWNN and WWPNs as the previous node. The multipathing device drivers should detect the recovery of paths that are available to the replacement node. The host multipathing device drivers take approximately 30 minutes to recover the paths. Therefore, do not upgrade the other node in the I/O group for at least 30 minutes after successfully upgrading the first node in the I/O group. If you have other nodes in other I/O groups to upgrade, you can perform that upgrade while you wait the 30 minutes noted above. 16.Repeat steps 2 on page 917 to 15 for each node you want to replace.
Expanding an existing SVC cluster In this section, we describe how to expand an existing SVC cluster with new nodes. An SVC cluster can only be expanded with node pairs, which means you always have to add at least two nodes to your existing cluster. The maximum number of nodes is eight. This task assumes the following situation: Your cluster contains six or less nodes. All nodes that are configured in the cluster are present. All errors in the cluster error log are fixed. All managed disks (MDisks) are online. You have a 2145 uninterruptible power supply-1U (2145 UPS-1U) unit for each new SAN Volume Controller 2145-8G4 node. There are no VDisks, MDisks, or controllers with a status of degraded or offline. The SVC configuration has been backed up through the CLI or GUI and the file saved to the master console.
920
Implementing the IBM System Storage SAN Volume Controller V4.3
Download, install, and run the latest “SVC Software Upgrade Test Utility” from http://www-1.ibm.com/support/docview.wss?rs=591&uid=ssg1S4000585 to verify there are no known issues with the current cluster environment before beginning the node upgrade procedure. Perform the following steps to add nodes to an existing cluster: 1. Depending on the model of node being added, it may be necessary to upgrade the existing SVC cluster software to a level that supports the hardware model: – The model 2145-8G4 requires Version 4.2.x or later. – The model 2145-8F4 requires Version 4.1.x or later. – The model 2145-8F2 requires Version 3.1.x or later. – The 2145-4F2 is the original model and thus is supported by Version 1 through Version 4. It is highly recommended that the existing cluster be upgraded to the latest level of SVC software available; however, the minimum level of SVC cluster software recommended for the 4F2 is Version 3.1.0.5. 2. Install additional nodes and UPSs in a rack. Do not connect to the SAN at this time. 3. Ensure each node being added has a unique WWNN. Duplicate WWNNs may cause serious problems on a SAN and must be avoided. Below is an example of how this could occur: The nodes came from cluster ABC where they were replaced by brand new nodes. The procedure to replace these nodes in cluster ABC required each brand new node’s WWNN to be changed to the old nodes’ WWNN. Adding these nodes now to the same SAN will cause duplicate WWNNs to appear with unpredictable results. You will need to power up each node separately while disconnected from the SAN and use the front panel to view the current WWNN. If necessary, change it to something unique on the SAN. If required, contact IBM Support for assistance before continuing. 4. Power up additional UPSs and nodes. Do not connect to the SAN at this time. 5. Ensure each node displays “Cluster:” on the front panel and nothing else. If something other then this is displayed, contact IBM Support for assistance before continuing. 6. Connect additional nodes to LAN. 7. Connect additional nodes to SAN fabric(s). Attention: Do not add the additional nodes to the existing cluster before the zoning and masking steps below are completed or SVC will enter a degraded mode and log errors with unpredictable results. 8. Zone additional node ports in the existing SVC only zone(s). There should be a SVC zone in each fabric with nothing but the ports from the SVC nodes in it. These zones are needed for initial formation of the cluster, as nodes need to see each other to form a cluster. This zone may not exist and the only way the SVC nodes see each other is through a storage zone that includes all the node ports. However, it is highly recommended to have a separate zone in each fabric with just the SVC node ports included to avoid the possibility of the nodes losing communication with each other if the storage zone(s) are changed or deleted. 9. Zone new node ports in existing SVC/Storage zone(s). There should be a SVC/Storage zone in each fabric for each disk subsystem used with SVC. Each zone should have all the SVC ports in that fabric along with all the disk subsystem ports in that fabric that will be used by SVC to access the physical disks.
Appendix D. Node replacement
921
Note: There are exceptions when EMC DMX/Symmetrix or HDS storage is involved. For further information, review the SVC Software Installation and Configuration Guide, available at: http://www.ibm.com/storage/support/2145 10.On each disk subsystem seen by the SVC, use its management interface to map LUNs that are currently used by SVC to all the new WWPNs of the new nodes that will be added to the SVC cluster. This is a critical step, as the new nodes must see the same LUNs as the existing SVC cluster nodes see before adding the new nodes to the cluster, otherwise problems may arise. Also note that all SVC ports zoned with the back-end storage must see all the LUNs presented to SVC through all those same storage ports or SVC will mark the devices as degraded. 11.Once all the above is done, then you can add the additional nodes to the cluster using the SVC GUI or CLI and the cluster should not mark anything degraded, as the new nodes will see the same cluster configuration, the same storage zoning, and the same LUNs as the existing nodes. 12.Check the status of the controller(s) and MDisks to ensure there is nothing marked degraded. If there is, then something is not configured properly, and this needs to be addressed immediately before doing anything else to the cluster. If it cannot be determined fairly quickly what is wrong, remove the newly added nodes from the cluster until the problem is resolved. You can contact IBM Support for assistance.
Moving VDisks to a new I/O group Once new nodes are added to a cluster, you may want to move VDisk ownership from one I/O group to another to balance the workload. This is currently a disruptive process. The host applications will have to be quiesced during the process. The actual moving of the VDisk in SVC is simple and quick; however, some host operating systems may need to have their file systems and volume groups varied off or removed along with their disks and multiple paths to the VDisks deleted and rediscovered. In effect, it would be the equivalent of discovering the VDisks again as when they were initially brought under SVC control. This is not a difficult process, but can take some time to complete, so you must plan accordingly. This task assumes the following situation: All steps described in “Expanding an existing SVC cluster” on page 920 are completed. All nodes that are configured in the cluster are present. All errors in the cluster error log are fixed. All managed disks (MDisks) are online. There are no VDisks, MDisks, or controllers with a status of degraded or offline. The SVC configuration has been backed up through the CLI or GUI and the file saved to the master console. Perform the following steps to move the VDisks: 1. Stop the host I/O. 2. Vary off your file system or shut down your host, depending on your operating system. 3. Move all of the VDisks from the I/O group of the nodes you are replacing to the new I/O group.
922
Implementing the IBM System Storage SAN Volume Controller V4.3
4. If you had your host shut down, start it again. 5. From each host, issue a rescan of the multipathing software to discover the new paths to the VDisks. 6. See the documentation that is provided with your multipathing device driver for information about how to query paths to ensure that all paths have been recovered. 7. Vary on your file system. 8. Restart the host I/O. 9. Repeat steps 1 to 8 for each vdisk in the cluster that you want to replace.
Replacing nodes disruptively (rezoning the SAN) You can replace SAN Volume Controller 2145-4F2, SAN Volume Controller 2145-8F2, or SAN Volume Controller 2145-8F4 nodes with SAN Volume Controller 2145-8G4 nodes. This task disrupts your environment because you must rezone your SAN and the host multipathing device drivers must discover new paths. Access to virtual disks (VDisks) is lost during this task. In fact, you can use this procedure to replace any model node with different model node. This task assumes that the following conditions exist: The cluster software is at V4.2.0 or higher. All nodes that are configured in the cluster are present. The new nodes that are configured are not powered on and not connected. All errors in the cluster error log are fixed. All managed disks (MDisks) are online. You have a 2145 uninterruptible power supply-1U (2145 UPS-1U) unit for each new SAN Volume Controller 2145-8G4 node. There are no VDisks, MDisks, or controllers with a status of degraded or offline. The SVC configuration has been backed up through the CLI or GUI and the file saved to the master console. Download, install, and run the latest “SVC Software Upgrade Test Utility” from http://www-1.ibm.com/support/docview.wss?rs=591&uid=ssg1S4000585 to verify there are no known issues with the current cluster environment before beginning the node upgrade procedure. Perform the following steps to replace nodes: 1. Quiesce all I/O from the hosts that access the I/O group of the node that you are replacing. 2. Delete the node that you want to replace from the cluster and I/O group. Note: The node is not deleted until the SAN Volume Controller cache is destaged to disk. During this time, the partner node in the I/O group transitions to write through mode. You can use the command-line interface (CLI) or the SAN Volume Controller Console to verify that the deletion process has completed. 3. Ensure that the node is no longer a member of the cluster. 4. Power off the node and remove it from the rack.
Appendix D. Node replacement
923
5. Install the replacement (new) node in the rack and connect the uninterruptible power supply (UPS) cables and the Fibre Channel cables. 6. Power on the node. 7. Rezone your switch zones to remove the ports of the node that you are replacing from the host and storage zones. Replace these ports with the ports of the replacement node. 8. Add the replacement node to the cluster and I/O group. Important: Both nodes in the I/O group cache data; however, the cache sizes are asymmetric. The replacement node is limited by the cache size of the partner node in the I/O group. Therefore, the replacement node does not utilize the full size of its cache. 9. From each host, issue a rescan of the multipathing software to discover the new paths to VDisks. Note: If your system is inactive, you can perform this step after you have replaced all nodes in the cluster. The host multipathing device drivers take approximately 30 minutes to recover the paths. 10.Refer to the documentation that is provided with your multipathing device driver for information about how to query paths to ensure that all paths have been recovered before proceeding to the next step. 11.Repeat steps 1 to 10 for the partner node in the I/O group. Note: After you have upgraded both nodes in the I/O group, the cache sizes are symmetric and the full 8 GB of cache is utilized. 12.Repeat steps 1 to 11 for each node in the cluster that you want to replace. 13.Resume host I/O.
924
Implementing the IBM System Storage SAN Volume Controller V4.3
Related publications The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this book.
IBM Redbooks For information about ordering these publications, see “How to get Redbooks” on page 926. Note that some of the documents referenced here may be available in softcopy only. Get More Out of Your SAN with IBM Tivoli Storage Manager, SG24-6687 IBM System Storage Business Continuity: Part 1 Planning Guide, SG24-6547 IBM System Storage Business Continuity: Part 2 Solutions Guide, SG24-6548 IBM System Storage SAN Volume Controller, SG24-6423 IBM Tivoli Storage Area Network Manager: A Practical Introduction, SG24-6848 Implementing an IBM/Brocade SAN with 8 Gbps Directors and Switches, SG24-6116 Introduction to Storage Area Networks, SG24-5470 SAN Volume Controller: Best Practices and Performance Guidelines, SG24-7521 Using the SVC for Business Continuity, SG24-7371
Other resources These publications are also relevant as further information sources: IBM System Storage Master Console: Installation and User’s Guide, GC30-4090 IBM System Storage Open Software Family SAN Volume Controller: CIM Agent Developers Reference, SC26-7545 IBM System Storage Open Software Family SAN Volume Controller: Command-Line Interface User's Guide, SC26-7544 IBM System Storage Open Software Family SAN Volume Controller: Configuration Guide, SC26-7543 IBM System Storage Open Software Family SAN Volume Controller: Host Attachment Guide, SC26-7563 IBM System Storage Open Software Family SAN Volume Controller: Installation Guide, SC26-7541 IBM System Storage Open Software Family SAN Volume Controller: Planning Guide, GA22-1052 IBM System Storage Open Software Family SAN Volume Controller: Service Guide, SC26-7542 IBM TotalStorage Multipath Subsystem Device Driver User's Guide, SC30-4096
© Copyright IBM Corp. 2003-2008. All rights reserved.
925
Referenced Web sites These Web sites are also relevant as further information sources: Cygwin Linux-like environment for Windows: http://www.cygwin.com Download site for Windows SSH freeware: http://www.chiark.greenend.org.uk/~sgtatham/putty IBM site to download SSH for AIX: http://oss.software.ibm.com/developerworks/projects/openssh IBM Tivoli Storage Area Network Manager site: http://www-306.ibm.com/software/sysmgmt/products/support/IBMTivoliStorageAreaNe tworkManager.html IBM TotalStorage home page: http://www.storage.ibm.com IBM TotalStorage Virtualization Home Page: http://www-1.ibm.com/servers/storage/software/virtualization/index.html Microsoft Knowledge Base Article 131658: http://support.microsoft.com/support/kb/articles/Q131/6/58.asp Microsoft Knowledge Base Article 149927: http://support.microsoft.com/support/kb/articles/Q149/9/27.asp Open source site for SSH for Windows and Mac: http://www.openssh.com/windows.html SAN Volume Controller supported platform: http://www-1.ibm.com/servers/storage/support/software/sanvc/index.html Sysinternals home page: http://www.sysinternals.com Subsystem Device Driver download site: http://www-1.ibm.com/servers/storage/support/software/sdd/index.html
How to get Redbooks You can search for, view, or download Redbooks, Redpapers, Technotes, draft publications and Additional materials, as well as order hardcopy Redbooks, at this Web site: ibm.com/redbooks
926
Implementing the IBM System Storage SAN Volume Controller V4.3
Help from IBM IBM Support and downloads ibm.com/support IBM Global Services ibm.com/services
Related publications
927
928
Implementing the IBM System Storage SAN Volume Controller V4.3
Index A abends 396 abends dump 396 access pattern 494 Accessing FlashCopy target 844 active quorum disk 62 active SVC cluster 177 add a new volume 218, 223 add a node 324 add additional ports 470 add an HBA 341 Add SSH Public Key 115 additional extents 59 admin password 314 administration tasks 321, 423, 460 administration using the GUI 401 advanced security 5 AIX and FlashCopy 844 AIX and Remote Copy 858 AIX host system 232 AIX LVM 844 AIX specific information 212 AIX toolbox 232 AIX-based hosts 212 allocation algorithm 50 amount of I/O 61 analysis 77, 525 application server guidelines 38 application servers 15 application testing 541 assign VDisks 171 assigned VDisk 218, 223 asymmetrical virtualization 6 asynchronous 670 asynchronous notifications 562 Asynchronous Peer-to-Peer Remote Copy 669 asynchronous remote 606, 670 asynchronous remote copy 606, 672 asynchronously 669 attributes 441, 446 authenticate 96 authentication 125 autoextend 20 automate tasks 902 automatic configuration 14 automatic Linux system 274 automatic update process 275 automatically 20 automatically discover 330 automatically restarted 519 automation 399 auxiliary 640, 678, 687, 708 auxiliary VDisk 670, 680, 687 availability 2
© Copyright IBM Corp. 2003-2008. All rights reserved.
available managed disks 328
B back-end application 10 back-end storage controllers 15 back-end storage guidelines 37 background copy 621, 628, 680, 687 background copy progress 635, 703 background copy rate 558–559 backup 390, 540 of data with minimal impact on production 543 backup speed 540 backup time 540 balance 83 bandwidth 44, 644, 678, 713 bandwidth impact 690 basic setup requirements 117 basic tasks 843 bat script 903 bind 301 bind address 19 bitmaps 542 block aggregation 3, 15 block size 48 block virtualization 46 block-for-block translation 48 boot 39 boot device 39 bottlenecks 78, 81 buffers 540 business requirements 77
C cable connections 31 cache 12, 540, 551, 670 cache algorithm 81 caching 80 caching capability 77 candidate node 325 cap 61 capacity 57, 231 capacity information 508 capacity measurement 480 capacity planning 77 capacity utilization 4 certificates 178–179, 407 change the IP addresses 315 channel extender 34 channels 677 check software levels 511 chpartnership 691 chrcconsistgrp 692 chrcrelationship 692 chunks 46, 750
929
CIMOM 96 CLI 97, 157, 303, 388, 695 commands 232 scripting for SVC task automation 399 cluster 13 adding nodes 174 administration 14 creation 158, 174 error log 385 IP address 103 shutting down 318–320, 421 time zone 161, 183, 315 time zone and time 418 viewing properties 309, 413 cluster error log 524 cluster partnership 611, 677 cluster properties 316 cluster state information 330 clustered IBM SAN appliance 5 clustered server resources 13 clusters 26 coarse grained striping 50 command syntax 307 command-line interface 157 common MDG 55 common platform 4 Compass architecture 12 complexity 2 concepts 9 concurrent commands 84 concurrent instances 749 concurrent software upgrade 376 configuration 209 and administration using the GUI 401 restoring 537 using the CLI 157 using the GUI 173 configuration backup and recovery 389 configuration data 389 configuration node 10, 14, 19, 158, 174 configure AIX 213 configure SDD 301 configuring the GUI 106 connected 615–616, 681–682 connected state 618, 682, 684 connectivity 14 consistency 540, 607, 672 consistency freeze 618, 684 consistency group 543–545 limits 546 consistency group zero 546 consistent 616–617, 682–683 consistent data set 540 Consistent Stopped state 614, 680 Consistent Synchronized state 614, 660, 680, 730 ConsistentDisconnected 620, 686 ConsistentStopped 618, 684 ConsistentSynchronized 619, 685 constrained link 679 container 46
930
control the node target ports 61 control traffic 678 controller, renaming 328 conventional storage 741 cookie crumbs recovery 399 cooling 27 copy bandwidth 690 copy process 626, 693 copy rate 547, 559 copy service 669 Copy Services 12 limitations with Windows 2000 865 managing 510 Windows 2000 860, 862 Windows NT 860 Windows Volume Sets 882 COPY_COMPLETED 562 copying state 568 copying the grain 547 core PID 43 corruption 541 counterpart SAN 34 create a FlashCopy 565 create a new VDisk 477 create an MDG 334, 451 create an SVC partnership 644, 712 create mapping command 564–565, 577, 579 create New Cluster 110 create SVC partnership 630, 697 create VDisks 192 Creating a host 61 creating a host 61–62, 338 creating a VDisk 167 creating managed disk groups 188 current cluster state 62 Cygwin 263
D Dark Fibre 44 data backup with minimal impact on production 543 consistency 860 moving and migration 540 data consistency 563, 670 data flow 33 data migration 26, 749 data migration and moving 540 data mining 541 data mover appliance 368 database log 675 degraded mode 36 delete a FlashCopy 571 a host 341 a host port 343 a port 472 a VDisk 358, 489, 509 ports 343 Delete consistency group command 572, 595 Delete mapping command 571, 595
Implementing the IBM System Storage SAN Volume Controller V4.3
dependent writes 545, 608–609, 674–675 destructive 387 detect the new MDisks 330 detected 330 device driver 85 device SCSI address 43 device specific modules 238 direct connection 37 directory map 52 dirty bit 621, 688 discard commands 84 disconnected 615–616, 681–682 disconnected state 682 discovering assigned VDisk 218, 223, 240 discovering newly assigned MDisks 435 disk 861 disk access profile 365 disk controller renaming 432 systems 327, 431 viewing details 327, 431 disk timeout value 295 disk zone 32 Diskpart 245 display summary information 328 displaying managed disks 187 distance 34, 604 distance limitations 605, 671 distributed redundant cache 15 DMP 298 documentation 26, 412 DSMs 238 dual site 82 dump I/O statistics 395 I/O trace 395 listing 394, 531 other nodes 397 dynamic pathing 298–299 dynamic shrinking 505 dynamic tracking 214 dynamic volumes 872
E effect of latency 81 empty MDG 336 empty state 621, 687 enlarging an extended basic volume 871 Enterprise Storage Server (ESS) 603 entire VDisk 543 ERP 85 error 332, 385, 524, 618, 681, 684 Error Code 520 error handling 560 error log 385, 524 analyzing 524 file 520 error notification 384, 522 error number 520 error priority 525
Error Recovery Process 85 ESS (Enterprise Storage Server) 603 ESS specialist 67 ESS storage 67 ESS to SVC 754 Ethernet 31 Ethernet connection 19, 39 event 385, 524 event log 394 events 613, 680 excludes 436 exclusive processor 331 Execute Metro Mirror 634, 702 expand 64 a VDisk 228, 244, 359 a volume 245 expand a space-efficient VDisk 360 expandvdisksize 360 expansion 59 extended disk 871 extended distance solutions 604 extended volume 871 extenders 44 extent 10, 46, 742 free 55 size 55 size rules 55 extent level 742 extent sizes 48 extents 48
F fabric local 34 remote 34 fabric interconnect 34 factory WWNN 916 failed node 62 failover 34, 298, 671 failover only 278 failover situation 605, 671 fan-in 34 fast fail 214 fast restore 540 FAStT 603 configuration 37 migration considerations 886 storage 72 favored hosts 86 feature log 388, 530 feature log dump 388 feature, licensing 528 features, licensing 387 featurization log 395 Featurization Settings 114 Fibre Channel port fan in 34 Fibre Channel ports 31 Fibre Channel switch 18 file aggregation 6 file system 281 Index
931
filtering 308, 407 filters 308 fixed error 385, 524 FlashCopy 539 accessing source, target on the same AIX host 846 accessing target with recreatevg 848 bitmap 547 commands 563 how it works 541–542 image mode disk 552 indirection layer 547 mapping 541 mapping events 553 rules 551 serialization of I/O 560 synthesis 559 FlashCopy functionality 844 FlashCopy indirection layer 547 FlashCopy mapping 543, 553 FlashCopy mapping states 555 Copying 556 Idling/Copied 555 Prepared 557 Preparing 556 Stopped 556 Suspended 556 FlashCopy mappings 546 FlashCopy properties 546 flexibility 77 flush the cache 589 focal point 612, 678 focal point node 678 forced deletion 470 foreground I/O latency 690 format 478, 485, 491, 500 free extents 49–50, 358 front-end application 10 front-end host 36 FTEDIT 863
G gateway IP address 104 GBICs 34 general housekeeping 413 generate some randomness 99 generating output 308 generator 100 geographically dispersed 82, 604 Global Mirror 669 Global Mirror relationship 673 Global Mirror remote copy technique 670 GM 669 gminterdelaysimulation 690 gmintradelaysimulation 690 gmlinktolerance 689 governing throttle 494 graceful manner 326 grain 10, 547, 560 grain is unsplit 548 grain size 547
932
grains 547, 559 granularity 48, 543 GUI 106, 123 signon 106
H hardware configuration 95 harmonic 29 harmonic distortion 29 HBA 35, 338 HBA fails 35 HBA ports 38 heartbeat signal 14 help 412 heterogeneous hosts 210 high availability 11, 13, 18, 26, 44 home directory 232 host and application server guidelines 38 configuration 209 creating 338 definitions 185 deleting 468 HBAs 37 information 338, 461 showing 373 systems 32 host adapter configuration settings 234 host bus adapter 338 host definitions 162 host key 122 host level 460 host objects 61 host workload 440 housekeeping 413 HP-UX support information 298–299
I I/O governing 365, 494 I/O governing rate 61, 365 I/O group 10–11, 14–16, 47 name 410 renaming 321, 423 viewing details 321 I/O pair 30 I/O per secs 26 I/O statistics dump 395 I/O threshold 61 I/O trace dump 395 ICAT 123 identical data 678 idling 619, 685 idling state 626, 693 IdlingDisconnected 620, 686 image mode 17, 48, 440, 751 image mode disk 552 image mode MDisk 752 image mode to image mode 775 image mode to managed mode 768
Implementing the IBM System Storage SAN Volume Controller V4.3
image mode VDisk 746 image mode virtual disk 48 image mode virtual disks 58 image-mode mapping 17 importvg 844 inappropriate zoning 38 in-band virtualization 3 inconsistent 616, 682 Inconsistent Copying state 614, 681 Inconsistent Stopped state 614, 659–660, 680, 730 InconsistentCopying 618, 684 InconsistentDisconnected 620, 686 InconsistentStopped 618, 684 increasing complexity 2 index number 436 indirection 50 indirection layer 547 indirection layer algorithm 548 informational error logs 562 initial considerations 886 input power 320 input voltage capture 29 install 25 Install Certificate 180 insufficient bandwidth 559 integrity 62, 544–545, 609, 675 Intel hardware 12 interaction with the cache 551 intercluster 44 intercluster communication and zoning 677 intercluster link 611, 677 intercluster link bandwidth 691 intercluster link maintenance 611–612, 677 intercluster Metro Mirror 604, 670 intercluster zoning 611–612, 677 internal resources 84 interswitch link (ISL) 33, 37 interval 316 intracluster 44 intracluster Metro Mirror 604, 670 IP address 19 modifying 314, 415 IP addresses 26, 416 IP subnet 39 ipconfig 131 IPv4 131, 140 IPv4 stack 140 IPv6 131 IPv6 address 138 IPv6 addresses 131 IPv6 connectivity 135 ISL (interswitch link) 33, 37 ISL count 37 ISL hop 33, 44 ISL hop count 604, 670 issue CLI commands 263
K kernel level 275 key files on AIX 232
L last extent 753 latency 81 LBA 49, 621, 688 LDM (logical disk manager) 865 LDM database 865 license 103 license feature 528 licensing feature 387 licensing feature settings 387, 528 limiting factor 78 Linux 232 Linux kernel 12 Linux on Intel 274 list dump 394 list of MDisks 458 list of VDisks 459 list the dumps 531 listing dumps 394, 531 Load balancing 278 local cluster 623, 689 local fabric 34 local fabric interconnect 34 locking the quorum disk 331 log 675 logged 385 Logical Block Address 49, 621, 688 logical configuration 18 logical configuration data 391 logical disk manager (LDM) 865 logical disks 56 logical SANs 33 logical unit numbers 165 logins 678 logs 674 lsrcrelationshipcandidate 692 LU 10 LUN masking 61–62 LUNs 3, 10, 46, 56, 65, 76 LVM data structures 847
M maintaining availability 2 maintaining passwords 414 maintenance levels 234 maintenance procedures 519 maintenance tasks 375, 510 managed disk 10, 15, 46, 56, 433 display 164 displaying 187 working with 327, 430 managed disk group 15, 165, 334 creating 188 viewing 191 managed disk group (MDG) 11 managed mode MDisk 752 managed mode to image mode 771 managed mode virtual disk 49, 58 management xviii, 77
Index
933
managing storage growth 2 map a VDisk 207 map a VDisk to a host 361 mapping 49, 60, 542 mapping events 553 mapping state 553 mapping table 50 maps 15 master 678, 687 master console 11, 27, 31, 121 master VDisk 680, 687 maximum capacity 48 MDG (managed disk group) 11 MDG information 507 MDG level 334 MDGs 26 MDisk 10, 15, 26, 187, 328, 433 adding 336, 456 discovering 330, 435 displaying 164 including 332, 436 information 328, 434 modes 752 name parameter 328 removing 336, 457 renaming 329, 434 showing 371, 458, 506 showing in group 337 working with 327 MDisk group 55 creating 334, 451 deleting 336, 454 name 410 renaming 335, 453 showing 333, 372, 437, 507 viewing information 334 memory 11 metadata 52 Metro Mirror 603 Metro Mirror consistency group 624–628, 691–695 Metro Mirror features 606, 672 Metro Mirror process 612, 678 Metro Mirror relationship 624–625, 628, 657, 673, 691–692, 695, 728 microcode 14 Microsoft Cluster 244 Microsoft Multi Path Input Output 238 migrate 741 migrate a VDisk 746 migrate between MDGs 746 migrate data 751 migrate VDisks 60, 367 migrating multiple extents 742 migration algorithm 750 functional overview 749 operations 742 overview 742 tips 754 migration activities 742
934
migration operations 55 migration phase 440 migration process 368 migration progress 748 migration scenarios 885 migration threads 742 minimal downtime 48 mirrored 670 mirrored copy 669 mkpartnership 690 mkrcconsistgrp 691 mkrcrelationship 691 modify a host 340 modifying a VDisk 59, 363 mount 281 mount point 281 moving and migrating data 540 MPIO 38, 66, 238 MSCS 244 multipath configuration 215 multipath I/O 38 multipath storage solution 238 multipathing device driver 38 multiple disk arrays 77 multiple extents 742 multiple virtual machines 289
N naming conventions 41 new code 519 new disks 220, 226 new mapping 361 no virtualization 48 node 11, 14, 321, 323, 424 adding 324, 425 adding to cluster 174 deleting 326, 427 failure 560 port 34 renaming 325, 427 shutting down 326 using the GUI 423 viewing details 323, 425 node details 323 node discovery 330, 436 node dumps 397 node level 323, 424 nodes 26 non-preferred path 298 non-redundant 34 N-port 33
O offline 57 offline rules 745 older disk systems 80 on screen content 308, 407 online 57 online help 412
Implementing the IBM System Storage SAN Volume Controller V4.3
on-screen content 308 OpenSSH 232 OpenSSH client 263 operating system versions 234 ordered list 50 ordering 545 organizing on-screen content 308 other node dumps 397 overall performance needs 26 oversubscription 33 overwritten 382, 542
P package numbering and version 375, 511 parallelism 749 partial last extent 753 partially used 48 partnership 611, 677, 689 passphrase 100 password maintenance 414 passwords 414 path failover 298 path failure 561 path offline 561 path offline for source VDisk 561 path offline for target VDisk 561 path offline state 561 paths 211 path-selection policy algorithms 278 peak 691 per cluster 749 per managed disk 749 performance 57 performance advantage 77 performance considerations 78, 88 performance improvement 77 performance requirements 26 performance throttling 494 physical location 26 physical planning 27 physical rules 30 physical site 27 physical storage 15 Physical Volume Links 299 PiT consistent data 540 PiT copy 547 PiT semantics 544 planning 82 planning chart 32 planning rules 26 plink 902 point-in-time copy 617, 683 policing 61 policy 50 policy decision 622, 688 pool 49 port adding 341, 470 address example 43 deleting 343, 472
port binding 301 port mapping 75 port mask 61 port masking 212 POSIX compliant 12 possible paths 211 Power Systems 232 Powerware 30 PPRC 5 background copy 621, 628, 687 commands 622, 689 configuration limits 688 detailed states 618, 684 preferred path 74, 298 pre-installation planning 26 prepare (pre-trigger) FlashCopy mapping command 567, 588 PREPARE_COMPLETED 562 preparing volumes 223, 228 pre-trigger 567, 588 primary 640, 671, 708 primary copy 687 priority 60, 368 priority setting 60, 368 private key 96, 100, 232, 902 production VDisk 687 productivity 3 provisioning 691 pseudo device driver 215 public key 96, 100, 232, 902 PuTTY 98, 123, 321 CLI session 127 default location 100 security alert 128 PuTTY application 127, 326 PuTTY Installation 263 PuTTY Key Generator 100–101 PuTTY Key Generator GUI 98 PuTTY Secure Copy 378 PuTTY session 101, 129 PuTTY SSH client software 263 PVIDs 847 PVLinks 39, 299
Q QLogic HBAs 275 QoS (Quality of Service) 5, 12 Quality of Service (QoS) 5, 12, 60 queue 84 queue depth 83 queue depth calculation 85–86 queue depth limit 85, 87, 89 queued commands 87 queueing 84 quickly 157 quiesce 320 quiesce time 589 quiesced 922 quorum candidates 62 quorum disk 55–56, 62, 330, 436 Index
935
setting 436 quorum index number 331
R RAID controller 12, 32 RAID protection 76 RDAC 65–66 real-time synchronized 604–605, 671 reassign the VDisk 362 reboot 860 recall commands 308 recommended levels 511 recovery algorithms 84 recreatevg command 844, 848 Redbooks Web site 926 Contact us xxi redundant 34 redundant SAN 34 redundant SAN fabrics 18 redundant SVC 178 redundant SVC environment 160 reform the cluster 331 registry 123, 861 relationship 543, 670, 678 relationship state diagram 614, 680 reliability 57 remote cluster 34 Remote Copy and AIX 858 Windows spanned volume 882 remote fabric 34 interconnect 34 remove a disk 260 remove a VDisk 232 remove an MDG 336 remove WWPN definitions 343 removed 62 rename a disk controller 432 rename an MDG 453 rename an MDisk 434 renaming an I/O group 423 repartitioning 57 rescan disks 242 reset function 123 Reset SSH Fingerprint 123 resiliency 82 restart the cluster 321 restart the node 327 restart the SVC node 429 restarting 638, 706 restore procedure 537 restore process 390 rmrcconsistgrp 694 rmrcrelationship 694 round robin 55, 57, 85, 278, 298
S sample script 905 SAN 3
936
SAN Boot Support 298, 300 SAN definitions 33 SAN design guidelines 35 SAN fabric 32, 36 SAN Integration Server 11 SAN interfaces 13 SAN interoperability 37 SAN planning 32 SAN Volume Controller 11 clustering 13 compatibility 19 documentation 412 general housekeeping 413 help 412 logical configuration 18 multipathing 17 virtualization 15 SAN Volume Controller (SVC) 11 SAN zoning 96 scalable 80 scalable cache 12 scalable solutions 5 scripting 399, 622, 688 scripts 245, 901 SCSI primitives 330 SDD 38, 66, 215, 222, 227, 300 SDD (Subsystem Device Driver) 17, 222, 227, 275, 300, 757 SDD Dynamic Pathing 298 SDD installation 216 SDD package version 215, 236 SDDDSM 238 secondary 671 secondary copy 687 secondary site 26 secure data flow 96 secure session 326 Secure Shell (SSH) 96 security 5, 123 security policy 61 separate zones 37 sequential 58, 170, 478, 485, 499, 501 sequential mapping 16 sequential policy 51 serial numbers 218, 225 serialization 560 serialization of I/O by FlashCopy 560 service password 314 service, maintenance using the GUI 510 set attributes 441, 446 set the cluster time zone 418 set up Metro Mirror 629, 642, 695, 710 SEV 366 shared 48 shells 399 show the MDG 507 show the MDisks 506 shrink a VDisk 505 shrinking 59, 505 shrinkvdisksize 369
Implementing the IBM System Storage SAN Volume Controller V4.3
shrunk 59 shut down 244, 326 shut down a single node 326, 429 shut down the cluster 320, 421 Signon page 106 Simple Network Management Protocol 332, 622, 688 simple volume 871 single name space 6 single point of failure 34 site 27 slew rate 29 SNIA 3 SNMP 332, 622, 688 SNMP alerts 436 SNMP manager 385 SNMP trap 562 software licensing 19 software upgrade 375, 511–512 software upgrade packages 511 solution 77 sort 410 sort criteria 410 sorting 410 source 559, 687 source virtual disks 542 space 48 space management 12 space-efficient 349, 359, 371 space-efficient VDisk 52, 364, 369–370, 440 space-efficient VDisks 484 space-efficient Virtual Disk 52 space-efficient volume 369 spanned volume 871 spanned volumes 882 special migration 753 split 547 splitting the SAN 34 SPoF 34 spreading the load 57 SSH 96, 129, 902 SSH (Secure Shell) 96 SSH client 232, 263 SSH client software 97 SSH keys 97, 123 SSH public key 121–122 SSH server 96 SSH-2 98 stack 751 stand-alone Metro Mirror relationship 633, 701 start (trigger) FlashCopy mapping command 568–569, 590–591 start a PPRC relationship command 625–626, 692–693 startrcrelationship 692 state 618, 684 connected 615, 681 consistent 616–617, 682–683 ConsistentDisconnected 620, 686 ConsistentStopped 618, 684 ConsistentSynchronized 619, 685 disconnected 615, 681
empty 621, 687 idling 619, 685 IdlingDisconnected 620, 686 inconsistent 616, 682 InconsistentCopying 618, 684 InconsistentDisconnected 620, 686 InconsistentStopped 618, 684 overview 613, 681 synchronized 617, 683 state fragments 616, 682 state overview 615, 688 state transitions 562, 681 states 553, 559, 613, 680 statistics 316 statistics collection 419 starting 419 stopping 317, 420 statistics dump 395 stop 681 stop FlashCopy consistency group 571, 593 stop FlashCopy mapping command 570 STOP_COMPLETED 562 stoprcconsistgrp 694 stoprcrelationship 693 storage capacity 26 storage growth 2 storage network 3 storage virtualization 11 stripe VDisks 77 striped 478, 485, 499, 501 striped mapping 16 striped policy 50 striped VDisk 170 subnet mask IP address 103 Subsystem Device Driver (SDD) 17, 222, 227, 275, 300, 757 Subsystem Device Driver DSM 238 SUN Solaris support information 298 superuser 106 supported switches 37 surviving node 326 suspended mapping 570 SVC 11 basic installation 102 cluster configuration backup and recovery 389 task automation 399 SVC cluster 174, 178, 330 SVC cluster candidates 644, 713 SVC cluster partnership 623, 689 SVC cluster software 513 SVC configuration 15, 18, 26 backing up 535 deleting the backup 537 restoring 537 SVC device 11 SVC installations 35 SVC intercluster 36 SVC intracluster 35 SVC master console 98 SVC node 14, 18, 35–36
Index
937
SVC PPRC functions 606 SVC setup 210 svcinfo 307, 329 svcinfo lsfreeextents 749 svcinfo lshbaportcandidate 342 svcinfo lsmdiskextent 749 svcinfo lsmigrate 748 svcinfo lsVDisk 333 svcinfo lsVDiskextent 749 svcinfo lsVDiskmember 371 svctask 307, 311, 329 svctask chlicense 387 svctask dumpinternallog 388 svctask finderr 381 svctask mkfcmap 564–565, 577, 579, 623–626, 689–693 switch configuration 37 switch zoning 211 switching copy direction 640, 667, 708, 737 switchrcconsistgrp 695 switchrcrelationship 695 symmetrical 1 symmetrical network 33 symmetrical virtualization 1 synchronized 617, 678, 683 synchronizing 678 synchronous reads 751 synchronous writes 751 synthesis 560
T target 687, 861 target reads 548 target server 861 target virtual disks 542 tasks 84 test new applications 541 threads parameter 495 threshold level 61 threshold quantity 61 throttles 494 throttling parameters 493 tie-break situations 62 tie-break solution 331, 436 time 184, 315, 418 time zone 161, 183, 315, 418 timeout 295 trace dump 395 traffic profile activity 26 transitions 752 trigger 568–569, 590–591
U unallocated capacity 247 unassign 490 unconfigured nodes 324 uneven performance 82 unfixed error 385, 524 uninterruptible power supply 11, 15, 30–31, 36, 320 unmanaged MDisk 752
938
unmap a VDisk 362 unrecognized certificates 178 unused space 49 up2date 274 update messages 415 updates 274 upgrade 511 upgrade precautions 376 upgrading software 511 use of Metro Mirror 621, 687 using SDD 222, 227, 275, 300
V VDisk 11, 458 assigning 207 assigning to host 171 creating 167, 192, 348, 477 creating in image mode 349, 440 deleting 358, 484, 489 discovering assigned 218, 223, 240 expanding 359 I/O governing 363 image mode migration concept 751 information 346, 475 mapped to this host 362 mapping to a host 361, 492 migrating 60, 367, 495 modifying 363, 493 path offline for source 561 path offline for target 561 showing 459 showing for MDisk 333, 438 showing map to a host 509 showing using group 337 shrinking 368, 496 working with 346 VDisk mirror 440 VDisk-to-host mapping 362 deleting 490 Veritas Volume Manager 298 View Certificate 180 View I/O Group details 321 viewing managed disk groups 191 virtual disk 11, 15, 50, 344, 346, 474, 543 creating 56 deleting 60 expanding 59 reducing 59 Virtual Machine File System 289 virtual pool 2 virtualization 15 virtualization device 4 Virtualization Limit 114 virtualization mapping 16 virtualization operations 55 virtualization overview 11 VLUN 10 VMFS 289–291 VMFS datastore 293 voltage regulation 15
Implementing the IBM System Storage SAN Volume Controller V4.3
volume group 228 volume management 15 Volume Sets 862 voting set 62 vpath configured 220, 226
W warning threshold 370 Web interface 301 Windows 2000 and Copy Services 860, 862 Windows 2000 based hosts 233 Windows 2000 host configuration 233, 287 Windows 2003 238 Windows host system CLI 263 Windows NT and 2000 specific information 233 Windows NT and 2000 specifics 860 Windows NT Volume Sets 862 Windows registration 122 Windows registry 123 Windows spanned volumes 882 Windows Volume Sets 860, 862, 882 with reboot 860 without reboot 860 working with managed disks 327, 430 worldwide node name 916 worldwide port name 215 write ordering 608, 673, 682 write through mode 36 writes 674 writes to source or target 549 write-through mode 14 WWNN 916 WWNs 162 WWPNs 61, 215, 338, 342, 464
Y YaST Online Update 274
Z zone 15, 18, 32 zoning 43 zoning capabilities 32 zoning recommendation 243, 257 zoning requirements 177
Index
939
940
Implementing the IBM System Storage SAN Volume Controller V4.3
Implementing the IBM System Storage SAN Volume Controller V4.3
Implementing the IBM System Storage SAN Volume Controller V4.3
Implementing the IBM System Storage SAN Volume Controller V4.3
Implementing the IBM System Storage SAN Volume Controller V4.3
(1.5” spine) 1.5”<-> 1.998” 789 <->1051 pages
Implementing the IBM System Storage SAN Volume Controller V4.3
Implementing the IBM System Storage SAN Volume Controller V4.3
Back cover
®
Implementing the IBM System Storage SAN Volume Controller V4.3 Install, use, and troubleshoot the SAN Volume Controller
This IBM Redbooks publication is an updated, detailed technical guide to the IBM System Storage SAN Volume Controller (SVC), a virtualization appliance solution that maps virtualized volumes visible to hosts and applications to physical volumes on storage devices.
Learn how to implement block virtualization
Each server within the SAN has its own set of virtual storage addresses, which are mapped to physical addresses. If the physical addresses change, the server continues running using the same virtual addresses it had before. This means that volumes or storage can be added or moved while the server is still running.
Create space-efficient VDisks
The IBM virtualization technology improves management of information at the block level in a network, enabling applications and servers to share storage devices on a network. This book covers the following areas: - Storage virtualization high-level overview - Architecture of the SVC - Implementing and configuring the SVC - Using virtualization and advanced copy services functions - Migrating existing storage to the SVC
®
INTERNATIONAL TECHNICAL SUPPORT ORGANIZATION
BUILDING TECHNICAL INFORMATION BASED ON PRACTICAL EXPERIENCE IBM Redbooks are developed by the IBM International Technical Support Organization. Experts from IBM, Customers and Partners from around the world create timely technical information based on realistic scenarios. Specific recommendations are provided to help you implement IT solutions more effectively in your environment.
For more information: ibm.com/redbooks SG24-6423-06
ISBN 073843163X