Home » Compaq Manuals » Servers » Compaq ProLiant 6400R » Manual Viewer

Compaq ProLiant 6400R Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 37

I/O Path Configurations for Redundant Fibre Channel Fabrics

Get Compaq ProLiant 6400R PDF manuals and user guides

View all Compaq ProLiant 6400R manuals

Add to My Manuals
Save this manual to your list of manuals

Page 37 highlights

Cluster Architecture 2-13 I/O Path Configurations for Redundant Fibre Channel Fabrics Overview of Fibre Channel Fabric SAN Topology Fibre Channel standards define a multi-layered architecture for moving data across the storage area network (SAN). This layered architecture can be implemented using the Fibre Channel Fabric or the Fibre Channel Arbitrated Loop (FC-AL) topology. The PDC/O5000 supports both topologies. A redundant Fibre Channel Fabric is two to four Fibre Channel SAN Switches installed between host adapters in a PDC/O5000's cluster nodes and the array controllers in the shared storage subsystems. Fibre Channel SAN Switches provide full 100 MBps bandwidth per switch port. Whereas the introduction of new devices to FC-AL Storage Hubs further divides their shared bandwidth, adding new devices to Fibre Channel SAN Switches increases the aggregate bandwidth. Redundant Fibre Channel Fabrics A redundant Fibre Channel Fabric consists of the PDC/O5000 cluster hardware used to connect host adapters to a particular set of shared storage devices using Fibre Channel SAN Switches. Each redundant Fibre Channel Fabric consists of the following hardware: I From two to four host adapters in each node I From two to four Fibre Channel SAN Switches I MA8000/EMA12000 Storage Subsystems, each containing two dual-port HSG80 Array Controllers I Fibre Channel cables used to connect the host adapters to the Fibre Channel SAN Switches and the Fibre Channel SAN Switches to the array controllers I GBIC-SW modules installed in host adapters, Fibre Channel SAN Switches, and array controllers IMPORTANT: For detailed information about cascading two Fibre Channel SAN Switches, refer to the latest Compaq StorageWorks documentation. This guide does not document cascaded configurations for the Fibre Channel SAN Switch.

Section	Page
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide	1
Notice	2
Contents	3
About This Guide	12
Purpose	12
Audience	12
Scope	13
Referenced Manuals	14
Supplemental Documents	15
Text Conventions	15
Symbols in Text	16
Symbols on Equipment	16
Rack Stability	17
Getting Help	17
Compaq Technical Support	18
Compaq Website	18
Compaq Authorized Reseller	19
Chapter 1: Clustering Overview	20
Clusters Defined	21
Availability	23
Scalability	23
Compaq Parallel Database Cluster€Overview	24
Chapter 2: Cluster Architecture	25
Compaq ProLiant Servers	26
HighAvailability Features of ProLiant Servers	27
Shared Storage Components	27
MA8000/EMA12000 Storage Subsystem	28
Controller Enclosure Components	28
Disk Enclosure Components	28
HSG80 Array Controller	28
Fibre Channel SAN Switch	29
Storage Hub	30
KGPSABC and KGPSA-CB Host Adapter	30
Gigabit Interface ConverterShortwave	31
Fibre Channel Cables	31
Configuring and Cabling the MA8000/EMA12000 Storage Subsystem Components	31
Configuring LUNS for Storagesets	31
SCSI Cabling Examples	32
SCSI Bus Addressing	32
UltraSCSI Cables	33
Using I/O Modules in the Controller Enclosure	33
Using DualBus I/O Modules in the Disk Enclosures	34
Using SingleBus I/O Modules in the Disk Enclosures	35
Connecting EMUs Between MA8000/EMA12000 Storage Subsystems	36
I/O Path Configurations for Redundant Fibre Channel Fabrics	37
Overview of Fibre Channel Fabric SAN Topology	37
Redundant Fibre Channel Fabrics	37
Multiple Redundant Fibre Channel Fabrics	39
Maximum Distances Between Nodes and Shared Storage Subsystem Components	40
I/O Data Paths in a Redundant Fibre Channel Fabric	41
Host Adapter-to-Fibre Channel SAN Switch I/O Data Paths	42
Fibre Channel SAN SwitchtoArray Controller I/O Data Paths	43
I/O Path Definitions for Redundant Fibre Channel Fabrics	44
I/O Path Configuration Examples for Redundant Fibre Channel Fabrics	45
Dual Redundancy Configuration Example	45
Quad Redundancy Configuration Example	46
Summary of I/O Path Failure and Failover Scenarios for Redundant Fibre Channel Fabrics	49
I/O Path Configurations for Redundant Fibre Channel Arbitrated Loops	50
Overview of FCAL SAN Topology	50
Redundant Fibre Channel Arbitrated Loops	50
Multiple Redundant Fibre Channel Arbitrated Loops	52
Maximum Distances Between Nodes and Shared Storage Subsystem Components	54
I/O Data Paths in a Redundant FCAL	56
Host AdaptertoStorage Hub I/O Data Paths	56
Storage HubtoArray Controller I/O Data Paths	57
I/O Path Definitions for Redundant FCALs	58
I/O Path Configuration Examples for Redundant FCALs	59
Dual Redundancy Configuration Example	59
Quad Redundancy Configuration Example	60
Summary of I/O Path Failure and Failover Scenarios for Redundant FCALs	62
Cluster Interconnect Requirements	63
Ethernet Cluster Interconnect	63
Ethernet Cluster Interconnect Components	64
Ethernet Cluster Interconnect Adapters	64
Ethernet Switch	64
Ethernet Cluster Interconnect Diagrams	65
\fLocal Area Network	66
Chapter 3: Cluster Software Components	67
Overview of the Cluster Software	67
Microsoft Windows 2000 Advanced Server	68
Compaq Software	68
Compaq SmartStart and Support Software	68
Compaq System Configuration Utility	69
Compaq Insight Manager	69
Compaq Insight Manager XE	69
Compaq Options ROMPaq	70
Compaq StorageWorks Command Console	70
Compaq StorageWorks Secure Path for Windows 2000	70
Compaq Operating System Dependent Modules	71
OSDs for Oracle8i Parallel Server	71
Oracle Software	71
Oracle8i Server Enterprise Edition	72
Oracle8i Server	72
Oracle8i Parallel Server Option	72
Oracle8i Enterprise Manager	73
Oracle8i Certification	73
Application Failover and Reconnection Software	73
Chapter 4: Cluster Planning	75
Site Planning	76
Capacity Planning for Cluster Hardware	76
Compaq ProLiant Servers	76
Planning Shared Storage Components for Redundant Fibre Channel Fabrics	77
Planning Shared Storage Components for Redundant Fibre Channel Arbitrated Loops	78
Planning Cluster Interconnect and Client LAN Components	80
Planning an Ethernet Cluster Interconnect	80
Planning the Client LAN	80
Planning Cluster Configurations for Redundant Fibre Channel Fabrics	81
Planning Dual Redundancy Configurations	81
Planning Quad Redundancy Configurations	83
Planning Cluster Configurations for Redundant Fibre Channel Arbitrated Loops	85
Planning Dual Redundancy Configurations	85
Planning Quad Redundancy Configurations	87
RAID Planning for the MA8000/EMA12000 Storage Subsystem	89
Supported RAID Levels	90
Raw Data Storage and Database Size	91
Selecting the Appropriate RAID Level	92
Planning the Grouping of Physical Disk Storage Space	93
Disk Drive Planning	94
Nonshared Disk Drives	94
Shared Disk Drives	94
Network Planning	95
Windows 2000 Advanced Server Host Files for an Ethernet Cluster Interconnect	95
Client LAN	96
Chapter 5: Installation and Configuration	97
Reference Materials for Installation	97
Installation Overview	98
Installing the Hardware	99
Setting Up the Nodes	99
Installing the KGPSA-BC and KGPSA-CB Host Adapters	100
Installing GBIC-SW Modules for the Host Adapters	100
Cabling the Host Adapters to Fibre Channel SAN Switches or Storage Hubs	101
Installing the Cluster Interconnect Adapters	102
Ethernet Cluster Interconnect	102
Installing the Client LAN Adapters	102
Installing GBIC-SW Modules for the Array Controllers	102
Cabling an Ethernet Cluster Interconnect	103
Cabling the Client LAN	105
Redundant Client LAN	105
Installing Hardware Into an MA8000/EMA12000 Storage Subsystem	106
Installing Hardware into a Controller Enclosure	106
Installing Hardware into a Disk Enclosure	106
Cabling the Controller Enclosure to Disk Enclosures	107
Cabling EMUs to Each Other	109
Cabling Array Controllers to Fibre Channel SAN Switches	110
Cabling Array Controllers to Storage Hubs	111
Installing Operating System Software	112
Guidelines for Clusters	113
Automated Installation Using SmartStart	113
Cluster-Specific SmartStart Installation	114
Automated Installation Steps	114
Setting up and Configuring an MA8000/EMA12000 Storage Subsystem	117
Designating a Server as a Maintenance Terminal	117
Powering On the MA8000/EMA12000 Storage Subsystem	117
Installing the StorageWorks Command Console (SWCC) Client	118
Configuring a Storage Subsystem for Secure Path Operation	118
Discovering Shared Physical Disks	122
Verifying Array Controller Properties	123
Displaying SCSI Devices in the Devices Windowpane	123
Verifying Controller Properties	123
Configuring a Storageset	125
Verifying Access	127
Installing Secure Path Software for Windows 2000	128
Overview of Secure Path Software Installation	128
Description of the Secure Path Software	129
Installing the Host Adapter Drivers	129
Installing the Fibre Channel Software Setup (FCSS) Utility	130
Installing the Secure Path Drivers, Secure Path Agent, and Secure Path Manager	131
Specifying the Preferred_Path for Storage Units	132
Powering Up All Other Fibre Channel SAN Switches or Storage Hubs	134
Creating Partitions	134
Installing Compaq OSDs	135
Verifying Cluster Communications	136
Mounting Remote Drives and Verifying Administrator Privileges	137
Installing the Ethernet OSDs	138
Installing Oracle Software	148
Configuring Oracle8i Software	149
Installing Object Link Manager	149
Additional Notes on Configuring Oracle Software	150
Creating a Symbolic Link for OPS_CMDISK	150
Verifying Access to Shared Storage from All Nodes	150
Verifying the Hardware and Software Installation	151
Cluster Communications	151
Access to Shared Storage from All Nodes	151
OSDs	151
Other Verification Tasks	152
Power Distribution and Power Sequencing Guidelines	152
Overview	152
Server Power Distribution	153
Storage Subsystem Power Distribution	153
Power Sequencing	154
Chapter 6: Cluster Management	155
Cluster Management Concepts	156
Powering Off a Node Without Interrupting Cluster Services	156
Managing a Cluster in a Degraded Condition	156
Managing Network Clients Connected to a Cluster	157
Cluster Events	157
Management Applications	158
Monitoring Server and Network Hardware	158
Monitoring Storage Subsystem Hardware	159
Managing Shared Storage	159
Using the Command Line Interface	159
Using the Compaq StorageWorks Command Console	160
Using Secure Path Manager	160
Using Disk Management	160
Monitoring the Database	161
Remotely Managing a Cluster	161
Software Maintenance	162
Deinstalling the OSDs	162
Upgrading Oracle8i Server	165
Upgrading the OSDs	165
Upgrading the Ethernet OSDs	165
Deinstalling a Partial OSD Installation	167
Managing Changes to Shared Storage Components	168
Adding New Storagesets to Increase Shared Storage Capacity	168
Expanding a Logical drive Within a Storageset	169
Replacing a Failed Drive in a Storage Subsystem	169
Replacing a Host Adapter	170
Determining a Host Adapter Problem	170
Replacing a Failed Host Adapter	171
Adding a Shared Storage Subsystem	173
Replacing a Cluster Node	173
Removing the Node	174
Adding the Replacement Node	174
Preparing the Replacement Node	175
Installing the Cluster Software	177
Adding a Cluster Node	178
Preparing the New Node	179
Preparing the Existing Cluster Nodes	181
Installing the Cluster Software	181
Monitoring Cluster Performance	183
Tools Overview	183
Using Secure Path Manager	184
Monitoring Storage Systems	185
Manipulating Storagesets and I/O Paths	185
Detecting I/O Path and Array Controller Failures	186
Identifying I/O Path and Array Controller Failures	186
Uninstalling Secure Path	187
Chapter 7: Troubleshooting	188
Basic Troubleshooting Tips	189
Power	189
Physical Connections	189
Access to Cluster Components	190
Software Revisions	190
Firmware Revisions	191
Troubleshooting Oracle8i and OSD Installation Problems and Error Messages	192
Potential Difficulties Installing the OSDs with the Oracle Universal Installer	192
Unable to Start OracleCMService	193
Unable to Start OracleNMService	193
Unable to Start the Database	194
Initialization of the Dynamic Link Library NM.DLL Failed	194
Troubleshooting Node-to-Node Connectivity Problems	194
Nodes Are Unable to Communicate with Each Other	194
When Using the Ethernet Cluster Interconnect	195
Unable to Ping the Cluster Interconnect or the Client LAN	195
When Using the Ethernet Cluster Interconnect	195
Troubleshooting Client-to-Cluster Connectivity Problems	196
A Network Client Cannot Communicate With the Cluster	196
Troubleshooting Shared Storage Subsystem Problems	197
Verifying Host Adapter Device Driver Installation	197
Verifying KGPSA-BC Device Driver Initialization	197
Verifying Connectivity to a Redundant Fibre Channel Fabric	199
Fibre Channel Fabric Connectivity Guidelines	199
MA8000/EMA12000 Storage Subsystem Connectivity to Fibre Channel FabrIc Connection Paths	199
Server Node Connectivity to Fibre Channel Fabric Connection Paths	200
Verifying Connectivity to a Redundant Fibre Channel Arbitrated Loop	200
Fibre Channel Arbitrated Loop Connectivity Guidelines	200
MA8000/EMA12000 Storage Subsystem Connectivity to FC-AL Connection Paths	201
Server Node Connectivity to FC-AL Connection Paths	201
A Cluster Node Cannot Connect to the Shared Drives	202
Disk Management Shows Storagesets With the Same Label (Dual Image)	202
Device or Devices Were Not Found by KGPSA-BC Device Driver	202
Devices on One I/O Connection Path Cannot Be Seen by the Cluster Nodes	203
For a Redundant Fibre Channel Fabric	203
For a Redundant Fibre Channel Arbitrated Loop	204
Troubleshooting Secure Path	205
Secure Path Guidelines for Windows 2000 Advanced Server	205
Secure Path Manager Shows Reversed Locations for Top and Bottom Array Controllers	207
Secure Path Manager Cannot Start With Hosts That Use Hyphenated Host Names	207
Secure Path Manager Is Delayed In Reporting Path Failure Information	207
The Addition of New LUNs Causes an Error	208
A Configuration of More Than 64 LUNs Prevents the Secure Path Agent From Starting	208
Appendix A: Diagnosing and Resolving Shared Disk Problems	209
Introduction	209
Run Object Link Manager on All Nodes	211
Restart All Affected Nodes in the Cluster	212
Rerun and Validate Object Link Manager On All Affected Nodes	212
Run and Validate Secure Path Manager On All Nodes	213
Run Disk Management On All Nodes	213
Run and Validate the StorageWorks Command Console From All Storage Subsystems	214
Perform Cluster Software and Firmware Checks	214
Perform Cluster Hardware Checks	215
Contact Your Compaq Support Representative	216
Glossary	217