Home » Compaq Manuals » Servers » Compaq ProLiant 6500 » Manual Viewer

Compaq ProLiant 6500 Compaq Parallel Database Cluster Model PDC/O2000 for Orac - Page 33

Fibre Channel Cables, I/O Path Configurations for Redundant Fibre Channel Fabrics

Get Compaq ProLiant 6500 PDF manuals and user guides

View all Compaq ProLiant 6500 manuals

Add to My Manuals
Save this manual to your list of manuals

Page 33 highlights

Cluster Architecture 2-9 Fibre Channel Cables Shortwave (multi-mode) fibre optic Fibre Channel cables are used to connect the nodes, the Storage Hubs, FC-AL Switches, or Fibre Channel SAN Switches, and RA4000/RA4100 Arrays in a PDC/O2000 cluster. I/O Path Configurations for Redundant Fibre Channel Fabrics Overview of Fibre Channel Fabric SAN Topology Fibre Channel standards define a multi-layered architecture for moving data across the storage area network (SAN). This layered architecture can be implemented using the Fibre Channel Fabric or the Fibre Channel Arbitrated Loop (FC-AL) topology. The PDC/O2000 supports both topologies. A redundant Fibre Channel Fabric is two to four Fibre Channel SAN Switches installed between Fibre Host Adapters in a PDC/O2000's cluster nodes and the array controllers in the shared storage subsystems. These hardware components cannot be shared by other redundant Fibre Channel Fabrics or redundant Fibre Channel Arbitrated Loops in the cluster. Fibre Channel SAN Switches provide full 100 MBps bandwidth per switch port. Whereas the introduction of new devices to FC-AL Storage Hubs further divides their shared bandwidth, adding new devices to Fibre Channel SAN Switches increases the aggregate bandwidth. Redundant Fibre Channel Fabrics A redundant Fibre Channel Fabric refers to the redundant hardware implemented to connect Fibre Host Adapters to a particular set of shared storage devices in a PDC/O2000 that uses Fibre Channel SAN Switches. Each redundant Fibre Channel Fabric consists of the following hardware: I Two or more Fibre Host Adapters in each node I Two to four Fibre Channel SAN Switches

Section	Page
Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide	1
Notice	2
Contents	3
About This Guide	13
Purpose	13
Audience	13
Scope	14
Referenced Manuals	15
Supplemental Documents	16
Text Conventions	17
Symbols in Text	17
Symbols on Equipment	18
Rack Stability	19
Getting Help	19
Compaq Technical Support	19
Compaq Website	20
Compaq Authorized Reseller	20
Chapter 1: Clustering Overview	21
Clusters Defined	22
Availability	23
Scalability	23
Compaq Parallel Database Cluster Overview	24
Chapter 2: Cluster Architecture	25
Compaq ProLiant Servers	26
HighAvailability Features of ProLiant Servers	27
Shared Storage Components	27
RA4000 Array	28
RA4100 Array	28
RA4000 Array Controllers	29
Fibre Channel SAN Switches	30
FC-AL Switches	31
Storage Hubs	31
Fibre Host Adapters	32
Gigabit Interface Converter-Shortwave Modules	32
Fibre Channel Cables	33
I/O Path Configurations for Redundant Fibre Channel Fabrics	33
Overview of Fibre Channel Fabric SAN Topology	33
Redundant Fibre Channel Fabrics	33
Multiple Redundant Fibre Channel Fabrics	35
Maximum Distances Between Nodes and Shared Storage Components in a Redundant Fibre Channel Fabric	37
I/O Data Paths in a Redundant Fibre Channel Fabric	38
Fibre Host Adapter to Fibre Channel SAN Switch Paths	38
Fibre Channel SAN Switch to Array Controller Paths	39
I/O Path Configuration Guidelines for Redundant Fibre Channel Fabrics	39
I/O Path Configuration Rules for Redundant Fibre Channel Fabrics	43
Active/Standby Configuration Examples for Redundant Fibre Channel Fabrics	44
Active/Standby Configuration with One RA4000/4100 Array	46
Active/Standby Configuration with Two RA4000/4100 Arrays	47
Active/Standby Configuration with Three RA4000/RA4100 Arrays	48
Active/Standby Configuration with Four RA4000/RA4100 Arrays	49
Active/Standby Configuration with Five RA4000/RA4100 Arrays	50
Active/Active Configuration Examples for Redundant Fibre Channel Fabrics	51
Active/Active Configuration with Two RA4000/RA4100 Arrays	53
Active/Active Configuration with Three RA4000/RA4100 Arrays	54
Active/Active Configuration with Four RA4000/RA4100 Arrays	55
Active/Active Configuration with Five RA4000/RA4100 Arrays	56
Summary of I/O Path Failure and Failover Scenarios for Redundant Fibre Channel Fabrics	57
I/O Path Configurations for Redundant Fibre Channel Arbitrated Loops	63
Overview of FCAL SAN Topology	63
Redundant Fibre Channel Arbitrated Loops	64
Multiple Redundant Fibre Channel Arbitrated Loops	66
Maximum Distances Between Nodes and Shared Storage Components in a Redundant FCAL	68
I/O Data Paths in a Redundant FCAL	69
Fibre Host AdaptertoStorage Hub/Switch Paths	69
Storage Hub/SwitchtoArray Controller Paths	70
I/O Path Configuration Guidelines for Redundant Fibre Channel Arbitrated Loops	71
I/O Path Configuration Rules for Redundant FCALs	74
Active/Standby Configuration Examples for Redundant FCALs	75
Active/Standby Configuration with One RA4000/4100 Array	77
Active/Standby Configuration with Two RA4000/4100 Arrays	78
Active/Standby Configuration with Three RA4000/RA4100 Arrays	79
Active/Standby Configuration with Four RA4000/RA4100 Arrays	80
Active/Standby Configuration with Five RA4000/RA4100 Arrays	81
Active/Active Configuration Examples for Redundant FCALs	82
Active/Active Configuration with Two RA4000/RA4100 Arrays	84
Active/Active Configuration with Three RA4000/RA4100 Arrays	85
Active/Active Configuration with Four RA4000/RA4100 Arrays	86
Active/Active Configuration with Five RA4000/RA4100 Arrays	87
Summary of I/O Path Failure and Failover Scenarios for Redundant FCALs	88
Cluster Interconnect Options	94
Ethernet Cluster Interconnect	94
Non-Redundant Ethernet Cluster Interconnect Components	95
Redundant Ethernet Cluster Interconnect Components	95
Ethernet Cluster Interconnect Adapters	96
Ethernet Switch	96
Ethernet Cluster Interconnect Diagrams	97
Local Area Network	100
Chapter 3: Cluster Software Components	101
Overview of the Cluster Software	101
Microsoft Windows 2000 Advanced Server	101
Compaq Software	102
Compaq SmartStart and Support Software	102
Compaq System Configuration Utility	103
Compaq Array Configuration Utility	103
Fibre Channel Fault Isolation Utility	103
Compaq Insight Manager	104
Compaq Insight Manager XE	104
Compaq Options ROMPaq	104
Compaq StorageWorks Secure Path for Windows 2000	105
Compaq Operating System Dependent Modules	105
OSDs for Oracle8i Parallel Server	105
Oracle Software	106
Oracle8i Server Enterprise Edition	106
Oracle8i Server	106
Oracle8i Parallel Server Option	106
Oracle8i Enterprise Manager	107
Oracle8i Certification	107
Application Failover and Reconnection Software	108
Chapter 4: Cluster Planning	109
Site Planning	110
Capacity Planning for Cluster Hardware	111
Compaq ProLiant Servers	111
Planning Shared Storage Components for Redundant Fibre Channel Fabrics	111
Planning Shared Storage Components for Redundant Fibre Channel Arbitrated Loops	113
Planning Cluster Interconnect and Client LAN Components	114
Planning an Ethernet Cluster Interconnect	114
Planning the Client LAN	114
Planning Cluster Configurations for Redundant Fibre Channel Fabrics	115
Sample Small Configuration in a Redundant Fibre Channel Fabric	115
Sample Large Configuration in a Redundant Fibre Channel Fabric	117
Planning Cluster Configurations for Redundant Fibre Channel Arbitrated Loops	119
Sample Small Configuration in a Redundant FCAL	119
Sample Large Configuration in a Redundant FCAL	121
RAID Planning	122
Supported RAID Levels	124
Raw Data Storage and Database Size	125
Selecting the Appropriate RAID Levels	125
Planning the Grouping of Physical Disk Storage Space	126
Disk Drive Planning	127
Nonshared Disk Drives	127
Shared Disk Drives	128
Network Planning	128
Windows 2000 Advanced Server Hosts Files for an Ethernet Cluster Interconnect	128
Client LAN	129
Chapter 5: Installation and Configuration	130
Installation Overview	131
Installing the Hardware	133
Setting Up the Nodes	133
Installing the Fibre Host Adapters	133
Installing GBIC-SW Modules for the Fibre Host Adapters	134
Cabling the Fibre Host Adapters to the Storage Hubs or Switches	134
Installing the Cluster Interconnect Adapters	135
Ethernet Cluster Interconnect Adapters	136
Installing the Client LAN Adapters	136
Setting Up the RA4000/RA4100 Arrays	136
Compaq Array Configuration Utility	137
Installing GBIC-SW Modules for the RA4000 Array Controllers	138
Cabling the Storage Hubs or Switches to the RA4000 Array Controllers	138
Cabling an Active/Standby Configuration	139
Cabling an Active/Active Configuration	140
Installing Additional Redundant Fibre Channel Fabrics	144
Installing Additional Redundant FC-ALs	144
Cabling the Ethernet Cluster Interconnect	145
Cabling the Client LAN	149
Redundant Client LAN	149
Installing the Operating System Software and Configuring the RA4000/RA4100 Arrays	150
Guidelines for Clusters	150
Automated Installation Using SmartStart	151
Cluster-Specific SmartStart Installation	151
Automated Installation Steps	152
Installing Secure Path Software for Windows 2000	155
Installing the Secure Path Server Software	155
Installing the Secure Path Client Software	156
Verifying Shared Disk Storage Using Secure Path Manager	156
Defining Active Array Controllers	157
Installing Compaq OSDs	159
Verifying Cluster Communications	160
Mounting Remote Drives and Verifying Administrator Privileges	161
Installing the Ethernet OSDs	162
Installing Oracle Software	173
Configuring Oracle Software	174
Installing Object Link Manager	174
Additional Notes on Configuring Oracle Software	175
Creating a Symbolic Link for OPS_CMDISK	175
Verifying Access to Shared Storage from All Nodes	175
Verifying the Hardware and Software Installation	176
Cluster Communications	176
Access to Shared Storage from All Nodes	176
OSDs	176
Other Verification Tasks	177
Power Distribution and Power Sequencing Guidelines	177
Server Power Distribution	178
RA4000/RA4100 Array Power Distribution	178
Power Sequencing	179
Chapter 6: Cluster Management	180
Cluster Management Concepts	181
Powering Off a Node Without Interrupting Cluster Services	181
Managing a Cluster in a Degraded Condition	181
Managing Network Clients Connected to a Cluster	182
Cluster Events	182
Management Applications	183
Monitoring Server and Network Hardware	183
Managing Shared Drives	184
Monitoring Redundant Fibre Channel Fabrics	184
Monitoring Redundant Fibre Channel Arbitrated Loops	185
Monitoring the Database	186
Remotely Managing a Cluster	186
Software Maintenance for Oracle8i	187
Deinstalling the OSDs	187
Upgrading Oracle8i Server	190
Upgrading the OSDs	190
Upgrading the Ethernet OSDs	191
Deinstalling a Partial OSD Installation	192
Upgrading Oracle8i Server	193
Managing Changes to Shared Storage Components	193
Replacing a Failed Disk	193
Adding Disk Drives to Increase Storage Capacity	194
Adding an RA4000/RA4100 Array	194
Replacing a Failed Fibre Host Adapter	195
Replacing a Cluster Node	196
Removing the Node	196
Adding the Replacement Node	197
Preparing the Replacement Node	197
Installing the Cluster Software	198
Adding a Cluster Node	200
Preparing the New Node	201
Preparing the Existing Cluster Nodes	202
Installing the Cluster Software for Oracle8i	202
Monitoring Cluster Operation	204
Tools Overview	204
Using Secure Path Manager	204
Monitoring Storage Systems	205
Manipulating Storagesets and I/O Paths	206
Detecting I/O Path and Array Controller Failures	206
Identifying I/O Path and Array Controller Failures	207
Uninstalling Secure Path	207
Chapter 7: Troubleshooting	208
Basic Troubleshooting Tips	209
Power	209
Physical Connections	209
Access to Cluster Components	210
Software Revisions	210
Firmware Revisions	211
Troubleshooting Oracle8i and OSD Installation Problems and Error Messages	212
Potential Difficulties Installing the OSDs with the Oracle Universal Installer	212
Unable to Start OracleCMService	213
Unable to Start OracleNMService	214
Unable to Start the Database	214
Initialization of the Dynamic Link Library NM.DLL Failed	215
Troubleshooting Node-to-Node Connectivity Problems	215
Nodes Are Unable to Communicate with Each Other	215
When Using the Ethernet Cluster Interconnect	215
Unable to Ping the Cluster Interconnect or the Client LAN	216
When Using the Ethernet Cluster Interconnect	216
Node or Nodes Unable to Rejoin the Cluster	216
Troubleshooting Client-to-Cluster Connectivity Problems	217
A Network Client Cannot Communicate with the Cluster	217
Troubleshooting Shared Storage Problems	218
Verifying Connectivity to a Redundant Fibre Channel Fabric	218
Verifying Connectivity to a Redundant Fibre Channel Arbitrated Loop	219
Shared Disks in the RA4000/RA4100 Arrays Are Not Recognized By One or More Nodes	219
A Cluster Node Cannot Connect to the Shared Drives	221
Troubleshooting Secure Path	221
Secure Path Guidelines for Windows 2000 Advanced Server	221
Secure Path Manager Cannot Start With Hosts That Use Hyphenated Host Names	222
Secure Path Manager Is Delayed In Reporting Path Failure Information	223
The Addition of New LUNs Causes an Error	223
A Configuration of More Than 64 LUNs Prevents the Secure Path Agent From Starting	223
Appendix A: Diagnosing and Resolving Shared Disk Problems	224
Introduction	224
Run Object Link Manager on All Nodes	226
Restart All Affected Nodes in the Cluster	227
Rerun and Validate Object Link Manager On All Affected Nodes	227
Run and Validate Secure Path Manager On All Nodes	228
Run Disk Management On All Nodes	228
Run and Validate the Array Configuration Utility On All Nodes	229
Perform Cluster Software and Firmware Checks	229
Perform Cluster Hardware Checks	230
Contact Your Compaq Support Representative	231
Glossary	232