Home » Compaq Manuals » Servers » Compaq ProLiant 6500 » Manual Viewer

Compaq ProLiant 6500 Compaq Parallel Database Cluster Model PDC/O2000 for Orac - Page 133

Installing the Hardware, Setting Up the Nodes, Installing the Fibre Host Adapters

Get Compaq ProLiant 6500 PDF manuals and user guides

View all Compaq ProLiant 6500 manuals

Add to My Manuals
Save this manual to your list of manuals

Page 133 highlights

5-4 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Installing the Hardware Setting Up the Nodes Physically preparing the nodes (servers) for a cluster is not very different than preparing them for individual use. You will install all necessary adapters and insert all internal hard disks. You will attach network cables and plug in SCSI and Fibre Channel cables. The primary difference is in setting up the shared storage subsystem. Set up the hardware on one node completely, then set up the rest of the nodes identically to the first one. Do not load any software on any cluster node until all the hardware has been installed in all cluster nodes. Before loading software, read "Installing the Operating System Software and Configuring the RA4000/RA4100 Arrays" in this chapter to understand the idiosyncrasies of configuring a cluster. IMPORTANT: With some possible exceptions, the servers in the cluster must be set up identically. The cluster components common to all nodes in the cluster must be identical, for example, the ProLiant server model, cluster interconnect adapters, amount of memory, cache, and number of CPUs must be the same for each cluster node. It also means the Fibre host adapters must be installed into the same PCI slots in each server. While setting up the physical hardware, follow the installation instructions in your Compaq ProLiant Server Setup and Installation Guide and in your Compaq ProLiant Server Installation poster. When you are ready to install the Fibre Host Adapters and your cluster interconnect adapters, refer to the instructions in the pages that follow. Installing the Fibre Host Adapters Each redundant Fibre Channel Fabric or redundant Fibre Channel Arbitrated Loop (FC-AL) requires at least two Fibre Host Adapters in each cluster node. Install these devices as you would any other PCI adapter. Install two Fibre Host Adapters on the same PCI bus in each server and into the same two PCI slots in each server. If you need specific instructions, see the Compaq StorageWorks Fibre Channel Host Bus Adapter Installation Guide or the Compaq StorageWorks 64-Bit/66 MHz Fibre Channel Host Adapter Installation Guide.

Section	Page
Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide	1
Notice	2
Contents	3
About This Guide	13
Purpose	13
Audience	13
Scope	14
Referenced Manuals	15
Supplemental Documents	16
Text Conventions	17
Symbols in Text	17
Symbols on Equipment	18
Rack Stability	19
Getting Help	19
Compaq Technical Support	19
Compaq Website	20
Compaq Authorized Reseller	20
Chapter 1: Clustering Overview	21
Clusters Defined	22
Availability	23
Scalability	23
Compaq Parallel Database Cluster Overview	24
Chapter 2: Cluster Architecture	25
Compaq ProLiant Servers	26
HighAvailability Features of ProLiant Servers	27
Shared Storage Components	27
RA4000 Array	28
RA4100 Array	28
RA4000 Array Controllers	29
Fibre Channel SAN Switches	30
FC-AL Switches	31
Storage Hubs	31
Fibre Host Adapters	32
Gigabit Interface Converter-Shortwave Modules	32
Fibre Channel Cables	33
I/O Path Configurations for Redundant Fibre Channel Fabrics	33
Overview of Fibre Channel Fabric SAN Topology	33
Redundant Fibre Channel Fabrics	33
Multiple Redundant Fibre Channel Fabrics	35
Maximum Distances Between Nodes and Shared Storage Components in a Redundant Fibre Channel Fabric	37
I/O Data Paths in a Redundant Fibre Channel Fabric	38
Fibre Host Adapter to Fibre Channel SAN Switch Paths	38
Fibre Channel SAN Switch to Array Controller Paths	39
I/O Path Configuration Guidelines for Redundant Fibre Channel Fabrics	39
I/O Path Configuration Rules for Redundant Fibre Channel Fabrics	43
Active/Standby Configuration Examples for Redundant Fibre Channel Fabrics	44
Active/Standby Configuration with One RA4000/4100 Array	46
Active/Standby Configuration with Two RA4000/4100 Arrays	47
Active/Standby Configuration with Three RA4000/RA4100 Arrays	48
Active/Standby Configuration with Four RA4000/RA4100 Arrays	49
Active/Standby Configuration with Five RA4000/RA4100 Arrays	50
Active/Active Configuration Examples for Redundant Fibre Channel Fabrics	51
Active/Active Configuration with Two RA4000/RA4100 Arrays	53
Active/Active Configuration with Three RA4000/RA4100 Arrays	54
Active/Active Configuration with Four RA4000/RA4100 Arrays	55
Active/Active Configuration with Five RA4000/RA4100 Arrays	56
Summary of I/O Path Failure and Failover Scenarios for Redundant Fibre Channel Fabrics	57
I/O Path Configurations for Redundant Fibre Channel Arbitrated Loops	63
Overview of FCAL SAN Topology	63
Redundant Fibre Channel Arbitrated Loops	64
Multiple Redundant Fibre Channel Arbitrated Loops	66
Maximum Distances Between Nodes and Shared Storage Components in a Redundant FCAL	68
I/O Data Paths in a Redundant FCAL	69
Fibre Host AdaptertoStorage Hub/Switch Paths	69
Storage Hub/SwitchtoArray Controller Paths	70
I/O Path Configuration Guidelines for Redundant Fibre Channel Arbitrated Loops	71
I/O Path Configuration Rules for Redundant FCALs	74
Active/Standby Configuration Examples for Redundant FCALs	75
Active/Standby Configuration with One RA4000/4100 Array	77
Active/Standby Configuration with Two RA4000/4100 Arrays	78
Active/Standby Configuration with Three RA4000/RA4100 Arrays	79
Active/Standby Configuration with Four RA4000/RA4100 Arrays	80
Active/Standby Configuration with Five RA4000/RA4100 Arrays	81
Active/Active Configuration Examples for Redundant FCALs	82
Active/Active Configuration with Two RA4000/RA4100 Arrays	84
Active/Active Configuration with Three RA4000/RA4100 Arrays	85
Active/Active Configuration with Four RA4000/RA4100 Arrays	86
Active/Active Configuration with Five RA4000/RA4100 Arrays	87
Summary of I/O Path Failure and Failover Scenarios for Redundant FCALs	88
Cluster Interconnect Options	94
Ethernet Cluster Interconnect	94
Non-Redundant Ethernet Cluster Interconnect Components	95
Redundant Ethernet Cluster Interconnect Components	95
Ethernet Cluster Interconnect Adapters	96
Ethernet Switch	96
Ethernet Cluster Interconnect Diagrams	97
Local Area Network	100
Chapter 3: Cluster Software Components	101
Overview of the Cluster Software	101
Microsoft Windows 2000 Advanced Server	101
Compaq Software	102
Compaq SmartStart and Support Software	102
Compaq System Configuration Utility	103
Compaq Array Configuration Utility	103
Fibre Channel Fault Isolation Utility	103
Compaq Insight Manager	104
Compaq Insight Manager XE	104
Compaq Options ROMPaq	104
Compaq StorageWorks Secure Path for Windows 2000	105
Compaq Operating System Dependent Modules	105
OSDs for Oracle8i Parallel Server	105
Oracle Software	106
Oracle8i Server Enterprise Edition	106
Oracle8i Server	106
Oracle8i Parallel Server Option	106
Oracle8i Enterprise Manager	107
Oracle8i Certification	107
Application Failover and Reconnection Software	108
Chapter 4: Cluster Planning	109
Site Planning	110
Capacity Planning for Cluster Hardware	111
Compaq ProLiant Servers	111
Planning Shared Storage Components for Redundant Fibre Channel Fabrics	111
Planning Shared Storage Components for Redundant Fibre Channel Arbitrated Loops	113
Planning Cluster Interconnect and Client LAN Components	114
Planning an Ethernet Cluster Interconnect	114
Planning the Client LAN	114
Planning Cluster Configurations for Redundant Fibre Channel Fabrics	115
Sample Small Configuration in a Redundant Fibre Channel Fabric	115
Sample Large Configuration in a Redundant Fibre Channel Fabric	117
Planning Cluster Configurations for Redundant Fibre Channel Arbitrated Loops	119
Sample Small Configuration in a Redundant FCAL	119
Sample Large Configuration in a Redundant FCAL	121
RAID Planning	122
Supported RAID Levels	124
Raw Data Storage and Database Size	125
Selecting the Appropriate RAID Levels	125
Planning the Grouping of Physical Disk Storage Space	126
Disk Drive Planning	127
Nonshared Disk Drives	127
Shared Disk Drives	128
Network Planning	128
Windows 2000 Advanced Server Hosts Files for an Ethernet Cluster Interconnect	128
Client LAN	129
Chapter 5: Installation and Configuration	130
Installation Overview	131
Installing the Hardware	133
Setting Up the Nodes	133
Installing the Fibre Host Adapters	133
Installing GBIC-SW Modules for the Fibre Host Adapters	134
Cabling the Fibre Host Adapters to the Storage Hubs or Switches	134
Installing the Cluster Interconnect Adapters	135
Ethernet Cluster Interconnect Adapters	136
Installing the Client LAN Adapters	136
Setting Up the RA4000/RA4100 Arrays	136
Compaq Array Configuration Utility	137
Installing GBIC-SW Modules for the RA4000 Array Controllers	138
Cabling the Storage Hubs or Switches to the RA4000 Array Controllers	138
Cabling an Active/Standby Configuration	139
Cabling an Active/Active Configuration	140
Installing Additional Redundant Fibre Channel Fabrics	144
Installing Additional Redundant FC-ALs	144
Cabling the Ethernet Cluster Interconnect	145
Cabling the Client LAN	149
Redundant Client LAN	149
Installing the Operating System Software and Configuring the RA4000/RA4100 Arrays	150
Guidelines for Clusters	150
Automated Installation Using SmartStart	151
Cluster-Specific SmartStart Installation	151
Automated Installation Steps	152
Installing Secure Path Software for Windows 2000	155
Installing the Secure Path Server Software	155
Installing the Secure Path Client Software	156
Verifying Shared Disk Storage Using Secure Path Manager	156
Defining Active Array Controllers	157
Installing Compaq OSDs	159
Verifying Cluster Communications	160
Mounting Remote Drives and Verifying Administrator Privileges	161
Installing the Ethernet OSDs	162
Installing Oracle Software	173
Configuring Oracle Software	174
Installing Object Link Manager	174
Additional Notes on Configuring Oracle Software	175
Creating a Symbolic Link for OPS_CMDISK	175
Verifying Access to Shared Storage from All Nodes	175
Verifying the Hardware and Software Installation	176
Cluster Communications	176
Access to Shared Storage from All Nodes	176
OSDs	176
Other Verification Tasks	177
Power Distribution and Power Sequencing Guidelines	177
Server Power Distribution	178
RA4000/RA4100 Array Power Distribution	178
Power Sequencing	179
Chapter 6: Cluster Management	180
Cluster Management Concepts	181
Powering Off a Node Without Interrupting Cluster Services	181
Managing a Cluster in a Degraded Condition	181
Managing Network Clients Connected to a Cluster	182
Cluster Events	182
Management Applications	183
Monitoring Server and Network Hardware	183
Managing Shared Drives	184
Monitoring Redundant Fibre Channel Fabrics	184
Monitoring Redundant Fibre Channel Arbitrated Loops	185
Monitoring the Database	186
Remotely Managing a Cluster	186
Software Maintenance for Oracle8i	187
Deinstalling the OSDs	187
Upgrading Oracle8i Server	190
Upgrading the OSDs	190
Upgrading the Ethernet OSDs	191
Deinstalling a Partial OSD Installation	192
Upgrading Oracle8i Server	193
Managing Changes to Shared Storage Components	193
Replacing a Failed Disk	193
Adding Disk Drives to Increase Storage Capacity	194
Adding an RA4000/RA4100 Array	194
Replacing a Failed Fibre Host Adapter	195
Replacing a Cluster Node	196
Removing the Node	196
Adding the Replacement Node	197
Preparing the Replacement Node	197
Installing the Cluster Software	198
Adding a Cluster Node	200
Preparing the New Node	201
Preparing the Existing Cluster Nodes	202
Installing the Cluster Software for Oracle8i	202
Monitoring Cluster Operation	204
Tools Overview	204
Using Secure Path Manager	204
Monitoring Storage Systems	205
Manipulating Storagesets and I/O Paths	206
Detecting I/O Path and Array Controller Failures	206
Identifying I/O Path and Array Controller Failures	207
Uninstalling Secure Path	207
Chapter 7: Troubleshooting	208
Basic Troubleshooting Tips	209
Power	209
Physical Connections	209
Access to Cluster Components	210
Software Revisions	210
Firmware Revisions	211
Troubleshooting Oracle8i and OSD Installation Problems and Error Messages	212
Potential Difficulties Installing the OSDs with the Oracle Universal Installer	212
Unable to Start OracleCMService	213
Unable to Start OracleNMService	214
Unable to Start the Database	214
Initialization of the Dynamic Link Library NM.DLL Failed	215
Troubleshooting Node-to-Node Connectivity Problems	215
Nodes Are Unable to Communicate with Each Other	215
When Using the Ethernet Cluster Interconnect	215
Unable to Ping the Cluster Interconnect or the Client LAN	216
When Using the Ethernet Cluster Interconnect	216
Node or Nodes Unable to Rejoin the Cluster	216
Troubleshooting Client-to-Cluster Connectivity Problems	217
A Network Client Cannot Communicate with the Cluster	217
Troubleshooting Shared Storage Problems	218
Verifying Connectivity to a Redundant Fibre Channel Fabric	218
Verifying Connectivity to a Redundant Fibre Channel Arbitrated Loop	219
Shared Disks in the RA4000/RA4100 Arrays Are Not Recognized By One or More Nodes	219
A Cluster Node Cannot Connect to the Shared Drives	221
Troubleshooting Secure Path	221
Secure Path Guidelines for Windows 2000 Advanced Server	221
Secure Path Manager Cannot Start With Hosts That Use Hyphenated Host Names	222
Secure Path Manager Is Delayed In Reporting Path Failure Information	223
The Addition of New LUNs Causes an Error	223
A Configuration of More Than 64 LUNs Prevents the Secure Path Agent From Starting	223
Appendix A: Diagnosing and Resolving Shared Disk Problems	224
Introduction	224
Run Object Link Manager on All Nodes	226
Restart All Affected Nodes in the Cluster	227
Rerun and Validate Object Link Manager On All Affected Nodes	227
Run and Validate Secure Path Manager On All Nodes	228
Run Disk Management On All Nodes	228
Run and Validate the Array Configuration Utility On All Nodes	229
Perform Cluster Software and Firmware Checks	229
Perform Cluster Hardware Checks	230
Contact Your Compaq Support Representative	231
Glossary	232

Match case Limit results 1 per page

5-4

Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide

Installing the Hardware

Setting Up the Nodes

Physically preparing the nodes (servers) for a cluster is not very different than

preparing them for individual use. You will install all necessary adapters and

insert all internal hard disks. You will attach network cables and plug in SCSI

and Fibre Channel cables. The primary difference is in setting up the shared

storage subsystem.

Set up the hardware on one node completely, then set up the rest of the nodes

identically to the first one. Do not load any software on any cluster node until

all the hardware has been installed in all cluster nodes. Before loading

software, read “Installing the Operating System Software and Configuring the

RA4000/RA4100 Arrays” in this chapter to understand the idiosyncrasies of

configuring a cluster.

IMPORTANT:

With some possible exceptions, the servers in the cluster must be set up

identically. The cluster components common to all nodes in the cluster must be identical,

for example, the ProLiant server model, cluster interconnect adapters, amount of memory,

cache, and number of CPUs must be the same for each cluster node. It also means the

Fibre host adapters must be installed into the same PCI slots in each server.

While setting up the physical hardware, follow the installation instructions in

your

Compaq ProLiant Server Setup and Installation Guide

and in your

Compaq ProLiant Server Installation poster. When you are ready to install the

Fibre Host Adapters and your cluster interconnect adapters, refer to the

instructions in the pages that follow.

Installing the Fibre Host Adapters

Each redundant Fibre Channel Fabric or redundant Fibre Channel Arbitrated

Loop (FC-AL) requires at least two Fibre Host Adapters in each cluster node.

Install these devices as you would any other PCI adapter.

Install two Fibre Host Adapters on the same PCI bus in each server and into

the same two PCI slots in each server. If you need specific instructions, see the

Compaq StorageWorks Fibre Channel Host Bus Adapter Installation Guide

the

Compaq StorageWorks 64-Bit/66 MHz

Fibre Channel Host Adapter

Installation Guide