Compaq ProLiant 6400R Compaq Parallel Database Cluster Model PDC/O1000 for Ora - Page 5

Contents

v

Chapter 3

Cluster Software Components

Overview of the Cluster Software

............................................................................

3-1

Microsoft Windows 2000 Advanced Server

............................................................

3-2

Compaq Software

....................................................................................................

3-2

Compaq SmartStart and Support Software

.......................................................

3-2

Compaq System Configuration Utility

.............................................................

3-3

Compaq Array Configuration Utility

................................................................

3-3

Fibre Channel Fault Isolation Utility

................................................................

3-3

Compaq Insight Manager

.................................................................................

3-4

Compaq Insight Manager XE

...........................................................................

3-4

Compaq Options ROMPaq

...............................................................................

3-4

Compaq Operating System Dependent Modules

..............................................

3-5

Oracle Software

.......................................................................................................

3-5

Oracle8

i

Server Enterprise Edition

...................................................................

3-5

Oracle8

i

Server

.................................................................................................

3-6

Oracle8

i

Parallel Server Option

........................................................................

3-6

Oracle8

i

Enterprise Manager

............................................................................

3-6

Oracle8

i

Certification

.......................................................................................

3-7

Application Failover and Reconnection Software

...................................................

3-7

Chapter 4

Cluster Planning

Site Planning

............................................................................................................

4-2

Capacity Planning for Cluster Hardware

.................................................................

4-3

Compaq ProLiant Servers

.................................................................................

4-3

Planning Shared Storage Components for Non-Redundant Fibre Channel

Fabrics

..............................................................................................................

4-4

Planning Shared Storage Components for Non-Redundant Fibre Channel

Arbitrated Loops

...............................................................................................

4-5

Planning Cluster Interconnect and Client LAN Components

...........................

4-6

Planning Cluster Configurations for Non-Redundant Fibre Channel Fabrics

.........

4-7

Sample Midsize Cluster with One Non-Redundant Fibre Channel Fabric

.......

4-8

Sample Large Cluster with One Non-Redundant Fibre Channel Fabric

...........

4-9

Planning Cluster Configurations for Non-Redundant Fibre Channel Arbitrated

Loops

.....................................................................................................................

4-10

Sample Midsize Cluster with One Non-Redundant FC-AL

...........................

4-10

Sample Large Cluster with One Non-Redundant FC-AL

...............................

4-11

RAID Planning

......................................................................................................

4-12

Supported RAID Levels

.................................................................................

4-14

Raw Data Storage and Database Size

.............................................................

4-15

Selecting the Appropriate RAID Levels

.........................................................

4-16

Planning the Grouping of Physical Disk Storage Space

........................................

4-17

Compaq ProLiant 6400R Compaq Parallel Database Cluster Model PDC/O1000 for Ora - Page 5

Cluster Software Components, Cluster Planning

Page 5 highlights

Section	Page
Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide	1
Notice	2
Contents	3
About This Guide	12
Purpose	12
Audience	12
Scope	13
Referenced Manuals	14
Supplemental Documents	15
Text Conventions	16
Symbols in Text	16
Symbols on Equipment	17
Rack Stability	18
Getting Help	18
Compaq Technical Support	18
Compaq Website	19
Compaq Authorized Reseller	19
Chapter 1: Clustering Overview	20
Clusters Defined	21
Availability	22
Scalability	22
Compaq Parallel Database Cluster Overview	23
Chapter 2: Architecture	24
Compaq ProLiant Servers	25
HighAvailability Features of ProLiant Servers	26
Shared Storage Components	26
RA4000 Array	27
RA4100 Array	27
RA4000 Array Controller	28
Fibre Channel SAN Switch	28
Storage Hub	29
FCAL Switch	30
Fibre Host Adapters	30
Gigabit Interface Converter-Shortwave Modules	31
Fibre Channel Cables	31
Availability Features of the Shared Storage Components	32
I/O Path Configurations in a NonRedundant Fibre Channel Fabric	32
Overview of Fibre Channel Fabric SAN Topology	32
Non-Redundant Fibre Channel Fabric	33
Using Multiple NonRedundant Fibre Channel Fabrics	34
Maximum Distances Between Cluster Nodes and Shared Storage Subsystem Components in a NonRedundant Fibre Channel Fabric	36
I/O Data Paths for a NonRedundant Fibre Channel Fabric	36
Fibre Host Adapter-to-Fibre Channel SAN Switch Paths	37
Fibre Channel SAN Switch-to-Array Controller Paths	38
I/O Path Configurations in a NonRedundant Fibre Channel Arbitrated Loop	39
Overview of FCAL SAN Topology	39
Non-Redundant Fibre Channel Arbitrated Loop	39
Using Multiple Non-Redundant Fibre Channel Arbitrated Loops	41
Maximum Distances Between Cluster Nodes and Shared Storage Subsystem Components in a NonRedundant FCAL	43
I/O Data Paths for a Non-Redundant FC-AL	43
Fibre Host Adapter-to-Storage Hub/FCAL Switch Paths	44
Storage Hub/FC-AL Switch-to-Array Controller Paths	45
Cluster Interconnect Options	46
Ethernet Cluster Interconnect	46
Non-Redundant Ethernet Cluster Interconnect Components	47
Redundant Ethernet Cluster Interconnect Components	47
Ethernet Cluster Interconnect Adapters	47
Ethernet Switch	48
Ethernet Cluster Interconnect Diagrams	49
Local Area Network	52
Chapter 3: Cluster Software Components	53
Overview of the Cluster Software	53
Microsoft Windows 2000 Advanced Server	54
Compaq Software	54
Compaq SmartStart and Support Software	54
Compaq System Configuration Utility	55
Compaq Array Configuration Utility	55
Fibre Channel Fault Isolation Utility	55
Compaq Insight Manager	56
Compaq Insight Manager XE	56
Compaq Options ROMPaq	56
Compaq Operating System Dependent Modules	57
OSDs for Oracle8i Parallel Server	57
Oracle Software	57
Oracle8i Server Enterprise Edition	57
Oracle8i Server	58
Oracle8i Parallel Server Option	58
Oracle8i Enterprise Manager	58
Oracle8i Certification	59
Application Failover and Reconnection Software	59
Chapter 4: Cluster Planning	60
Site Planning	61
Capacity Planning for Cluster Hardware	62
Compaq ProLiant Servers	62
Planning Shared Storage Components for NonRedundant Fibre Channel Fabrics	63
Planning Shared Storage Components for NonRedundant Fibre Channel Arbitrated Loops	64
Planning Cluster Interconnect and Client LAN Components	65
Planning an Ethernet Cluster Interconnect	65
Planning the Client LAN	66
Planning Cluster Configurations for NonRedundant Fibre Channel Fabrics	66
Sample Midsize Cluster with One NonRedundant Fibre Channel Fabric	67
Sample Large Cluster with One Non-Redundant Fibre Channel Fabric	68
Planning Cluster Configurations for NonRedundant Fibre Channel Arbitrated Loops	69
Sample Midsize Cluster with One NonRedundant FCAL	69
Sample Large Cluster with One Non-Redundant FCAL	70
RAID Planning	71
Supported RAID Levels	73
Raw Data Storage and Database Size	74
Selecting the Appropriate RAID Levels	75
Planning the Grouping of Physical Disk Storage Space	76
Disk Drive Planning	77
Nonshared Disk Drives	77
Shared Disk Drives	77
Network Planning	78
Windows 2000 Advanced Server Hosts Files for an Ethernet Cluster Interconnect	78
Client LAN	79
Chapter 5: Installation and Configuration	80
Installation Overview	81
Installing the Hardware	82
Setting Up the Nodes	82
Installing the Fibre Host Adapters	83
Installing GBIC-SW Modules for the Fibre Host Adapters	83
Cabling the Fibre Host Adapters to the Storage Hub, FC-AL Switch, or Fibre Channel SAN Switch	84
Installing the Cluster Interconnect Adapters	85
Ethernet Cluster Interconnect Adapters	86
Installing the Client LAN Adapters	86
Setting Up the RA4000/RA4100 Arrays	87
Compaq Array Configuration Utility	87
Installing GBIC-SW Modules for the RA4000 Array Controller	88
Cabling the Storage Hub, FC-AL Switch, or Fibre Channel SAN Switch to the RA4000 Array Controllers	88
Installing Additional Fibre Channel Fabrics or FCALs	89
Cabling the Ethernet Cluster Interconnect	90
Cabling the Client LAN	95
Redundant Client LAN	95
Installing Operating System Software and Configuring the RA4000/RA4100 Arrays	96
Guidelines for Clusters	96
Automated Installation Using SmartStart	97
Cluster-Specific SmartStart Installation	97
Automated Installation Steps	98
Installing Compaq OSDs	101
Verifying Cluster Communications	101
Mounting Remote Drives and Verifying Administrator Privileges	102
Installing the Ethernet OSDs	103
Installing Oracle Software	115
Configuring Oracle Software	115
Installing Object Link Manager	115
Additional Notes on Configuring Oracle Software	116
Creating a Symbolic Link for OPS_CMDISK	116
Verifying Access to Shared Storage from All Nodes	116
Verifying the Hardware and Software Installation	117
Cluster Communications	117
Access to Shared Storage from All Nodes	117
OSDs	117
Other Verification Tasks	118
Power Distribution and Power Sequencing Guidelines	118
Server Power Distribution	119
RA4000/RA4100 Array Power Distribution	119
Power Sequencing	120
Chapter 6: Cluster Management	121
Cluster Management Concepts	122
Powering Off a Node Without Interrupting Cluster€Services	122
Managing a Cluster in a Degraded Condition	122
Managing Network Clients Connected to a Cluster	123
Cluster Events	123
Management Applications	124
Monitoring Server and Network Hardware	124
Managing Shared Drives	125
Monitoring NonRedundant Fibre Channel Fabrics	125
Monitoring NonRedundant Fibre Channel Arbitrated Loops	126
Monitoring the Database	126
Remotely Managing a Cluster	127
Software Maintenance for Oracle8i	128
Deinstalling the OSDs	128
Upgrading Oracle8i Server	131
Upgrading the OSDs	131
Upgrading the Ethernet OSDs	132
Deinstalling a Partial OSD Installation	133
Upgrading Oracle8i Server	134
Managing Changes to Shared Storage Components	134
Replacing a Failed Disk	134
Adding Disk Drives to Increase Shared Storage Capacity	135
Adding an RA4000/RA4100 Array	136
Replacing a Failed Fibre Host Adapter	137
Replacing a Cluster Node	138
Removing the Node	138
Adding the Replacement Node	139
Preparing the Replacement Node	139
Installing the Cluster Software	140
Adding a Cluster Node	141
Preparing the New Node	142
Preparing the Existing Cluster Nodes	143
Installing the Cluster Software for Oracle8i	143
Monitoring Cluster Operation	145
Tools Overview	145
Chapter 7: Troubleshooting	146
Basic Troubleshooting Tips	147
Power	147
Physical Connections	147
Access to Cluster Components	148
Software Revisions	148
Firmware Revisions	149
Troubleshooting Oracle8i and OSD Installation Problems and Error Messages	150
Potential Difficulties Installing the OSDs with the Oracle Universal Installer	150
Unable to Start OracleCMService	151
Unable to Start OracleNMService	152
Unable to Start the Database	152
Initialization of the Dynamic Link Library NM.DLL Failed	153
Troubleshooting Node-to-Node Connectivity Problems	153
Nodes Are Unable to Communicate with Each Other	153
When Using an Ethernet Cluster Interconnect	154
Unable to Ping the Cluster Interconnect or the Client LAN	154
When Using an Ethernet Cluster Interconnect	154
Node or Nodes Unable to Rejoin the Cluster	155
Troubleshooting Client-to-Cluster Connectivity Problems	155
A Network Client Cannot Communicate with the Cluster	155
Troubleshooting Shared Storage Problems	156
Verifying Connectivity to a NonRedundant Fibre Channel Fabric	156
Verifying Connectivity to a NonRedundant Fibre Channel Arbitrated Loop	157
Shared Disks in the RA4000/RA4100 Arrays Are Not Recognized By One or More Nodes	157
A Cluster Node Cannot Connect to the Shared Drives	159
Appendix A: Diagnosing and Resolving Shared Disk Problems	160
Introduction	160
Run Object Link Manager On All Nodes	162
Restart All Affected Nodes in the Cluster	163
Rerun and Validate Object Link Manager On All Affected Nodes	163
Run Disk Management On All Nodes	164
Run and Validate the Array Configuration Utility On All Nodes	164
Perform Cluster Software and Firmware Checks	165
Perform Cluster Hardware Checks	165
Contact Your Compaq Support Representative	166
Glossary	167