Compaq ProLiant 6500 Compaq Parallel Database Cluster Model PDC/O1000 for Orac
Compaq ProLiant 6500 Manual
View all Compaq ProLiant 6500 manuals
Add to My Manuals
Save this manual to your list of manuals |
Compaq ProLiant 6500 manual content summary:
- Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 1
Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Second Edition (June 2001) Part Number 225083-002 Compaq Computer Corporation - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 2
Compaq Computer Corporation Compaq, the Compaq logo, Compaq Insight Manager, SmartStart, Rompaq, ProLiant, and StorageWorks Registered in U.S. Patent and Trademark Office. ActiveAnswers is a trademark of Compaq be trademarks of their respective companies. Compaq shall not be liable for technical or - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 3
About This Guide Purpose ...xiii Audience...xiii Scope ...xiv Referenced Manuals xv Supplemental Documents xvi Text Conventions...xvii Symbols in Text...xvii Symbols on Equipment xviii Rack Stability ...xix Getting Help ...xix Compaq Technical Support xix Compaq Website xx Compaq Authorized - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 4
Guide Chapter 2 Architecture Compaq ProLiant Servers 2-2 High-Availability Features of ProLiant Servers 2-3 Shared Storage Components 2-3 RA4000 Array 2-4 RA4100 Array 2-4 RA4000 Array Controller Fibre Channel Fabric 2-13 I/O Path Configurations in a Non-Redundant Fibre Channel Arbitrated - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 5
2000 Advanced Server 3-2 Compaq Software ...3-2 Compaq SmartStart and Support Software 3-2 Compaq System Configuration Utility 3-3 Compaq Array Configuration Utility 3-3 Fibre Channel Fault Isolation Utility 3-3 Compaq Insight Manager 3-4 Compaq Insight Manager XE 3-4 Compaq Options ROMPaq - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 6
5-11 Cabling the Client LAN 5-16 Installing Operating System Software and Configuring the RA4000/RA4100 Arrays...5-17 Guidelines for Clusters 5-17 Automated Installation Using SmartStart 5-18 Installing Compaq OSDs 5-22 Verifying Cluster Communications 5-22 Mounting Remote Drives and Verifying - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 7
Installation and Configuration continued Verifying the Hardware and Software Installation 5-38 Cluster Communications 5-38 Access to Shared Storage from All Nodes 5-38 OSDs ...5-38 Other Verification Tasks 5-39 Power Distribution and Power Sequencing Guidelines 5-39 Server Power Distribution - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 8
6-25 Tools Overview 6-25 Chapter 7 Troubleshooting Basic Troubleshooting Tips 7-2 Power ...7-2 Physical Connections 7-2 Access to Cluster Components 7-3 Software Revisions 7-3 Firmware Revisions 7-4 Troubleshooting Oracle8i and OSD Installation Problems and Error Messages ..... 7-5 Potential - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 9
Link Manager On All Affected Nodes A-4 Run Disk Management On All Nodes A-5 Run and Validate the Array Configuration Utility On All Nodes A-5 Perform Cluster Software and Firmware Checks A-6 Perform Cluster Hardware Checks A-6 Contact Your Compaq Support Representative A-7 Glossary Index - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 10
for Oracle8i and Windows 2000 Administrator Guide List of Figures Figure 1-1. Example of a six-node Compaq Parallel Database Cluster Model PDC/O1000 I/O data paths ...2-14 Figure 2-6. Fibre Channel SAN Switch-to-array controller I/O data path 2-15 Figure 2-7. Two-node PDC/O1000 cluster with one - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 11
Redundant Ethernet cluster interconnect for a two-node PDC/O1000 cluster 5-15 Figure 5-7. Server power distribution in a three-node cluster 5-40 Figure A-1. Tasks for diagnosing and resolving shared disk problems A-2 List of Tables Table 2-1 High-Availability Components of ProLiant Servers 2-3 - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 12
, configuration, and maintenance information that can be valuable for a variety of users. If you are installing the PDC/O1000 but will not be administering the cluster on a daily basis, please make this guide available to the person or persons who will be responsible for the clustered servers after - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 13
high-availability features of the PDC/O1000 I Planning and designing a PDC/O1000 cluster configuration to meet your business needs I Installing and configuring PDC/O1000 hardware and software I Managing the PDC/O1000 I Troubleshooting the PDC/O1000 The following summarizes the contents of this guide - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 14
These related manuals include, but are not limited to: I Documentation related to the ProLiant servers you are clustering (for example, guides, posters, and performance and tuning guides) I Compaq StorageWorks documentation G Compaq StorageWorks RAID Array 4000 User Guide G Compaq StorageWorks RAID - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 15
Guide G Oracle Enterprise Manager Configuration Guide G Oracle Enterprise Manager Concepts Guide Supplemental Documents The following technical documents contain important supplemental information for the Compaq Parallel Database Cluster Model PDC/O1000: I Supported Ethernet Interconnects for Compaq - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 16
in bodily harm or loss of life. CAUTION: Text set off in this manner indicates that failure to follow directions could result in damage to equipment or loss of information. IMPORTANT: Text set off in this manner presents clarifying information or specific instructions. NOTE: Text set off in this - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 17
exists. WARNING: To reduce the risk of injury from a hot component, allow the surface to cool before touching. Power Supplies or Systems marked with these symbols indicate the equipment is supplied by multiple sources of power. WARNING: To reduce the risk of injury from electrical shock, remove all - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 18
If you have a problem and have exhausted the information in this guide, you can get further information and other help in the following locations. Compaq Technical Support In North America, call the Compaq Technical Phone Support Center at 1-800-OK-COMPAQ. This service is available 24 hours a day - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 19
/O1000 for Oracle8i and Windows 2000 Administrator Guide Compaq Website The Compaq website has information on this product as well as the latest drivers and Flash ROM images. You can access the Compaq website by logging on to the Internet at www.compaq.com Compaq Authorized Reseller For the name of - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 20
mission controls-many critical applications can reach the desired levels of availability and scalability with non-proprietary client/server-based clustering. These clustering solutions use industry-standard hardware and software, thereby providing key clustering features at a lower price than - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 21
Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i on Windows 2000 Administrator Guide Clusters Defined A cluster is an integration of software and hardware products that enables a set of loosely coupled servers servers and shared storage subsystem components offers a level of availability - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 22
topologies support the availability is the measure of how well a computer system can continuously deliver services to clients. Availability new system. In a stand-alone server configuration, scalable systems allow increased throughput by adding processors or more memory. In a cluster configuration - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 23
, proprietary cluster solutions, at a fraction of the cost. The PDC/O1000 combines the Microsoft Windows 2000 Advanced Server operating system and the industry-leading Oracle8i Parallel Server with award-winning Compaq ProLiant servers and shared storage subsystems. Together, these hardware and - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 24
. The hardware products include: I Compaq Proliant servers I Shared storage components G Compaq StorageWorks RAID Array 4100s (RA4100 Arrays) or Compaq StorageWorks RAID Array 4000s (RA4000 Arrays) G Compaq StorageWorks RAID Array 4000 Controller (RA4000 Array Controller) installed in each RA4000 - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 25
is a Compaq ProLiant server. With some exceptions, all nodes in a PDC/O1000 cluster must be identical in model. In addition, all components common to all nodes in a cluster, such as memory, number of CPUs, and the interconnect adapters, must be identical and identically configured. NOTE: Certain - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 26
power supplies, redundant cooling fans, and Error Checking and Correcting (ECC) memory. The high-availability features of ProLiant servers are a critical foundation of Compaq clustering products. Table 2-1 lists the high-availability features found in many ProLiant servers Array Controller in each - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 27
Arrays. Each RA4000 Array contains one single-port RA4000 Array Controller. The array controller connects the RA4000 Array to one Storage Hub, FC-AL information about the RA4000 Array, refer to the Compaq StorageWorks RAID Array 4000 User Guide. RA4100 Array The RA4100 Array is another shared - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 28
which it has been configured. Because the array controller processes the I/O requests, Controller, refer to the Compaq StorageWorks RAID Array 4000 User Guide or the Compaq StorageWorks RAID Array 4100 User Guide 100 MBps bandwidth on every port. Adding new devices to ports on the Fibre Channel SAN - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 29
Guide For further information, refer to these manuals provided with the Fibre Channel SAN Switch: I Compaq StorageWorks Fibre Channel SAN Switch 8 Installation and Hardware Guide I Compaq one Fibre Host Adapter in each node with the array controller in the RA4000/RA4100 Array. The Fibre Host Adapter - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 30
in each node with the array controller in the RA4000/RA4100 Array. supports 11 ports. For further information, refer to the Compaq StorageWorks Fibre Channel Arbitrated Loop Switch (FC-AL Switch) User Guide. Fibre Host Adapters Fibre Host Adapters are the interface between the cluster nodes (servers - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 31
Channel Host Bus Adapter Installation Guide or the Compaq StorageWorks 64-Bit/66-MHz Fibre Channel Host Adapter Installation Guide. Gigabit Interface Converter-Shortwave Modules A Gigabit Interface Converter-Shortwave (GBIC-SW) module must be installed at both ends of a Fibre Channel cable. One - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 32
accepts redundant, hot-pluggable power supplies and a hot-pluggable fan module. The RA4000 Array Controller supports pre-failure notification on hard drives and provides an Array Accelerator made with ECC memory. The Array Accelerator is backed with onboard rechargeable batteries, ensuring that the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 33
Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Non-Redundant Fibre Channel Fabric The PDC/O1000 supports Fibre Host Adapter in each cluster node and the RA4000 Array Controller in each RA4000/RA4100 Array for that fabric. These hardware components - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 34
ProLiant Server Figure 2-2. Two-node PDC/O1000 cluster with one non-redundant Fibre Channel Fabric Using Multiple Non-Redundant Fibre Channel Fabrics The PDC/O1000 supports to: I Increase the amount of shared storage available to the cluster's servers when the Fibre Channel SAN Switch in your first - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 35
the number of Fibre Host Adapters your Compaq ProLiant servers support. Refer to the Compaq ProLiant server documentation for this information. Figure 2-3 shows two non-redundant Fibre Channel Fabrics In Figure 2-3, the original non-redundant Fibre Channel Fabric connects to eight RA4000/RA4100 - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 36
Fibre Host Adapter in each node to the Fibre Channel SAN Switch. I Another path runs from the Fibre Channel SAN Switch to the RA4000 Array Controller in each RA4000/RA4100 Array of that Fibre Channel Fabric. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 37
Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide ProLiant Servers Switch (Cluster Interconnect) ProLiant Servers It affects only those users who are connected to RA4100 Array, the status of the array controller in that RA4000/RA4100 Array changes to - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 38
/RA4100 Array of a Fibre Channel Fabric. There is one path for each array controller. Fibre Host Adapters Client LAN Fibre Host Adapters ProLiant Servers Switch (Cluster Interconnect) ProLiant Servers Fibre Channel SAN Switch RA4000/4100 Array Figure 2-6. Fibre Channel SAN Switch-to-array - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 39
Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide I/O Path Configurations is available for each port. When you use a FC-AL Switch, the FC-AL SAN supports multiple each cluster node and the RA4000 Array Controller in each RA4000/RA4100 Array for that - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 40
FC-AL Switch and the Storage Hub or FC-AL Switch to each array controller Figure 2-7 shows a two-node PDC/O1000 with one non-redundant FC-AL. RA4000/4100 RA4000/4100 RA4000/4100 Array Array Array Fibre Host Adapter ProLiant Server Node 1 Storage Hub/ FC-AL Switch Node 2 Fibre Host Adapter - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 41
ALs in a PDC/O1000 cluster to: I Increase the amount of shared storage available to the cluster's servers when the Storage Hub or FC-AL Switch in your first FC-AL is of Fibre Host Adapters your Compaq ProLiant servers support. Refer to the Compaq ProLiant server documentation for this information. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 42
Storage Hubs/ FC-AL Switches Fibre Host Adapters RA4000/4100 Arrays (4) Figure 2-8. PDC/O1000 cluster with two non-redundant FC-ALs In Figure 2-8, the original non-redundant FC-AL connects to eight RA4000/RA4100 Arrays. The second non-redundant FC-AL connects to four RA4000/RA4100 Arrays. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 43
2-20 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Maximum Distances Between Cluster Nodes and Shared Storage Subsystem Components Storage Hub or FC-AL Switch to the RA4000 Array Controller in each RA4000/RA4100 Array of that non-redundant FC-AL. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 44
Adapters ProLiant Servers Switch (Cluster Interconnect) ProLiant Servers Storage It affects only those users who are connected to through another node. Note that Compaq Insight Manager monitors the health of RA4100 Array, the status of the array controller in that RA4000/RA4100 Array changes to - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 45
in each RA4000/RA4100 Array of the FC-AL. There is one path for each array controller. Fibre Host Adapters Client LAN Fibre Host Adapters ProLiant Servers Switch (Cluster Interconnect) ProLiant Servers Storage Hub/FC-AL Switch RA4000/4100 Array Figure 2-11. Storage Hub/FC-AL Switch-to - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 46
paper Supported Ethernet Interconnects for Compaq Parallel Database Clusters Using Oracle Parallel Server (ECG062/0299) for detailed information about configuring redundant and non-redundant Ethernet cluster interconnects. This document is available at www.compaq.com/support/techpubs/whitepapers - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 47
2-24 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Non-Redundant Ethernet Cluster cables to a separate Ethernet adapter in each server. NOTE: In a redundant Ethernet cluster configuration, one Ethernet crossover cable must be installed between - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 48
to the technical white paper Supported Ethernet Interconnects for Compaq Parallel Database Clusters Using Oracle Parallel Server for detailed information about configuring redundant Ethernet cluster interconnects. This document is available at www.compaq.com/support/techpubs/whitepapers For detailed - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 49
2-26 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Ethernet Cluster Interconnect Diagrams Figure 2-12 shows the non-redundant Ethernet cluster interconnect components used in a two-node PDC/O1000 cluster. These components include a dual- - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 50
Architecture 2-27 Figure 2-13 shows another option for a non-redundant Ethernet cluster interconnect in a two-node PDC/O1000 cluster. These include an Ethernet adapter in each node connected by Ethernet cables to an Ethernet switch or hub. RA4000/4100 Array Storage Hub/Switch Ethernet Adapter - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 51
for Oracle8i and Windows 2000 Administrator Guide Because Ethernet switches are required in PDC/O1000 clusters with three or more nodes, using an Ethernet switch instead of a crossover cable makes it easier to upgrade the cluster interconnect if more servers are added to the cluster. IMPORTANT - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 52
communicate. In the case of a cluster, the hardware components of the client LAN are no different than in a standalone server configuration. The software components used by network clients should have the ability to detect node failures and automatically reconnect the client to another cluster - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 53
modules (OSDs) and this Oracle software: I Oracle8i Enterprise Edition with the Oracle8i Parallel Server Option NOTE: For information about currently-supported software revisions for the, refer to the Compaq Parallel Database Cluster Model PDC/O1000 for Windows 2000 Certification Matrix at www - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 54
the Compaq ProLiant servers in a PDC/O1000 cluster. SmartStart uses an automated step-by-step process to configure the operating system and load the system software. The Compaq SmartStart and Support Software CD also contains device drivers and utilities that enable you to take advantage of specific - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 55
addresses, boot order of disk controllers, and so on. For information about the System Configuration Utility, see the Compaq Server Setup and Management pack. Compaq Array Configuration Utility The Compaq Array Configuration Utility, found on the Compaq SmartStart and Support Software CD, is used to - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 56
optional Remote Insight controller, and SmartStart. In Compaq servers, each hardware subsystem, such as non-shared disk storage, system memory, and system processor, has a robust set of management capabilities. Compaq Full Spectrum Fault Management notifies the end user of impending fault conditions - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 57
the OSD layer interacts with Oracle8i Parallel Server, refer to the Oracle8i Parallel Server Setup and Configuration Guide. Also refer to "Installing Compaq OSDs" in Chapter 5, "Installation and Configuration." The OSD software is found on the Compaq Parallel Database Cluster Clustering Software for - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 58
Oracle8i Parallel Server Setup and Configuration Guide I Other Oracle documentation for Oracle8i Server and Oracle8i Parallel Server provided with cluster. Hardware components are managed with Compaq Insight Manager. To conserve space for cluster resources (memory, processes), you should not install - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 59
series of test suites designed to stress an Oracle8i Parallel Server Compaq will support any implementation of Oracle8i Parallel Server that does not strictly conform to the configurations certified with this process. For a complete list of certified Compaq servers, see the Compaq longer available and - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 60
Cluster Planning Before connecting any cables or powering on any hardware on your Compaq Parallel Database Cluster Model PDC/O1000 cluster hardware I Planning cluster configurations for non-redundant Fibre Channel Fabrics I Planning cluster configurations for non-redundant Fibre Channel Arbitrated - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 61
Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Availability of appropriate equipment and qualified personnel I Space for unpacking, installing, and servicing temperature, humidity, and air quality I Power, including voltage, current, grounding, noise - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 62
Cluster. For a current list of PDC/O1000-certified servers and details on supported configurations, refer to the Compaq Parallel Database Cluster Model PDC/O1000 Certification Matrix for Windows 2000. This document is available on the Compaq website at www.compaq.com/solutions/enterprise/ha-pdc.html - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 63
single-port Compaq StorageWorks RAID Array 4000 Controller (RA4000 Array Controller) installed in each RA4000/RA4100 Array NOTE: For more information about non-redundant Fibre Channel Fabrics in a PDC/O1000 cluster, see Chapter 2, "Cluster Architecture." The Fibre Channel SAN Switch is available in - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 64
Compaq StorageWorks Storage Hub (Storage Hub) or Compaq StorageWorks FC-AL Switch (FC-AL Switch) I One or more RA4000 Arrays or RA4100 Arrays I One single-port RA4000 Array Controller AL Switch is available in an 8-port model that can be expanded to 11 ports. The Storage Hub is available in 7-port - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 65
to the technical white paper Supported Ethernet Interconnects for Compaq Parallel Database Clusters Using Oracle Parallel Server for detailed information about configuring redundant Ethernet cluster interconnects. This document is available at www.compaq.com/support/techpubs/whitepapers Before you - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 66
Once you have investigated your requirements with respect to particular parts of the cluster (ProLiant servers, shared storage components, cluster interconnect components, client LAN components), you need to plan the configuration of the entire PDC/O1000 cluster. This section describes sample - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 67
4-8 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Sample Midsize Cluster with Channel Fabric The sample midsize cluster configuration shown in Figure 4-1 contains these hardware components: I Four ProLiant servers (cluster nodes) I One Fibre - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 68
) Client LAN Figure 4-2. Larger PDC/O1000 cluster with one non-redundant Fibre Channel Fabric The larger configuration shown in Figure 4-2 contains these hardware components: I Six ProLiant servers (cluster nodes) I One Fibre Host Adapter in each cluster node I One Fibre Channel SAN Switch to - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 69
4-10 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Planning Cluster Configurations for Non-Redundant Fibre Channel Arbitrated Loops This section describes sample configurations for midsize and larger clusters that use non-redundant Fibre Channel - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 70
Cluster Planning 4-11 The sample midsize cluster configuration shown in Figure 4-3 contains these hardware components: I Four ProLiant servers (cluster nodes) I One Fibre Host Adapter in each cluster node I One Storage Hub or FC-AL Switch installed between the Fibre Host Adapters and the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 71
Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide The larger configuration shown in Figure 4-4 contains these hardware components: I Six ProLiant servers in the array without affecting the availability of data to users. RAID also uses the technique of - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 72
Adapters that can be installed in the ProLiant server model you will be using. Refer to the server documentation for this information. I The appropriate making copies of the data or calculating checksums. Use the Compaq Array Configuration Utility to implement RAID on your logical disks. NOTE: Do - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 73
information about RAID configuration contained in the Compaq StorageWorks RAID Array 4000 User Guide or the Compaq StorageWorks RAID Array 4100 User Guide. I Refer to the Compaq white paper Configuring Compaq RAID Technology for Database Servers, #ECG 011/0598, available at the Compaq website at www - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 74
Cluster Planning 4-15 Raw Data Storage and Database Size Raw data storage is the amount of storage available before any RAID levels have been configured. It is called raw data storage because RAID volumes require some overhead. The maximum size of a database stored in a RAID system will always be - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 75
Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Selecting the Appropriate RAID Levels Many factors affect which RAID levels you select for your cluster database. These include the specific availability Server of disks and controllers. The database - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 76
Server database. RA4000/4100 Array Disk Disk Disk Disk Disk Disk Disk Disk Drive Drive Drive Drive Drive Drive Drive Drive Create logical drive arrays with Compaq Array Configuration the Compaq Array Configuration Utility, group the RA4000/RA4100 Array disk drives into RAID disk arrays at specific - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 77
a Compaq SMART-2 controller or by using dedicated RA4000/RA4100 Arrays for nonshared storage. Shared Disk Drives Shared disk drives are contained in the RA4000/RA4100 Arrays and are accessible to each node in the PDC/O1000. If a logical drive is configured with a RAID level that does not support - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 78
2000 Advanced Server Hosts Files for an Ethernet Cluster Interconnect When an Ethernet cluster interconnect is installed between cluster nodes, the Compaq operating system dependent modules (OSDs) require a unique entry in the hosts and lmhosts files located at %SystemRoot%\system32\drivers\etc for - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 79
4-20 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Client LAN Physically, the structure of the client network is no different than that used for a nonclustered configuration. To ensure continued access to the database when a cluster node is - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 80
your cluster, have the following materials available during installation. You will find references to them throughout this chapter. I User guides for the clustered Compaq ProLiant servers I Installation posters for the clustered ProLiant servers I Installation guides for the cluster interconnect and - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 81
Guide I Compaq StorageWorks Fibre Channel SAN Switch Management Guide I Compaq SmartStart Installation poster I Compaq SmartStart and Support Software CD I Microsoft Windows 2000 Advanced Server Administrator's Guide I Microsoft Windows 2000 Advanced Server CD with Service Pack 1 or later I Compaq - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 82
Installation and Configuration 5-3 I Installing and configuring operating system software, including: G SmartStart 4.9 or later G Microsoft Windows 2000 Advanced Server with Service Pack 1 or later I Installing and configuring the Compaq operating system dependent modules (OSDs), including: G Using - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 83
type must be installed into the same PCI slots in each server. While setting up the physical hardware, follow the installation instructions in your Compaq ProLiant Server Setup and Installation Guide and in your Compaq ProLiant Server installation poster. When you are ready to install the Fibre Host - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 84
Installation and Configuration 5-5 To install GBIC-SW modules: 1. Verify that a GBIC-SW module has been installed in each Fibre Host Adapter in a server. 2. Insert a GBIC Fibre Channel cables on cable management arms. Support the Fibre Channel cable so that a bend radius at the cable connector is not - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 85
(Cluster Interconnect) ProLiant Servers Storage Hub/Switch RA4000/4100 Array Figure 5-1. Connecting Fibre Host Adapters to a Storage Hub, FC-AL Switch, or Fibre Channel SAN Switch For more information about the Storage Hubs, see the Compaq StorageWorks Storage Hub 7 Installation Guide and the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 86
Matrix for Windows 2000 at www.compaq.com/solutions/enterprise/ha-pdc.html If you need specific instructions on how to install an Ethernet adapter, refer to the documentation of the Ethernet adapter you are installing or refer to the user guide of the ProLiant server you are using. Refer to the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 87
SmartStart installation, review the instructions in the "Installing Operating System Software and Configuring the RA4000/RA4100 Arrays" section of this chapter. These instructions include clustering information that is not included in the Compaq StorageWorks RAID Array 4000 User Guide or the Compaq - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 88
Switch, or Fibre Channel SAN Switch connected to the RA4000 Array Controller in one RA4000/RA4100 Array. Fibre Host Adapters Client LAN Fibre Host Adapters ProLiant Servers Switch (Cluster Interconnect) ProLiant Servers Storage Hub/Switch RA4000/4100 Array Figure 5-2. Cabling a Storage Hub - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 89
Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide including: I One Fibre Host Adapter in each server I One Fibre Channel SAN Switch for each non Switch, or Fibre Channel SAN Switch to the array controller in each RA4000/RA4100 Array I GBIC-SW modules - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 90
Installation and Configuration 5-11 Figure 5-3 shows a PDC/O1000 with two non-redundant Fibre cable or one 100-Mbit/second Ethernet switch or hub and standard Ethernet cables to connect the two servers. G For PDC/O1000 clusters with three or more nodes, you use one 100-Mbit/second Ethernet switch - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 91
5-12 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide The following components are used cables to a separate Ethernet adapter in each server. NOTE: In a redundant Ethernet cluster configuration, one Ethernet crossover cable must be installed between - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 92
Installation and Configuration 5-13 Figure 5-4 shows the non-redundant Ethernet cluster interconnect components used in a two-node PDC/O1000 cluster. These components include a dual-port Ethernet adapter in - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 93
5-14 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Figure 5-5 shows another the cluster interconnect. 2. Connect the other end of one Ethernet cable to an Ethernet hub or switch. Connect the other end of the second Ethernet cable to the second - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 94
, Compaq requires configuring Ethernet connections in a redundant or non-redundant cluster interconnect, including enabling failover from one Ethernet path to another, see Supported Ethernet Interconnects for Compaq Parallel Database Clusters Using Oracle Parallel Server at www.compaq.com/support - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 95
is not supported. For information on configuring Ethernet connections in a redundant client LAN, including enabling failover from one Ethernet path to another, see Supported Ethernet Interconnects for Compaq Parallel Database Clusters Using Oracle Parallel Server at www.compaq.com/support/techpubs - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 96
. I Be sure your servers, adapters, hubs, and switches are installed and cabled before you install the software. I Power on the cluster as instructed later in this chapter in "Power Distribution and Power Sequencing Guidelines." I SmartStart runs the Compaq Array Configuration Utility, which is used - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 97
I SmartStart and Support Software CD 4.9 or later I Microsoft Windows 2000 Advanced Server with Service Pack 1 or later I SmartStart Installation poster I Server Profile diskette Cluster-Specific SmartStart Installation The SmartStart Installation poster describes the general flow of configuring and - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 98
Channel SAN Switches G Ethernet hubs or switches 2. Power up a cluster node and put the SmartStart and Support Software CD into the CD-ROM drive. 3. Select results of the shared drives configured during the first node's setup. NOTE: Do not install Microsoft Cluster Server. 7. From Start, select - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 99
5-20 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide 9. Enter unique IP addresses and node names for each node in the hosts and lmhosts files located at %SystemRoot%\system32\drivers\etc. Record this information: G For the Ethernet cluster - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 100
them. You will need this information later when installing Compaq OSDs. 12. If you are installing node1, open information on creating partitions, see the Oracle8i Parallel Server Setup and Configuration Guide. 13. If you are installing a node 16. Power down the cluster nodes and client LAN Ethernet switches - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 101
Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Installing Compaq OSDs Use the Oracle Universal Installer (OUI) program to install the Compaq operating system dependent modules (OSDs) for Oracle8i Parallel Server. Compaq supplies a software package with the OUI that installs - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 102
hosts and lmhosts files at %SystemRoot%\system32\drivers\etc. For example, the command ping Administrator Privileges Because the OUI installs and configures OSDs on the other nodes in the cluster used to mount drives on remote nodes. The user running the net use command must have administrator - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 103
in different domains, create a local user on each server with the same Windows Server user name and password on each node. Then make that local user a member of the local Administrators group on each node before installing the OSDs. 2. Insert the Compaq Parallel Database Cluster Clustering Software - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 104
Installation and Configuration 5-25 3. If autorun is enabled, the Oracle Universal Installer Welcome screen appears. If autorun is disabled, enter d:\setup at a C: prompt. Substitute the letter assigned to the CD drive for d. The Oracle Universal Installer Welcome screen appears. NOTE: The OUI - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 105
5-26 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide 4. Click Next to continue. The License screen appears. 5. Click Accept to accept the terms of the license agreement. The File Locations screen appears. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 106
Installation and Configuration 5-27 6. Click Next to continue. The Installation Types screen appears. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 107
Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide 7. Select Typical to install the Ethernet OSDs into the default location, C:\Compaq\OPS. Skip to step 12. 8. Select Custom to install the Ethernet OSDs into a location of your choice. The Component Locations - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 108
Installation and Configuration 5-29 9. If you want to accept the default location, skip to step 13. 10. If you want to specify an alternate location and you know - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 109
5-30 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide 12. Browse to the location where you want to install the OSDs. Click OK. The Component Locations screen appears indicating the alternate location you specified. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 110
Installation and Configuration 5-31 13. Click Next to continue. The Cluster Members screen appears. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 111
5-32 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide 14. The Cluster Members screen indicates the host name of the installing machine. Define the node names for the client LAN by adding the node - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 112
Installation and Configuration 5-33 16. The Cluster Members screen now shows the node names you specified on the first Cluster Members screen with "_san" appended to each name, - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 113
5-34 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide 18. If the information on the OSD Summary screen is not correct, click Previous to modify the previous screens. When the information is correct, click Next to continue. The Summary screen appears. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 114
and Configuration 5-35 bar as the installation progresses. 21. When the installation completes, the End of Installation screen appears. 22. Click Exit to quit the OUI OracleCMService, on all nodes in the cluster. This service must be started before installing Oracle software. See the Oracle documentation - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 115
, the Oracle8i documentation instructs you to install vendor-supplied operating system dependent modules (OSDs). You have already completed this step. Configuring Oracle Software Configuring Oracle software includes configuring Oracle8i Parallel Server, Oracle8i Parallel Server Manager, Oracle8i - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 116
creating a database. For instructions on installing and using Object Link Manager, see the readme file on the Compaq Parallel Database Cluster Clustering Software for Oracle8i on Microsoft Windows 2000 CD at \Object Link Manager\readme.txt Additional Notes on Configuring Oracle Software In addition - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 117
SystemRoot%\system32\drivers\etc. service did not start successfully, make sure that the OracleCMService is started and the SID is in the Oracle registry. There are two logs that can provide information about why the Oracle service did not start. These logs are: I \Compaq\OPS\cmsrvr.log I \Compaq - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 118
the Services icon in the Control Panel. Once there, look for Oracle Object Service in the list of services and Compaq power distribution unit (PDU). PDUs connect to an uninterruptible power supply (UPS) or building power. The PDUs and UPSs are the only cluster components connected to a building power - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 119
/RA4100 Array in the cluster has two power supplies, the power distribution (with respect to connecting them to PDUs and UPSs) can be configured similarly to the example shown for server power distribution. By substituting RA4000/RA4100 Arrays for ProLiant servers in Figure 5-7, you can provide two - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 120
-AL Switches, or Fibre Channel SAN Switches (Power is applied to a Storage Hub when the AC power cord is plugged in.) 3. Ethernet hubs or switches 4. ProLiant servers Be sure to power down the cluster components in the following order: 1. ProLiant servers 2. Ethernet hubs or switches 3. Storage Hubs - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 121
I Software maintenance I Managing changes to shared storage I Replacing a cluster node I Adding a cluster node I Managing cluster operation NOTE: The procedures in this chapter contain high-level instructions. The instructions summarize what has either already been discussed in this guide, Oracle - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 122
6-2 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Cluster Management Concepts Powering Off a Node Without Interrupting Cluster Services At some time during the life of your cluster you will need to perform an operation on a cluster node that will - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 123
user ability to access the database. Since users will experience some disruption of service and possibly a performance degradation during a node eviction, they might become concerned about the availability the System log or within specific error log files. The Oracle8i Server events are sent to the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 124
Remotely controls servers In Compaq servers, each hardware subsystem, such as network adapters, system processors, and system memory, has a robust set of management capabilities. Compaq Full Spectrum Fault Management notifies of impending fault conditions. Several steps are required to configure the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 125
add new drive arrays. The ACU is available on the SmartStart and Support Software CD, which is found in the Compaq Server Setup and Management pack. For additional information about the ACU, see the Compaq StorageWorks RAID Array 4000 User Guide or the Compaq StorageWorks RAID Array 4100 User Guide - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 126
Controller installed in each RA4000 Array or RA4100 Array I One Compaq StorageWorks Storage Hub (Storage Hub) or Compaq StorageWorks FC-AL Switch (FC-AL Switch) I One Fibre Host Adapter installed in each server FC-AL. FFIU is on the SmartStart and Support Software CD. Monitoring the Database Oracle8i - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 127
Enterprise Manager Administrator's Guide I Oracle Enterprise Manager Configuration Guide I Oracle Enterprise Manager Concepts Guide I Oracle8i Parallel Server Setup and Configuration Guide I Oracle8i Parallel Server Concepts I Oracle8i Parallel Server Administration, Deployment, and Performance - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 128
instance on all nodes and stop the Oracle services running on all nodes prior to removing the OSDs. To deinstall the OSDs: 1. Log in as a user who has administrator privileges on all nodes in the cluster. 2. Insert the Compaq Parallel Database Cluster Clustering Software for Oracle8i on Microsoft - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 129
Cluster Management 6-9 4. If autorun is disabled, enter d:\setup at a C: prompt. Substitute the letter assigned to the CD-ROM drive for d. The Oracle Universal Installer Welcome screen appears. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 130
Windows 2000 Administrator Guide 5. Click Deinstall Products. Next, a window containing an inventory of the installed components appears. The following shows the Inventory screen that is displayed when Ethernet is used for the cluster interconnect. 6. To deinstall OSDs, select Compaq Ethernet OSDs - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 131
of Oracle8i Server, consult your Compaq and Oracle service representatives. Make sure that the new release is certified and supported on the PDC/O1000, and get assistance to determine which procedure you should follow to perform the upgrade properly. Upgrading the OSDs During the life of your - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 132
6-12 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Upgrading the Ethernet OSDs To upgrade the Ethernet OSDs: 1. Insert the CD containing the new revision of the OSDs into the CD-ROM drive on the primary node. (This is the node that was used for - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 133
has finished, restart all nodes. 5. Install the new Ethernet OSDs from the new by following the procedure described under "Installing Compaq OSDs" in Chapter 5, "Installation and Configuration." 6. After installing the new Ethernet OSDs, restart OracleCMService and OracleNMService on all nodes - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 134
will likely affect the operation of the cluster. Before upgrading to any new release of Oracle8i Server, consult your Compaq and Oracle service representatives. Make sure that the new release is certified and supported on the PDC/O1000, and get assistance to determine which procedure you should - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 135
User Guide or the Compaq StorageWorks RAID Array 4100 User Guide can result in loss of data. Adding Disk Drives to Increase Shared Storage Capacity During the life the new data files to the database. Refer to the Oracle8i Parallel Server Setup and Configuration Guide for more detailed instructions. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 136
Stop the Oracle services running on each node. 3. Shut down Windows 2000 Advanced Server on each node and power off all the cluster nodes. 4. Power off all of the the new data files to the database. Refer to the Oracle8i Parallel Server Setup and Configuration Guide for more detailed instructions. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 137
Shut down the Oracle services running on the node. 3. Shut down the server. 4. Power off the node. new one. 7. From the new Fibre Host Adapter, connect the Fibre Channel cable that runs to its Fibre Channel SAN Switch, FC-AL Switch, or Storage Hub. 8. Power on the node. 9. Restart the Oracle services - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 138
Guide Replacing a Cluster Node Before replacing a node, verify that the new configuration is supported by the PDC/O1000 cluster. To determine if your new server is supported, see the Compaq Stop all Oracle services on the node to be removed. 5. Shut down the server on that node and power it off. 6. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 139
to configure the server's local non-shared disks with RAID. 5. Install and configure Windows 2000 Advanced Server with Service Pack 1 or later (whichever is installed on the other cluster nodes). 6. Install necessary Compaq drivers and utilities from the Compaq SmartStart and Support Software - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 140
Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide 7. Install the Fibre Host Adapter or Fibre Host Adapters in the replacement node in the same server Software Now that the replacement node is configured and physically connected to the cluster, the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 141
this point, all further operations are specific to Oracle configuration. Refer to the Oracle setup and configuration documentation for information on configuring Oracle software. Adding a Cluster Node Before adding a node, verify that the new configuration is supported by the PDC/O1000. To determine - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 142
disks with RAID. 5. Install and configure Windows 2000 Advanced Server with Service Pack 1 or later (whichever is installed on the other cluster nodes). 6. Install all necessary Compaq drivers and utilities from the Compaq SmartStart and Support Software CD, including the drivers for the Fibre Host - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 143
SystemRoot%\system32\drivers\etc. 2. Verify TCP/IP connectivity between the new node and the existing cluster nodes. Run the ping utility from the new node, . Installing the Cluster Software for Oracle8i Now that the new node is configured and physically connected to the cluster, the next procedure - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 144
only the new node name when asked which node to install the software on. See the Oracle8i Parallel Server Setup and Configuration Guide for details on installing and configuring the Oracle software. 7. Once the configuration of the Oracle software is complete, start the Oracle services and Oracle - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 145
Examples of monitored objects are memory, processor, hard drive performance, network software, and application software. I Oracle8i Performance Manager captures, computes, and presents performance data about the Oracle database. Specific data about Oracle8i Parallel Server metrics are monitored and - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 146
chapter addresses potential problems you might encounter as you install, configure, test, and operate your Compaq Parallel Database Cluster Model PDC/O1000 (referred to here as the PDC/O1000). This chapter is organized as follows: I Basic troubleshooting tips I Troubleshooting Oracle8i and operating - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 147
Oracle8i and Windows 2000 Administrator Guide Basic Troubleshooting Tips Power Verify that the following are all powered on: I ProLiant servers (cluster nodes) I StorageWorks RAID Array 4000s (RA4000 Arrays) or StorageWorks RAID Array 4100s (RA4100 Arrays) I Compaq StorageWorks Fibre Channel SAN - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 148
Cluster Communications" in Chapter 5, "Installation and Configuration." I To confirm that the shared storage is Server Release 8.1.7 with Oracle8i Parallel Server Option Release 8.1.7 I RA4000 Array Controller drivers I Compaq Ethernet OSDs I Microsoft Windows 2000 Advanced Server, including Service - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 149
Controllers For information about currently supported firmware revisions for the PDC/O1000, refer to the Compaq Parallel Database Cluster Model PDC/O1000 Certification Matrix for Windows 2000 at www.compaq.com/solutions/enterprise/ha-pdc.html Depending on your specific configuration, other firmware - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 150
Troubleshooting 7-5 Troubleshooting Oracle8i and OSD Installation Problems installation. If difficulties persist, contact your support representative for assistance. I The OUI uses on the primary node: C: net use \\machine_name\C$ * /User: in which the "*" results in the system prompting - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 151
Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide G The user running the properly configured. Service, OracleNMService, is running. If not, see the error logs in C:\compaq\ops\nm.log. If OracleNMService is running, check the CM Server logs in C:\compaq - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 152
scan the list of services and confirm that the OracleService is started. I Verify that the database links are properly set up on all nodes and that correct entries exist for all datafiles, including OPS_CMDISK. Refer to the Oracle8i Parallel Server Setup and Configuration Guide for further details. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 153
, then the OracleCMService. From Administrative Tools, open the Services screen. Locate each service in the displayed list, highlight it, then stop it. Finally, highlight the service and restart it. Troubleshooting Node-to-Node Connectivity Problems Nodes Are Unable to Communicate with Each Other - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 154
are properly connected and that the Ethernet switches or hubs are powered on. In a redundant Ethernet cluster interconnect, verify that all Make sure that the Ethernet drivers are properly installed and configured on the cluster nodes. Verify that the Ethernet drivers for the cluster interconnect - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 155
all cluster nodes, refer to Appendix A, "Diagnosing and Resolving Shared Disk Problems." Troubleshooting Client-to-Cluster Connectivity Problems A Network Client Cannot Communicate with the Cluster I Verify that the user ID has sufficient rights to perform the desired action. I Make sure proper - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 156
/RA4100 Array problems that are specific to the product itself or to the use of the RA4000/RA4100 Array in a stand-alone server configuration. For those issues, refer to the Compaq StorageWorks RAID Array 4000 User Guide or the Compaq StorageWorks RAID Array 4100 User Guide. Verifying Connectivity - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 157
Fibre Channel Host Adapter Installation Guide or the Compaq StorageWorks 64-bit/66-MHz Fibre Channel Host Adapter Installation Guide for installation instructions. I Determine whether or not hardware errors or communications transport failures are causing the problem. Use the Event Viewer to - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 158
Fibre Channel cables, refer to the Compaq StorageWorks RAID Array 4000 User Guide or the Compaq StorageWorks RAID Array 4100 User Guide. I If all drives are recognized by the Array Configuration Utility and are configured correctly, start Windows 2000 Advanced Server and run Disk Management. Verify - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 159
Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Switch, or Storage Hub, and each RA4000 Array Controller. G Verify that all Fibre Channel cables are components must be powered on before the servers. If any cluster node was powered on before the Fibre - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 160
StorageWorks RAID Array 4000s (RA4000 Arrays) or Compaq StorageWorks RAID Array 4100s (RA4100 Arrays) is preventing one or more nodes from accurately viewing or successfully accessing the available shared storage resources. I A connectivity problem exists either within the cluster interconnect or - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 161
Manager On All Affected Nodes Exit. Problem Fixed or No Further Disk Problems Perform Cluster Software OK and Firmware Checks Problem Persists Problem Persists Perform Cluster Hardware Checks OK Problem Persists Contact Your Compaq Support Representative Figure A-1. Tasks for diagnosing and - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 162
Shared Disk Problems A-3 Run Object Link Manager On All Nodes Object Link Manager simplifies the creation and maintenance of symbolic All Affected Nodes in the Cluster" in this appendix for further instructions. For instructions on installing and using Object Link Manager, see the readme file: - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 163
A-4 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Restart All Affected Nodes in service is properly installed on the node. For instructions on installing and using Object Link Manager, refer to this readme file contained on the Compaq Parallel - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 164
Diagnosing and Resolving Shared Disk Problems A-5 Run Disk Management On All Configuration Utility On All Nodes The Array Configuration Utility is used to configure the hardware aspects of disk drives connected to the RA4000 Array Controllers. The Array Configuration Utility allows you to configure - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 165
PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Perform Cluster Software and Firmware Checks Use the Compaq SmartStart and Support Software CD to perform these checks on the cluster software and firmware: 1. Verify that all necessary drivers have been installed. For example, verify that - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 166
4100 User Guide. 7. Verify that all Fibre Host Adapters, RA4000 Array Controllers, and RA4000/RA4100 Array shared disk modules are properly installed and seated in their receptacles. Contact Your Compaq Support Representative If you are unable to resolve the problem, contact your Compaq support - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 167
that enables a set of loosely coupled servers and storage to present a single system image to clients and to operate as a single system. Clustering is a way to improve availability and scalability. The period of time that a computer system is unable to service clients' requests. A standard network - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 168
2 Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Fibre Channel Fibre Channel Arbitrated Loop (FC-AL) Fibre Channel Fabric GBIC-SW module Heartbeat High availability Hot-pluggable Hot spare Interconnect Integrated Distributed Lock Manager An IEEE - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 169
Controller. A board that allows a computer to be connected with a network, and works with the network operating system to control the flow of information over the network. An individual server in a cluster. New database support. An additional license is required for Oracle8i Parallel Server. An - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 170
Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide Reliability Resource Scalability Service Shared resource Shared storage System UPS The continuous integrity of a system (server or an SMP system. Example: a server. Uninterruptible power supply. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 171
23 application failover software for Oracle8i 3-7 Array Configuration Utility See Compaq Array Configuration Utility automated installation for Windows 2000 Advanced Server 5-18 availability defined 1-3 features of Compaq ProLiant Servers 2-3 C cabling client LAN 5-16 Ethernet cluster interconnect - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 172
Compaq Parallel Database Cluster Model PDC/O1000 for Oracle8i and Windows 2000 Administrator Guide , troubleshooting 7-9 cluster degradation, troubleshooting 6-2 supported servers 2-2 cluster operation, monitoring 6-25 cluster planning capacity planning 4-3 client LAN 4-20 cluster configurations - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 173
1-1 PDC/O1000 example 1-2 scalability in 1-3 Compaq Array Configuration Utility configuring RA4000/RA4100 Arrays for Windows 2000 Advanced Server 5-17 managing shared drives with 6-5 overview 3-3 Index 3 running to check for shared disk problems A-5 troubleshooting shared storage with 7-13 using to - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 174
Administrator Guide Compaq Options ROMPaq, overview 3-4 Compaq Parallel Database Cluster, overview 1-4 Compaq SmartStart and Support software 3-2 Compaq SmartStart and Support Software CD 3-2 Compaq software overview Compaq Array Configuration Utility 3-3 Compaq Insight Manager 3-4 Compaq Insight - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 175
to check for shared disk problems A-5 verifying shared storage 5-21 4-6 Index 5 F FC-AL SAN I/O path configuration guidelines 2-16 installing additional nonredundant FC-ALSs 5-10 multiple 2-16 topology overview 2-16 FC-AL Switches See Compaq StorageWorks FC-AL Switches description 2-7 FFIU See Fibre - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 176
Administrator Guide Fibre Host Adapters See Compaq StorageWorks Fibre Channel Host Bus Adapters replacing 6-17 firmware revisions to array controller 2-22 I/O data paths in a non-redundant FC-AL 2-20 I/O data paths in a non-redundant Fibre Channel Fabric 2-13 I/O path configuration guidelines FC- - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 177
system dependent modules See Compaq operating system dependent modules Index 7 operating system software installing for Windows 2000 Advanced Server 5-17 versions supported 3-2 Oracle software certification for Oracle8i 3-7 configuring for Oracle8i 5-36 installation troubleshooting for Oracle8i - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 178
units for Oracle8i 5-39 power sequencing for Oracle8i 5-41 ProLiant servers high-availability features 2-3 power distribution 5-40 supported servers 2-2 R RA4000 Array Controllers See Compaq StorageWorks RAID Array 4000 Array Controllers RA4000 Arrays See Compaq StorageWorks RAID Array 4000s RA4000 - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 179
System Configuration Utility Service Packs for Windows 2000 Advanced Server supported for the PDC/O1000 5-18 versions supported 3-2 shared disk problems contacting your Compaq support representative A-7 introduction A-1 performing cluster hardware checks A-6 performing cluster software and firmware - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/O1000 for Orac - Page 180
of potential problems 7-1 unable to start database for Oracle8i 7-7 troubleshooting tips accessibility of resources 7-3 firmware revisions 7-4 physical connections 7-2 power 7-2 software revisions 7-3 U uninterruptible power supplies for Oracle8i 5-39 upgrading Oracle8i Server 6-11 V verifying
Parallel Database Cluster Model
PDC/O1000 for Oracle8
i
and
Windows 2000
Administrator Guide
Second Edition (June 2001)
Part Number 225083-002
Compaq Computer Corporation