Compaq ProLiant 8500 Compaq Parallel Database Cluster Model PDC/05000 for Orac
Compaq ProLiant 8500 Manual
View all Compaq ProLiant 8500 manuals
Add to My Manuals
Save this manual to your list of manuals |
Compaq ProLiant 8500 manual content summary:
- Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 1
Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Second Edition (June 2001) Part Number 225081-002 Compaq Computer Corporation - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 2
Compaq Computer Corporation Compaq, the Compaq logo, Compaq Insight Manager, SmartStart, ROMPaq, ProLiant, and StorageWorks Registered in U.S. Patent and Trademark Office. ActiveAnswers is a trademark of Compaq be trademarks of their respective companies. Compaq shall not be liable for technical or - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 3
...1-2 Availability ...1-4 Scalability ...1-4 Compaq Parallel Database Cluster Overview 1-5 Chapter 2 Cluster Architecture Compaq ProLiant Servers 2-2 High-Availability Features of ProLiant Servers 2-3 Shared Storage Components 2-3 MA8000/EMA12000 Storage Subsystem 2-4 HSG80 Array Controller - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 4
Modules in the Controller Enclosure 2-9 Connecting EMUs Between MA8000/EMA12000 Storage Subsystems........ 2-12 I/O Path Configurations for Redundant Fibre Advanced Server 3-2 Compaq Software...3-2 Compaq SmartStart and Support Software 3-2 Compaq System Configuration Utility 3-3 Compaq Insight - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 5
4-2 Compaq ProLiant Servers 4-2 Planning Shared Storage Components for Redundant Fibre Channel Fabrics ...4-3 Planning Shared Storage Components for Redundant Fibre Channel Arbitrated Loops 4-4 Planning Cluster Interconnect and Client LAN Components 4-6 Planning Cluster Configurations for - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 6
Array Controllers to Storage Hubs 5-15 Installing Operating System Software 5-16 Guidelines for Clusters 5-17 Automated Installation Using SmartStart 5-17 Setting up and Configuring an MA8000/EMA12000 Storage Subsystem 5-21 Designating a Server as a Maintenance Terminal 5-21 Powering On - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 7
Remotely Managing a Cluster 6-7 Software Maintenance 6-8 Deinstalling the OSDs 6-8 Upgrading Oracle8i Server 6-11 Upgrading the OSDs 6-11 Deinstalling a Partial OSD Installation 6-13 Managing Changes to Shared Storage Components 6-14 Adding New Storagesets to Increase Shared Storage Capacity - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 8
Oracle8i and Windows 2000 Administrator Guide Chapter 7 Troubleshooting Basic Troubleshooting Tips 7-2 Power ...7-2 Physical Connections 7-2 Access to Cluster Components 7-3 Software Revisions 7-3 Firmware Revisions 7-4 Troubleshooting Oracle8i and OSD Installation Problems and Error Messages - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 9
Manager On All Nodes A-5 Run Disk Management On All Nodes A-5 Run and Validate the StorageWorks Command Console From All Storage Subsystems ...A-6 Perform Cluster Software and Firmware Checks A-6 Perform Cluster Hardware Checks A-7 Contact Your Compaq Support Representative A-8 Glossary Index - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 10
x Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide List of Figures Figure 1-1. Example of a two-node PDC/O5000 cluster 1-3 Figure 2-1. SCSI bus numbers for I/O modules in the controller enclosure ......... 2-10 Figure 2-2. UltraSCSI cabling between - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 11
5-7. Dual redundancy configuration for a redundant FC-AL 5-16 Figure 5-8. Server power distribution in a three-node cluster 5-57 Figure A-1. Tasks for diagnosing and resolving shared storage problems A-2 List of Tables Table 2-1 High-Availability Components of ProLiant Servers 2-3 Table 2-2 SCSI - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 12
, configuration, and maintenance information that can be valuable for a variety of users. If you are installing the PDC/O5000 but will not be administering the cluster on a daily basis, please make this guide available to the person or persons who will be responsible for the clustered servers after - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 13
high-availability features of the PDC/O5000 I Planning and designing a PDC/O5000 cluster configuration to meet your business needs I Installing and configuring PDC/O5000 hardware and software I Managing the PDC/O5000 I Troubleshooting the PDC/O5000 The following summarizes the contents of this guide - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 14
refer to documentation related to the specific hardware and software components of the Compaq Parallel Database Cluster. These related manuals include, but are not limited to: I Documentation related to the ProLiant servers you are clustering (for example, guides, posters, and performance and tuning - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 15
Configuring Compaq RAID Technology for Database Servers, at www.compaq.com/highavailability I Various technical white papers on Oracle and cluster sizing, which are available from Compaq ActiveAnswers website, at www.compaq Type When you are instructed to type information, type the information without pressing - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 16
life. CAUTION: Text set off in this manner indicates that failure to follow directions could result in damage to equipment or loss of information. IMPORTANT: Text set off in this manner presents clarifying information or specific instructions area contains no operator serviceable parts. WARNING: To - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 17
cool before touching. Power Supplies or Systems marked with these symbols indicate the equipment is supplied by multiple sources of power. WARNING: To reduce Getting Help If you have a problem and have exhausted the information in this guide, you can get further information and other help - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 18
as long you own the product. A technical support specialist will help you diagnose the problem or guide you to the next step in the warranty process. In North America, call the Compaq Technical Phone Support Center at 1-800-OK-COMPAQ1. This service is available 24 hours a day, 7 days a week. Outside - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 19
PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Compaq Authorized Reseller For the name of your nearest Compaq Authorized Reseller: I In the United States, call 1-800-345-1518. I In Canada, call 1-800-263-5868. I Elsewhere, see the Compaq website for locations and telephone numbers. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 20
mission controls, many critical applications can reach the desired levels of availability and scalability with non-proprietary client/server-based clustering. These clustering solutions use industry-standard hardware and software, thereby providing key clustering features at a lower price than - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 21
Components" to determine the specific releases your cluster kit supports. Figure 1-1 shows an example of a PDC/O5000 that includes two nodes (Compaq ProLiant™ servers), two sets of storage subsystems, two Compaq StorageWorksTM Fibre Channel SAN Switches or Compaq StorageWorks Fibre Channel Storage - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 22
P1 P2 A P1 P2 B P1 P2 Storage Subsystem #1 Storage Subsystem #2 Figure 1-1. Example of a two-node PDC/O5000 cluster The PDC/O5000 can support redundant Fibre Channel Fabric Storage Area Network (SAN) and redundant Fibre Channel Arbitrated Loop (FC-AL) SAN topologies. In the example shown in - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 23
components to an existing system without the expense of adding an entire new system. In a stand-alone server configuration, scalable systems allow increased throughput by adding processors or more memory. In a cluster configuration, this result is usually obtained by adding cluster nodes. Not only - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 24
found in expensive, proprietary cluster solutions, at a fraction of the cost. The PDC/O5000 combines the popular Microsoft Windows 2000 Advanced Server operating system and the industry-leading Oracle8i Parallel Server with award-winning Compaq ProLiant servers and shared storage subsystems. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 25
Compaq ProLiant servers I Shared storage components G Compaq StorageWorks Modular Array 8000 Fibre Channel Storage Subsystems or the Compaq StorageWorks Enterprise Modular Array 12000 Fibre Channel Storage Subsystems (MA8000/EMA12000 Storage Subsystems) G Compaq StorageWorks HSG80 Array Controllers - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 26
each node is a Compaq ProLiant server. With some exceptions, all nodes in a PDC/O5000 must be identical in model. In addition, all components common to all nodes in a cluster, such as memory, number of CPUs, and the interconnect adapters, should be identical and identically configured. NOTE: Certain - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 27
power supplies, redundant cooling fans, and Error Checking and Correcting (ECC) memory. The high-availability features of ProLiant servers are a critical foundation of Compaq clustering products. Table 2-1 lists the high-availability features found in many ProLiant servers Array Controllers I Fibre - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 28
MA8000/EMA12000 Storage Subsystem houses the two HSG80 Array Controllers, one cache module for each controller, an environmental monitoring unit (EMU), one or two power supplies, and three dual-speed fans. In addition, the controller enclosure houses the six I/O modules that connect the enclosure - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 29
documentation provided with the array controllers. Fibre Channel SAN Switch IMPORTANT: For detailed information about cascading two Fibre Channel SAN Switches, refer to the latest Compaq StorageWorks documentation. This guide does not document cascaded configurations for the Fibre Channel SAN - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 30
PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide For further information, refer to these manuals provided with each Fibre Channel SAN Switch: I Compaq StorageWorks Fibre Channel SAN Switch 8 Installation and Hardware Guide I Compaq StorageWorks Fibre Channel SAN Switch 16 Installation and - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 31
are used to connect the server nodes, Fibre Channel SAN Switches or Storage Hubs, and array controllers in a PDC/O5000. Configuring and Cabling the MA8000/EMA12000 Storage Subsystem Components Configuring LUNS for Storagesets If you install the optional Compaq StorageWorks Large LUN utility on - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 32
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide If controller A port 1, LUNS 2 and 3 to array controller A port 2, LUNS 4 and 5 to array controller B port 1, and LUNs 6 and 7 to array controller B port 2. In this configuration, a single controller - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 33
buses, with each bus controlling half of the total available disk drive slots in the Compaq documentation provided with the storage subsystem. UltraSCSI Cables Compaq recommends using the shortest cable length possible to connect disk enclosures to the controller enclosure. The maximum supported - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 34
2-10 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide SCSI Bus 1 SCSI Bus 2 SCSI Bus 3 SCSI Bus 4 A B SCSI Bus 5 SCSI Bus 6 Controller Enclosure (rear view) Figure 2-1. SCSI bus numbers for I/O modules in the controller enclosure Using Dual- - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 35
enclosure and three disk enclosures Using Single-Bus I/O Modules in the Disk Enclosures Figure 2-3 shows UltraSCSI cables installed between a controller enclosure and six disk enclosures. When four or more disk enclosures are present in a storage subsystem, use one single-bus I/O module in - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 36
2-12 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Single-Bus I/O Module Single-Bus I/O Module Disk Enclosure Disk Enclosure Disk Enclosure Disk Enclosure Disk Enclosure Disk Enclosure I/O Modules (6) A B Controller Enclosure Figure 2-3. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 37
supports new devices to FC-AL Storage Hubs further divides their shared bandwidth, adding new controllers IMPORTANT: For detailed information about cascading two Fibre Channel SAN Switches, refer to the latest Compaq StorageWorks documentation. This guide does not document cascaded configurations - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 38
2-14 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide A redundant Fibre Channel failover capabilities of Compaq Secure Path software, this redundant Fibre Channel Fabric configuration gives cluster resources increased availability and fault tolerance. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 39
by the number of ports available on the Fibre Channel SAN Switches used. The storage subsystems are available only to the host adapters is restricted by the number of host adapters your Compaq servers support. Refer to the Compaq server documentation for this information. Figure 2-5 shows a two - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 40
2-16 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Storage Subsystem #1 A P1 P2 B P1 P2 Redundant following maximum distances apply: I Each MA8000/EMA12000 Storage Subsystem's controller enclosure can be placed up to 500 meters from the Fibre - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 41
Fabric I/O Data Paths in a Redundant Fibre Channel Fabric Within a redundant Fibre Channel Fabric, an I/O path connection exists between each host adapter and all four array controller ports in each storage subsystem. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 42
2-18 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Host Adapter-to-Fibre Channel SAN Switch I/O Data Paths Figure 2-7 highlights the I/O data paths that run between the host adapters and the Fibre Channel SAN - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 43
SAN Switch # 2 A P1 P2 B P1 P2 Storage Subsystem #1 A P1 P2 B P1 P2 Storage Subsystem #2 Figure 2-8. Fibre Channel SAN Switch-to-array controller I/O data paths If any component along an active path fails, Secure Path detects the failure and automatically transfers all I/O activity to the - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 44
2-20 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I/O Path Definitions for Redundant their two array controllers, the only bit of address information that will be different across the I/O paths from a given node to a specific storageset is the - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 45
the correct method for cabling the I/O path components in a dual redundancy configuration. In this example, both Fibre Channel SAN Switches connect to port 1 on one array controller and to port 2 on the other array controller in each MA8000/EMA12000 Storage Subsystem. If one Fibre Channel SAN Switch - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 46
and Windows 2000 Administrator Guide Host Adapters (2) Node 1 Fibre Channel SAN Switch # 1 Host Adapters (2) Node 2 Fibre Channel SAN Switch # 2 A P1 P2 B P1 P2 Storage Subsystem #1 A P1 P2 B P1 P2 Storage Subsystem #2 Figure 2-9. Dual redundancy configuration for a redundant Fibre - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 47
method for cabling the I/O path components in a quad redundancy configuration. In this example, the following connections are made: I controllers. Fibre Channel SAN Switch #1 connects to array controller A, port 1 in each storage subsystem. Fibre Channel SAN Switch #3 connects to array controller - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 48
Guide Host Adapters (4) Node 1 Node 2 Node 3 Node 4 Fibre Channel SAN Switch #1 Fibre Channel SAN Switch #3 A P1 P2 B P1 P2 Fibre Channel SAN Switch #2 Fibre Channel SAN Switch #4 A P1 P2 B P1 P2 Storage Subsystem #1 Storage Subsystem #2 Figure 2-10. Quad redundancy configuration - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 49
failure events for both dual redundancy or quad redundancy configurations in a redundant Fibre Channel Fabric and the failover the host adapter in the node that is connected to the defined backup array controller port takes over all I/O activity. The I/O path from the host adapter in the - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 50
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I/O Path Configurations The PDC/O5000 supports both topologies. bandwidth that is available for each port controllers I GBIC-SW modules installed in host adapters, Storage Hubs, and array controllers - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 51
with a four-loop redundant Fibre Channel Arbitrated Loop Used in conjunction with the I/O path failover capabilities of the Secure Path software, this redundant FC-AL configuration gives cluster resources increased availability and fault tolerance. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 52
redundant FC-AL. The maximum number of redundant FC-ALs you can install in a PDC/O5000 is restricted by the number of host adapters your Compaq servers support. Refer to the Compaq server documentation for this information. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 53
Cluster Architecture 2-29 Figure 2-12 shows a two-node PDC/O5000 that contains two redundant FC-ALs. In this example, each redundant FC-AL has its own pair of host adapters in each node, a pair of Storage Hubs, and two MA8000/EMA12000 Storage Subsystems. In Figure 2-12, the hardware components that - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 54
Administrator Guide Maximum Distances Between Nodes and Shared Storage Subsystem Components By using standard short-wave Fibre Channel cables with Gigabit Interface Converter-Shortwave (GBIC-SW) modules, the following maximum distances apply: I Each MA8000/EMA12000 Storage Subsystem's controller - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 55
Cluster Architecture 2-31 Figure 2-13 illustrates these maximum cable distances for a redundant FC-AL. Host Adapters (2) 500 meter maximum Host Adapters (2) Node 1 Node 2 Storage Hub #1 Storage Hub #2 500 meter maximum A P1 P2 B P1 P2 A P1 P2 B P1 P2 Storage Subsystem #1 Storage - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 56
2-32 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I/O Data Paths in a Redundant FC-AL Within a redundant FC-AL, an I/O path connection exists between each host adapter and all four array controller ports in each storage subsystem. Host Adapter-to - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 57
that path to the defined backup path. Storage Hub-to-Array Controller I/O Data Paths Figure 2-15 highlights the I/O data paths that run Storage Subsystem. There is one I/O connection path to and from each array controller port. Host Adapters (2) Host Adapters (2) Node 1 Storage Hub #1 Node - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 58
by the Windows 2000 Advanced Server operating system. Except for the LUN, the rest of the SCSI address is created within the host adapter miniport driver and is derived from the Arbitrated Loop Physical Address (ALPA) assigned to each of the four array controller ports in a shared storage subsystem - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 59
physical disk drive. I The LUN of the device (for disk devices, the LUN is always 0). I/O Path Configuration Examples for Redundant FC-ALs Every I/O path connection between host adapters and array controller ports is active at the same time. An active port does not become inactive unless it fails - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 60
for Oracle8i and Windows 2000 Administrator Guide Host Adapters (2) Node 1 Storage Hub #1 Host Adapters (2) Node 2 Storage Hub #2 A P1 P2 B P1 P2 A P1 P2 B P1 P2 Storage Subsystem #1 Storage Subsystem #2 Figure 2-16. Dual redundancy configuration for a redundant FC-AL Quad Redundancy - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 61
to array controller B, port 2 in each storage subsystem. Host Adapters (4) Node 1 Node 2 Node 3 Node 4 Storage Hub #1 Storage Hub #3 Storage Hub #2 Storage Hub #4 A P1 P2 B P1 P2 A P1 P2 B P1 P2 Storage Subsystem #1 Storage Subsystem #2 Figure 2-17. Quad redundancy configuration for - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 62
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Summary of I/O Path Failure and Failover Scenarios for Redundant FC-ALs Table 2-4 identifies possible I/O path failure events for both dual redundancy and quad configurations controller port takes - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 63
the technical white paper Supported Ethernet Interconnects for Compaq Parallel Database Clusters Using Oracle Parallel Server (ECG062/0299) for detailed information about configuring redundant Ethernet cluster interconnects. This document is available at www.compaq.com/support/techpubs/whitepapers - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 64
2-40 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Ethernet Cluster Interconnect Components cables to a separate Ethernet adapter in each server. NOTE: In a redundant Ethernet cluster configuration, one Ethernet crossover cable must be installed - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 65
Cluster Architecture 2-41 Ethernet Cluster Interconnect Diagrams Figure 2-18 shows the redundant Ethernet cluster interconnect components used in a two-node PDC/O5000 cluster. Ethernet Switch/Hub #2 for Cluster Interconnect Dual-port Ethernet Adapters (2) Ethernet Switch/Hub #1 for Cluster - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 66
2-42 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Local Area Network NOTE: For components of the client LAN are no different than in a standalone server configuration. The software components used by network clients should have the ability to - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 67
(OSDs) and this Oracle software: I Oracle8i Enterprise Edition with the Oracle8i Parallel Server Option NOTE: For information about currently-supported software revisions for the PDC/O5000, refer to the Compaq Parallel Database Cluster Model PDC/O5000 Certification Matrix for Windows 2000 at www - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 68
Server on a PDC/O5000 cluster. SmartStart uses an automated step-by-step process to configure the operating system and load the system software. The Compaq SmartStart and Support Software CD also contains device drivers and utilities that enable you to take advantage of specific capabilities - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 69
optional Remote Insight controller, and SmartStart. In Compaq servers, each hardware subsystem, such as non-shared disk storage, system memory, and system processor, has a robust set of management capabilities. Compaq Full Spectrum Fault Management notifies the end user of impending fault conditions - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 70
the ROM Firmware images for Compaq System product options, such as array controllers, disk drives, and tape drives used for non-shared storage. Compaq StorageWorks Command Console The Compaq StorageWorks Command Console (SWCC) is a graphical user interface (GUI) used to create, configure, and manage - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 71
to access up to 64 LUNs per target (array controller port). Compaq Operating System Dependent Modules Compaq supplies low-level services, called operating system dependent modules (OSDs), which are required by Oracle8i Parallel Server. The OSD layer monitors critical clustering hardware components - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 72
. It interacts with the OSDs, exchanging information about the state of each cluster node. For additional information, refer to: I Oracle8i Parallel Server Setup and Configuration Guide I Oracle8i Parallel Server Concepts I Oracle8i Parallel Server Administration, Deployment, and Performance - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 73
a series of test suites designed to stress an Oracle8i Parallel Server implementation Compaq will support any implementation of Oracle8i Parallel Server that does not strictly conform to the configurations certified with this process. For a complete list of certified Compaq servers, see the Compaq - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 74
3-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide To available and then connecting the client to one of the remaining, operational cluster nodes. NOTE: For complete information on how to ensure client auto-reconnect in an Oracle Parallel Server - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 75
Cluster Planning Before connecting any cables or powering on any hardware on your Compaq Parallel Database Cluster Model PDC/O5000 for cluster hardware I Planning cluster configurations for redundant Fibre Channel Fabrics I Planning cluster configurations for redundant Fibre Channel Arbitrated Loops - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 76
Availability of appropriate equipment and qualified personnel I Space for unpacking, installing, and servicing power requirements for your cluster components to identify special electrical supply needs in advance. Capacity Planning for Cluster Hardware Compaq ProLiant Servers The number of servers - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 77
For a current list of PDC/O5000certified servers and details on supported configurations, refer to the Compaq Parallel Database Cluster Model PDC/O5000 Certification Matrix for Windows 2000. This document is available on the Compaq website at www.compaq.com/solutions/enterprise/ha-pdc.html Planning - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 78
4-4 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide To determine which Fibre Channel SAN I Total number of host adapters in all nodes I Total number of array controller ports in all of the storage subsystems in the redundant Fibre Channel Fabric You - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 79
installed in each storage subsystem's controller enclosure. NOTE: For more information about redundant Fibre Channel Arbitrated Loops (FC-ALs) in a PDC/O5000, see Chapter 2, "Cluster Architecture." The Storage Hubs are available in two models: I Storage Hub 7 (7 ports) I Storage Hub 12 (12 - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 80
technical white paper Supported Ethernet Interconnects for Compaq Parallel Database Clusters Using Oracle Parallel Server for detailed information about configuring redundant Ethernet cluster interconnects. This document is available at www.compaq.com/support/techpubs/whitepapers Before installing - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 81
a PDC/O5000 cluster's availability by providing redundant components along configuration provides two of each component along the I/O paths. These components include: I At least two cluster nodes I Two host adapters in each node I Two Fibre Channel SAN Switches I Two dual-port array controllers - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 82
4-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Figure 4-1 shows the access to port 1 on one array controller and port 2 on the other array controller in each storage subsystem. This configuration ensures that the host adapters can still - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 83
between the nodes and the Fibre Channel SAN Switches, and between the Fibre Channel SAN Switches and the storage subsystems In a quad redundancy configuration, both ports on both array controllers in each storage subsystem access the same storagesets. Thus, up to three of the four paths between the - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 84
4-10 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I The two even-numbered Fibre Channel SAN Switches connect to port 2 of the array controllers. Fibre Channel SAN Switch #2 connects to array controller A, port 2 in each storage subsystem. Fibre - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 85
-AL increases a PDC/O5000 cluster's availability by providing redundant components along the configuration provides two of each component along the I/O paths. These components include: I At least two cluster nodes I Two host adapters in each node I Two Storage Hubs I Two dual-port array controllers - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 86
4-12 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Figure 4-3 shows the correct method for cabling the I/O path components for dual redundancy in a redundant FC-AL. In Figure 4-3, both Storage Hubs connect to port 1 on one array controller and to - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 87
for the connections between the nodes and the switches or hubs, and between the switches or hubs and the storage subsystems In a quad redundancy configuration, both ports on both array controllers in each storage subsystem access the same storagesets. Thus, up to three of the four paths between the - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 88
4-14 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I The two even-numbered Storage Hubs connect to port 2 of the array controllers. Storage Hub #2 connects to array controller A, port 2 in each storage subsystem. Storage Hub #4 connects to array - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 89
tuning database cluster servers for optimal performance. Efforts to plan, configure, and tune a disk in the array without affecting the availability of data to users. RAID also uses the technique of installed in the ProLiant server model you will be using. Refer to the server documentation for this - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 90
contained in the StorageWorks documentation provided with the HSG80 Array Controller. I Refer to the Compaq white paper Configuring Compaq RAID Technology for Database Servers, #ECG 011/0598, available at the Compaq website at www.compaq.com I Refer to the various white papers on Oracle8i, which - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 91
Cluster Planning 4-17 Raw Data Storage and Database Size Raw data storage is the amount of storage available before any RAID levels have been configured. It is called raw storage because RAID volumes require some overhead. The maximum size of a database stored in a RAID system will always be less - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 92
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Selecting the Appropriate RAID Level Many factors will affect the RAID levels you select for your cluster database. These include the specific availability of disks and controllers. The database - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 93
Storage Subsystem disk enclosure could be grouped for an Oracle8i Parallel Server database. Shared Disk Storage Array Disk Disk Disk Disk Disk Disk Storage Subsystem contains up to six disk enclosures. You are advised to configure all the drives in each enclosure as a single separate logical drive - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 94
form and provides a more user-friendly tool for executing CLI commands. IMPORTANT: To perform partitioning, all required drivers must already be installed for each server. These include the Compaq KGPSA driver for the host adapters, the HSG80 Array Controller (HSZDISK) driver, and the Secure Path - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 95
in that logical drive will disrupt service to all Oracle databases that are dependent on that disk drive. See "Selecting the Appropriate RAID Level" earlier in this chapter. The array controllers are equipped with RAID. The array controller RAID is configured with RAID storageset commands issued - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 96
4-22 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide The following list identifies the of the client network is no different than that used for a nonclustered configuration. To ensure continued access to the database when a cluster node is evicted - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 97
this chapter. I User guides for the clustered Compaq ProLiant servers I Installation posters for the clustered ProLiant servers I Installation guides for the cluster interconnect and client LAN interconnect adapters I Compaq SmartStart Installation Poster I Compaq SmartStart and Support Software CD - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 98
and disk enclosure components, including power supplies, I/O modules, disk drives, array controllers, and cache memory modules I Installing and configuring operating system software, including: G SmartStart 4.9 or later G Windows 2000 Advanced Server with Service Pack 1 or later I Setting up - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 99
I Installing and configuring Secure Path for Windows 2000, including: G Installing host adapter drivers G Installing the Fibre Channel Software Setup (FCSS) utility G Installing the Secure Path Driver, Secure Path Agent, and Secure Path Manager G Installing the optional Compaq StorageWorks Large LUN - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 100
document is available on the Compaq website at www.compaq.com/solutions/enterprise/ha-pdc.html While setting up the physical hardware, follow the installation instructions in your Compaq ProLiant Server Setup and Installation Guide and in your Compaq ProLiant Server Installation Poster. Installing - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 101
Configurations for Redundant Fibre Channel Arbitrated Loops" in Chapter 2, "Cluster Architecture." To cable the host adapters to the Fibre Channel SAN Switches or Storage Hubs: 1. Using Fibre Channel cables, connect host adapter #1 in each server refer to the Compaq documentation provided with them. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 102
client LAN. If you need specific instructions on how to install an adapter, refer to the documentation for the adapter you are installing or refer to the user guide of the ProLiant server you are using. Installing GBIC-SW Modules for the Array Controllers Each MA8000/EMA12000 Storage Subsystem - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 103
to a separate Ethernet adapter in each server. NOTE: In a redundant Ethernet cluster configuration, one Ethernet crossover cable must be installed cluster interconnect. 2. Connect the other end of one Ethernet cable to an Ethernet hub or switch. Connect the other end of the second Ethernet cable to - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 104
on configuring Ethernet connections in a redundant or non-redundant cluster interconnect, including enabling failover from one Ethernet path to another, see Supported Ethernet Interconnects for Compaq Parallel Database Clusters Using Oracle Parallel Server at www.compaq.com/support/techpubs - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 105
. For information on configuring Ethernet connections in a redundant client LAN, including enabling failover from one Ethernet path to another, see Supported Ethernet Interconnects for Compaq Parallel Database Clusters Using Oracle Parallel Server at www.compaq.com/support/techpubs/whitepapers - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 106
(EMU), one or two power supplies, and three dual-speed fans. In addition, the controller enclosure houses the six I/O modules that connect the enclosure's six SCSI buses to up to six disk enclosures. For detailed instructions on installing hardware components in a controller enclosure, refer to the - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 107
Installation and Configuration 5-11 Cabling the Controller Enclosure to Disk Enclosures Compaq recommends using the shortest UltraSCSI cable length possible to connect disk enclosures to the controller enclosure. The maximum supported cable length is 2 meters (6.6 ft). Refer to the documentation - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 108
5-12 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Figure 5-4 shows UltraSCSI cables installed between a controller enclosure and three disk enclosures. When three or fewer disk enclosures are present in a storage subsystem, you can install one - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 109
Installation and Configuration 5-13 Figure 5-5 shows UltraSCSI cables installed between a controller enclosure and six disk enclosures. When four or more disk enclosures are present in a storage subsystem, you must install one single-bus I/O module in each - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 110
5-14 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Cabling Array Controllers to Fibre Channel SAN Switches In a PDC/O5000 with a redundant Fibre Channel Fabric, a dual redundancy configuration is the minimum allowable configuration that provides - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 111
5-7 shows the correct method for cabling the I/O path components in a dual redundancy configuration. In this example, both Storage Hubs connect to port 1 on one array controller and to port 2 on the other array controller in each MA8000/EMA12000 Storage Subsystem. If one Storage Hub fails, the other - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 112
P1 P2 B P1 P2 Storage Subsystem #1 Storage Subsystem #2 Figure 5-7. Dual redundancy configuration for a redundant FC-AL Installing Operating System Software You will follow an automated procedure using Compaq SmartStart to install the operating system software on the cluster nodes for the PDC - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 113
it is being installed on new servers. If there is any existing data on the servers, it will be destroyed. You will need the following during SmartStart installation for Oracle8i: I SmartStart and Support Software CD 4.9 or later I Microsoft Windows 2000 Advanced Server with Service Pack 1 or later - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 114
5-18 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Cluster-Specific SmartStart Installation The SmartStart Installation Poster describes the general flow of configuring and installing software on a single server. The installation for a PDC/O5000 - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 115
Configuration lmhosts files located at %SystemRoot%\system32\drivers\etc. Record this information: G For is the IP address that Windows 2000 Advanced Server assigned to the client LAN. 11. If using Service Pack 1 or later: a. Double-click the Network and Dial-up Connections icon in the Control Panel - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 116
when installing Compaq OSDs. 12. If you are installing node1, open Windows 2000 Advanced Server Disk Management information on creating partitions, see the Oracle8i Parallel Server Setup and Configuration Guide. 13. If you are installing a node 16. Power down the cluster nodes and client LAN Ethernet switches - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 117
also connect the second array controller to the server through its maintenance port so you can configure it for I/O path failover. NOTE: Only one server in the cluster should be designated as the maintenance server. To connect the maintenance server to both array controllers in the storage subsystem - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 118
Intel Client. Double-click on Setup. 5. Follow the setup wizard instructions to complete the installation. 6. Restart the server. Configuring a Storage Subsystem for Secure Path Operation Each new MA8000/EMA12000 Storage Subsystem is preconfigured with specific parameters. You may need to change - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 119
label.) NOTE: The "other" controller will shut down and must be manually restarted by momentarily pressing the reset button on the controller's front panel. Wait for two minutes for the controller to boot before proceeding. 11. Configure both array controllers for multibus failover mode. a. Type - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 120
5-24 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide d. Verify that both controllers are in multibus failover mode by entering the following commands and viewing the results: show this_controller and show other_controller e. Close the CLI window. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 121
specified hexadecimal numerals for the ALPA addresses. For the array controllers, first use the middle-range ALPA addresses. When middle range ALPA addresses are no longer available, use the higher-range ALPA addresses. If neither is available, use the lower-range ALPA addresses. Make sure that you - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 122
5-26 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 17. If "this_controller" is in a redundant Fibre controller CLI window: set this mirrored_cache Both array controllers flush their cache and start mirroring each other's cache. Both controllers - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 123
. The disk drives will be named in the DISK##### format. Refer to the Compaq StorageWorks documentation for the naming conventions for storage subsystem devices. 6. Close the CLI window. Verifying Array Controller Properties Displaying SCSI Devices in the Devices Windowpane 1. Click on the Windows - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 124
and Windows 2000 Administrator Guide Table 5-1 Controller Properties Description General Host Ports Cache Communications LUN Connections Battery Values Allocation class: 0 SCSI version: SCSI-2 Host Port 1 Requested Topology: Loop Hard Host Port 1 Requested Port Address: (user defined) Host Port - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 125
SVR1BOT1 SVR1BOT2 When an installed host adapter has to be replaced, the default connection name for the new host adapter must also be changed to the PDC connection name format. For details, refer to Chapter 6, "Cluster Management." Configuring a Storageset You must perform each of the steps in - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 126
Guide 4. Click Next for step 3 of the wizard. 5. Select the maximum capacity for the virtual disk. IMPORTANT: Secure Path requires maximum capacity partitions. 6. Click Next for step 4 of the wizard. 7. Enter the virtual disk name. G If the optional Compaq the Save controller configuration to virtual - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 127
Installation and Configuration 5-31 12. Click Finish to create the virtual disk. The click OK to display the Connect Serial dialog box. 5. Select the COM port (the port to which an array controller is connected). 6. Select a baud rate of 9600. 7. Click Connect to display the CLI window. 8. Click - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 128
of the software configuration procedures required for your Secure Path storage environment. For detailed procedures, refer to the Compaq StorageWorks Secure Path for Windows 2000 on RAID Array 4000/4100 Version 3.1. I Install the host adapter drivers I Install the Fibre Channel Software Setup (FCSS - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 129
Secure Path software (Secure Path drivers, Secure Path Agent, and Secure Path Manager) I Specify the preferred_paths of the storagesets (units) to the array controllers I Create extended partitions After performing these procedures in sequence, the software configuration of your Secure Path storage - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 130
5-34 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 6. Select the Driver tab. Click Update driver. The Update Device Driver Wizard appears. 7. Select Display a list of the know drivers for the device so I can choose a specific driver. 8. Select the - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 131
Configuration 5-35 3. Follow the instructions in the setup program to complete the installation. The installation process copies the drivers subsystems. To install the Secure Path Raidisk filter driver, RAIDisk driver, and agent: 1. Insert the Compaq StorageWorks Secure Path Software CD into your CD - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 132
, it is recommended that you balance the available storagesets between array controller ports in each storage subsystem. As storage demands are defined and individual drive throughput requirements are understood, adjustments to the disk I/O path configuration can be made using the Secure Path - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 133
ports. For example, assign LUNs 0 and 1 to array controller A port 1, LUNS 2 and 3 to array controller A port 2, LUNS 4 and 5 to array controller B port 1, and LUNs 6 and 7 to array controller B port 2. In this configuration, a single controller port can access all eight LUNs in the event of a path - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 134
resources. One logical partition is required for each Oracle Parallel Server shared database resource. See the Oracle8i Parallel Server Setup and Configuration Guide for information on Oracle8i Parallel Server database guidelines and requirements for subdividing the large extended partitions into - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 135
and Configuration blink when being formatted. Label the blinking drive with the driver letter assigned to it so it is easily located later. Compaq operating system dependent modules (OSDs) for Oracle8i Parallel Server. Compaq supplies an Oracle8i software package with the OUI that installs Compaq - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 136
nodes. From one node in the cluster, the OUI installs and configures OSDs on the other nodes in the cluster, provided that: I The servers are communicating through a local area network using a TCP/IP protocol. I The user running the OUI program has administrator privileges on every node. Verifying - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 137
Mounting Remote Drives and Verifying Administrator Privileges Because the OUI installs and configures OSDs on the other nodes in the cluster from the installing machine, such as installing device drivers and changing registry entries. To install the OSDs, the user must have administrator privileges - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 138
in different domains, create a local user on each server with the same Windows Server user name and password on each node. Then make that local user a member of the local Administrators group on each node before installing the OSDs. 2. Insert the Compaq Parallel Database Cluster Clustering Software - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 139
Installation and Configuration 5-43 4. Click Next to continue. The License screen appears. 5. Click Accept to accept the terms of the license agreement. The File Locations screen appears. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 140
Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 6. Click Next to continue. The Installation Types screen appears. 7. Select Typical to install the Ethernet OSDs into the default location, C:\Compaq\OPS. Skip to step 12. 8. Select Custom to install the Ethernet - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 141
Installation and Configuration 5-45 9. If you want to accept the default location, skip to step 13. 10. If you want to specify an alternate location and you know the exact directory path, specify it by typing over the default location and skip to step 12. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 142
5-46 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 11. If you want to specify an alternate location but do not know the exact directory path, click Change Location. The Choose Directory screen appears. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 143
Installation and Configuration 5-47 12. Browse to the location where you want to install the OSDs. Click OK. The Component Locations screen appears indicating the alternate location you specified. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 144
5-48 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 13. Click Next to continue. The Cluster Members screen appears. 14. The Cluster Members screen indicates the host name of the installing machine. Define the - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 145
Installation and Configuration 5-49 15. Click Next to continue. The Cluster Members screen reappears. 16. The Cluster Members screen now shows the node names you specified on the - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 146
5-50 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 17. Click Next to continue. The OSD Summary screen appears. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 147
Installation and Configuration 5-51 18. If the information on the OSD Summary screen is not correct, click Previous to modify the previous screens. When the information is correct, - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 148
on all nodes in the cluster. This service must be started before installing Oracle software. See the Oracle documentation for information on starting the cluster manager. Installing Oracle Software Using the Oracle8i Parallel Server Setup and Configuration Guide, follow the steps to install Oracle8i - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 149
software, the Oracle manual instructs you to install vendor-supplied operating system dependent modules (OSDs). You have already completed this step. Configuring Oracle8i Software Configuring Oracle software includes configuring Oracle8i Parallel Server, Oracle8i Parallel Server Manager, Oracle8i - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 150
creating a database. For instructions on installing and using Object Link Manager, see the readme file on the Compaq Parallel Database Cluster Clustering Software for Oracle8i on Microsoft Windows 2000 CD at \Object Link Manager\readme.txt Additional Notes on Configuring Oracle Software In addition - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 151
Configuration %\system32\drivers\etc. service did not start successfully, make sure that the OracleCMService is started and the SID is in the Oracle registry. There are two logs that can provide information about why the Oracle service did not start. These logs are: I \Compaq\OPS\cmsrvr.log I \Compaq - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 152
: I If the Object Link Manager service is running, run the Object Link Manager to see if the link OPS_CMDISK is in the list of links. Power Distribution and Power Sequencing Guidelines Overview It is recommended that you connect most cluster components to a Compaq power distribution unit (PDU). PDUs - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 153
PDUs and UPSs, can be configured similarly to the example shown for server power distribution. Each enclosure in an MA8000/EMA12000 Storage Subsystem has two AC power inlets. Substituting power inlets for power supplies and storage subsystem enclosures for ProLiant servers shown in Figure 5-8, you - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 154
previously in this chapter might have instructed you to perform a different sequence. Be sure to power down the cluster components in the following order: 1. ProLiant servers 2. Ethernet switches 3. MA8000/EMA12000 Storage Subsystem controller enclosure 4. MA8000/EMA12000 Storage Subsystem disk - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 155
I Software maintenance I Managing changes to shared storage I Replacing a cluster node I Adding a cluster node I Managing cluster operation NOTE: The procedures in this chapter contain high-level instructions. The instructions summarize what has either already been discussed in this guide, Oracle - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 156
6-2 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Cluster Management Concepts Powering Off a Node Without Interrupting Cluster Services At some time during the life of your cluster you will need to perform an operation on a cluster node that will - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 157
user ability to access the database. Since users will experience some disruption of service and possibly performance degradation during a node eviction, they might become concerned about the availability the Event Log or within specific error log files. Oracle8i Server events are sent to the - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 158
Remotely controls servers In Compaq servers, each hardware subsystem, such as network adapters, system processors, and system memory, has a robust set of management capabilities. Compaq Full Spectrum Fault Management notifies of impending fault conditions. Several steps are required to configure the - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 159
. The CLI supports dozens of commands to configure, modify, and diagnose problems in the storage subsystem. For example, the CLI enables you to configure the failover setting of the array controllers, set the RAID levels of the drives, and view the connections between the servers and the storage - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 160
Guide Using the Compaq StorageWorks Command Console The StorageWorks Command Console (SWCC) is a graphical user interface used to create and manage storage subsystems for the PDC/O5000. While the CLI window provides very detailed control drive arrays have been configured by the MA8000/EMA12000 - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 161
Enterprise Manager Administrator's Guide I Oracle Enterprise Manager Configuration Guide I Oracle Enterprise Manager Concepts Guide I Oracle8i Parallel Server Setup and Configuration Guide I Oracle8i Parallel Server Concepts I Oracle8i Parallel Server Administration, Deployment, and Performance - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 162
instance on all nodes and stop the Oracle services running on all nodes prior to removing the OSDs. To deinstall the OSDs: 1. Log in as a user who has administrator privileges on all nodes in the cluster. 2. Insert the Compaq Parallel Database Cluster Clustering Software for Oracle8i on Microsoft - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 163
Cluster Management 6-9 3. If autorun is enabled, the Oracle Universal Installer Welcome screen appears. 4. If autorun is disabled, enter d:\setup at a C: prompt. Substitute the letter assigned to the CD drive for d. The Oracle Universal Installer Welcome screen appears. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 164
Oracle8i and Windows 2000 Administrator Guide 5. Click Deinstall Products. Next, a window containing an inventory of the installed components appears. The following shows the Inventory screen that is displayed with an Ethernet cluster interconnect. 6. Checkmark Compaq Ethernet OSDs and click Remove - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 165
of Oracle8i Server, consult your Compaq and Oracle service representatives. Make sure that the new release is certified and supported on the PDC/O5000, and get assistance to determine which procedure you should follow to perform the upgrade properly. Upgrading the OSDs During the life of your - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 166
has finished, restart all nodes. 5. Install the new Ethernet OSDs from the new by following the procedure described under "Installing Compaq OSDs" in Chapter 5, "Installation and Configuration." 6. After installing the new Ethernet OSDs, restart OracleCMService and OracleNMService on all nodes - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 167
communicate with all other nodes in the cluster. I The user running the Oracle Universal Installer does not have administrator privileges on again. To remove a partial OSD installation: 1. Insert the Compaq Parallel Database Cluster Clustering Software for Oracle8i on Microsoft Windows 2000 CD - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 168
. This enables Secure Path to recognize the newly-added logical drive volumes and partitions. 7. Perform the necessary Oracle commands to associate the new data files to the database. See the Oracle setup and configuration documentation provided with the Oracle software for detailed instructions. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 169
Configuring a Storageset" in Chapter 5, "Installation and Configuration." G "Creating Partitions" in Chapter 5, "Installation and Configuration in a RAIDset or mirrorset, the array controller automatically places it into the failed set and power off the cluster components to replace a failed drive. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 170
3. Follow the instructions in the Compaq StorageWorks documentation to remove the failed drive and to install a new drive. 4. Enable Autospare by typing the following CLI command and pressing Enter: set failedset autospare diskname The array controller automatically initializes the new drive and - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 171
a new one. 7. Power up the node and run the System Configuration Utility. Take the default settings for the host adapter and save them. Restart the node when prompted. 8. From the Device Manager, open the entry SCSI Adapter and RAID Controllers and verify that the Compaq KGPSA device driver is - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 172
(node 1, top controller, port 1), SVR1TOP2, SVR1BOT1, and SVR1BOT2. To rename the default connection names for the new host adapter using the 2 on each array controller are the same as offsets you defined when the storageset was originally configured. NOTE: If you have configured the I/O paths for - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 173
Oracle instance on each node, stop the Oracle services on each node, shut down each node, and power off the servers and storage subsystems. After you have configured the new storage subsystem, you need to make the additional space available to Oracle by performing the necessary Oracle commands to - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 174
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Removing the Node 1. Record pertinent information that will be needed when configuring 4. Stop all Oracle services on the node to be removed. 5. Shut down the server on that node and power it off. 6. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 175
the node's local non-shared disks with RAID. 5. Install and configure Windows 2000 Advanced Server with Service Pack 1 or later. 6. Install necessary Compaq drivers and utilities from the Compaq SmartStart and Support Software CD. 7. Enter unique IP addresses and node names for each node in - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 176
(node 1, top controller, port 1), SVR1TOP2, SVR1BOT1, and SVR1BOT2. To rename the default connection names for the new host adapter using the each array controller are the same as the offsets you defined when the storageset was originally configured. NOTE: If the I/O paths are configured for quad - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 177
replacment node is configured and physically connected to from a single server, the primary originally installed the OSDs, do the following on all original cluster nodes (not the replacement node): a. Shut down the Oracle instance. b. Stop all Oracle services. c. On the node that was originally - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 178
. For a current list of PDC/O5000-certified servers and details on supported configurations, refer to the Compaq Parallel Database Cluster Model PDC/O5000 Certification Matrix for Windows 2000. During the life of the PDC/O5000, you might need to add a new node to the cluster. A desire to increase - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 179
instructions you should read Chapter 5, "Installation and Configuration." Preparing the New configure Windows 2000 Advanced Server with Service Pack 1 or later (whichever is installed on the other nodes). 6. Install necessary Compaq drivers and utilities from the Compaq SmartStart and Support - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 180
6-26 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 8. Install the host adapter drivers. 9. Install the FCSS utility. 10. Install and configure Secure Path for Windows 2000. 11. Using Fibre Channel cables, connect the host adapters in the new node - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 181
SystemRoot%\system32\drivers\etc. Verify TCP/IP connectivity between the new node and the existing cluster nodes. Run the ping utility from the new node, all other nodes. Installing the Cluster Software Now that the new node is configured and physically connected to the cluster, the next procedure - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 182
only the new node name when asked which node to install the software on. See the Oracle8i Parallel Server Setup and Configuration Guide for details on installing and configuring the Oracle software. 7. Once the configuration of the Oracle software is complete, start the Oracle services and Oracle - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 183
location and state of all configured storage units on each of memory, processor, hard drive performance, network software, and application software. I Oracle8i Performance Manager captures, computes, and presents performance data about the Oracle database. Specific data about Oracle8i Parallel Server - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 184
monitoring. IMPORTANT: If you make changes to the Secure Path Agent Configuration, the changes will not go into effect until you stop and restart the Secure Path Agent Service from the Services screen. NOTE: If Secure Path Manager shows a controller failure, be sure you match the displayed failed - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 185
enable I/O operations to a path that you have previously disabled. I Change the Preferred I/O path. Use this action when Load Distribution is disabled, there are multiple I/O paths available to a storageset on the same controller and you wish to select a new Preferred Path for normal I/O operations. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 186
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide access to a failed path after the problem has been corrected. Refer to the I/O Path and Array Controller Failures Secure Path Manager protect path availability. The RaiDisk device driver running on - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 187
Secure Path To remove Secure Path and Fibre Channel software and hardware from your cluster: 1. Open the Control Panel and choose Add/Remove Programs. 2. Select SANWorks Secure Path Server and click Add/Remove. 3. Select SANWorks Secure Path Client and click Add/Remove. 4. Click Yes to stop - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 188
chapter addresses potential problems you might encounter as you install, configure, test, and operate your Compaq Parallel Database Cluster Model PDC/O5000 (referred to here as the PDC/O5000). This chapter is organized as follows: I Basic Troubleshooting Tips I Troubleshooting Oracle8i and operating - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 189
7-2 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Basic Troubleshooting Tips Power Verify hubs I ProLiant servers NOTE: The Storage Hubs and some network hub or switch devices do not have on/off buttons; they receive power when the power cord is - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 190
Server with Oracle8i Parallel Server Option I MA8000/EMA12000 Storage Subsystem drivers I Compaq Insight Manager I Compaq Array Configuration Utility I Compaq Secure Path for Windows 2000 I Compaq Ethernet OSDs I Cluster interconnect drivers I Microsoft Windows 2000 Advanced Server with Service - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 191
currently supported firmware revisions for the PDC/O5000, refer to the Compaq Parallel Database Cluster Model PDC/O5000 Certification Matrix for Windows 2000. This document is available at www.compaq.com/solutions/enterprise/ha-pdc.html Depending on your specific configuration, other firmware might - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 192
Troubleshooting 7-5 Troubleshooting Oracle8i and OSD Installation Problems installation. If difficulties persist, contact your support representative for assistance. I The OUI uses on the primary node: C: net use \\machine_name\C$ * /User: in which the "*" results in the system prompting - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 193
5, "Installation and Configuration" for an overview of Oracle8i software installation. Refer to the documentation provided with the software for detailed installation instructions. I Verify the connections and accessibility of the cluster components. Refer to the "Basic Troubleshooting Tips" section - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 194
the list of services and check to see if the OracleService is started. I Verify that the database links are properly set up on all nodes and that correct entries exist for all datafiles, including OPS_CMDISK. Refer to the Oracle8i Parallel Server Setup and Configuration Guide for further details - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 195
Guide I The cluster interconnect must be on a separate subnet than the client LAN. Check this by viewing the IP addresses assigned to all interconnect adapters and to all client LAN adapters. I If Compaq Insight Manager is loaded and configured switches or hubs are powered on. In the redundant - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 196
all cluster nodes, refer to Appendix A, "Diagnosing and Resolving Shared Disk Problems." Troubleshooting Client-to-Cluster Connectivity Problems A Network Client Cannot Communicate With the Cluster I Verify that the user ID has sufficient rights to perform the desired action. I Make sure proper - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 197
section does not address storage subsystem problems that are specific to the MA8000/EMA12000 Storage Subsystem itself or to the use of the storage subsystem in a standalone server configuration. For detailed coverage of those issues, refer to the Compaq StorageWorks documentation provided with the - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 198
Troubleshooting 7-11 To fix this problem, first do the following: 1. From the Event Viewer, locate the event log entry marked CPQKGPSA to find the host adapter device driver errors. 2. Restart the server. 3. If the driver error message persists, reinstall the KGPSA-BC Host Adapter device driver. If - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 199
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Verifying Connectivity to a Redundant Fibre Channel Fabric Verify that the array controllers . I For each host adapter pair installed in the server, verify that the upper (or leftmost) host adapter is - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 200
Troubleshooting 7-13 The topology for Port 1 on each array controller should be: I Fabric Up state (not the Standby state) I FABRIC Server Node Connectivity to Fibre Channel Fabric Connection Paths To verify that your servers are connected to all Fibre Channel Fabric paths, open a CLI window and - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 201
for more information about how to configure an MA8000/EMA12000 Storage Subsystem. The topology for Port 1 on each array controller should be: I Loop Up state (not the Standby state) I LOOP_HARD Server Node Connectivity to FC-AL Connection Paths To verify that your servers are connected to all FC-AL - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 202
2 accesses a different range of storage devices than port 1. Refer to "Verifying Array Controller Properties" in Chapter 5, "Installation and Configuration." Device or Devices Were Not Found by KGPSA-BC Device Driver All of the I/O path devices (storagesets) present on one of the I/O paths should - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 203
Switch or Storage Hub and out to its array controllers. Do the following: I Confirm that the KGPSA-BC device driver was properly installed. Refer to "Installing Secure Path Software for Windows 2000" in Chapter 5, "Installation and Configuration." I From the Secure Path Manager GUI, confirm that all - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 204
that are configured to the other array controller in the storage subsystem. I If the array controllers have already been set to Multibus_Failover mode, then check the integrity of the components along the unseen FC-AL path. Do the following: G Confirm that the Compaq KGPSA device driver was properly - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 205
1 and port 2 on each array controller are active. Troubleshooting Secure Path For detailed information about troubleshooting Secure Path, refer to the troubleshooting documentation provided with the Secure Path software. Secure Path Guidelines for Windows 2000 Advanced Server The following rules and - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 206
D0 through D63 for port 1 array controllers and D100 through D163 for port 2 array controllers. G Distribute the LUNs evenly across all four array controller ports. I There is no support for Windows 2000 dynamic disks. I Windows NT fault-tolerant driver (for the host-based RAIDsets) cannot - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 207
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Secure Path Manager Shows Reversed Locations for Top and Bottom Array Controllers The Secure Path Manager GUI detects and displays the two HSG80 Array Controllers the Agent configuration program, and - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 208
Advanced Server, the addition of new LUN(s) may cause the Add/Remove Hardware Wizard to display an error upon initializing the LUN(s) and yellow warning indicators to be displayed from the Device Manager. To cancel these error displays, reboot the affected server node or nodes. A Configuration of - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 209
Problems Introduction When one or more nodes in a Compaq Parallel Database Cluster Model PDC/O5000 (referred to here as the PDC/O5000) cannot properly join the cluster, you need to consider two possible causes: I A problem the available shared storage resources. I A connectivity problem exists - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 210
Software OK and Firmware Checks Problem Persists Perform Cluster Hardware Checks OK Problem Persists Rerun and Validate Object Link OK Manager On All Affected Nodes Problem Persists Exit. Problem Fixed or No Further Disk Problems Contact Your Compaq Support Representative Figure A-1. Tasks - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 211
Shared Disk Problems A-3 Run Object Link Manager on All Nodes Object Link Manager simplifies the creation and maintenance of symbolic All Affected Nodes in the Cluster" in this appendix for further instructions. For instructions on installing and using Object Link Manager, see the readme file: - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 212
Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Restart All Affected Nodes in the Cluster If Object Manager service is properly installed on the node. For instructions on installing and using Object Link Manager, refer to this readme file contained on the Compaq Parallel - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 213
Resolving Shared Disk Problems A-5 Run and Validate Secure Path Manager On All Nodes The Secure Path Manager screens provide information about the shared storage resources in the cluster, including MA8000/EMA12000 Storage Subsystem storagesets, array controller path configurations, drive lettering - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 214
consistent across all nodes. 3. Verify that the RAID configurations of shared disks are consistent across all nodes. 4. Repeat steps 1 through 3 for all nodes in the cluster. Perform Cluster Software and Firmware Checks Use the Compaq SmartStart and Support Software CD to perform these checks on the - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 215
Diagnosing and Resolving Shared Disk Problems A-7 3. Verify that all firmware versions, including those for the system ROM on each server, the host adapter firmware, and array controller firmware, are correct and consistent across all nodes. NOTE: Refer to the Compaq Parallel Database Cluster Model - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 216
Guide Contact Your Compaq Support Representative If you are unable to resolve the problem, contact your Compaq support representative for assistance. In North America, call the Compaq Technical Support Phone Center at 1-800-OK-COMPAQ. Outside North America, call the nearest Compaq Technical Support - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 217
which is dedicated to servicing information requests to and servers and storage to present a single system image to clients and to operate as a single system. Clustering is a way to improve availability and scalability. The interface to the command set used to configure an HSG80 Array Controller - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 218
Guide Fabric Fault tolerance FC-AL Fibre Channel Fibre Channel Arbitrated Loop (FC-AL) Fibre Channel Fabric Fibre Channel Fabric Setup Software (FCSS) utility Heartbeat High availability Hot-pluggable Hot spare A complex set of interconnections through which there can be multiple and (to the user - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 219
available to the nodes and improving the cluster's I/O performance. Network Interface Controller. A board that allows a computer to be connected with a network, and works with the network operating system to control the flow of information over the network. An individual server in a cluster. New - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 220
responsible for enabling parallel database support. An additional license is required for Oracle8i Parallel Server. Power-On Self-Test. A server, storage, network, or cluster). A software or hardware entity upon which a client/server application or service is dependent. As it pertains to the Compaq - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 221
storage subsystem for the PDC/O5000. A group of disk drives configured with RAID techniques to operate as a single container and ensure single storage unit to the cluster nodes. A graphical user interface (GUI) used to create and manage MA8000 system. Example: a server. Uninterruptible power supply. - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 222
new nodes 6-24 administrator privileges when installing ODSs for Oracle8i 5-40 when installing OSDs for Oracle8i 5-41 ALPAs 5-23 application failover software for Oracle8i 3-7 arbitrated loop physical addresses 5-23 ASR timer value 5-4 availability definition 1-4 features of Compaq ProLiant Servers - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 223
29 preparing a new node 6-25 preparing a replacement node 6-21 replacing a node 6-19 shutting down cleanly 6-2 supported servers 2-2 cluster performance Compaq software overview 3-2 Oracle software 3-6 overview 3-1 cluster troubleshooting basic tips 7-2 client to cluster connectivity problems - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 224
availability in 1-4 defined 1-2 overview 1-1 PDC/O5000 example 1-2 scalability in 1-4 Command Line Interface controlling storage subsystems 6-5 Compaq authorized reseller xx Compaq Full Spectrum Fault Management 3-3 Compaq Insight Manager monitoring cluster performance 6-29 monitoring server - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 225
/O5000 for Oracle8i and Windows 2000 Administrator Guide Compaq System Configuration Utility 3-3 Compaq StorageWorks Command Console 3-4 Compaq StorageWorks Fibre Channel Storage Hubs description 2-6 Compaq System Configuration Utility 3-3 configuring MA8000/EMA12000 Storage Subsystem 5-22 Oracle8i - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 226
SAN dual redundancy configuration example 2-21 Fibre Channel SAN Switch to array controller I/O paths 2-19 configurations 4-7 planning quad redundancy configurations 4-9 Fibre Channel SAN Switches description 2-5 firmware support telephone numbers xviii host adapters description 2-6 host server - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 227
Path 5-22 configuring a storageset 5-29 creating partitions 5-38 designating a server as a maintenance terminal 5-21 discovering shared physical disks 5-26 installing hardware into a controller enclosure 5-10 installing hardware into a disk enclosure 5-10 installing host adapter device drivers 5-33 - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 228
Path Agent 5-35 Secure Path drivers 5-35 Secure Path software 5-32 SmartStart 5-17 specifying Preferred_Path for storage units 5-36 verifying access to all virtual disks 5-31 verifying array controller properties 5-27 Integrated Management Log, using with Compaq Insight Manager 3-3 K KGPSA Host - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 229
disk problems A-3 operating system dependent modules See Compaq operating system dependent modules operating system software installation for Oracle8i 5-17 versions supported 3-2 Oracle software certification for Oracle8i 3-7 configuring for Oracle8i 5-53 installation troubleshooting for Oracle8i - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 230
5-57 storage subsystems 5-57 power distribution units 5-56 power sequencing 5-58 ProLiant servers ASR timer value 5-4 high-availability features 2-3 power distribution 5-57 supported servers 2-2 properties verify for HSG80 Array Controllers 5-27 Q quad redundancy configuration example for FC-AL SAN - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 231
shared disk problems A-5 Service Packs supported 5-17 Service Packs for Windows 2000 Advanced Server versions supported 3-2 shared disk problems contacting your Compaq support representative A-8 introduction A-1 performing cluster hardware checks A-7 performing cluster software and firmware checks - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 232
xvii System Configuration Utility See Compaq System Configuration Utility System Configuration Utility host adapter replacement 6-17 T TCP/IP required by Secure Path 5-33 required for OSDs for Oracle8i 5-40 technical support xix telephone numbers xx text conventions xvi troubleshooting basic tips - Compaq ProLiant 8500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 233
verifying node connectivity to FC-AL paths 7-14 troubleshooting tips accessibility of resources 7-3 firmware revisions 7-4 physical connections 7-2 power 7-2 software revisions 7-3 U uninterruptible power source 5-56 upgrading Oracle8i Server 6-11 V verifying IP addresses for Oracle8i 5-19 virtual
Parallel Database Cluster Model
PDC/O5000 for Oracle8
i
and
Windows 2000
Administrator Guide
Second Edition (June 2001)
Part Number 225081-002
Compaq Computer Corporation