Compaq ProLiant 6500 Compaq Parallel Database Cluster Model PDC/05000 for Orac
Compaq ProLiant 6500 Manual
View all Compaq ProLiant 6500 manuals
Add to My Manuals
Save this manual to your list of manuals |
Compaq ProLiant 6500 manual content summary:
- Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 1
Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Second Edition (June 2001) Part Number 225081-002 Compaq Computer Corporation - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 2
Compaq Computer Corporation Compaq, the Compaq logo, Compaq Insight Manager, SmartStart, ROMPaq, ProLiant, and StorageWorks Registered in U.S. Patent and Trademark Office. ActiveAnswers is a trademark of Compaq Information Technologies Group, L.P. in the United States and other countries Compaq - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 3
Help ...xviii Compaq Technical Support xix Compaq Website xix Compaq Authorized Reseller xx Chapter 1 Clustering Overview Clusters Defined ...1-2 Availability ...1-4 Scalability ...1-4 Compaq Parallel Database Cluster Overview 1-5 Chapter 2 Cluster Architecture Compaq ProLiant Servers 2-2 High - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 4
Interconnect 2-39 Local Area Network 2-42 Chapter 3 Cluster Software Components Overview of the Cluster Software 3-1 Microsoft Windows 2000 Advanced Server 3-2 Compaq Software...3-2 Compaq SmartStart and Support Software 3-2 Compaq System Configuration Utility 3-3 Compaq Insight Manager 3-3 - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 5
Site Planning...4-2 Capacity Planning for Cluster Hardware 4-2 Compaq ProLiant Servers 4-2 Planning Shared Storage Components for Redundant Fibre Planning Quad Redundancy Configurations 4-13 RAID Planning for the MA8000/EMA12000 Storage Subsystem 4-15 Supported RAID Levels 4-16 Raw Data Storage - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 6
15 Installing Operating System Software 5-16 Guidelines for Clusters 5-17 Automated Installation Using SmartStart 5-17 Setting up and Configuring an MA8000/EMA12000 Storage Subsystem 5-21 Designating a Server as a Maintenance Terminal 5-21 Powering On the MA8000/EMA12000 Storage Subsystem 5-21 - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 7
Power Distribution and Power Sequencing Guidelines 5-56 Overview ...5-56 Server Power Distribution 5-57 Storage Subsystem Power Distribution 5-57 Power Management Concepts 6-2 Powering Off a Node Without Interrupting Cluster Services 6-2 Managing a Cluster in a Degraded Condition 6-2 Managing - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 8
Cluster Interconnect or the Client LAN 7-8 Troubleshooting Client-to-Cluster Connectivity Problems 7-9 A Network Client Cannot Communicate With the Cluster 7-9 Troubleshooting Shared Storage Subsystem Problems 7-10 Verifying Host Adapter Device Driver Installation 7-10 Verifying KGPSA-BC Device - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 9
All Nodes A-5 Run and Validate the StorageWorks Command Console From All Storage Subsystems ...A-6 Perform Cluster Software and Firmware Checks A-6 Perform Cluster Hardware Checks A-7 Contact Your Compaq Support Representative A-8 Glossary Index - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 10
x Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide List of Figures Figure 1-1. Example of a two-node PDC/O5000 cluster 1-3 Figure 2-1. SCSI bus numbers for I/O modules in the controller enclosure ......... 2-10 Figure 2-2. UltraSCSI cabling - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 11
power distribution in a three-node cluster 5-57 Figure A-1. Tasks for diagnosing and resolving shared storage problems A-2 List of Tables Table 2-1 High-Availability Components of ProLiant Servers 2-3 Table 2-2 SCSI bus address ID assignments for the MA8000/EMA12000 Storage Subsystem 2-9 Table - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 12
, configuration, implementation, management, and troubleshooting of the Compaq Parallel Database Cluster Model PDC/O5000 on Oracle8i software running on the Microsoft Windows 2000 Advanced Server operating system. Audience The expected audience of this guide consists primarily of MIS professionals - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 13
the PDC/O5000. I Chapter 7, "Troubleshooting," contains troubleshooting information for the PDC/O5000. I Appendix A, "Diagnosing and Resolving Shared Disk Problems," describes procedures to diagnose and resolve shared disk problems. I Glossary contains definitions of terms used in this guide. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 14
refer to documentation related to the specific hardware and software components of the Compaq Parallel Database Cluster. These related manuals include, but are not limited to: I Documentation related to the ProLiant servers you are clustering (for example, guides, posters, and performance and tuning - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 15
2000, at www.compaq.com/solutions/enterprise/ha-pdc.html I Configuring Compaq RAID Technology for Database Servers, at www.compaq.com/highavailability I elements appear in uppercase. Type When you are instructed to type information, type the information without pressing the Enter key. Enter When - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 16
: Text set off in this manner presents clarifying information or specific instructions. NOTE: Text set off in this manner presents commentary, symbols indicates the presence of electrical shock hazards. Enclosed area contains no operator serviceable parts. WARNING: To reduce the risk of injury from - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 17
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Any surface or area jacks are extended to the floor. I The full weight of the rack rests on the leveling jacks. I a problem and have exhausted the information in this guide, you can get - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 18
for as long you own the product. A technical support specialist will help you diagnose the problem or guide you to the next step in the warranty process. In North America, call the Compaq Technical Phone Support Center at 1-800-OK-COMPAQ1. This service is available 24 hours a day, 7 days a week - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 19
PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Compaq Authorized Reseller For the name of your nearest Compaq Authorized Reseller: I In the United States, call 1-800-345-1518. I In Canada, call 1-800-263-5868. I Elsewhere, see the Compaq website for locations and telephone numbers. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 20
features, availability and scalability, have been moving into client/server products for the last few years. The absorption of clustering than proprietary clustering systems. Before examining the features and benefits of the Compaq Parallel Database Cluster Model PDC/O5000 (referred to here as the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 21
Components" to determine the specific releases your cluster kit supports. Figure 1-1 shows an example of a PDC/O5000 that includes two nodes (Compaq ProLiant™ servers), two sets of storage subsystems, two Compaq StorageWorksTM Fibre Channel SAN Switches or Compaq StorageWorks Fibre Channel Storage - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 22
B P1 P2 Storage Subsystem #1 Storage Subsystem #2 Figure 1-1. Example of a two-node PDC/O5000 cluster The PDC/O5000 can support redundant Fibre Channel Fabric Storage Area Network (SAN) and redundant Fibre Channel Arbitrated Loop (FC-AL) SAN topologies. In the example shown in Figure 1-1, the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 23
to an existing system without the expense of adding an entire new system. In a stand-alone server configuration, scalable systems allow increased throughput by adding processors or more memory. In a cluster configuration, this result is usually obtained by adding cluster nodes. Not only must the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 24
found in expensive, proprietary cluster solutions, at a fraction of the cost. The PDC/O5000 combines the popular Microsoft Windows 2000 Advanced Server operating system and the industry-leading Oracle8i Parallel Server with award-winning Compaq ProLiant servers and shared storage subsystems. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 25
solution to your computing environment. The hardware products include: I Compaq ProLiant servers I Shared storage components G Compaq StorageWorks Modular Array 8000 Fibre Channel Storage Subsystems or the Compaq StorageWorks Enterprise Modular Array 12000 Fibre Channel Storage Subsystems (MA8000 - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 26
Advanced Server with Service Pack 1 or later I Compaq drivers and utilities I Oracle8i Enterprise Edition with the Oracle8i Parallel Server Option For a description of the software products used with the PDC/O5000, refer to Chapter 3, "Cluster Software Components." Compaq ProLiant Servers A primary - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 27
which offer excellent reliability through redundant power supplies, redundant cooling fans, and Error Checking and Correcting (ECC) memory. The high-availability features of ProLiant servers are a critical foundation of Compaq clustering products. Table 2-1 lists the high-availability features found - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 28
Guide MA8000/EMA12000 Storage Subsystem The MA8000/EMA12000 Storage Subsystem is the shared storage solution for the PDC/O5000. Each storage subsystem consists of one controller enclosure and up to six disk enclosures. For detailed information about storage subsystem components, refer to the Compaq - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 29
storage unit called a storageset. RAID technology ensures that every unpartitioned storageset are simultaneously active, and access to a specific logical unit number (LUN) is distributed Compaq StorageWorks documentation. This guide port Fibre Channel SAN Switch are supported. From two to four Fibre - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 30
Guide For further information, refer to these manuals provided with each Fibre Channel SAN Switch: I Compaq StorageWorks Fibre Channel SAN Switch 8 Installation and Hardware Guide I Compaq more Storage Hubs provides fault tolerance and supports the redundant architecture implemented by the PDC/O5000 - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 31
Shortwave (multi-mode) fibre optic cables are used to connect the server nodes, Fibre Channel SAN Switches or Storage Hubs, and array in each storage subsystem. This 16-LUN restriction is imposed by the Compaq Secure Path software and the MA8000/EMA12000 Storage Subsystems. IMPORTANT: When offsets - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 32
2-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide If you do not install the optional Large LUN utility in you cluster, there are two accepted methods for distributing LUNs for storagesets: I You can - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 33
documentation provided with the storage subsystem. UltraSCSI Cables Compaq recommends using the shortest cable length possible to connect disk enclosures to the controller enclosure. The maximum supported cable length is 2 meters (6.6 ft). Refer to the documentation provided with your storage - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 34
2-10 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide SCSI Bus 1 SCSI Bus 2 SCSI Bus 3 SCSI Bus subsystem, use one dual-bus I/O module in each disk enclosure to support up to 14 disk drives in each disk enclosure. Using single-bus I/O - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 35
Cluster Architecture 2-11 Dual-Bus I/O Module Disk Enclosure Disk Enclosure Disk Enclosure I/O Modules (6) A B Controller Enclosure Figure 2-2. UltraSCSI cabling between a controller enclosure and three disk enclosures Using Single-Bus I/O Modules in the Disk Enclosures Figure 2-3 shows UltraSCSI - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 36
2-12 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Single-Bus I/O Module Single-Bus I/O Module Disk Enclosure Disk Enclosure Disk Enclosure Disk Enclosure Disk Enclosure Disk Enclosure I/O Modules (6) A B Controller Enclosure Figure 2-3. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 37
the Fibre Channel Arbitrated Loop (FC-AL) topology. The PDC/O5000 supports both topologies. A redundant Fibre Channel Fabric is two to four Fibre Fibre Channel SAN Switches, refer to the latest Compaq StorageWorks documentation. This guide does not document cascaded configurations for the Fibre - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 38
2-14 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide A redundant Fibre Channel Fabric consists Fabric Used in conjunction with the I/O path failover capabilities of Compaq Secure Path software, this redundant Fibre Channel Fabric configuration gives - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 39
of redundant Fibre Channel Fabrics you can install in a PDC/O5000 is restricted by the number of host adapters your Compaq servers support. Refer to the Compaq server documentation for this information. Figure 2-5 shows a two-node PDC/O5000 with two redundant Fibre Channel Fabrics. In this example - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 40
2-16 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Storage Subsystem #1 A P1 P2 B P1 P2 Redundant Fibre Channel Fabric #1 Fibre Channel SAN Switch #1 Host Adapters (4) Node 1 Fibre Channel SAN Switch #1 Storage Subsystem #2 A P1 P2 B - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 41
Cluster Architecture 2-17 Figure 2-6 illustrates these maximum cable distances for a redundant Fibre Channel Fabric. Host Adapters (2) 500 meter maximum Host Adapters (2) Node 1 Node 2 Fibre Channel SAN Switch # 1 500 meter maximum Fibre Channel SAN Switch # 2 A P1 P2 B P1 P2 A P1 P2 B - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 42
2-18 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Host Adapter-to-Fibre Channel SAN Switch I/O Data Paths Figure 2-7 highlights the I/O data paths that run between the host adapters and the Fibre Channel SAN - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 43
Cluster Architecture 2-19 Fibre Channel SAN Switch-to-Array Controller I/O Data Paths Figure 2-8 highlights the I/O data paths that run between the Fibre Channel SAN Switches and the two dual-port array controllers in each MA8000/EMA12000 Storage Subsystem. There is one I/O connection path to and - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 44
Guide I/O Path Definitions for Redundant Fibre Channel Fabrics In a redundant Fibre Channel Fabric, devices are accessed by the operating system using conventional SCSI addressing terminology. An I/O path consists of the complete physical interconnections from a given host (node) to a specific RAID - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 45
Cluster Architecture 2-21 I/O Path Configuration Examples for Redundant Fibre Channel Fabrics Every I/O path connection between host adapters and array controller ports is active at the same time. An active port does not become inactive unless it fails, whereupon all I/O activity on the failed path - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 46
2-22 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Host Adapters (2) Node 1 Fibre Channel SAN Switch # 1 Host Adapters (2) Node 2 Fibre Channel SAN Switch # 2 A P1 P2 B P1 P2 Storage Subsystem #1 A P1 P2 B P1 P2 Storage - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 47
Cluster Architecture 2-23 Figure 2-10 shows an acceptable method for cabling the I/O path components in a quad redundancy configuration. In this example, the following connections are made: I Each of four host adapters in each node connects to the same-numbered Fibre Channel SAN Switch (host adapter - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 48
2-24 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Host Adapters (4) Node 1 Node 2 Node 3 Node 4 Fibre Channel SAN Switch #1 Fibre Channel SAN Switch #3 A P1 P2 B P1 P2 Fibre Channel SAN Switch #2 Fibre Channel SAN - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 49
Cluster Architecture 2-25 Summary of I/O Path Failure and Failover Scenarios for Redundant Fibre Channel Fabrics Table 2-3 identifies possible I/O path failure events for both dual redundancy or quad redundancy configurations in a redundant Fibre Channel Fabric and the failover response implemented - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 50
a multi-layered architecture for moving data across the storage area network (SAN). This layered architecture can be implemented using the Fibre Channel Fabric or the Fibre Channel Arbitrated Loop (FC-AL) topology. The PDC/O5000 supports both topologies. Storage Hubs are used to create redundant FC - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 51
Cluster Architecture 2-27 Host Adapters (2) Node 1 Storage Hub #1 Host Adapters (2) Node 2 Storage Hub #2 A P1 P2 B P1 P2 Storage Subsystem #1 A P1 P2 B P1 P2 Storage Subsystem #2 Figure 2-11. Two-node PDC/O5000 with a four-loop redundant Fibre Channel Arbitrated Loop Used in - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 52
2-28 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Multiple Redundant Fibre Channel Arbitrated Loops The PDC/O5000 supports the the number of host adapters your Compaq servers support. Refer to the Compaq server documentation for this information. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 53
Cluster Architecture 2-29 Figure 2-12 shows a two-node PDC/O5000 that contains two redundant FC-ALs. In this example, each redundant FC-AL has its own pair of host adapters in each node, a pair of Storage Hubs, and two MA8000/EMA12000 Storage Subsystems. In Figure 2-12, the hardware components that - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 54
2-30 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Maximum Distances Between Nodes and Shared Storage Subsystem Components By using standard short-wave Fibre Channel cables with Gigabit Interface Converter-Shortwave (GBIC-SW) modules, - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 55
Cluster Architecture 2-31 Figure 2-13 illustrates these maximum cable distances for a redundant FC-AL. Host Adapters (2) 500 meter maximum Host Adapters (2) Node 1 Node 2 Storage Hub #1 Storage Hub #2 500 meter maximum A P1 P2 B P1 P2 A P1 P2 B P1 P2 Storage Subsystem #1 Storage - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 56
2-32 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I/O Data Paths in a Redundant FC-AL Within a redundant FC-AL, an I/O path connection exists between each host adapter and all four array controller ports in - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 57
Cluster Architecture 2-33 Secure Path monitors the status of the components along each active path. If Secure Path detects the failure of a host adapter, Fibre Channel cable, or Storage Hub along an active path, it automatically transfers all I/O activity on that path to the defined backup path. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 58
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide given host (node) to a specific RAID storageset in a shared storage subsystem Server operating system. Except for the LUN, the rest of the SCSI address is created within the host adapter miniport driver - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 59
Cluster Architecture 2-35 While cluster nodes use the unit number (Dxxyy) to identify and access a storageset or a single disk drive unit in the shared storage subsystems, the array controllers use a Port-Target-LUN (PTL) address to identify and access these resources in their storage subsystem. For - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 60
2-36 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Host Adapters (2) Node 1 Storage Hub #1 Host Adapters (2) Node 2 Storage Hub #2 A P1 P2 B P1 P2 A P1 P2 B P1 P2 Storage Subsystem #1 Storage Subsystem #2 Figure 2-16. Dual - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 61
Cluster Architecture 2-37 I The two odd-numbered Storage Hubs connect to port 1 of the array controllers. Storage Hub #1 connects to array controller A, port 1 in each storage subsystem. Storage Hub #3 connects to array controller B, port 1 in each storage subsystem. I The two even-numbered Storage - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 62
2-38 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Summary of I/O Path Failure and Failover Scenarios for Redundant FC-ALs Table 2-4 identifies possible I/O path failure events for both dual redundancy and quad configurations for a - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 63
Others are: private LAN, private interconnect, system area network (SAN), and private network. Throughout this guide, the term cluster interconnect is used. Compaq Parallel Database Clusters like the PDC/O5000 that use Oracle8i Parallel Server support an Ethernet cluster interconnect. In a PDC/O5000 - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 64
2-40 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Ethernet Cluster Interconnect They cannot be connected to the client network (LAN) or to servers that are not part of the PDC/O5000 cluster. When an Ethernet cluster interconnect is used - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 65
Cluster Architecture 2-41 Ethernet Cluster Interconnect Diagrams Figure 2-18 shows the redundant Ethernet cluster interconnect components used in a two-node PDC/O5000 cluster. Ethernet Switch/Hub #2 for Cluster Interconnect Dual-port Ethernet Adapters (2) Ethernet Switch/Hub #1 for Cluster - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 66
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Local Area server application requires a local area network, or LAN, over which client machines and servers auto-reconnect in an Oracle8i Parallel Server environment, contact your Oracle representative. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 67
(OSDs) and this Oracle software: I Oracle8i Enterprise Edition with the Oracle8i Parallel Server Option NOTE: For information about currently-supported software revisions for the PDC/O5000, refer to the Compaq Parallel Database Cluster Model PDC/O5000 Certification Matrix for Windows 2000 at www - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 68
to take advantage of specific capabilities offered on Compaq products. These drivers are provided for use with Compaq hardware only. The PDC/O5000 requires version 4.9 of the SmartStart and Support Software CD. For information about SmartStart, refer to the Compaq Server Setup and Management pack - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 69
Insight Manager. This log is used with the Compaq Integrated Management Display (IMD), the optional Remote Insight controller, and SmartStart. In Compaq servers, each hardware subsystem, such as non-shared disk storage, system memory, and system processor, has a robust set of management capabilities - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 70
Guide Compaq Options ROMPaq The Compaq Options ROMPaq diskettes allow a user to upgrade the ROM Firmware images for Compaq Compaq StorageWorks Secure Path for Windows 2000 Compaq StorageWorks Secure Path for Windows 2000 (Secure Path) must be installed on each server path StorageWorks RAID array - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 71
Server Setup and Configuration Guide provided with the Oracle8i software. The OSD software is found on the Compaq Parallel Database Cluster Clustering Software for Oracle8i on Microsoft Windows 2000 CD. This CD is included in the cluster kit for the PDC/O5000. Oracle Software The PDC/O5000 supports - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 72
3-6 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Oracle8i Server Enterprise Edition The Oracle8i Server Enterprise Edition provides the following: I Oracle8i Server I Oracle8i Parallel Server Option I Oracle8i Enterprise Manager Oracle8i Server - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 73
who choose to deliver platforms for use with Oracle8i Parallel Server must demonstrate the successful completion of the Oracle8i Parallel Server for Windows 2000 Certification. Neither Oracle nor Compaq will support any implementation of Oracle8i Parallel Server that does not strictly conform to the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 74
3-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide To implement this application and connection failover, a complete information on how to ensure client auto-reconnect in an Oracle Parallel Server environment, contact your Oracle representative. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 75
Before connecting any cables or powering on any hardware on your Compaq Parallel Database Cluster Model PDC/O5000 (referred to here as the Fabrics I Planning cluster configurations for redundant Fibre Channel Arbitrated Loops I RAID planning I Planning the grouping of physical disk storage space I - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 76
to the installation area I Availability of appropriate equipment and qualified personnel I Space for unpacking, installing, and servicing the computer equipment in advance. Capacity Planning for Cluster Hardware Compaq ProLiant Servers The number of servers you install in a PDC/O5000 should take - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 77
and memory size are all determined by several factors. These include the types of database applications being used and the number of clients connecting to the servers. NOTE: Certain restrictions apply to the server models and server configurations that are supported by the Compaq Parallel Database - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 78
4-4 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide To determine which Fibre depends on the amount of shared storage space required by the database, the hardware RAID levels used on the shared storage disks, and the number and storage capacity - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 79
and shared disk drives used in a PDC/O5000 redundant FC-AL depends on the amount of shared storage space required by the database, the hardware RAID levels used on the shared storage disks, and the number and storage capacity of disk drives installed in the enclosures. Refer to "Raw Data Storage - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 80
white paper Supported Ethernet Interconnects for Compaq Parallel Database Clusters Using Oracle Parallel Server for detailed information . Planning the Client LAN Every client/server application requires a local area network (LAN) over which client machines and servers communicate. In the case of a - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 81
Cluster Planning 4-7 Planning Cluster Configurations for Redundant Fibre Channel Fabrics A redundant Fibre Channel Fabric is two or more Fibre Channel SAN Switches installed between host adapters in cluster nodes and array controllers in the shared storage subsystems. Each redundant Fibre Channel - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 82
4-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Figure 4-1 shows the correct method for cabling the I/O path components for dual redundancy in a redundant Fibre Channel Fabric. In Figure 4-1, both Fibre Channel SAN Switches - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 83
Cluster Planning 4-9 Planning Quad Redundancy Configurations For the highest possible level of I/O path redundancy, you can configure your cluster for quad redundancy. This configuration provides four host adapters in each of at least four nodes and four Fibre Channel SAN Switches for each redundant - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 84
4-10 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I The two even-numbered Fibre Channel SAN Switches connect to port 2 of the array controllers. Fibre Channel SAN Switch #2 connects to array controller A, port 2 in - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 85
Cluster Planning 4-11 Planning Cluster Configurations for Redundant Fibre Channel Arbitrated Loops A redundant Fibre Channel Arbitrated Loop (FC-AL) is two or more Storage Hubs installed between host adapters in cluster nodes and array controllers in the shared storage subsystems. Each redundant FC- - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 86
4-12 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Figure 4-3 shows the correct method for cabling the I/O path components for dual redundancy in a redundant FC-AL. In Figure 4-3, both Storage Hubs connect to port 1 - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 87
Cluster Planning 4-13 Planning Quad Redundancy Configurations For the highest possible level of I/O path redundancy, you can configure your cluster for quad redundancy. This configuration provides four host adapters in each of at least four nodes and four Storage Hubs for each redundant FC-AL. The - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 88
4-14 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I The two even-numbered Storage Hubs connect to port 2 of the array controllers. Storage Hub #2 connects to array controller A, port 2 in each storage subsystem. Storage - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 89
disk in the array without affecting the availability of data to users. RAID also uses the technique of striping, which involves partitioning each drive's adapters that can be installed in the ProLiant server model you will be using. Refer to the server documentation for this information. I The - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 90
O5000 supports these RAID levels: I RAID 1 I RAID 0+1 I RAID 3/5 I RAID 5 For RAID level definitions and information about configuring hardware RAID, the Compaq white paper Configuring Compaq RAID Technology for Database Servers, #ECG 011/0598, available at the Compaq website at www.compaq.com - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 91
.2-GB Ultra3 disk drives provides a maximum storage capacity of 1.3 TB. The amount of shared disk space required for a given database size is affected by the RAID levels you select and the overhead required for indexes, I/O buffers, and logs. Consult your Oracle representative for further details. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 92
4-18 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Selecting the Appropriate RAID Level Many factors will affect the RAID levels you select for your cluster database. These include the specific availability, performance, reliability, and recovery - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 93
EMA12000 Storage Subsystem disk enclosure could be grouped for an Oracle8i Parallel Server database. Shared Disk Storage Array Disk Disk Disk Disk Disk Disk Disk with Disk Management Create logical partitions with Disk Management RAIDset RAID Logical Drive Extended Partition . . . . . . Figure - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 94
IMPORTANT: To perform partitioning, all required drivers must already be installed for each server. These include the Compaq KGPSA driver for the host adapters, the HSG80 Array Controller (HSZDISK) driver, and the Secure Path (RAIDISK.SYS) driver. Disk Drive Planning Nonshared Disk Drives Nonshared - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 95
Cluster Planning 4-21 If a logical drive is configured with a RAID level that does not support fault tolerance (for example, RAID 0), then the failure of the shared disk drives in that logical drive will disrupt service to all Oracle databases that are dependent on that disk drive. See "Selecting - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 96
4-22 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide The following list identifies the format of the hosts and lmhosts files for a four-node PDC/O5000 cluster with an Ethernet cluster interconnect: IP address - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 97
this chapter. I User guides for the clustered Compaq ProLiant servers I Installation posters for the clustered ProLiant servers I Installation guides for the cluster interconnect and client LAN interconnect adapters I Compaq SmartStart Installation Poster I Compaq SmartStart and Support Software CD - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 98
, including power supplies, I/O modules, disk drives, array controllers, and cache memory modules I Installing and configuring operating system software, including: G SmartStart 4.9 or later G Windows 2000 Advanced Server with Service Pack 1 or later I Setting up and configuring an MA8000/EMA12000 - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 99
G Installing the Secure Path Driver, Secure Path Agent, and Secure Path Manager G Installing the optional Compaq StorageWorks Large LUN utility G Installing the Hardware Setting Up the Nodes Physically preparing the nodes (servers) for a cluster is not very different than preparing them for - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 100
com/solutions/enterprise/ha-pdc.html While setting up the physical hardware, follow the installation instructions in your Compaq ProLiant Server Setup and Installation Guide and in your Compaq ProLiant Server Installation Poster. Installing the KGPSA-BC and KGPSA-CB Host Adapters To install KGPSA-BC - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 101
1 and 2 for all other host adapters in each server and additional Fibre Channel SAN Switches or Storage Hubs. Connect like-numbered host adapters across servers to the same Fibre Channel SAN Switch or Storage Hub. SAN Switches and Storage Hubs, refer to the Compaq documentation provided with them. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 102
5-6 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Installing the Cluster specific instructions on how to install an adapter, refer to the documentation for the adapter you are installing or refer to the user guide of the ProLiant server you - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 103
switches or hubs G For two-node PDC/O5000 clusters, you can use two 100-Mbit/sec Ethernet switches or hubs with cables to connect the servers. G For PDC/O5000 clusters with three or more nodes, you use two 100-Mbit/sec Ethernet switches connected by Ethernet cables to a separate Ethernet adapter - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 104
5-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Figure 5-2 shows the redundant Ethernet cluster see Supported Ethernet Interconnects for Compaq Parallel Database Clusters Using Oracle Parallel Server at www.compaq.com/support/techpubs/ - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 105
TCP/IP network to connect to a client LAN. The following procedure contains instructions for cabling an Ethernet client LAN. To cable an Ethernet client LAN: 1. see Supported Ethernet Interconnects for Compaq Parallel Database Clusters Using Oracle Parallel Server at www.compaq.com/support/techpubs - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 106
5-10 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Installing Hardware Into fans. In addition, the controller enclosure houses the six I/O modules that connect the enclosure's six SCSI buses to up to six disk enclosures. For detailed instructions - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 107
and Configuration 5-11 Cabling the Controller Enclosure to Disk Enclosures Compaq recommends using the shortest UltraSCSI cable length possible to connect disk enclosures to the controller enclosure. The maximum supported cable length is 2 meters (6.6 ft). Refer to the documentation provided - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 108
5-12 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Figure 5-4 shows UltraSCSI cables installed between a controller enclosure and three disk enclosures. When three or fewer disk enclosures are present in a storage subsystem, you can - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 109
EMUs to Each Other For information about connecting the EMUs between enclosures in an MA8000/EMA12000 Storage Subsystem and between different subsystems, refer to the Compaq documentation provided with the storage subsystem. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 110
5-14 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Cabling Array Controllers to Fibre Channel SAN Switches In a PDC/O5000 with a redundant Fibre Channel Fabric, a dual redundancy configuration is the minimum allowable configuration that - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 111
Installation and Configuration 5-15 Host Adapters (2) Node 1 Fibre Channel SAN Switch # 1 Host Adapters (2) Node 2 Fibre Channel SAN Switch # 2 A P1 P2 B P1 P2 Storage Subsystem #1 A P1 P2 B P1 P2 Storage Subsystem #2 Figure 5-6. Dual redundancy configuration for a redundant Fibre - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 112
5-16 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Storage Hub still has access Installing Operating System Software You will follow an automated procedure using Compaq SmartStart to install the operating system software on the cluster nodes for - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 113
data on the servers, it will be destroyed. You will need the following during SmartStart installation for Oracle8i: I SmartStart and Support Software CD 4.9 or later I Microsoft Windows 2000 Advanced Server with Service Pack 1 or later I SmartStart Installation Poster I Server Profile Diskette - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 114
5-18 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Cluster-Specific SmartStart Installation The SmartStart Installation Poster describes the general flow of configuring and installing software on a single server. The installation for a PDC/O5000 - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 115
and lmhosts files located at %SystemRoot%\system32\drivers\etc. Record this information: G For system area network.) Due to the complexity of Windows 2000 Advanced Server and multipleNIC servers, you box for the client LAN port and you are using Service Pack 1 or later: a. Double-click the Network - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 116
information later when installing Compaq OSDs. 12. If you are installing node1, open Windows 2000 Advanced Server Disk Management to create For more information on creating partitions, see the Oracle8i Parallel Server Setup and Configuration Guide. 13. If you are installing a node other than node1 - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 117
connector on a second communications cable to the maintenance port on the second array controller. 4. Connect the 9-pin serial connector on the communications cable to the unused COM port on the server. NOTE: Be careful to note which serial port is connected to which array controller. You will need - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 118
22 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide instructions to complete the installation. 6. Restart the server. Configuring a Storage Subsystem for Secure Path Operation Each new MA8000/EMA12000 Storage Subsystem is preconfigured with specific - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 119
: NOTE: Refer to the Compaq StorageWorks documentation for more detailed information -xxxx xx (xxxx-xxxx-xxxx-xxxx is the bar-coded World Wide ID located in the controller enclosure. The "other" controller will shut down and must be manually restarted by momentarily pressing the reset button on the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 120
5-24 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide d. Verify that both controllers are in multibus failover mode by entering the following commands and viewing the results: show this_controller and show other_controller e. Close the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 121
Installation and Configuration 5-25 set this_controller port_2_topology=loop_hard set other_controller port_2_topology=loop_hard IMPORTANT: The ALPA addresses of port 1 on both array controllers must be identical; the ALPA addresses of port 2 on both array controllers must be identical. Use only the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 122
5-26 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 17. If "this_controller" is in a redundant Fibre Channel Fabric, verify the following: a. Port_1_topology=fabric b. Port_2_topology=fabric 18. If "this_controller" is in a redundant FC-AL, verify - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 123
discovered are consistent with the installed disk drives. The disk drives will be named in the DISK##### format. Refer to the Compaq StorageWorks documentation for the naming conventions for storage subsystem devices. 6. Close the CLI window. Verifying Array Controller Properties Displaying SCSI - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 124
5-28 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Table 5-1 Controller Properties condition: selected Confirm that the screen is grayed out (disabled); if not, return to the "Disable Communication LUN" section. Operating System: Windows 2000 Unit - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 125
to begin step 1 of the Add Virtual Disk Wizard. NOTE: A virtual disk is also known as a logical drive. 2. Select a RAID level setting. Click Next for step 2 of the wizard. Supported RAID levels are: RAID1, RAID0+1, RAID3/5, and RAID5. NOTE: If the disks are being used for the first time, a dialog - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 126
2000 Administrator Guide 4. Click Next for step 3 of the wizard. 5. Select the maximum capacity for the virtual disk. IMPORTANT: Secure Path requires maximum capacity partitions. 6. Click Next for step 4 of the wizard. 7. Enter the virtual disk name. G If the optional Compaq StorageWorks Large - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 127
the virtual disk you created. The hourglass on the disk icon indicates that the storageset is being initialized. The drives you used to create the RAID set are highlighted in the Devices window. 13. Repeat steps 1 through 12 to create other virtual disks. 14. Wait for the virtual disk initialization - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 128
5-32 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 11. If not, type the following and press , refer to the Compaq StorageWorks Secure Path for Windows 2000 on RAID Array 4000/4100 Version 3.1. I Install the host adapter drivers I Install the Fibre - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 129
(Secure Path drivers, Secure Path : I Server software (Raidisk filter driver, RAIDisk driver and SecurePathAgent server management application that requires TCP/IP to be installed. You must install the Secure path drivers the Host Adapter Drivers To install the host adapter drivers: 1. From the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 130
5-34 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 6. Select the Driver tab. Click Update driver. The Update Device Driver Wizard appears. 7. Select Display a list of the know drivers for the device so I can choose a specific driver. 8. Select the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 131
35 3. Follow the instructions in the setup program to complete the installation. The installation process copies the drivers into the proper driver, and agent: 1. Insert the Compaq StorageWorks Secure Path Software CD into your CD-ROM drive. 2. If you have CD AUTORUN enabled on your server - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 132
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide To install Secure Path Manager: 1. Insert the StorageWorks Secure Path Software CD in your CD-ROM drive. 2. If you have CD AUTORUN enabled on your server assigns units to a specific port (1 or 2) on - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 133
Installation and Configuration 5-37 If you do not install the optional Large LUN utility on your cluster, there are two accepted methods for distributing LUNs for storagesets: I You can distribute eight LUNs between the same-numbered ports on both array controllers so that a total of 16 LUNs are - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 134
resources. One logical partition is required for each Oracle Parallel Server shared database resource. See the Oracle8i Parallel Server Setup and Configuration Guide for information on Oracle8i Parallel Server database guidelines and requirements for subdividing the large extended partitions into - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 135
. Drives blink when being formatted. Label the blinking drive with the driver letter assigned to it so it is easily located later. 5. Assign Compaq operating system dependent modules (OSDs) for Oracle8i Parallel Server. Compaq supplies an Oracle8i software package with the OUI that installs Compaq - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 136
that: I The servers are communicating through a local area network using a TCP option to install Oracle8i Parallel Server Database software. The Compaq operating system dependent modules to each cluster node while you were using SmartStart to install the operating system software. If drivers\etc. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 137
also verifies the administrator privileges of the installer. Administrator privileges are required for most installation tasks, such as installing device drivers and changing registry entries. To install the OSDs, the user must have administrator privileges on each node from the installing machine - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 138
domains, create a local user on each server with the same Windows Server user name and password on each node. Then make that local user a member of the local Administrators group on each node before installing the OSDs. 2. Insert the Compaq Parallel Database Cluster Clustering Software for Oracle8i - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 139
Installation and Configuration 5-43 4. Click Next to continue. The License screen appears. 5. Click Accept to accept the terms of the license agreement. The File Locations screen appears. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 140
Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 6. Click Next to continue. The Installation Types screen appears. 7. Select Typical to install the Ethernet OSDs into the default location, C:\Compaq\OPS. Skip to step 12. 8. Select Custom to install the Ethernet - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 141
Installation and Configuration 5-45 9. If you want to accept the default location, skip to step 13. 10. If you want to specify an alternate location and you know the exact directory path, specify it by typing over the default location and skip to step 12. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 142
5-46 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 11. If you want to specify an alternate location but do not know the exact directory path, click Change Location. The Choose Directory screen appears. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 143
Installation and Configuration 5-47 12. Browse to the location where you want to install the OSDs. Click OK. The Component Locations screen appears indicating the alternate location you specified. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 144
5-48 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 13. Click Next to continue. The Cluster Members screen appears. 14. The Cluster Members screen indicates the host name of the installing machine. Define the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 145
Installation and Configuration 5-49 15. Click Next to continue. The Cluster Members screen reappears. 16. The Cluster Members screen now shows the node names you specified on the first Cluster Members screen with "_san" appended to each name, indicating it as a cluster interconnect node name. If - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 146
5-50 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 17. Click Next to continue. The OSD Summary screen appears. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 147
Installation and Configuration 5-51 18. If the information on the OSD Summary screen is not correct, click Previous to modify the previous screens. When the information is correct, click Next to continue. The Summary screen appears. 19. Click Install to install the components listed in the summary. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 148
on all nodes in the cluster. This service must be started before installing Oracle software. See the Oracle documentation for information on starting the cluster manager. Installing Oracle Software Using the Oracle8i Parallel Server Setup and Configuration Guide, follow the steps to install Oracle8i - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 149
does not offer the option to install Oracle8i Parallel Server, it could mean that the OSDs did not install successfully or that the OracleCMService is not running. After installing Oracle software, the Oracle manual instructs you to install vendor-supplied operating system dependent modules - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 150
Guide online changes, Oracle8i Server can find the correct data files and can properly start the Oracle database. You must install Object Link Manager before installing the Oracle database software to ensure that the symbolic links are defined before creating a database. For instructions - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 151
SystemRoot%\system32\drivers\etc. service did not start successfully, make sure that the OracleCMService is started and the SID is in the Oracle registry. There are two logs that can provide information about why the Oracle service did not start. These logs are: I \Compaq\OPS\cmsrvr.log I \Compaq - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 152
Guide Other Verification Tasks Perform these other verification tasks: I Verify that Object Link Manager has been installed and is running. I Verify that the OPS_CMDISK symbolic link has actually been set. To verify that the Object Link Manager has been installed and is running, go to the Services - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 153
to UPS #1, and PDU #2 is connected to UPS #2. Each UPS is connected to building power. ProLiant Servers Power Supply #1 Power Supply #2 PDU #1 UPS #1 Building Power PDU #2 UPS #2 Figure 5-8. Server power distribution in a three-node cluster Having two PDUs and UPSs in the cluster provides - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 154
5-58 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Power Sequencing The following power sequencing have instructed you to perform a different sequence. Be sure to power down the cluster components in the following order: 1. ProLiant servers 2. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 155
ongoing activities. This chapter describes these management activities for the Compaq Parallel Database Cluster Model PDC/O5000 (referred to here as this chapter contain high-level instructions. The instructions summarize what has either already been discussed in this guide, Oracle documentation, or - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 156
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Cluster Management Concepts Powering Off a Node Without Interrupting Cluster Services node. 3. Shut down the server. The node can now be powered to follow this troubleshooting process: 1. Determine - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 157
the affected nodes and wait until a non-peak time to service the problem. c. If the problem is critical and affects all the cluster nodes, shut down the can be viewed either within the Event Log or within specific error log files. Oracle8i Server events are sent to the Application Event Log and can - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 158
and interconnect adapters I Forwards server alert fault conditions I Remotely controls servers In Compaq servers, each hardware subsystem, such as network adapters, system processors, and system memory, has a robust set of management capabilities. Compaq Full Spectrum Fault Management notifies - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 159
diagnose problems in the storage subsystem. For example, the CLI enables you to configure the failover setting of the array controllers, set the RAID levels of the drives, and view the connections between the servers and the storage subsystem. For detailed information, see the Compaq StorageWorks - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 160
PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Using the Compaq StorageWorks Command Console The StorageWorks Command Console ( of the shared storage system. Secure Path Manager is a component of the Compaq StorageWorks Secure Path software that is included in the PDC/O5000 cluster kit - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 161
Guide I Oracle8i Parallel Server Concepts I Oracle8i Parallel Server Administration, Deployment, and Performance Oracle8i Enterprise Manager and its corresponding documentation are shipped as part of Oracle8i Enterprise Edition. Remotely Managing a Cluster Oracle8i Enterprise Manager, Compaq - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 162
6-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Software Maintenance Deinstalling the OSDs Be sure to shut down the Oracle instance on all nodes and stop the Oracle services running on all nodes prior to removing the OSDs. To deinstall the OSDs: - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 163
Cluster Management 6-9 3. If autorun is enabled, the Oracle Universal Installer Welcome screen appears. 4. If autorun is disabled, enter d:\setup at a C: prompt. Substitute the letter assigned to the CD drive for d. The Oracle Universal Installer Welcome screen appears. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 164
Oracle8i and Windows 2000 Administrator Guide 5. Click Deinstall Products. Next, a window containing an inventory of the installed components appears. The following shows the Inventory screen that is displayed with an Ethernet cluster interconnect. 6. Checkmark Compaq Ethernet OSDs and click Remove - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 165
likely affect the operation of the cluster. Before upgrading to any new release of Oracle8i Server, consult your Compaq and Oracle service representatives. Make sure that the new release is certified and supported on the PDC/O5000, and get assistance to determine which procedure you should follow to - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 166
Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 2. Click Deinstall Products. The Inventory screen showing a list of installed components is displayed: 3. Checkmark Compaq Ethernet OSDs and click Remove. The Oracle Universal Installer removes all Ethernet - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 167
installation (by removing it) before you can successfully run the Oracle Universal Installer again. To remove a partial OSD installation: 1. Insert the Compaq Parallel Database Cluster Clustering Software for Oracle8i on Microsoft Windows 2000 CD into the CD-ROM drive of a cluster node. 2. From - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 168
6-14 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Managing Changes to Shared Storage Components Adding New Storagesets to Increase Shared Oracle setup and configuration documentation provided with the Oracle software for detailed instructions. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 169
operate without one of its members. IMPORTANT: If the failed drive is not part of a fault-tolerant drive array, you can lose some or all of the data on the failed drive. RAID level 0 does not provide fault tolerance. With RAID employed, the effects of replacing a drive are felt only by the storage - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 170
the instructions in the Compaq StorageWorks driver or a with the host adapter itself. To identify the source of the problem and fix it, do the following: 1. From the Device Manager, open the entry SCSI Adapter and RAID Controllers. 2. Confirm that there are no red or yellow marks next to entry Compaq - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 171
Oracle instance on the node. 2. Shut down the Oracle services running on the node. 3. Shut down the server. 4. Power off the node. NOTE: The KGPSA-BC SCSI Adapter and RAID Controllers and verify that the Compaq KGPSA device driver is started. If it is not started, reload the driver. 9. Connect the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 172
6-18 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 10. From CLI, use the Rename command to rename the connections between the configuration seen from other nodes in the cluster. 14. Start the Oracle services and Oracle instance on the cluster node. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 173
the first storage subsystem. The majority of the instructions to add a storage subsystem are found in stop the Oracle services on each node, shut down each node, and power off the servers and storage subsystems supported by the PDC/O5000. To determine if your new server is supported, see the Compaq - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 174
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide to be removed. 4. Stop all Oracle services on the node to be removed. 5. Shut down the server on that node and power it off. need detailed instructions you should read through Chapter 5, "Installation - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 175
Utility to configure the node's local non-shared disks with RAID. 5. Install and configure Windows 2000 Advanced Server with Service Pack 1 or later. 6. Install necessary Compaq drivers and utilities from the Compaq SmartStart and Support Software CD. 7. Enter unique IP addresses and node names for - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 176
6-22 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 11. Using Fibre Channel cables, connect the host adapters in the replacement node to the appropriate Storage Hubs or Fibre Channel SAN Switches. 12. If, - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 177
further simplified in that almost all steps are performed from a single server, the primary cluster node. To replace a cluster node, you must not the replacement node): a. Shut down the Oracle instance. b. Stop all Oracle services. c. On the node that was originally used to install the OSDs, use the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 178
configuring Oracle software. Adding a Cluster Node Before adding a node, verify that the new configuration is supported by the PDC/O5000. For a complete list of servers supported by the PDC/O5000, see the Compaq Parallel Database Cluster Model PDC/O5000 Certification Matrix for Windows 2000 at www - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 179
to follow. If you need detailed instructions you should read Chapter 5, " RAID. 5. Install and configure Windows 2000 Advanced Server with Service Pack 1 or later (whichever is installed on the other nodes). 6. Install necessary Compaq drivers and utilities from the Compaq SmartStart and Support - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 180
6-26 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 8. Install the host adapter drivers. 9. Install the FCSS utility. 10. Install and configure Secure Path for Windows 2000. 11. Using Fibre Channel cables, connect the host adapters in the new - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 181
of each existing cluster node. These files are located at %SystemRoot%\system32\drivers\etc. Verify TCP/IP connectivity between the new node and the existing cluster application-level cluster software, including Oracle8i Server with the Oracle8i Parallel Server Option. The cluster must be brought - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 182
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Shut down the Oracle instance. b. Stop all Oracle services. c. Deinstall the OSDs by running the OUI on on. See the Oracle8i Parallel Server Setup and Configuration Guide for details on installing and - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 183
Examples of monitored objects are memory, processor, hard drive performance, network software, and application software. I Oracle8i Performance Manager captures, computes, and presents performance data about the Oracle database. Specific data about Oracle8i Parallel Server metrics are monitored and - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 184
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Using Secure Path Manager StorageWorks Secure Path is a high availability software product that provides continuous data access for MA8000/EMA12000 Storage Subsystems. Redundant hardware, advanced RAID - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 185
the storageset. I The array controller servicing the path. I The physical port number of the host adapter servicing the path. This is a relative Alternate. Use this action when you have Load Distribution enabled and you want to disable I/O operations to one or more I/O paths. I Make an I/O path - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 186
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide to restore access to a failed path after the problem has been corrected. Refer to the Secure Path protect path availability. The RaiDisk device driver running on cluster nodes handles Secure Path - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 187
Secure Path and Fibre Channel software and hardware from your cluster: 1. Open the Control Panel and choose Add/Remove Programs. 2. Select SANWorks Secure Path Server and click Add/Remove. 3. Select SANWorks Secure Path Client and click Add/Remove. 4. Click Yes to stop the Secure Path Agent so that - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 188
chapter addresses potential problems you might encounter as you install, configure, test, and operate your Compaq Parallel Database Cluster Model PDC/O5000 (referred to here as the PDC/O5000). This chapter is organized as follows: I Basic Troubleshooting Tips I Troubleshooting Oracle8i and operating - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 189
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Basic Troubleshooting Tips Power Verify that the cluster nodes, storage subsystems, and Fibre Channel SAN Switches or Storage Hubs for the storage area switches/hubs I ProLiant servers NOTE: The - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 190
Server with Oracle8i Parallel Server Option I MA8000/EMA12000 Storage Subsystem drivers I Compaq Insight Manager I Compaq Array Configuration Utility I Compaq Secure Path for Windows 2000 I Compaq Ethernet OSDs I Cluster interconnect drivers I Microsoft Windows 2000 Advanced Server with Service - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 191
Guide For information about currently supported software revisions for the PDC/O5000, refer to the Compaq Parallel Database Cluster Model PDC/O5000 Certification Matrix for Windows 2000. This document is available at www.compaq.com/solutions/enterprise/ha-pdc.html Depending on your specific - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 192
Troubleshooting 7-5 Troubleshooting Oracle8i and OSD Installation Problems and Error Messages Potential Difficulties will likely result in an incomplete installation. If difficulties persist, contact your support representative for assistance. I The OUI uses the network connections to transfer files - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 193
documentation provided with the software for detailed installation instructions. I Verify the connections and accessibility of the cluster components. Refer to the "Basic Troubleshooting Tips" section in this chapter. I Check the Windows 2000 Advanced Server event log for errors. Be sure to check - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 194
the service and restart it. Troubleshooting Node-to-Node Connectivity Problems Nodes Are Unable to Communicate with Each Other I Check the System log and Applications log for errors. I Check the OracleCMService error log for information. The default location for the error log is C:\compaq\ops - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 195
Guide I The cluster interconnect must be on a separate subnet than the client LAN. Check this by viewing the IP addresses assigned to all interconnect adapters and to all client LAN adapters. I If Compaq drivers are properly installed and configured on the cluster nodes. Verify the Ethernet drivers - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 196
and all other Oracle services have started without error. From the Services screen, check to see if the services have started. From the A, "Diagnosing and Resolving Shared Disk Problems." Troubleshooting Client-to-Cluster Connectivity Problems A Network Client Cannot Communicate With the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 197
section does not address storage subsystem problems that are specific to the MA8000/EMA12000 Storage Subsystem itself or to the use of the storage subsystem in a standalone server configuration. For detailed coverage of those issues, refer to the Compaq StorageWorks documentation provided with the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 198
Troubleshooting 7-11 To fix this problem, first do the following: 1. From the Event Viewer, locate the event log entry marked CPQKGPSA to find the host adapter device driver errors. 2. Restart the server. 3. If the driver error message persists, reinstall the KGPSA-BC Host Adapter device driver. If - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 199
7-12 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Verifying Connectivity to locations across all cluster nodes. I For each host adapter pair installed in the server, verify that the upper (or leftmost) host adapter is connected by Fibre Channel - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 200
Troubleshooting 7-13 The topology for Port 1 on each array controller should be: I Fabric Up state (not the Standby state) I FABRIC Server Node Connectivity to Fibre Channel Fabric Connection Paths To verify that your servers are connected to all Fibre Channel Fabric paths, open a CLI window and - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 201
7-14 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I For the array Loop Up state (not the Standby state) I LOOP_HARD Server Node Connectivity to FC-AL Connection Paths To verify that your servers are connected to all FC-AL paths, open a CLI - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 202
Troubleshooting 7-15 A Cluster Node Cannot Connect to the Shared Drives Verify connections and accessibility between the servers storagesets, as will Disk Management. To correct this problem, do the following: I Ensure that the Not Found by KGPSA-BC Device Driver All of the I/O path devices (storagesets) - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 203
Guide To determine whether or not the storagesets can be viewed by a KGPSA-BC device driver, do the following: 1. From the Device Manager, select SCSI Adapters. 2. View the list of installed Hard Drives and manually Storage Subsystem, then Windows 2000 Advanced Server will have 4 I/O paths to each - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 204
Troubleshooting 7-17 I If the array controllers have already been set to Multibus_Failover mode, then check the integrity of the components along the unseen Fibre Channel Fabric path. Do the following: G Confirm that the Compaq KGPSA device driver was properly installed. Refer to "Installing Secure - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 205
The following rules and guidelines apply to the operation of Secure Path software in a PDC/O5000 cluster running Windows 2000 Advanced Server: I If you install the optional Compaq StorageWorks Large LUN utility on your PDC/O5000 cluster, you can distribute a maximum of 256 LUNs among the four array - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 206
Troubleshooting 7-19 G You controller ports. I There is no support for Windows 2000 dynamic disks. I Windows NT fault-tolerant driver (for the host-based RAIDsets) device volume labels with the Disk Management, you must first manually stop the Secure Path Agent. After the modifications are competed - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 207
7-20 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Secure Path Manager Shows Reversed Locations for Top and Bottom Array Controllers , the Secure Path Manager screen is refreshed every 60 seconds. To perform a manual refresh, press the F5 key. - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 208
Troubleshooting 7-21 The Addition of New LUNs Causes an Error In a PDC/O5000 cluster running Windows 2000 Advanced Server, the addition of from automatically starting. If this occurs, do the following: 1. From the Services screen, right click on Secure Path Agent. 2. Select Properties, then select - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 209
A Appendix Diagnosing and Resolving Shared Disk Problems Introduction When one or more nodes in a Compaq Parallel Database Cluster Model PDC/O5000 (referred to here as the PDC/O5000) cannot properly join the cluster, you need to consider two possible causes: I A problem related to the shared disk - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 210
OK and Firmware Checks Problem Persists Perform Cluster Hardware Checks OK Problem Persists Rerun and Validate Object Link OK Manager On All Affected Nodes Problem Persists Exit. Problem Fixed or No Further Disk Problems Contact Your Compaq Support Representative Figure A-1. Tasks for - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 211
Shared Disk Problems A-3 order in which disk drives are brought online changes, Oracle8i Server can find the correct data files and can properly start the Affected Nodes in the Cluster" in this appendix for further instructions. For instructions on installing and using Object Link Manager, see the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 212
Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Restart All Affected Nodes in the Cluster If Object Manager service is properly installed on the node. For instructions on installing and using Object Link Manager, refer to this readme file contained on the Compaq Parallel - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 213
Shared Disk Problems A-5 Run for Secure Path Manager is correct. Refer to the Compaq Parallel Database Cluster Model PDC/O5000 Certification Matrix for Windows Subsystem" in Chapter 5, "Installation and Configuration," for installation instructions. 4. Repeat steps 1 through 3 for all nodes in the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 214
Guide Run and Validate the StorageWorks Command Console From All Storage Subsystems The Compaq Compaq SmartStart and Support Software CD to perform these checks on the cluster software and firmware: 1. Verify that all necessary drivers have been installed. For example, verify that the correct driver - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 215
Event Viewer to verify that no hardware errors or transport problems exist. Check the event log for disk I/O error messages or indications of problems with the communications transport. 2. Determine whether or not the Compaq servers support the physical SCSI disks that are installed in the MA8000 - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 216
Guide Contact Your Compaq Support Representative If you are unable to resolve the problem, contact your Compaq support representative for assistance. In North America, call the Compaq Technical Support Phone Center at 1-800-OK-COMPAQ. Outside North America, call the nearest Compaq Technical Support - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 217
100 percent being the best possible rating. A local area network dedicated to servicing requests between network client machines and between the cluster of software and hardware technologies that enables a set of loosely coupled servers and storage to present a single system image to clients and to - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 218
area network (SAN) topology that uses Fibre Channel SAN Switches to connect cluster nodes with shared storage resources. Installs the HSZDISK driver and other shared storage subsystem device drivers identify a computer system that can continuously deliver services to its clients 99.9 percent of the - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 219
Compaq Parallel Database Cluster interconnect is dedicated to servicing be statically or dynamically assigned. Local Area Network. The Compaq StorageWorks Large LUN utility allows PDC/O5000 the flow of information over the network. An individual server in a cluster. New Technology File System. A file - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 220
Guide Operating System Dependent (OSD) layer Oracle8i Parallel Server Option POST Raw Partition RAID Reliability Resource Scalability Secure Path Service is responsible for enabling parallel database support. An additional license is required for Oracle8i Parallel Server. Power-On Self-Test. A set - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 221
subsystem for the PDC/O5000. A group of disk drives configured with RAID techniques to operate as a single container and ensure that every partitioned and manage MA8000/EMA12000 Storage Subsystem resources, including hardware RAID implementation, disk mirroring, and disk partitioning. A complete - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 222
failover software for Oracle8i 3-7 arbitrated loop physical addresses 5-23 ASR timer value 5-4 availability definition 1-4 features of Compaq ProLiant Servers 2-3 C cabling client LAN 5-9 Ethernet cluster interconnect for Oracle8i 5-8 capacity planning client LAN 4-6 cluster interconnect components - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 223
down cleanly 6-2 supported servers 2-2 cluster performance RAID planning 4-15 shared disk drives 4-20 site planning 4-2 cluster software for Oracle8i Compaq software overview 3-2 Oracle software 3-6 overview 3-1 cluster troubleshooting basic tips 7-2 client to cluster connectivity problems - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 224
operating system dependent modules for Oracle8i overview 3-5 Compaq Options ROMPaq, overview 3-4 Compaq Parallel Database Cluster, overview 1-5 Compaq SmartStart and Support software 3-2 Compaq SmartStart and Support Software CD 3-2 Compaq software Secure Path for Windows 2000 3-4 StorageWorks - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 225
Guide Compaq System Configuration Utility 3-3 Compaq StorageWorks Command Console 3-4 Compaq StorageWorks Fibre Channel Storage Hubs description 2-6 Compaq installation 6-13 device driver initialization verifying 7-10 disk running to check for shared disk problems A-5 distributed lock manager 2-39 - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 226
for Oracle8 5-6 installing for Oracle8i 5-4 grouping physical disk storage space 4-19 H HAL 7-10 hardware installation verification for Oracle8i 5-55 help additional sources xviii technical support telephone numbers xviii host adapters description 2-6 host server verifying connectivity to FC-AL 7-14 - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 227
for Oracle8i and Windows 2000 Administrator Guide verifying connectivity to Fibre Channel 5-29 creating partitions 5-38 designating a server as a maintenance terminal 5-21 discovering shared a disk enclosure 5-10 installing host adapter device drivers 5-33 installing SWCC 5-22 KGPSA-BC and KGPSA - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 228
drivers 5-35 Secure Path software 5-32 SmartStart Compaq Insight Manager 3-3 K KGPSA Host Adapters description 2-6 replacing 6-16 KGPSA-BC and KGPSA-CB Host adapter installation for Oracle8i 5-4 KGPSA-BC Host adapter device driver 19 local area network (LAN Oracle8i Parallel Server database resources - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 229
disk problems A-3 operating system dependent modules See Compaq operating system dependent modules operating system software installation for Oracle8i 5-17 versions supported 3-2 Oracle software certification for Oracle8i 3-7 configuring for Oracle8i 5-53 installation troubleshooting for Oracle8i - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 230
for redundant Fibre Channel Fabrics 4-3 site planning 4-2 supported RAID levels 4-16 port topology 5-23 power distribution 5-56 ProLiant servers 5-57 storage subsystems 5-57 power distribution units 5-56 power sequencing 5-58 ProLiant servers ASR timer value 5-4 high-availability features 2-3 power - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 231
33 troubleshooting 7-18 Secure Path Agent installation for Oracle8i 5-35 Secure Path Manager installation vs operation 5-33 running to check for shared disk problems A-5 Service Packs supported 5-17 Service Packs for Windows 2000 Advanced Server versions supported 3-2 shared disk problems contacting - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 232
site planning 4-2 SmartStart See Compaq SmartStart and Support Software installation for Oracle8i 5-17 SmartStart and Support Software CD 5-17 5-40 technical support xix telephone numbers xx text conventions xvi troubleshooting basic tips 7-2 client to cluster connectivity problems 7-9 cluster cannot - Compaq ProLiant 6500 | Compaq Parallel Database Cluster Model PDC/05000 for Orac - Page 233
potential problems 7-1 unable to start database for Oracle8i 7-7 verifying connectivity to FC-AL paths 7-13 verifying connectivity to Fibre Channel Fabric paths 7-12 verifying KGPSA-BC driver initialization 7-10 verifying node connectivity to Fibre Channel Fabric paths 7-13 troubleshooting verifying
Parallel Database Cluster Model
PDC/O5000 for Oracle8
i
and
Windows 2000
Administrator Guide
Second Edition (June 2001)
Part Number 225081-002
Compaq Computer Corporation