Compaq ProLiant 6400R Compaq Parallel Database Cluster Model PDC/05000 for Ora
Compaq ProLiant 6400R Manual
View all Compaq ProLiant 6400R manuals
Add to My Manuals
Save this manual to your list of manuals |
Compaq ProLiant 6400R manual content summary:
- Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 1
Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Second Edition (June 2001) Part Number 225081-002 Compaq Computer Corporation - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 2
of Microsoft Corporation in the United States and other countries. All other product names mentioned herein may be trademarks of their respective companies. Compaq shall not be liable for technical or editorial errors or omissions contained herein. The information in this document is provided "as is - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 3
About This Guide Purpose ...xiii Audience...xiii Scope ...xiv Referenced Manuals xv Supplemental Documents xvi Text Conventions...xvi Symbols in Text...xvii Symbols on Equipment xvii Rack Stability ...xviii Getting Help ...xviii Compaq Technical Support xix Compaq Website xix Compaq Authorized - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 4
iv Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Storage Hub ...2-6 KGPSA-BC and KGPSA- 2000 Advanced Server 3-2 Compaq Software...3-2 Compaq SmartStart and Support Software 3-2 Compaq System Configuration Utility 3-3 Compaq Insight Manager 3-3 - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 5
Capacity Planning for Cluster Hardware 4-2 Compaq ProLiant Servers 4-2 Planning Shared Storage Planning for the MA8000/EMA12000 Storage Subsystem 4-15 Supported RAID Levels 4-16 Raw Data Storage and Database Size Server Host Files for an Ethernet Cluster Interconnect ...4-21 Client LAN ...4-22 - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 6
vi Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Chapter 5 Installation and Configuration Reference Materials for Installation 5-1 Installation Overview 5-2 Installing the Hardware 5-3 Setting Up the Nodes 5-3 Installing the KGPSA-BC and KGPSA - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 7
5-57 Power Sequencing 5-58 Chapter 6 Cluster Management Cluster Management Concepts 6-2 Powering Off a Node Without Interrupting Cluster Services 6-2 Managing a Cluster in a Degraded Condition 6-2 Managing Network Clients Connected to a Cluster 6-3 Cluster Events 6-3 Management Applications - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 8
Oracle8i and Windows 2000 Administrator Guide Chapter 7 Troubleshooting Basic Troubleshooting Tips 7-2 Power ...7-2 Physical Connections 7-2 Access to Cluster Components 7-3 Software Revisions 7-3 Firmware Revisions 7-4 Troubleshooting Oracle8i and OSD Installation Problems and Error Messages - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 9
All Nodes A-5 Run and Validate the StorageWorks Command Console From All Storage Subsystems ...A-6 Perform Cluster Software and Firmware Checks A-6 Perform Cluster Hardware Checks A-7 Contact Your Compaq Support Representative A-8 Glossary Index - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 10
x Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide List of Figures Figure 1-1. Example of a two-node PDC/O5000 cluster 1-3 Figure 2-1. SCSI bus numbers for I/O modules in the controller enclosure ......... 2-10 Figure 2-2. UltraSCSI cabling - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 11
FC-AL 5-16 Figure 5-8. Server power distribution in a three-node cluster 5-57 Figure A-1. Tasks for diagnosing and resolving shared storage problems A-2 List of Tables Table 2-1 High-Availability Components of ProLiant Servers 2-3 Table 2-2 SCSI bus address ID assignments for the MA8000/EMA12000 - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 12
About This Guide Purpose This administrator guide provides information about the planning, installation, configuration, implementation, management, and troubleshooting of the Compaq Parallel Database Cluster Model PDC/O5000 on Oracle8i software running on the Microsoft Windows 2000 Advanced Server - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 13
the PDC/O5000. I Chapter 7, "Troubleshooting," contains troubleshooting information for the PDC/O5000. I Appendix A, "Diagnosing and Resolving Shared Disk Problems," describes procedures to diagnose and resolve shared disk problems. I Glossary contains definitions of terms used in this guide. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 14
software components of the Compaq Parallel Database Cluster. These related manuals include, but are not limited to: I Documentation related to the ProLiant servers you are clustering (for example, guides, posters, and performance and tuning guides) I Compaq StorageWorks documentation provided with - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 15
Guide Supplemental Documents The following technical documents contain important supplemental information for the Compaq Parallel Database Cluster Model PDC/O5000: I Compaq typeface and in uppercase. FILENAMES File names appear in uppercase italics. are instructed to type information, type the information - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 16
in the text of this guide. They have the following meanings in this manner presents clarifying information or specific instructions. NOTE: Text set off in this manner presents of electrical shock hazards. Enclosed area contains no operator serviceable parts. WARNING: To reduce the risk of injury from - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 17
xviii Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Any surface or area of the any reason. Getting Help If you have a problem and have exhausted the information in this guide, you can get further information and other help in the following - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 18
for as long you own the product. A technical support specialist will help you diagnose the problem or guide you to the next step in the warranty process. In North America, call the Compaq Technical Phone Support Center at 1-800-OK-COMPAQ1. This service is available 24 hours a day, 7 days a week - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 19
PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Compaq Authorized Reseller For the name of your nearest Compaq Authorized Reseller: I In the United States, call 1-800-345-1518. I In Canada, call 1-800-263-5868. I Elsewhere, see the Compaq website for locations and telephone numbers. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 20
software, thereby providing key clustering features at a lower price than proprietary clustering systems. Before examining the features and benefits of the Compaq Parallel Database Cluster Model PDC/O5000 (referred to here as the PDC/O5000), it is helpful to understand the concepts and terminology - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 21
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide supports. Figure 1-1 shows an example of a PDC/O5000 that includes two nodes (Compaq ProLiant™ servers), two sets of storage subsystems, two Compaq StorageWorksTM Fibre Channel SAN Switches or Compaq - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 22
P1 P2 A P1 P2 B P1 P2 Storage Subsystem #1 Storage Subsystem #2 Figure 1-1. Example of a two-node PDC/O5000 cluster The PDC/O5000 can support redundant Fibre Channel Fabric Storage Area Network (SAN) and redundant Fibre Channel Arbitrated Loop (FC-AL) SAN topologies. In the example shown in - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 23
for Oracle8i and Windows 2000 Administrator Guide Availability When computer systems experience outages, the amount of time the system is unavailable is referred to as downtime. Downtime has several primary causes: hardware faults, software faults, planned service, operator error, and environmental - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 24
Database Cluster Overview As traditional clustering technology has moved into the open systems of client/server computing, Compaq has provided innovative, customer-focused solutions. The PDC/O5000 moves client/server computing one step closer to the capabilities found in expensive, proprietary - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 25
StorageWorks Enterprise Modular Array 12000 Fibre Channel Storage Subsystems (MA8000/EMA12000 Storage Subsystems) G Compaq StorageWorks HSG80 Array Controllers (HSG80 Array Controllers) G Compaq StorageWorks Fibre Channel SAN Switches (Fibre Channel SAN Switches) for redundant Fibre Channel Fabrics - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 26
Guide G Gigabit Interface Converter-Shortwave (GBIC-SW) modules G Fibre Channel cables I Cluster interconnect components G Ethernet NIC adapters G Ethernet cables G Ethernet switches/hubs The software products include: I Microsoft Windows 2000 Advanced Server with Service Pack 1 or later I Compaq - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 27
cooling fans, and Error Checking and Correcting (ECC) memory. The high-availability features of ProLiant servers are a critical foundation of Compaq clustering products. Table 2-1 lists the high-availability features found in many ProLiant servers. Table 2-1 High-Availability Components of ProLiant - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 28
Guide MA8000/EMA12000 Storage Subsystem The MA8000/EMA12000 Storage Subsystem is the shared storage solution for the PDC/O5000. Each storage subsystem consists of one controller enclosure and up to six disk enclosures. For detailed information about storage subsystem components, refer to the Compaq - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 29
Fibre Channel SAN Switches, refer to the latest Compaq StorageWorks documentation. This guide does not document cascaded configurations for the Fibre 8-port Fibre Channel SAN Switch and 16-port Fibre Channel SAN Switch are supported. From two to four Fibre Channel SAN Switches can be used in each - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 30
Guide For further information, refer to these manuals provided with each Fibre Channel SAN Switch: I Compaq StorageWorks Fibre Channel SAN Switch 8 Installation and Hardware Guide I Compaq more Storage Hubs provides fault tolerance and supports the redundant architecture implemented by the PDC/O5000 - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 31
you can distribute a maximum of 16 LUNs among the four array controller ports in each storage subsystem. This 16-LUN restriction is imposed by the Compaq Secure Path software and the MA8000/EMA12000 Storage Subsystems. IMPORTANT: When offsets are used, always assign LUNs 0-99 to port 1 on each HSG80 - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 32
2-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide If you do not install the optional Large LUN utility in you cluster, there are two accepted methods for distributing LUNs for storagesets: I You can - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 33
documentation provided with the storage subsystem. UltraSCSI Cables Compaq recommends using the shortest cable length possible to connect disk enclosures to the controller enclosure. The maximum supported cable length is 2 meters (6.6 ft). Refer to the documentation provided with your storage - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 34
2-10 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide SCSI Bus 1 SCSI Bus 2 SCSI Bus 3 SCSI Bus subsystem, use one dual-bus I/O module in each disk enclosure to support up to 14 disk drives in each disk enclosure. Using single-bus I/O - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 35
Cluster Architecture 2-11 Dual-Bus I/O Module Disk Enclosure Disk Enclosure Disk Enclosure I/O Modules (6) A B Controller Enclosure Figure 2-2. UltraSCSI cabling between a controller enclosure and three disk enclosures Using Single-Bus I/O Modules in the Disk Enclosures Figure 2-3 shows UltraSCSI - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 36
2-12 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Single-Bus I/O Module Single-Bus I/O Module Disk Enclosure Disk Enclosure Disk Enclosure Disk Enclosure Disk Enclosure Disk Enclosure I/O Modules (6) A B Controller Enclosure Figure 2-3. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 37
the Fibre Channel Arbitrated Loop (FC-AL) topology. The PDC/O5000 supports both topologies. A redundant Fibre Channel Fabric is two to four Fibre Fibre Channel SAN Switches, refer to the latest Compaq StorageWorks documentation. This guide does not document cascaded configurations for the Fibre - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 38
2-14 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide A redundant Fibre Channel Fabric consists Fabric Used in conjunction with the I/O path failover capabilities of Compaq Secure Path software, this redundant Fibre Channel Fabric configuration gives - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 39
of redundant Fibre Channel Fabrics you can install in a PDC/O5000 is restricted by the number of host adapters your Compaq servers support. Refer to the Compaq server documentation for this information. Figure 2-5 shows a two-node PDC/O5000 with two redundant Fibre Channel Fabrics. In this example - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 40
2-16 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Storage Subsystem #1 A P1 P2 B P1 P2 Redundant Fibre Channel Fabric #1 Fibre Channel SAN Switch #1 Host Adapters (4) Node 1 Fibre Channel SAN Switch #1 Storage Subsystem #2 A P1 P2 B - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 41
Cluster Architecture 2-17 Figure 2-6 illustrates these maximum cable distances for a redundant Fibre Channel Fabric. Host Adapters (2) 500 meter maximum Host Adapters (2) Node 1 Node 2 Fibre Channel SAN Switch # 1 500 meter maximum Fibre Channel SAN Switch # 2 A P1 P2 B P1 P2 A P1 P2 B - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 42
2-18 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Host Adapter-to-Fibre Channel SAN Switch I/O Data Paths Figure 2-7 highlights the I/O data paths that run between the host adapters and the Fibre Channel SAN - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 43
Cluster Architecture 2-19 Fibre Channel SAN Switch-to-Array Controller I/O Data Paths Figure 2-8 highlights the I/O data paths that run between the Fibre Channel SAN Switches and the two dual-port array controllers in each MA8000/EMA12000 Storage Subsystem. There is one I/O connection path to and - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 44
2-20 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I/O Path Definitions for Redundant Fibre Channel Fabrics In a redundant Fibre Channel Fabric, devices are accessed by the operating system using conventional SCSI addressing terminology. An I/O - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 45
Cluster Architecture 2-21 I/O Path Configuration Examples for Redundant Fibre Channel Fabrics Every I/O path connection between host adapters and array controller ports is active at the same time. An active port does not become inactive unless it fails, whereupon all I/O activity on the failed path - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 46
2-22 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Host Adapters (2) Node 1 Fibre Channel SAN Switch # 1 Host Adapters (2) Node 2 Fibre Channel SAN Switch # 2 A P1 P2 B P1 P2 Storage Subsystem #1 A P1 P2 B P1 P2 Storage - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 47
Cluster Architecture 2-23 Figure 2-10 shows an acceptable method for cabling the I/O path components in a quad redundancy configuration. In this example, the following connections are made: I Each of four host adapters in each node connects to the same-numbered Fibre Channel SAN Switch (host adapter - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 48
2-24 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Host Adapters (4) Node 1 Node 2 Node 3 Node 4 Fibre Channel SAN Switch #1 Fibre Channel SAN Switch #3 A P1 P2 B P1 P2 Fibre Channel SAN Switch #2 Fibre Channel SAN - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 49
Cluster Architecture 2-25 Summary of I/O Path Failure and Failover Scenarios for Redundant Fibre Channel Fabrics Table 2-3 identifies possible I/O path failure events for both dual redundancy or quad redundancy configurations in a redundant Fibre Channel Fabric and the failover response implemented - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 50
2-26 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I/O Path Configurations for Fabric or the Fibre Channel Arbitrated Loop (FC-AL) topology. The PDC/O5000 supports both topologies. Storage Hubs are used to create redundant FC-ALs with a total - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 51
Cluster Architecture 2-27 Host Adapters (2) Node 1 Storage Hub #1 Host Adapters (2) Node 2 Storage Hub #2 A P1 P2 B P1 P2 Storage Subsystem #1 A P1 P2 B P1 P2 Storage Subsystem #2 Figure 2-11. Two-node PDC/O5000 with a four-loop redundant Fibre Channel Arbitrated Loop Used in - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 52
2-28 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Multiple Redundant Fibre Channel Arbitrated Loops The PDC/O5000 supports the use of multiple redundant FC-ALs within the same cluster. You would install additional redundant FC-ALs in a PDC/O5000 - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 53
Cluster Architecture 2-29 Figure 2-12 shows a two-node PDC/O5000 that contains two redundant FC-ALs. In this example, each redundant FC-AL has its own pair of host adapters in each node, a pair of Storage Hubs, and two MA8000/EMA12000 Storage Subsystems. In Figure 2-12, the hardware components that - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 54
2-30 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Maximum Distances Between Nodes and Shared Storage Subsystem Components By using standard short-wave Fibre Channel cables with Gigabit Interface Converter-Shortwave (GBIC-SW) modules, - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 55
Cluster Architecture 2-31 Figure 2-13 illustrates these maximum cable distances for a redundant FC-AL. Host Adapters (2) 500 meter maximum Host Adapters (2) Node 1 Node 2 Storage Hub #1 Storage Hub #2 500 meter maximum A P1 P2 B P1 P2 A P1 P2 B P1 P2 Storage Subsystem #1 Storage - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 56
2-32 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I/O Data Paths in a Redundant FC-AL Within a redundant FC-AL, an I/O path connection exists between each host adapter and all four array controller ports in - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 57
Cluster Architecture 2-33 Secure Path monitors the status of the components along each active path. If Secure Path detects the failure of a host adapter, Fibre Channel cable, or Storage Hub along an active path, it automatically transfers all I/O activity on that path to the defined backup path. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 58
2-34 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide If any component along an active path fails, Secure Path detects the failure and automatically transfers all I/O activity to the defined backup path. I/O Path Definitions - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 59
Cluster Architecture 2-35 While cluster nodes use the unit number (Dxxyy) to identify and access a storageset or a single disk drive unit in the shared storage subsystems, the array controllers use a Port-Target-LUN (PTL) address to identify and access these resources in their storage subsystem. For - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 60
2-36 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Host Adapters (2) Node 1 Storage Hub #1 Host Adapters (2) Node 2 Storage Hub #2 A P1 P2 B P1 P2 A P1 P2 B P1 P2 Storage Subsystem #1 Storage Subsystem #2 Figure 2-16. Dual - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 61
Cluster Architecture 2-37 I The two odd-numbered Storage Hubs connect to port 1 of the array controllers. Storage Hub #1 connects to array controller A, port 1 in each storage subsystem. Storage Hub #3 connects to array controller B, port 1 in each storage subsystem. I The two even-numbered Storage - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 62
2-38 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Summary of I/O Path Failure and Failover Scenarios for Redundant FC-ALs Table 2-4 identifies possible I/O path failure events for both dual redundancy and quad configurations for a - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 63
, private interconnect, system area network (SAN), and private network. Throughout this guide, the term cluster interconnect is used. Compaq Parallel Database Clusters like the PDC/O5000 that use Oracle8i Parallel Server support an Ethernet cluster interconnect. In a PDC/O5000, the Ethernet cluster - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 64
2-40 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Ethernet Cluster Interconnect Components The following components are used in a redundant Ethernet cluster interconnect: I Two Ethernet adapters in each cluster node I Ethernet cables and switches - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 65
Cluster Architecture 2-41 Ethernet Cluster Interconnect Diagrams Figure 2-18 shows the redundant Ethernet cluster interconnect components used in a two-node PDC/O5000 cluster. Ethernet Switch/Hub #2 for Cluster Interconnect Dual-port Ethernet Adapters (2) Ethernet Switch/Hub #1 for Cluster - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 66
2-42 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Local Area Network NOTE: For the PDC/O5000, the client LAN and the cluster interconnect must be treated as separate networks. Do not use either - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 67
(OSDs) and this Oracle software: I Oracle8i Enterprise Edition with the Oracle8i Parallel Server Option NOTE: For information about currently-supported software revisions for the PDC/O5000, refer to the Compaq Parallel Database Cluster Model PDC/O5000 Certification Matrix for Windows 2000 at www - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 68
3-2 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Microsoft Windows 2000 Advanced Server This version of the PDC/O5000 supports Microsoft Windows 2000 Advanced Server with Service Pack 1 or later. NOTE: The PDC/O5000 does not work in conjunction - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 69
System Configuration Utility The SmartStart and Support Software CD also contains the Compaq System Configuration Utility. This utility is the primary means to configure hardware devices within your servers, such as I/O addresses, boot order of disk controllers, and - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 70
Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Compaq Options ROMPaq The Compaq Options ROMPaq diskettes allow a user to upgrade the ROM Firmware images for Compaq System product options, such as array controllers, disk drives, and tape drives used for - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 71
per target (array controller port). Compaq Operating System Dependent Modules Compaq supplies low-level services, called operating system dependent modules (OSDs Setup and Configuration Guide provided with the Oracle8i software. The OSD software is found on the Compaq Parallel Database Cluster - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 72
3-6 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Oracle8i Server Enterprise Edition The Oracle8i Server Enterprise Edition provides the following: I Oracle8i Server I Oracle8i Parallel Server Option I Oracle8i Enterprise Manager Oracle8i Server - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 73
Parallel Server must demonstrate the successful completion of the Oracle8i Parallel Server for Windows 2000 Certification. Neither Oracle nor Compaq will support any implementation of Oracle8i Parallel Server that does not strictly conform to the configurations certified with this process. For - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 74
3-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide To implement this application and connection failover, a software interface between the Oracle software and the client must be written. Such a software interface would be responsible - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 75
4 Chapter Cluster Planning Before connecting any cables or powering on any hardware on your Compaq Parallel Database Cluster Model PDC/O5000 (referred to here as the PDC/O5000), it is important that you understand how all the various cluster components - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 76
4-2 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Site Planning You must carefully select and and qualified personnel I Space for unpacking, installing, and servicing the computer equipment I Sufficient floor strength for the computer equipment - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 77
apply to the server models and server configurations that are supported by the Compaq Parallel Database Cluster. For a current list of PDC/O5000certified servers and details on supported configurations, refer to the Compaq Parallel Database Cluster Model PDC/O5000 Certification Matrix for Windows - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 78
4-4 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide To determine which Fibre Channel components: I Two or more host adapters in each node. I Two or more Compaq StorageWorks Fibre Channel Storage Hubs (Storage Hubs), each of which connects all of the - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 79
Cluster Planning 4-5 I MA8000/EMA12000 Storage Subsystems, each of which holds up to 72 disk drives. I Two dual-port HSG80 Array Controllers installed in each storage subsystem's controller enclosure. NOTE: For more information about redundant Fibre Channel Arbitrated Loops (FC-ALs) in a PDC/O5000, - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 80
4-6 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Planning Cluster Interconnect and Client LAN cluster interconnects. This document is available at www.compaq.com/support/techpubs/whitepapers Before installing the Ethernet cluster interconnect in - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 81
Cluster Planning 4-7 Planning Cluster Configurations for Redundant Fibre Channel Fabrics A redundant Fibre Channel Fabric is two or more Fibre Channel SAN Switches installed between host adapters in cluster nodes and array controllers in the shared storage subsystems. Each redundant Fibre Channel - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 82
4-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Figure 4-1 shows the correct method for cabling the I/O path components for dual redundancy in a redundant Fibre Channel Fabric. In Figure 4-1, both Fibre Channel SAN Switches - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 83
Cluster Planning 4-9 Planning Quad Redundancy Configurations For the highest possible level of I/O path redundancy, you can configure your cluster for quad redundancy. This configuration provides four host adapters in each of at least four nodes and four Fibre Channel SAN Switches for each redundant - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 84
4-10 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I The two even-numbered Fibre Channel SAN Switches connect to port 2 of the array controllers. Fibre Channel SAN Switch #2 connects to array controller A, port 2 in - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 85
Cluster Planning 4-11 Planning Cluster Configurations for Redundant Fibre Channel Arbitrated Loops A redundant Fibre Channel Arbitrated Loop (FC-AL) is two or more Storage Hubs installed between host adapters in cluster nodes and array controllers in the shared storage subsystems. Each redundant FC- - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 86
4-12 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Figure 4-3 shows the correct method for cabling the I/O path components for dual redundancy in a redundant FC-AL. In Figure 4-3, both Storage Hubs connect to port 1 - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 87
Cluster Planning 4-13 Planning Quad Redundancy Configurations For the highest possible level of I/O path redundancy, you can configure your cluster for quad redundancy. This configuration provides four host adapters in each of at least four nodes and four Storage Hubs for each redundant FC-AL. The - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 88
4-14 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I The two even-numbered Storage Hubs connect to port 2 of the array controllers. Storage Hub #2 connects to array controller A, port 2 in each storage subsystem. Storage - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 89
Cluster Planning 4-15 RAID Planning for the MA8000/EMA12000 Storage Subsystem Storage subsystem performance is one of the most important aspects of tuning database cluster servers for optimal performance. Efforts to plan, configure, and tune a PDC/O5000 cluster should focus on getting the most out - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 90
the RAID sets, use the Compaq StorageWorks Command Console (SWCC) or the Command Line Interface (CLI) window for the HSG80 Array Controllers. NOTE: Do not use the software RAID offered by the operating system to configure your shared storage disks. Supported RAID Levels RAID provides several fault - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 91
Cluster Planning 4-17 Raw Data Storage and Database Size Raw data storage is the amount of storage available before any RAID levels have been configured. It is called raw storage because RAID volumes require some overhead. The maximum size of a database stored in a RAID system will always be less - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 92
4-18 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Selecting the Appropriate RAID Level Many factors will affect the RAID levels you select for your cluster database. These include the specific availability, performance, reliability, - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 93
Cluster Planning 4-19 Planning the Grouping of Physical Disk Storage Space Figure 4-5 shows an example of how the physical storage space for one 12-drive array in an MA8000/EMA12000 Storage Subsystem disk enclosure could be grouped for an Oracle8i Parallel Server database. Shared Disk Storage Array - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 94
Guide logical partitions. For detailed information about these procedures, refer to "Configuring a Storageset" and "Creating Partitions" in Chapter 5, "Installation and Configuration." You use either the CLI window for the HSG80 Array Controllers or the Compaq Server application files are stored - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 95
that does not support fault tolerance (for example, RAID 0), then the failure of the shared disk drives in that logical drive will disrupt service to all Oracle nodes, the Compaq operating system dependent modules (OSDs) require a unique entry in the hosts and lmhosts files located at %SystemRoot - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 96
4-22 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide The following list identifies the format of the hosts and lmhosts files for a four-node PDC/O5000 cluster with an Ethernet cluster interconnect: IP address node1 IP address node1_san IP address - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 97
interconnect and client LAN interconnect adapters I Compaq SmartStart Installation Poster I Compaq SmartStart and Support Software CD I Microsoft Windows 2000 Advanced Server Administrator's Guide I Microsoft Windows 2000 Advanced Server CD with Service Pack 1 or later I StorageWorks documentation - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 98
O5000 for Oracle8i and Windows 2000 Administrator Guide I StorageWorks Secure Path Software CD I Compaq Parallel Database Cluster Clustering Software for including: G SmartStart 4.9 or later G Windows 2000 Advanced Server with Service Pack 1 or later I Setting up and configuring an MA8000/EMA12000 - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 99
StorageWorks Large LUN utility G Specifying preferred_path units G Creating partitions I Installing and configuring the Compaq operating system dependent modules (OSDs), including: G Using Oracle Universal Installer to install OSDs for an Ethernet cluster interconnect I Installing and configuring - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 100
supported by the Compaq Parallel Database Cluster. For a current list of PDC-certified servers and details on supported configurations, refer to the Compaq follow the installation instructions in your Compaq ProLiant Server Setup and Installation Guide and in your Compaq ProLiant Server Installation - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 101
5-1. Connecting host adapters to Fibre Channel SAN Switches or Storage Hubs For more information about Fibre Channel SAN Switches and Storage Hubs, refer to the Compaq documentation provided with them. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 102
5-6 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Installing the Cluster you need specific instructions on how to install an adapter, refer to the documentation for the adapter you are installing or refer to the user guide of the ProLiant server - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 103
Installation and Configuration 5-7 To install GBIC-SW modules: 1. Verify that a GBIC-SW module is installed into both ports on each array controller. 2. Insert a GBIC-SW module into the correct port on each Storage Hub or Fibre Channel SAN Switch (two modules for each array controller). 3. Repeat - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 104
5-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Figure 5-2 shows the redundant Ethernet cluster from one Ethernet path to another, see Supported Ethernet Interconnects for Compaq Parallel Database Clusters Using Oracle Parallel Server at - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 105
any TCP/IP network to connect to a client LAN. The following procedure contains instructions for cabling an Ethernet client LAN. To cable an Ethernet client LAN: 1. one Ethernet path to another, see Supported Ethernet Interconnects for Compaq Parallel Database Clusters Using Oracle Parallel Server at - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 106
5-10 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Installing Hardware Into an MA8000/EMA12000 six SCSI buses to up to six disk enclosures. For detailed instructions on installing hardware components in a controller enclosure, refer to the - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 107
and Configuration 5-11 Cabling the Controller Enclosure to Disk Enclosures Compaq recommends using the shortest UltraSCSI cable length possible to connect disk enclosures to the controller enclosure. The maximum supported cable length is 2 meters (6.6 ft). Refer to the documentation provided - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 108
5-12 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Figure 5-4 shows UltraSCSI cables installed between a controller enclosure and three disk enclosures. When three or fewer disk enclosures are present in a storage subsystem, you can - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 109
EMUs to Each Other For information about connecting the EMUs between enclosures in an MA8000/EMA12000 Storage Subsystem and between different subsystems, refer to the Compaq documentation provided with the storage subsystem. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 110
5-14 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Cabling Array Controllers to Fibre Channel SAN Switches In a PDC/O5000 with a redundant Fibre Channel Fabric, a dual redundancy configuration is the minimum allowable configuration that - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 111
Installation and Configuration 5-15 Host Adapters (2) Node 1 Fibre Channel SAN Switch # 1 Host Adapters (2) Node 2 Fibre Channel SAN Switch # 2 A P1 P2 B P1 P2 Storage Subsystem #1 A P1 P2 B P1 P2 Storage Subsystem #2 Figure 5-6. Dual redundancy configuration for a redundant Fibre - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 112
5-16 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Storage Hub still has access Installing Operating System Software You will follow an automated procedure using Compaq SmartStart to install the operating system software on the cluster nodes for - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 113
addresses and node names for each cluster node (server) in the hosts and lmhosts files located at %SystemRoot%\system32\drivers\etc. G One IP address and node name is Oracle8i: I SmartStart and Support Software CD 4.9 or later I Microsoft Windows 2000 Advanced Server with Service Pack 1 or later I - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 114
5-18 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Cluster-Specific SmartStart hubs or switches 2. Power up a cluster node and put the SmartStart and Support Software CD into the CD-ROM drive. 3. Select the Assisted Integration installation path - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 115
IP addresses and node names for each node in the hosts and lmhosts files located at %SystemRoot%\system32\drivers\etc. Record this information: G For TCP/IP Properties dialog box for the client LAN port and you are using Service Pack 1 or later: a. Double-click the Network and Dial-up Connections - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 116
5-20 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide e. With the client LAN NIC For more information on creating partitions, see the Oracle8i Parallel Server Setup and Configuration Guide. 13. If you are installing a node other than node1, open Disk - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 117
Installation and Configuration 5-21 Setting up and Configuring an MA8000/EMA12000 Storage Subsystem Designating a Server as a Maintenance Terminal To set up and configure an MA8000/EMA12000 Storage Subsystem in a PDC/O5000, you must connect a cluster server (node) to an HSG80 Array Controller in - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 118
5-22 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 2. Connect each storage subsystem's controller Client. Double-click on Setup. 5. Follow the setup wizard instructions to complete the installation. 6. Restart the server. Configuring a Storage - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 119
controllers, perform these steps: NOTE: Refer to the Compaq StorageWorks documentation for more detailed information about changing the port label.) NOTE: The "other" controller will shut down and must be manually restarted by momentarily pressing the reset button on the controller's front panel. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 120
5-24 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide d. Verify that both controllers are in multibus failover mode by entering the following commands and viewing the results: show this_controller and show other_controller e. Close the - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 121
Installation and Configuration 5-25 set this_controller port_2_topology=loop_hard set other_controller port_2_topology=loop_hard IMPORTANT: The ALPA addresses of port 1 on both array controllers must be identical; the ALPA addresses of port 2 on both array controllers must be identical. Use only the - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 122
5-26 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 17. If "this_controller" is in a redundant Fibre Channel Fabric, verify the following: a. Port_1_topology=fabric b. Port_2_topology=fabric 18. If "this_controller" is in a redundant FC-AL, verify - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 123
discovered are consistent with the installed disk drives. The disk drives will be named in the DISK##### format. Refer to the Compaq StorageWorks documentation for the naming conventions for storage subsystem devices. 6. Close the CLI window. Verifying Array Controller Properties Displaying SCSI - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 124
5-28 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Table 5-1 Controller Properties Description General Host Ports Cache Communications LUN Connections Battery Values Allocation class: 0 SCSI version: SCSI-2 Host Port 1 Requested Topology: Loop - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 125
Add Virtual Disk Wizard. NOTE: A virtual disk is also known as a logical drive. 2. Select a RAID level setting. Click Next for step 2 of the wizard. Supported RAID levels are: RAID1, RAID0+1, RAID3/5, and RAID5. NOTE: If the disks are being used for the first time, a dialog box appears prompting you - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 126
2000 Administrator Guide 4. Click Next for step 3 of the wizard. 5. Select the maximum capacity for the virtual disk. IMPORTANT: Secure Path requires maximum capacity partitions. 6. Click Next for step 4 of the wizard. 7. Enter the virtual disk name. G If the optional Compaq StorageWorks Large - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 127
Installation and Configuration 5-31 12. Click Finish to create the virtual disk. The Storage window displays the virtual disk you created. The hourglass on the disk icon indicates that the storageset is being initialized. The drives you used to create the RAID set are highlighted in the Devices - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 128
5-32 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 11. If not, type the following and for your Secure Path storage environment. For detailed procedures, refer to the Compaq StorageWorks Secure Path for Windows 2000 on RAID Array 4000/4100 Version - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 129
: 1. From the Start menu, select Programs. Open Administrative Tools. 2. Select Computer Management. 3. Double-click on System Tools. 4. Double-click on Display Adapters. 5. Double-click on Compaq LP6000/LP7000/LP8000. A Properties screen appears. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 130
5-34 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 6. Select the Driver tab. Click Update IMPORTANT: It is recommended you run this utility before any New Technology File System (NTFS) volumes are created on the MA8000/EMA12000 Storage Subsystem. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 131
3. Follow the instructions in the setup Raidisk filter driver, RAIDisk driver, and agent: 1. Insert the Compaq StorageWorks Secure Path Software CD into your CD-ROM drive. 2. If configure TCP/IP on your network. They include a) host files on servers and clients and b) DNS, with NetBios using DNS - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 132
5-36 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide To install Secure the four array controller ports. The 16-LUN restriction is imposed by the Compaq Secure Path software and the MA8000/EMA12000 Storage Subsystems. IMPORTANT: When offsets are - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 133
Installation and Configuration 5-37 If you do not install the optional Large LUN utility on your cluster, there are two accepted methods for distributing LUNs for storagesets: I You can distribute eight LUNs between the same-numbered ports on both array controllers so that a total of 16 LUNs are - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 134
and Windows 2000 Administrator Guide 3. Verify that the above changes were made by entering the following command: show units full 4. Repeat steps 1 through 3 for each storage subsystem in the PDC/O5000. You have completed the software configuration required to support your Secure Path environment - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 135
OSDs Use the Oracle Universal Installer (OUI) program to install the Compaq operating system dependent modules (OSDs) for Oracle8i Parallel Server. Compaq supplies an Oracle8i software package with the OUI that installs Compaq OSDs for an Ethernet cluster interconnect. The OUI runs from one node - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 136
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I Sets up registry entries for file not offer the option to install Oracle8i Parallel Server Database software. The Compaq operating system dependent modules (OSDs) are installed from one node in - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 137
Installation and Configuration 5-41 For example, the command ping node1 should return the following type of information. Pinging node1.loc1.yoursite.com [127.0.0.1] with 32 bytes of data: Reply from 127.0.0.1: bytes=32 time - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 138
5-42 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide If the local Administrators group on each node before installing the OSDs. 2. Insert the Compaq Parallel Database Cluster Clustering Software for Oracle8i on Microsoft Windows 2000 CD into the - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 139
Installation and Configuration 5-43 4. Click Next to continue. The License screen appears. 5. Click Accept to accept the terms of the license agreement. The File Locations screen appears. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 140
Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 6. Click Next to continue. The Installation Types screen appears. 7. Select Typical to install the Ethernet OSDs into the default location, C:\Compaq\OPS. Skip to step 12. 8. Select Custom to install the Ethernet - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 141
Installation and Configuration 5-45 9. If you want to accept the default location, skip to step 13. 10. If you want to specify an alternate location and you know the exact directory path, specify it by typing over the default location and skip to step 12. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 142
5-46 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 11. If you want to specify an alternate location but do not know the exact directory path, click Change Location. The Choose Directory screen appears. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 143
Installation and Configuration 5-47 12. Browse to the location where you want to install the OSDs. Click OK. The Component Locations screen appears indicating the alternate location you specified. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 144
5-48 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 13. Click Next to continue. The Cluster Members screen appears. 14. The Cluster Members screen indicates the host name of the installing machine. Define the - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 145
it as a cluster interconnect node name. If necessary, retype the cluster interconnect node names to match the names you entered into the hosts and lmhosts files for the Ethernet cluster interconnect. Do not use special characters. Separate the node names with a space. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 146
5-50 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 17. Click Next to continue. The OSD Summary screen appears. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 147
Installation and Configuration 5-51 18. If the information on the OSD Summary screen is not correct, click Previous to modify the previous screens. When the information is correct, click Next to continue. The Summary screen appears. 19. Click Install to install the components listed in the summary. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 148
on all nodes in the cluster. This service must be started before installing Oracle software. See the Oracle documentation for information on starting the cluster manager. Installing Oracle Software Using the Oracle8i Parallel Server Setup and Configuration Guide, follow the steps to install Oracle8i - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 149
After installing Oracle software, the Oracle manual instructs you to install vendor-supplied operating Server Setup and Configuration Guide and the Oracle8i Enterprise maintains the symbolic links between disk partitions and Oracle data files. Failure to install Object Link Manager can result in - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 150
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide online changes, Oracle8i Server can find the correct data files a database. For instructions on installing and using Object Link Manager, see the readme file on the Compaq Parallel Database Cluster - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 151
and lmhosts files at %SystemRoot service did not start successfully, make sure that the OracleCMService is started and the SID is in the Oracle registry. There are two logs that can provide information about why the Oracle service did not start. These logs are: I \Compaq\OPS\cmsrvr.log I \Compaq - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 152
Guide Other Verification Tasks Perform these other verification tasks: I Verify that Object Link Manager has been installed and is running. I Verify that the OPS_CMDISK symbolic link has actually been set. To verify that the Object Link Manager has been installed and is running, go to the Services - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 153
Installation and Configuration 5-57 Server Power Distribution Figure 5-8 shows an example of server power distribution for a three-node cluster. Power supply #1 of each server is connected to PDU #1. Power supply #2 of each server is connected to PDU #2. PDU #1 is connected to UPS #1, and PDU #2 is - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 154
/O5000 for Oracle8i and Windows 2000 Administrator Guide Power Sequencing The following power sequencing information applies to existing clusters that are up and running. The cluster installation steps presented previously in this chapter might have instructed you to perform a different sequence. Be - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 155
ongoing activities. This chapter describes these management activities for the Compaq Parallel Database Cluster Model PDC/O5000 (referred to here as this chapter contain high-level instructions. The instructions summarize what has either already been discussed in this guide, Oracle documentation, or - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 156
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Cluster Management Concepts Powering Off a Node Without Interrupting Cluster Services degraded condition, it is helpful to follow this troubleshooting process: 1. Determine what caused the degradation. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 157
the affected nodes and wait until a non-peak time to service the problem. c. If the problem is critical and affects all the cluster nodes, shut down the can be viewed either within the Event Log or within specific error log files. Oracle8i Server events are sent to the Application Event Log and can - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 158
PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide The following log files are used: I The Node Manager (NM), provided by Compaq, logs messages to C:\compaq\ops\nm.log I By default, the OracleCMService sends messages to C:\compaq\ops\cmsrvr.log Management Applications Monitoring Server and - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 159
components. Compaq Insight Manager XE is an optional CD available upon request from the Compaq System Management website at www.compaq.com/ the MA8000/EMA12000 Storage Subsystem. The CLI supports dozens of commands to configure, modify, and diagnose problems in the storage subsystem. For example, - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 160
PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Using the Compaq StorageWorks Command Console The StorageWorks Command Console ( of the shared storage system. Secure Path Manager is a component of the Compaq StorageWorks Secure Path software that is included in the PDC/O5000 cluster kit - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 161
Guide I Oracle8i Parallel Server Concepts I Oracle8i Parallel Server Administration, Deployment, and Performance Oracle8i Enterprise Manager and its corresponding documentation are shipped as part of Oracle8i Enterprise Edition. Remotely Managing a Cluster Oracle8i Enterprise Manager, Compaq - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 162
6-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Software Maintenance Deinstalling the OSDs Be sure to shut down the Oracle instance on all nodes and stop the Oracle services running on all nodes prior to removing the OSDs. To deinstall the OSDs: - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 163
Cluster Management 6-9 3. If autorun is enabled, the Oracle Universal Installer Welcome screen appears. 4. If autorun is disabled, enter d:\setup at a C: prompt. Substitute the letter assigned to the CD drive for d. The Oracle Universal Installer Welcome screen appears. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 164
Oracle8i and Windows 2000 Administrator Guide 5. Click Deinstall Products. Next, a window containing an inventory of the installed components appears. The following shows the Inventory screen that is displayed with an Ethernet cluster interconnect. 6. Checkmark Compaq Ethernet OSDs and click Remove - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 165
the operation of the cluster. Before upgrading to any new release of Oracle8i Server, consult your Compaq and Oracle service representatives. Make sure that the new release is certified and supported on the PDC/O5000, and get assistance to determine which procedure you should follow to perform the - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 166
Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 2. Click Deinstall Products. The Inventory screen showing a list of installed components is displayed: 3. Checkmark Compaq Ethernet OSDs and click Remove. The Oracle Universal Installer removes all Ethernet - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 167
installation (by removing it) before you can successfully run the Oracle Universal Installer again. To remove a partial OSD installation: 1. Insert the Compaq Parallel Database Cluster Clustering Software for Oracle8i on Microsoft Windows 2000 CD into the CD-ROM drive of a cluster node. 2. From - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 168
6-14 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Managing Changes to Shared Storage 7. Perform the necessary Oracle commands to associate the new data files to the database. See the Oracle setup and configuration documentation provided with - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 169
Cluster Management 6-15 Expanding a Logical drive Within a Storageset Expanding the size of a logical drive within a storageset is a complex process and should be attempted only by the site database administrator. The major steps in expanding a logical drive within a storageset are: 1. Back up the - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 170
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide delete failedset diskname 3. Follow the instructions in the Compaq StorageWorks documentation to remove the failed make sure the problem is with the host adapter and not with any of the - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 171
4. If the System log still reports a problem, then it is probably caused by the host the Oracle instance on the node. 2. Shut down the Oracle services running on the node. 3. Shut down the server. 4. Adapter and RAID Controllers and verify that the Compaq KGPSA device driver is started. If it is - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 172
6-18 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 10. From CLI, use the Rename command to rename the connections between the configuration seen from other nodes in the cluster. 14. Start the Oracle services and Oracle instance on the cluster node. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 173
instructions to files to the database. Refer to the documentation provided with the Oracle software for detailed guidelines. Replacing a Cluster Node Before replacing a node, verify that the new configuration is supported by the PDC/O5000. To determine if your new server is supported, see the Compaq - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 174
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide file server that can be restored onto the replacement node. 3. Stop the Oracle instance on the node to be removed. 4. Stop all Oracle services instructions you should read through Chapter 5, - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 175
with Service Pack 1 or later. 6. Install necessary Compaq drivers and utilities from the Compaq SmartStart and Support Software CD . 7. Enter unique IP addresses and node names for each node in the hosts and lmhosts files - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 176
6-22 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 11. Using Fibre Channel cables, connect the host adapters in the replacement node to the appropriate Storage Hubs or Fibre Channel SAN Switches. 12. If, - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 177
installed the OSDs, do the following on all original cluster nodes (not the replacement node): a. Shut down the Oracle instance. b. Stop all Oracle services. c. On the node that was originally used to install the OSDs, use the OUI to deinstall the OSDs. d. Restart all nodes to complete the - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 178
6-24 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 2. If the replacement node is the node on which you originally installed the OSDs, do the following on all nodes: a. Shut down the Oracle instance. b. Stop all Oracle services. c. Uninstall the - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 179
outline of the steps to follow. If you need detailed instructions you should read Chapter 5, "Installation and Configuration." Preparing the Service Pack 1 or later (whichever is installed on the other nodes). 6. Install necessary Compaq drivers and utilities from the Compaq SmartStart and Support - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 180
6-26 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 8. Install the host adapter drivers. 9. Install the FCSS utility. 10. Install and configure Secure Path for Windows 2000. 11. Using Fibre Channel cables, connect the - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 181
nodes for adding a new node, add the unique IP addresses and node names for the new node in the hosts and lmhosts files of each existing cluster node. These files are located at %SystemRoot%\system32\drivers\etc. Verify TCP/IP connectivity between the new node and the existing cluster nodes. Run the - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 182
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide the Oracle instance. b. Stop all Oracle services. c. Deinstall the OSDs by running the OUI files and additional rollback segments to the database. After the new partitions are created for the log files - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 183
being performed by each node, which signifies a need to adjust the distribution of the workload so that the overall cluster performance is maximized. I Compaq Insight Manager can be used to gather low-level, in-depth hardware statistics for the server and networking hardware components used in the - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 184
6-30 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Using Secure Path Manager StorageWorks go into effect until you stop and restart the Secure Path Agent Service from the Services screen. NOTE: If Secure Path Manager shows a controller failure, be - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 185
The Secure Path host system (node) that has an established access path to the storageset. I The array controller servicing the path. I The physical port number of the host adapter servicing the path. This is a relative number determined by Windows 2000's order of discovery for host adapters on that - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 186
6-32 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I Make an I/O path Offline. you want Secure Path Manager to restore access to a failed path after the problem has been corrected. Refer to the Secure Path documentation provided with the Secure - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 187
can proceed properly. 5. Restart the node. 6. Repeat Steps 1 through 5 for all nodes in the cluster. NOTE: To aid in the reinstallation of Secure Path, the file "client.ini" is not removed. The uninstallation process is complete. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 188
chapter addresses potential problems you might encounter as you install, configure, test, and operate your Compaq Parallel Database Cluster Model PDC/O5000 (referred to here as the PDC/O5000). This chapter is organized as follows: I Basic Troubleshooting Tips I Troubleshooting Oracle8i and operating - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 189
7-2 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Basic Troubleshooting Tips Power Verify that the cluster nodes, storage subsystems, and Fibre Channel SAN Switches or Storage Hubs for the storage area network (SAN) and the - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 190
Troubleshooting "Diagnosing and Resolving Shared Disk Problems," for more details. Software Compaq Insight Manager I Compaq Array Configuration Utility I Compaq Secure Path for Windows 2000 I Compaq Ethernet OSDs I Cluster interconnect drivers I Microsoft Windows 2000 Advanced Server with Service - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 191
Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide For information about currently supported software revisions for the PDC/O5000, refer to the Compaq Parallel Database Cluster Model PDC/O5000 Certification Matrix for Windows 2000. This document is available - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 192
Troubleshooting 7-5 Troubleshooting Oracle8i and OSD Installation Problems and Error Messages Potential Difficulties Installing the OSDs with the Oracle Universal Installer I The Oracle Universal Installer (OUI) uses an inventory file to track which components of the OSDs have been installed on each - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 193
Service, OracleNMService, is running. If not, see the error logs in C:\compaq\ops\nm.log. If OracleNMService is running, check the CM Server logs in C:\compaq\ installation instructions. I Verify the connections and accessibility of the cluster components. Refer to the "Basic Troubleshooting Tips" - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 194
the service and restart it. Troubleshooting Node-to-Node Connectivity Problems Nodes Are Unable to Communicate with Each Other I Check the System log and Applications log for errors. I Check the OracleCMService error log for information. The default location for the error log is C:\compaq\ops - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 195
Guide I The cluster interconnect must be on a separate subnet than the client LAN. Check this by viewing the IP addresses assigned to all interconnect adapters and to all client LAN adapters. I If Compaq IP addresses, ensure that the hosts file and lmhosts files are properly set up. When Using - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 196
Oracle services have started without error. From the Services screen, check to see if the services have started Resolving Shared Disk Problems." Troubleshooting Client-to-Cluster Connectivity Problems A Network Client Cannot Make sure that the TNSNames.ora file is properly configured on client and - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 197
7-10 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I Make sure that the Oracle TNSListener is running on the cluster nodes. From the Services screen, verify that it is started. I Make sure that the Oracle Listener files are set up on the cluster - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 198
Troubleshooting 7-11 To fix this problem, first do the following: 1. From the Event Viewer, and not to reassign PCI resources. 1. Use Notepad or Edit to edit the BOOT.INI file as follows: a. Remove the read-only file attribute (for example, attrib -r -h -s c:\boot.ini). b. Add the /PCILOCK option - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 199
7-12 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Verifying Connectivity to a Redundant Fibre Channel Fabric Verify that the array controllers and the cluster nodes are properly connected to all paths of the redundant - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 200
Troubleshooting 7-13 The topology for Port 1 on each array controller should be: I Fabric Up state (not the Standby state) I FABRIC Server Node Connectivity to Fibre Channel - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 201
7-14 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I For the array controller pair installed in each storage subsystem, verify that the two ports on each array controller are connected to different Storage Hubs. I - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 202
Troubleshooting 7-15 A Cluster Node Cannot Connect to the Shared Drives Verify connections both port 1 and port 2 will see the same storagesets, as will Disk Management. To correct this problem, do the following: I Ensure that the affected array controller is set to multibus_failover mode. Refer to - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 203
/O5000 for Oracle8i and Windows 2000 Administrator Guide To determine whether or not the storagesets can be viewed by a KGPSA-BC device driver, do the following: 1. From the Device Manager, select SCSI Adapters. 2. View the list of installed Hard Drives and manually count them. The total number of - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 204
Troubleshooting 7-17 I If the array controllers have already been set to Multibus_Failover mode, then check the integrity of the components along the unseen Fibre Channel Fabric path. Do the following: G Confirm that the Compaq KGPSA device driver was properly installed. Refer to "Installing Secure - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 205
Guide G Verify that the Fibre Channel cables have been properly installed from the Storage Hubs to both ports on the array controller. G Use the CLI to obtain status information about the array controller. Verify that both port 1 and port 2 on each array controller are active. Troubleshooting Compaq - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 206
Troubleshooting 7-19 G You across all four array controller ports. I There is no support for Windows 2000 dynamic disks. I Windows NT fault-tolerant Secure Path Agent's list of authorized clients or passwords, you must first manually stop the Secure Path Agent. Be sure to restart the Secure Path - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 207
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide and DNS will not be able to resolve the name correctly. Hyphenated host file names must be changed to unhyphenated names on all client systems. To correct host a manual refresh, press the F5 key. - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 208
Troubleshooting 7-21 The Addition of New LUNs Causes an Error In a PDC/O5000 inhibit the Secure Path Agent from automatically starting. If this occurs, do the following: 1. From the Services screen, right click on Secure Path Agent. 2. Select Properties, then select Recovery. 3. Select Recovery to - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 209
A Appendix Diagnosing and Resolving Shared Disk Problems Introduction When one or more nodes in a Compaq Parallel Database Cluster Model PDC/O5000 (referred to here as the PDC/O5000) cannot properly join the cluster, you need to consider two possible causes: I A problem related to the shared disk - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 210
OK and Firmware Checks Problem Persists Perform Cluster Hardware Checks OK Problem Persists Rerun and Validate Object Link OK Manager On All Affected Nodes Problem Persists Exit. Problem Fixed or No Further Disk Problems Contact Your Compaq Support Representative Figure A-1. Tasks for - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 211
Nodes in the Cluster" in this appendix for further instructions. For instructions on installing and using Object Link Manager, see the readme file: \Object Link Manager\readme.txt This file is located on the Compaq Parallel Database Cluster Clustering Software for Oracle8i on Microsoft Windows - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 212
Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Restart All Affected Nodes in the Cluster If Object Manager service is properly installed on the node. For instructions on installing and using Object Link Manager, refer to this readme file contained on the Compaq Parallel - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 213
Shared Disk Problems A-5 Run for Secure Path Manager is correct. Refer to the Compaq Parallel Database Cluster Model PDC/O5000 Certification Matrix for Windows Subsystem" in Chapter 5, "Installation and Configuration," for installation instructions. 4. Repeat steps 1 through 3 for all nodes in the - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 214
2000 Administrator Guide Run and Validate the StorageWorks Command Console From All Storage Subsystems The Compaq StorageWorks Command Console the cluster. Perform Cluster Software and Firmware Checks Use the Compaq SmartStart and Support Software CD to perform these checks on the cluster software - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 215
Event Viewer to verify that no hardware errors or transport problems exist. Check the event log for disk I/O error messages or indications of problems with the communications transport. 2. Determine whether or not the Compaq servers support the physical SCSI disks that are installed in the MA8000 - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 216
Guide Contact Your Compaq Support Representative If you are unable to resolve the problem, contact your Compaq support representative for assistance. In North America, call the Compaq Technical Support Phone Center at 1-800-OK-COMPAQ. Outside North America, call the nearest Compaq Technical Support - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 217
and the disk drives to which it is physically and logically connected. The period of time that a computer system is unable to service clients' requests. A standard network protocol that operates mostly on a physical level, using network interface cards and cabling to transmit data between computers - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 218
Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Fabric Fault tolerance FC-AL Fibre Channel Fibre Channel nodes are operating. A term used to identify a computer system that can continuously deliver services to its clients 99.9 percent of the time (no more than 8.5 hours of - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 219
the cluster heartbeat. The Compaq Parallel Database Cluster interconnect is dedicated to servicing requests only between cluster nodes. the network. An individual server in a cluster. New Technology File System. A file organization system by which data is stored and accessed in a Microsoft Windows - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 220
responsible for enabling parallel database support. An additional license is required file system formatting information. An unformatted partition. Oracle Parallel Server requires its database files /server application or service is dependent. As it pertains to the Compaq Parallel Database Cluster, - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 221
Glossary 5 Shared storage Storageset StorageWorks Command Console (SWCC) System UPS A storage system that can be simultaneously accessed by multiple nodes in a PDC cluster. The MA8000/EMA12000 Storage Subsystem is the shared storage subsystem for the PDC/O5000. A group of disk drives configured - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 222
components for redundant Fibre Channel Fabrics 4-3 CDs StorageWorks Secure Path Software 5-35 certification Oracle8i software 3-7 CIM See Compaq Insight Manager CLI Window 3-4 client LAN adapters installation for Oracle 8.1.7 5-6 cabling 5-9 overview 2-42 planning 4-6, 4-22 redundant 5-6 redundant - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 223
node 6-21 replacing a node 6-19 shutting down cleanly 6-2 supported servers 2-2 cluster performance monitoring 6-29 cluster planning client LAN 4-22 Compaq software overview 3-2 Oracle software 3-6 overview 3-1 cluster troubleshooting basic tips 7-2 client to cluster connectivity problems - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 224
operating system dependent modules for Oracle8i overview 3-5 Compaq Options ROMPaq, overview 3-4 Compaq Parallel Database Cluster, overview 1-5 Compaq SmartStart and Support software 3-2 Compaq SmartStart and Support Software CD 3-2 Compaq software Secure Path for Windows 2000 3-4 StorageWorks - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 225
Guide Compaq System Configuration Utility 3-3 Compaq StorageWorks Command Console 3-4 Compaq StorageWorks Fibre Channel Storage Hubs description 2-6 Compaq managing storage subsystems 6-6 running to check for shared disk problems A-5 distributed lock manager 2-39 downtime 1-4 dual redundancy - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 226
for Oracle8 5-6 installing for Oracle8i 5-4 grouping physical disk storage space 4-19 H HAL 7-10 hardware installation verification for Oracle8i 5-55 help additional sources xviii technical support telephone numbers xviii host adapters description 2-6 host server verifying connectivity to FC-AL 7-14 - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 227
6 Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide verifying connectivity to Fibre Channel Fabrics 7-13 hosts files for Ethernet cluster interconnect 4-21 for Oracle 8.1.7 5-40 for Oracle8i 5-19 HSG80 Array Controllers configuring multibus failover 5-23 - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 228
31 verifying array controller properties 5-27 Integrated Management Log, using with Compaq Insight Manager 3-3 K KGPSA Host Adapters description 2-6 replacing 6-16 40 for Oracle8i 5-19 local area network (LAN) 2-42 log files 6-4 for Oracle8i 5-55 logical drives expanding capacity 6-15 logical - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 229
disk problems A-3 operating system dependent modules See Compaq operating system dependent modules operating system software installation for Oracle8i 5-17 versions supported 3-2 Oracle software certification for Oracle8i 3-7 configuring for Oracle8i 5-53 installation troubleshooting for Oracle8i - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 230
troubleshooting 7-6 OracleNMService, troubleshooting 7-6 OSDs See Compaq Compaq Parallel Database Cluster 1-5 Compaq software 3-2 installation for Oracle8i 5-2 Oracle8i Server Enterprise Edition 3-6 Oracle8i software 3-5 shared disk problems 4-3 site planning 4-2 supported RAID levels 4-16 port - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 231
storage subsystems 5-32 overview 6-6 removing 6-33 troubleshooting 7-18 Secure Path Agent installation for Oracle8i 5-35 Secure Path Manager installation vs operation 5-33 running to check for shared disk problems A-5 Service Packs supported 5-17 Service Packs for Windows 2000 Advanced Server - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 232
7-3 storage hubs powering on 5-21, 5-38 Storage Hubs See Compaq StorageWorks Fibre Channel Storage Hubs description 2-6 storage subsystem connectivity to 40 technical support xix telephone numbers xx text conventions xvi troubleshooting basic tips 7-2 client to cluster connectivity problems 7-9 - Compaq ProLiant 6400R | Compaq Parallel Database Cluster Model PDC/05000 for Ora - Page 233
potential problems 7-1 unable to start database for Oracle8i 7-7 verifying connectivity to FC-AL paths 7-13 verifying connectivity to Fibre Channel Fabric paths 7-12 verifying KGPSA-BC driver initialization 7-10 verifying node connectivity to Fibre Channel Fabric paths 7-13 troubleshooting verifying
Parallel Database Cluster Model
PDC/O5000 for Oracle8
i
and
Windows 2000
Administrator Guide
Second Edition (June 2001)
Part Number 225081-002
Compaq Computer Corporation