HP Cluster Platform Interconnects v2010 Quadrics QsNetII Interconnect
HP Cluster Platform Interconnects v2010 Manual
 |
View all HP Cluster Platform Interconnects v2010 manuals
Add to My Manuals
Save this manual to your list of manuals |
HP Cluster Platform Interconnects v2010 manual content summary:
- HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 1
Information: Version 1.1 Product Version: HP Cluster Platform This manual describes how to install, operate, and manage the Quadrics QsNetII interconnect used in certain models of HP Cluster Platform. It also provides instructions for replacing interconnect components and describes the available - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 2
contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 3
steps can save time and prevent problems. This manual limits detailed information and procedures Service representatives. This guide references skilled tasks and describes important safety considerations and is not intended as a training aide for untrained personnel. The information in this manual - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 4
interconnect control menu. Describes diagnostic procedures and system tests. Provides troubleshooting procedures for nodes and links, including the QM500 PCI card. • Cluster Platform Overview • Cluster Platform Site Preparation Guide • Core Components • Server Overview Platform-Specific Documents - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 5
HP Web site has information on this product. You can access the HP Web site at the following URL: http://www.hp.com About This Manual v - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 6
Text set off in this manner presents clarifying information or specific instructions. Note Text set off in this manner presents commentary, sidelights, or interesting points of information. Conventions This manual uses the following typographic conventions: Monospace type This type denotes - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 7
Platform Overview and Site Preparation Guide to obtain specific information on safety module. There are no user-serviceable parts inside. The laser module should be serviced by service personnel only. Do not attempt Refer to the cluster documentation for instructions on how to move components in - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 8
its on/off switch, then remove the power cord before removing the component's cover. Remove the Power Protection Device cables before any servicing operation. Always replace the cover before switching the component on again. Using Cellular Phones and Other Wireless Technology Cellular telephones and - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 9
Recycling Shipping an integrated cluster generates far less packaging than the individual components that it contains. However, large clusters use a substantial amount of packaging material that is not reusable. The bulk of the packaging material is recyclable, and is labeled as such. You should - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 10
- HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 11
see Section 1.4) 1.1 Cluster Components and Terminology The Core Components Guide describes components that are generic to all HP Cluster Platform clock card) has an integral control processor (AB998A). Each interconnect chassis can support up to 2 clock cards, (clock A, clock B) installed in the - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 12
cluster racks are then disconnected from each other, packed and mounted on shipping pallets for delivery to a customer site. The Site Preparation Guide describes the assembly status of the cluster when it is delivered to an installation site. It also describes the following procedures: • Removing - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 13
running diagnostics (where available) to ensure correct assembly. 8. Troubleshooting the cluster, if necessary. 9. Configuring the software, as the cluster as described in the HP XC System Software Installation Guide. These manuals are available on the documentation CD-ROMs that are included with - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 14
tabs to disconnect a cable. • Provide a service loop from the bundle to the connector. • Support cables so that bundles cannot pull directly on various Cable Management Installation Guides provided on the documentation CD-ROM and follow the cable routing instructions therein. Always fasten cables - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 15
against the metal edges of the rack. Use the cable management brackets and guides. • When bundling signal cables along the outside of the rack rails, do rack is to be positioned next to other enclosures, maintain a sufficient service loop in any connecting cables to allow the rack to be moved out - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 16
in the cluster and identify the cable management systems used for the server models and for the interconnect (Quadrics). The cable management installation guides are provided on the documentation CD-ROM. 3. Complete any rack assembly work and organize the racks as described in the Site Preparation - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 17
to the control (and optionally service) nodes, as described in the Core Components Guide. 17. Connect the cables from the service nodes to the storage components (optional). 18. Consult the operating system software (such as HP XC) for power-up instructions before applying power through the PDU - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 18
- HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 19
provides transport for user application communication between the service and application nodes. Applications communicate across the as follows: • A midplane design interconnect (enclosure). Each side of the enclosure supports four switch cards with up to 64 link ports, and a controller card in - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 20
part numbers might change as models are revised. Always verify that a part and its firmware (if any) is of the correct revision and supports your current configuration. 2.2 Interconnect Network Topology Each switch chip in a switch card has eight link ports and can connect up to eight nodes - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 21
or greater, interconnects are connected in four stages; for larger networks, the interconnects are connected in five stages. Note HP Cluster Platform supports only the number of nodes specified for any given release. The maximum number of nodes is further constrained by the operating environment - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 22
: name = QRrailNumber[N|T]ICNumber • The integer railNumber indicates the rail number: 0 for single rail systems; 0 or 1 for dual rail systems. Note Dual-rail configurations are not supported in HP Cluster Platforms at the time of publication. 2-4 Quadrics Interconnect Overview - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 23
• The letter N or T indicates the type of interconnect: N for node-level; T for top-level. • The integer ICNumber is the interconnect's identification number based on its position in the hierarchy. The numbering of both node-level and top-level interconnects starts at 0 (zero). Table 2-1 shows the - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 24
- HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 25
The 128-port QS64A chassis in an HP Cluster Platform is modular, enabling the installation of different combinations of midplanes and cards to support varying cluster configurations. Configurations are determined by the number of nodes in the cluster, the density of the nodes, and other options such - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 26
3X-CM5A0) midplane, you can fully populate the interconnect enclosure with QM511C switch card modules. In this configuration, a single interconnect enclosure can support a cluster of up to 128 nodes, each fitted with a QM500 network adapter. Various combinations of QM511C and QM511L switch cards are - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 27
configurations require difference midplanes and combinations of QM501 and QM502 switch cards. For complete information on the interconnect configurations supported for HP Cluster Platform, cabling tables. 3.2.1 Interconnect Fan and Power Supply Modules The interconnect enclosure has a 17 U high, 510 - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 28
of 256, 512, 1024 or a theoretical maximum of 4096 nodes. The Cabling Tables provide definitions of the interconnect configurations that are supported for HP Cluster Platform. 3.2.3.1 Interconnect Configuration for 16-128 Node Clusters For configurations of 16-128 nodes, QM511C and QM511L switch - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 29
Figure 3-3: Interconnect, Rear View with Slot Numbering Slot 0 Slot 1 Fan A/CtrlA Slot 2 Slot 3 Fan A/CtrlA Slot 0 Slot 1 Slot 2 Slot 3 HPTC-0028 3.2.3.2 Interconnect Configuration for Federated Clusters For clusters with greater than 128 nodes, the interconnect enclosure is configured as a - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 30
3.2.3.2.1 Node-Level Interconnect (NLI) Node level interconnects use the 3X-CS5A0-AF midplane. 4 QM502 switch cards are inserted in slots 0 through 3 in the rear of the enclosure. 4 QM501 switch cards are inserted in slots 4 through 7 in the front of the enclosure. Each of the QM501 cards has 16 - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 31
3-6, is a switch card containing seven Elite switch components. It provides 16 ports and implements all three stages in the fat tree topology of a single interconnect, supporting a cluster of up to 128 nodes. 128-Port Interconnect Modules 3-7 - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 32
QM503 controller card, shown in Figure 3-8, distributes clock signals and JTAG information to the switch cards. It also hosts a daughter card that supports management and diagnostic processes. Figure 3-8: QM503 Controller Card Fan B / Ctrl B Pri Clk Ok Sec Clk Ok B Clk Active Clk Error Federated - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 33
connecting to its console using either the serial port or, if IP configuration is complete, a telnet connection. An RPC interface is also supported by the management software, this simplifies the task of controlling a large number of switch units. 3.5 QM580 Clock Distribution Box Clusters with more - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 34
2. Clock in ports. Clock in A is the top port, Clock in B is the bottom port and is not used. 3. Mode select switch. 4. Power switch. 5. Fuse. 6. Power inlet. By linking two or more clock boxes together, it is possible to provide clock signals for more than 24 interconnects. The QM580 has a 1 U high - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 35
6. Repeat steps 1-5 until all interconnect controller cards are cabled to their respective clock boxes. 7. Connect the synchronization cable between the clock boxes, as specified in the cabling tables. 8. Ensure that the clock mode switches are set to master for clock box A and Slave for clock box - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 36
- HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 37
QS32A 16/32-port interconnect. The following information is provided for this component: • An overview of the interconnect is provided in Section 4.1 • Instructions for cabling the 16/32-port interconnect are provided in Section 4.2 • An explanation of the 16/32-port interconnect signal LEDs is - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 38
cables assembles comply with the specifications for Quadrics links. • Ensure that the cables are fully inserted, locked in position and have the required support to ensure connector mating angles are within the acceptable 10 degree limit. • Replace cables if they show any signs of damage. • Cover - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 39
• Install spring connector clips on all ports in use. Ensure that all the spring fingers are in contact with the cable connector housing. Replace the clips if they show any signs of damage. 4.2.1 Connecting Cables to the Interconnect Use the following procedure to install cables: 1. Undo the knurled - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 40
tasks. Clock cable connections are not used in HP Cluster Platform configurations because the 16/32-port interconnect is not supported for federated configurations and therefore timing signal synchronization is not required. 4.2.2 Disconnecting and Removing the Interconnect from the Rack Use - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 41
Color Interpre- Normal Status tation Alternate Status Green Link Illuminated if the link is function functioning. Unlit if the link has a problem or is disconnected. Yellow Link Illuminated or flashing if data Unlit if no data is passing activity traffic is passing through the link. through - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 42
- HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 43
service the QM500. It should be read in conjunction with the instructions supplied with the node computer for fitting a PCI X card. See the Servers and Workstations Guide for specific instructions Platform, it must be installed in a PCI X slot that supports a clock speed of 133 MHz, a 64-bit wide bus - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 44
the port on the interconnect. 9. Power up and reboot the node. 5.2.2 General Instructions for Removing the QM500 Network Adapter The procedure for removing a QM500 is as follows that the weight of the link cable is supported so that it does not cause the cable plug and QM500 port to be - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 45
6 QM580 Clock Distribution Box This chapter describes the installation, operation, and maintenance of the QM580 clock distribution box. The QM580 clock distribution box provides a global, external clock source for a federated network of interconnects. 6.1 Replacing a QM580 Clock Distribution Box - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 46
0 should be connected to port 0, node switch 1 to port 1, and so on. Refer to the cabling tables for specific port-to-port cabling instructions. Caution CAUTION: When connecting clock distribution cables, it is essential that Ctrl A and Ctrl B signals are not mixed between interconnects; that is - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 47
6.3 Powering Up a QM580 Clock Distribution Box The QM580 clock distribution box uses a Universal power supply operating in the range 90-254 V AC (61-49 Hz). Power is supplied through a standard IEC320 socket on the front panel. To power up a QM580 clock distribution box, perform the following steps: - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 48
- HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 49
of the 128-Port Interconnect This chapter describes how to install modules and service the 128-port interconnect. The installation procedures are similar for all interconnect baying kit, as described in the Site Planning Guide. Installation and Maintenance of the 128-Port Interconnect 7-1 - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 50
A rack mounting kit is supplied with each interconnect. It has three sets of components with fasteners as follows: • An enclosure tray • Four tray slides (two front and two rear) • Two mounting brackets • Fasteners: - 12 cage nuts, washers, and machine screws to attach the rail kit to the rack - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 51
Use the following procedure to install the enclosure in the rack: 1. Unpack the interconnect enclosure and the rack mounting kit, ensuring that all components and fasteners are present. 2. Attach a bracket to each side of the interconnect enclosure using the 16 screws supplied, setting the position - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 52
6. The tray can move freely on the slides. Slide it all the way to the rear of the rack Note The tray might move as you are lifting in the interconnect enclosure. Hold the tray as you slide the enclosure forward. 7. Remove the protective covers and the nuts and washers from the two threaded studs - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 53
7.3.1 Installing a QM501 or QM502 Switch Card Color-coded keys, known as midplane polarizing keys, are fitted to the switch cards and to the midplane of the interconnect. These keys prevent the switch cards from being fully inserted in an incorrect slot. Polarizing keys function as follows: • QM501 - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 54
so that you can reconnect them to the correct ports. 2. Starting at the top port, disconnect all the link cables and remove them from their support hooks on cable management bar. Take care not to bend the cables beyond their minimum bend radius. 3. Unclip and lower the hinged cable management bar - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 55
of explosion if the Lithium battery is incorrectly replaced. Replace only with a type CR2032 battery or equivalent. Dispose of used batteries according to the manufacturer's instructions. Installation and Maintenance of the 128-Port Interconnect 7-7 - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 56
Figure 7-6: Replacing the Lithium Battery QUaDrics QsNet" PSU A Fan B/Ctrl B PSU B Batter and Holder HPTC-0034 Use the following procedure to replace the battery: 1. Remove the QM503 controller card from the interconnect. 2. Carefully slide the Lithium battery from under the metal clip - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 57
unit. A QM566 power supply blank must be fitted to slot B if only one QM561 power supply unit is fitted. Figure 7-7: Inserting or Removing a Power Supply QUaDrics QsNet" PSU A PSU B PSU A Latch HPTC-0033 Install the QM561 power supply units as follows: 1. Unpack the QM561 power supply units - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 58
Use the following procedure to replace a power supply: Caution If you remove a QM561 while the interconnect is powered up, do not leave the system running unattended and do not insert anything other than a replacement QM561 in the vacant slot. 1. Unpack the new QM561 power supply unit, keeping - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 59
will operate normally for up to 120 seconds without fan cooling. Service personnel must ensure that the replacement task is completed within this time Bars Rear of Rack Front of Rack See Cable Management Kit Installation Guide, part number: AA-RVWCA-TE, or later revision. 7.3.8 Powering Up - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 60
functioning, you can swap the other out for maintenance with no disruption to the operation of the interconnect. When powered up, normal operation of each QM561 power supply unit is indicated by the illumination of the following LEDs: Label AC OK DC OK Fault Over Temp Color Green Green Red Red - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 61
2. Verify that the Main menu is displayed, pressing Return to redraw the screen as necessary. 3. Select menu option Set Module Mode by pressing 4 followed by Return. 4. In response to the prompt Set Federated "y" or "n", press y for a federated switch or n for a standalone Switch, followed by Return - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 62
to prevent a connector angle displacement of more than 10 degrees from the horizontal line. Ensure that the cables are supported by the cable management bars. 3. Support the cable using available cable management systems to ensure that the connector and port remain aligned, and there is no stress - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 63
the replacement cable first. Note You might not want to go to the trouble of completely removing a bad cable unless it is to be returned for replacement the LED should cease to glow. If it persists, there is a problem with the link. 7.6.4 Connecting the Interconnect to the Control Network You - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 64
fully inserted, locked in position and have the required cable management support to ensure connector mating angles are within the acceptable limits. 4. the installation instructions supplied with the midplane kit. The instructions supplied with the kit might supersede the instructions in this - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 65
and all blank cards. 6. Disconnect the two QM573-x midplane wiring looms from the top of the midplane. If necessary, carefully remove one of the card guides to gain access. Position the cables out of the way. 7. Fit two midplane handles to the board. 8. Slacken off all the captive fasteners holding - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 66
9. Tilt the bottom of the midplane towards you until it will clear the top of the module cage then, keeping it at this angle, slowly and carefully remove the midplane from the interconnect. 10. Take off the midplane handles and fit Midplane protective covers (if supplied with the replacement - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 67
layout and cabling tables for your model of HP Cluster platform. To ensure that the correct cooling airflow is maintained, follow these installation instructions and take care that you do not block any of the perforated vents. The rack mount kit contains the components shown in Figure 8-1. Figure - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 68
Table 8-1: 16/32-Port Interconnect Rail Kit Callout Item Description 1 Front track, which slides into the rear track and mounts on the front rack columns 2 Rear track, which mounts on the rear rack columns 3 Slide rails that mount on the interconnect chassis 4 Cable management slides (Left and right - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 69
Figure 8-2: 16/32-port chassis Cable Management Assembly ` _ X ] ^ \ Y Z [ X Y Use the following procedure to first assemble the cable management components: 1. On each cable management slide (callout 1) , screw 2 M4 x 10 mm knurled-head bolts (callout 2) loosely into the two outer holes, giving - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 70
9. Secure both the cable management upper retainer and the cable management lower retainer to the right hand cable management slide by using an M4 x 10 mm knurled-head bolt. When you have assembled the cable management components, you mount it on the rack kit as shown in . Figure 8-3: Installing the - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 71
6. Fasten the two front tracks to the rack columns by using four screws (two in each column). Do not tighten the screws at this time. (No screw thread should show, but the screws should be loose enough to allow for minor adjustment of the track). 7. Adjust the position of the sliding cable - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 72
the Infiniband ports. As part of the cabling procedure, you might need to adjust the position of the cable management assembly so that it adequately supports the cables and also enables easy removal of a cable when required. The chassis is now ready for cabling. Use the cabling tables for your model - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 73
in figure identify the following features: 1. Four captive screws that secure the module in the chassis. 2. Fan units. 3. Power supply (not user serviceable). Removing the fan and PSU module provides access to the controller card for maintenance operations such as replacing the battery or flash card - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 74
8.4 Replacing the Switch Card Module Use the following procedure to replace the switch card module: 1. Unpack the replacement module, retaining all the packaging materials. 2. Switch off the power and disconnect the power cord at the inlet. 3. Disconnect all the cables and remove the interconnect - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 75
Figure 8-8: Controller Module [ Y Z X \ \ \ \ The following features of the controller module are relevant to servicing: 1. The connector for the ribbon data cable. The other end of this cable connects to the switch card module. 2. The power cable connector. 3. The replaceable - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 76
or equivalent. Dispose of used batteries according to the manufacturer's instructions. Warning There is a danger of explosion if the you do card only from HP. You need only perform this operation if HP authorizes and supports an update to the firmware. 2. Remove the fan and PSU module from the - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 77
environment as described in the Administration Guide for the HP XC Operating environment. Additional diagnostic tests and procedures might be provided in the HP XC documentation. This manual covers diagnostics, performance analysis, and troubleshooting for the following cluster components only - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 78
This manual does not cover diagnostics and troubleshooting for other cluster components, such as servers (used as Administration Guide Supported diagnostics are provided as part of the HP XC distribution kit. If you are not using HP XC, see Chapter 12 and Chapter 13 for troubleshooting procedures - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 79
• Connect a dumb terminal to the port labelled on the QM503 controller card. Connect a keyboard to the port labeled Mouse Kybd. These ports are shown by callouts 5 and 3 in Figure 10-1. • Connect a PC or laptop to the port labeled COM 1 on the QM503 controller card. Use terminal emulation software - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 80
in the HP Cluster Platform Overview and Site Planning Guide, and in the cabling tables. The integer xx same procedures when diagnosing cluster problems or when replacing an interconnect and thus requires more than one interconnect chassis to service the network. • Redundant - If a second controller - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 81
be connected from the clock box to the input labeled Pri Clk (primary clock) on the controller card. This is the hardware configuration that is supported by the redundant mode setting. (Do not use the port labeled Sec Clk). See Figure 10-1 for the location of the clock inputs, and Chapter - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 82
5. Enter 0 (zero) for the rail number in response to the following prompt: Enter rail: 0 6. Enter the letter (in upper or lower case) that corresponds to the type of the interconnect in response to the following prompt: Enter type (N for Node, T for Top) (q to abort): N 7. In response to the - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 83
do the following: a. Backup the file by copying it to a new filename, so that you can restore the original file in the event of a problem. b. Edit the file with a text editor such as vi, adding interconnect names and TCP/IP addresses as appropriate. 9.4 Postinstallation Verification Procedure Use - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 84
jtest> quit All firmware revisions should be the same for all controller card. Use only the firmware revision that is supported by HP for your cluster model and operating environment. 5. If you need to upgrade the firmware, open a connection to the controller card and select the - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 85
12. On systems not running HP XC, use the following procedure: a. Determine whether your system is a full bandwidth or reduced bandwidth configuration. If the cluster has a reduced bandwidth configuration, follow the additional test rules defined in Section 12.11. b. Run the qsnet2_dmatest on the - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 86
9-1: Diagnostic Flowchart Start Verify the configuration of the interconnects Interconnects are configured and pass the self test. No Configure or fix any problems Yes Verify all LEDs in the cluster with nodes powered on Run the qselantest on all nodes Did all nodes pass the qselantest - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 87
10 Using Component LEDs The LEDs in an interconnect network indicate the operational status of the network components. The following sections describe the location of the LEDs and how to check their status to confirm the operation of the network: • Finding the location of LEDs (see Section 10.1). • - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 88
Depending on the type of switch card installed in a slot, its port LEDs show the following status: - The status of the link between the interconnect and the QM500 PCI card in the node - The status of the link between the node-level interconnect and the top-level interconnect in a federated cluster. - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 89
determine the proper operation of hardware and thus isolate a potential software problem. Link LEDs are those that enable you to verify the status of or the link cable might be faulty. The fault might only be an installation problem, such as a card that is not correctly seated in the slot or a - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 90
green LEDs are illuminated. An illuminated red LED indicates that a component or the link cable might be faulty. The fault might only be an installation problem, such as a card that is not correctly seated in the slot or a cable that is not securely seated in a port. During a normal (error free - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 91
supply or DC output indicator. The PSU temperature indicator. This might illuminate if OFF Red the PSU runs too hot, perhaps due to a fan problem. The DC power output indicator. This might illuminate if the DC voltage drops below a specified level. Green Amber 10.2.4 Clock Generator LEDs The - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 92
Master Error This clock is in master mode. An error is detected in the clock generator box. Amber OFF N/A (Status only) Red 10.3 Diagnosing Link Problems by Using the LEDs You can use the LEDs to diagnose the following links: • Preparing to run a diagnosis (see Section 10.3.1). • Diagnosing - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 93
Note Ignore illuminated red LEDs on connected links when the interconnect is first powered up. These will be cleared by the interconnect manager software. You can also Ignore red LEDs on unconnected links. If the green LED is lit at one end of the link cable but not the other, it is most likely - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 94
degrees from horizontal at their connection point with a port. To avoid problems at port connections, ensure that all cables are correctly hooked into components with a known good component, or verify the configuration as instructed. Ensure that you perform each stop in the order specified. Check - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 95
be 650M/S, replace the clock distribution cables that are connected to each interconnect. c. Replace the link cable. d. Replace the switch card in the top-level interconnect. e. Replace the switch card in node-level interconnect. By the end of this procedure, you will have isolated and replaced the - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 96
- HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 97
settings) so that you can perform management operations remotely. You can manage the interconnect either by using one of the interconnect network services, such as jtest and swctrl, or by using a telnet connection. You can make a direct connection to the controller card during initial configuration - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 98
Quadrics Switch Control Network Settings Name : QR0N00 MAC address : 004053072380 IP address : 172.20.66.1 Netmask : 255.255.255.0 Broadcast : 172.20.255.255 Protocol : STATIC TFTP Server : 194.202.174.19 Upgrade file : 503-upgrade.tar Gateway : 172.16.20.1 press return to return to menu: 11.2 - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 99
times. Options other than those listed below can cause jobs to crash, and might also cause damage to cluster components. The command options supported for HP Cluster Platform are as follows: • Operations are applied to selected chips on the selected boards as follows: boards id ... chips id - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 100
Platform and for any installed operating environment such as HP XC. Do not install any controller card firmware revisions that are not supported for your cluster's configuration. To upgrade the controller card firmware, you must perform the following tasks: 11-4 Accessing and Using the Interconnect - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 101
• Each controller card is connected to the management network by an Ethernet cable. Before you can connect to the controller cards over the management network, you must assign an IP address to each controller card and ensure that it is configured for upgrades via TFTP protocol, as described in - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 102
7. To confirm that the upgrade has completed successfully, reconnect to the interconnect as follows: cp6000sms# telnet QR0T00 8. The Quadrics Switch Control menu is displayed again, as shown in step 1. 9. Choose option 3 to run the jtest command , as follows cp6000sms# jtest QR0T00 jtest: - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 103
you to diagnose and test your HP Cluster platform to resolve problems or as part of scheduled maintenance. The following procedures are described utility, (see Section 12.1). • Using the qsnetstat script and manual procedures for environmental monitoring, (see Section 12.2). • Using the qselantest - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 104
as PASSED, with no errors detected. If the output from selftest contains any error messages, use the following procedures to interpret and correct the problems. ERROR can read lm75 on empty slot, suspected JTAG failure on board X • Location: On all boards. - Possible Diagnoses: JTAG failure on the - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 105
board is unknown to the installed version of the self test software. 3 Correction: Update the controller card firmware to an appropriate and supported revision. ERROR Fantray X is missing or faulty • Location: Fan tray A or fan tray B. - Possible Diagnoses: Slot 0, 1, 2, 3 and/or Ctrl A is populated - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 106
power supply fan. Table 12-1 provides additional information for problems specific to the power supply. Table 12-1: Interpreting PSU you can use a software script. You can also optionally run a series of manual checks and test commands as described in the following sections: • Run qsnetstat to - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 107
on a production system. The following component status is tested: • Controller card. • Fans • Power supply units • Clock source • Firmware • Error registers Typical problems found include the following: • Incorrect IP or hostnames in the /etc/hosts file or in the supplied argument list. • Incorrect - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 108
QR1T06 192.168.180.135 QS2_16X8 42-4022508 A/656 O/O OOOOOO 26'C QR1T07 192.168.180.136 QS2_16X8 42-4022508 A/656 O/O OOOOOO 27'C -Links In Reset QR1N02 6:0:7 Held In Reset QR1N06 7:2:5 QR1N09 3:5:5 Held In Reset QR1N09 5:2:3 Held In Reset . . . 1T02 5:2:3 QR1T03 - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 109
jtest command from the cluster's control node. Interconnect management scripts and commands are located in the /opt/qsnet/bin directory. You can manually determine the environmental status of an interconnect by using the jtest utility as follows 1. Launch the /opt/qsnet/bin/jtest utility remotely - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 110
Clock - Both source and frequency (truncated) in the format A/656 or B/656. • Clock Errors - This relates to data traffic timing errors, not problems with the clock sources. 12.2.3.2 Clock Source Information from the jtest Utility Refer to the output display from the jtest utility, shown in Example - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 111
12.2.4 Verifying the View of the Network To ensure that there is a correct and consistent view of the interconnect network, you must verify the following: • That the QM500 PCI adapters in each node can see the correct network size. • Each node has the correct network position. To verify the network - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 112
bandwidth. • Data consistency. The diagnostics software provides an initialization script named /etc/init.d/qspost that enables you to run qselantest as a service when a node boots. Configure the script by using the following command: # chkconfig -add qspost This command adds symbolic links in the - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 113
is extensive and a sample is provided in Appendix D. 12.3.2 Running the qselantestp Utility Under HP XC The HP XC operating environment supports a wrapper script named qselantestp (the p stands for parallel). The script automatically spawns a qselantest process on all the nodes, directing the output - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 114
- No Errors 12.5 Using the qsnet2_level_test Utility When running the HP XC operating environment, you can use the qsnet2_level_test utility for diagnosing interconnect problems. To run the command on all nodes simultaneously, use the /cluster/bin/crsh command. If all nodes do not complete the test - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 115
by badly seated link cables between a node level interconnect and a top level interconnect. Check and reseat both ends of the affected link cables. If the problems persist, replace the affected link cables. This test is designed to run in parallel on every node and test every possible link in the - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 116
• The -noparse option enables you to run the qsnet2_level_test utility so that it records the log files in the specified directory without parsing the log files (defers analysis of the logs). This option is useful during drain time testing where you have a limited time to perform preventative - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 117
all required and optional arguments that are supported by the benchmarking program. The qsnet2soaktest script process is killed by a timeout due to system problems). On completion, the script takes a second snapshot to syslog The swmlogger service enables swmserver interconnect management daemons - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 118
with an inconsistent firmware revision. It compares firmware revisions to that of the first module that signs in. ": problem: power/control problem" This message indicates that an interconnect module of an unknown type has tried to establish communication with swmlogger. ": link - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 119
": clock: " This message indicates a change in the interconnect clock signal. A clock failure means the clock frequency has drifted +/-1.0 MHz from 656 MHz, while clock ok means the clock frequency has returned to within specification. 12.7.2 Using swmlogger for Production - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 120
on transaction (packet and transaction error). This indicates some bit errors in one of the transactions. • RcvLk Receiver lock error (low level line error). Problems with the received or local clock. • Dskew Deskew error (low level line error). Only likely to be caused by a hard failing data bit - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 121
.5. 6. qsnet2_level_test at level3, as described in Section 12.5. 7. qsnet2_level_test at level4, as described in Section 12.5. As an alternative (and if it is supported by your configuration) you can run the qsnetsoaktest with its duration time set to 10 minutes as described in Section 12.6. 12.10 - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 122
• qsnet2_level_test level2 • qsnet2_level_test level3 If the cluster is a federated configuration (more than 128 nodes), the qsnet2_drain_test script also runs the following tests: • /usr/bin/qsportmap. • qsnet2_level_test level4. Note Clusters might be configured with full bandwidth or reduced - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 123
xc4n15: tping -f gex 8k xc4n16: duplex ping complete xc4n16: tping -f gex 8k xc4n12: duplex ping complete xc4n12: tping -f gex 8k xc4n12: duplex ping complete xc4n12: tping -f gex 8k xc4n13: duplex ping complete xc4n13: tping -f gex 8k xc4n14: duplex ping complete xc4n14: tping -f gex 8k xc4n14: - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 124
Table 12-2: Node ID Pattern for Testing Reduced Bandwidth Clusters (cont.) 0-4 Test this set of node IDs 5-7 Do not test 8 - 11 Test this set of node IDs 12 - 15 Do not test 16 - 19 Test this set of node IDs 20 - 23 Do not test Test and Verify the QM500 PCI Adapter in a Node 1. Verify - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 125
c. Run the qsnet2_level_test level3 on all nodes connected to the interconnect, as described in Section 12.5. d. If configured as a federated solution, run the qsnet2_level_test level4 on all nodes connected to the interconnect, as described in Section 12.5. Note For systems not running HP XC, - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 126
card is configured as described in Section 9.3.2 and any additional configuration required by the operating environment (and described in the software manuals) is complete. 3. Open a telnet connection to the interconnect. When the main menu is displayed, choose the option titled 9. Self Test. An - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 127
address if you replace a control processor card. The TCP/IP address for the new interconnect should remain the same. 12.12 Collecting QM500 Manufacturer Data (VPD) The manufacturing (VPD) data, such as its part number and serial number, for each QM500 PCI card is recorded in a file named vpd on each - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 128
sql> quit The cluster name is shown in the val column of the output. For a cluster called cp6000 the specified database should be qsdiagshost:rms_cp6000. Supported network types are: QMS64, QMF256 and QMF1024 or you can use the -n and -t options to specify the number of interconnects in your system - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 129
to discard link traffic. If a link has a high error rate, you can configure it out until you can determine the cause of the problem during a scheduled maintenance period. If you do not specify command line options, the command verifies the network against the network definition stored in the - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 130
# qsportmap name Where name is an interconnect name, such as QR0T03. The command displays a table of boards and ports on the top-level interconnect. Each location shows the port on the node-level interconnect module to which a port on the top-level interconnect is connected. Typical output follows: - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 131
are intended to assist you in identifying symptoms of common problems and to direct you towards appropriate diagnostics: • Node troubleshooting, (see Section 13.1). • Link cable troubleshooting, (see Section 13.2). 13.1 Troubleshooting Node Problems Node powers up but is otherwise dead or fails - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 132
as the following: Rev A switch detected... ...change in network level..... You might see these messages in conjunction with a nodeset problem, as described in the preceding troubleshooting symptoms. Proceed as follows: 1. The QM500 network adapter is either faulty or needs to be reseated in its PCI - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 133
and qsnetcabletest to verify their integrity. Using a tool such as pdsh with dshbak is helpful for sorting the diagnostic output. 13.2 Troubleshooting Link Problems A high occurrence of network errors seen using the qsnetstat or qsneterr diagnostics Network errors are often due to a badly seated or - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 134
not behaving as expected then the network is probably not the cause. Escalate the problem to the next level of support. You suspect that a link cable replacement is required You suspect that a link then reconnect the original cable and remove the temporary link. 13-4 Troubleshooting Nodes and Links - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 135
A Components Required The tables in this appendix specify the quantities of components that are required to build a cluster, depending on the number of nodes and the bandwidth. Each table is preceded by a parts count for the parts quantities that do not vary. The table headings provide the following - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 136
Bounded Cluster to 128 Nodes, Reduced Bandwidth The following parts list specifies parts that are the same for all node counts: Part Bounded interconnect enclosure (chassis) Quantity 1 The following table specifies the additional parts and count required, depending number of nodes in the cluster: - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 137
Federated Clusters of 129-256 Nodes, Reduced Bandwidth The following parts list specifies the parts that are the same for all node counts: Part Top level interconnect enclosure (uses a transpose midplane that has no link connectors). Transpose midplane with no link connectors Top level interconnect - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 138
Federated Clusters of 257 to 512 Nodes, Reduced Bandwidth The following parts list specifies the parts that are the same for all node counts: Part Top level interconnect enclosure (uses a transpose midplane that has no link connectors). Transpose midplane with no link connectors Top level - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 139
Federated Clusters of 257 to 512 Nodes, Full Bandwidth The following parts list specifies the parts that are the same for all node counts: Part Top level interconnect enclosure (uses a transpose midplane that has no link connectors). Transpose midplane with no link connectors Top level interconnect - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 140
Federated Clusters of 513-1024 Nodes, Reduced Bandwidth The following parts list specifies the parts that are the same for all node counts: Part Top level interconnect enclosure (uses a transpose midplane that has no link connectors). Transpose midplane with no link connectors Top level - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 141
Federated Clusters of 513-1024 Nodes, Full Bandwidth The following parts list specifies the parts that are the same for all node counts: Part Top level interconnect enclosure (uses a transpose midplane that has no link connectors). Transpose midplane with no link connectors Top level interconnect, - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 142
- HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 143
-001 3X-CM584-15 Note The interconnect wiring loom is identified in some interconnect diagnostics as a replaceable component. HP does not support the wiring loom as a replaceable component. If the wiring loom fails, replace the entire chassis enclosure, repopulating the replacement chassis with - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 144
- HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 145
C Component Specifications This appendix provides operating and performance specifications for cluster components. Specifications might change for individual components as products are improved. Consult the user information for a specific components to obtain the latest specification and operation - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 146
Operating and Performance Specifications Idle or Operating LWAd, bels 7.2 B LpAm, dBA 56 dBA C.2 Quadrics® QsNetII™ Model QS32A 16-port or 32-port Interconnect The following table presents the operating and performance specifications for the QsNetII interconnect. Operating and Performance - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 147
• The dimensions of the QM500 are as follows: • Length 178 mm Width 125 mm • Depth 22 mm Weight 0.13 Kg • The dimensions of container in which the QM500 is shipped are as follows: • Length 300 mm Width 125 mm • Depth 25 mm Weight 0.25 Kg • Environmental Characteristics • The maximum ambient room - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 148
Physical Specifications Storage temperature Operating humidity Storage humidity Maximum operating altitude Maximum shipping altitude Operating vibration Operating shock Nominal heat dissipation Maximum heat dissipation Airflow Not tested 20C to 80C 10C to 90C 3037 m (10,000 ft) 12190 m (40,000 - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 149
as follows: # /usr/opt/rms/diag/bin/elandebug device_error_stats ... CorrectableErrors 0(0) UncorrectableErrors 0(0) MultipleErrors 0(0) If you are escalating a problem report to a support organization, use the following command to obtain information about the card: # cat /proc/qsnet/elan4/device0 - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 150
MV: 1.00 MS: 00000000 DT: 00 DV: 00 DS: 00 DG: 00 DD: 00 Replace the QM500 PCI adapter under the following circumstances: • An uncorrectable error occurs. • The driver reports multiple errors. • A high rate of correctable errors occur on the same QM500 network adapter in a short period of time. D.1 - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 151
Show_pci: reading data... VendorID: 14fc DeviceID: 0001 Command: 00000156 EnableMemory EnableBusMaster Invalidate Parity EnableSERR Status: 00000230 Capabilities 66Mhz Devsel=1 ClassCode 28000 RevisionID 01 CacheLineSize: 20 Latency Timer: 40 Header Type: 00 BIST: 00 BARs[0]: d000000c BARs[1]: - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 152
tmemtest: allocated main condition vars tmemtest: thread init tmemtest: waiting for thread tests to start... EVEN_CHECKERBOARD test over 33554432 bytes: 100% complete ODD_CHECKERBOARD test over 33554432 bytes: 100% complete ADDRESS_LINE_TEST test over 33554432 bytes: 100% complete - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 153
Addrs: 200000000061a6a0 - 2000000000632d38 qselantest Output qsnet2_probe ...STARTED Mode Switched NodeId 0 NumLevels 2 NumNodes 16 qsnet2_probe ...PASSED qsnet2_dmatest -e -RM -se -dm -n 1000000 -S 0 ...STARTED mnfr_dmatest: master self=0 peer=0 source 0x2000000000604000 dest 0x2000000004606000 - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 154
Addrs: 200000000460a000 - 2000000004622698 mnfr_dmatest: Pattern: ffffffffffffffff words: 12500 Addrs: 20000000046226a0 - 200000000463ad38 mnfr_dmatest: Pattern: 5555555555555555 words: 12500 Addrs: 200000000463ad40 - 20000000046533d8 mnfr_dmatest: Pattern: aaaaaaaaaaaaaaaa words: 12500 Addrs: - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 155
workstations. Wear the straps on both feet when standing on conductive floors or dissipating floor mats. • Use conductive field service tools. • Use a portable field service kit with a folding static-dissipating work mat. • If the suggested equipment for proper grounding is not available, have an - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 156
- HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 157
F Cable Types This appendix explains the types of cables used in the interconnect , and illustrates the default wiring topology for standalone and federated networks. There are four types of interconnect cables: JTAG The JTAG cable connects the parallel port on the switch-managing node to the - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 158
- HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 159
systems, 1-4 port mapping, 12-27 routing, 1-4 testing level with qsnet2_level_test, 12-12 testing with qsnet2_dmatest, 12-11 troubleshooting, 13-3 types, F-1 cable management, 4-3, 8-2 cable management bars, 7-11 cables CAT-V, 1-3 Ethernet, 2-2 interconnect link, 1-3 QM581, 1-3 QM584, 1-3 cabling - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 160
8-8 controller card, 3-8 configuring, 9-3 installing, 7-6 LEDs, 10-5 Index-2 main menu, 9-3 mode, 9-4 reboot, 11-1 security, 11-1 upgrading firmware, 11-1 Core Components Guide, 1-7 count, node, 1-5 creating a diagnostic database, 12-25 Ctrl Run LED, 10-2 D data QM500, 12-25 DC Good LED, 10-2 dense - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 161
E elan, 2-1 electrostatic discharge overview, E-1 prevention measures, E-1 elite, 2-1, 2-2, 2-3, 3-1 EMI, 5-2, 7-16 environmental monitoring, 12-4 error link, 12-17 PSU messages, 12-4 selftest messages, 12-2 Error LED (QM580), 10-3 ESD grounding, E-1 preventing, E-1 Ethernet cable, 2-2, 4-4 Ethernet - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 162
node boot LEDs, 10-4 port mapping, 12-27 ports, 2-1 testing level with qsnet2_level_test, 12-12 testing with qsnet2_dmatest, 12-11 troubleshooting, 13-3 verifying, 7-15 lithium battery installing, 7-7 logging messages, 12-15 M main menu, 9-3 maintenance 16/32-port chassis, 8-1 collecting QM500 data - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 163
settings, 11-1 NLI, 2-4, 2-5t, 3-1, 3-5t node count, 1-5 identifier, 12-9 levels, 12-9 link LEDs at boot, 10-4 number, 12-9 numbers, 2-3 troubleshooting, 13-1 node level link diagnosis using LEDs, 10-6 node-level interconnect, 2-4, 3-5t nodeid, 12-9 numlevels, 12-9 numnodes, 12-9 O Over Temp LED - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 164
PSU error messages, 12-4 query controller using qsctrl, 12-26 self test, 12-1 self test error messages, 12-2 soak test, 12-25 soak testing, 12-28 testing modules, 12-21 testing QM500, 12-22 testing QM501, 12-22 testing QM502, 12-22 testing QM503, 12-24 testing QM511C, 12-23 testing QM511CL, 12-24 - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 165
settings, 11-1 site preparation preventing electrostatic damage, E-1 Site Preparation Guide, 1-6 slot numbers, 3-4 soak test, 12-15, 12-25 , 10-8 top-level interconnect, 2-4, 3-5t, 3-6 topology, 1-6, 2-1, 2-2 troubleshooting, 13-1 clock source, 12-8 diagnostics, 12-1 drain time testing, 12-19 - HP Cluster Platform Interconnects v2010 | Quadrics QsNetII Interconnect - Page 166
query controller using qsctrl, 12-26 selftest command, 12-1 selftest error messages, 12-2 soak test, 12-25 soak testing, 12-15 testing level with qsnet2_level_test, 12-12 using LEDs, 10-3 verify interconnect, 12-4 two-stage network, 2-2 U user interface accessing, 11-1 V verifying clock source,
-
1 -
2 -
3 -
4 -
5 -
6 -
7 -
8
-
9
-
10
-
11
-
12
-
13
-
14
-
15
-
16
-
17
-
18
-
19
-
20
-
21
-
22
-
23
-
24
-
25
-
26
-
27
-
28
-
29
-
30
-
31
-
32
-
33
-
34
-
35
-
36
-
37
-
38
-
39
-
40
-
41
-
42
-
43
-
44
-
45
-
46
-
47
-
48
-
49
-
50
-
51
-
52
-
53
-
54
-
55
-
56
-
57
-
58
-
59
-
60
-
61
-
62
-
63
-
64
-
65
-
66
-
67
-
68
-
69
-
70
-
71
-
72
-
73
-
74
-
75
-
76
-
77
-
78
-
79
-
80
-
81
-
82
-
83
-
84
-
85
-
86
-
87
-
88
-
89
-
90
-
91
-
92
-
93
-
94
-
95
-
96
-
97
-
98
-
99
-
100
-
101
-
102
-
103
-
104
-
105
-
106
-
107
-
108
-
109
-
110
-
111
-
112
-
113
-
114
-
115
-
116
-
117
-
118
-
119
-
120
-
121
-
122
-
123
-
124
-
125
-
126
-
127
-
128
-
129
-
130
-
131
-
132
-
133
-
134
-
135
-
136
-
137
-
138
-
139
-
140
-
141
-
142
-
143
-
144
-
145
-
146
-
147
-
148
-
149
-
150
-
151
-
152
-
153
-
154
-
155
-
156
-
157
-
158
-
159
-
160
-
161
-
162
-
163
-
164
-
165
-
166
 |
 |
HP Cluster Platform
Quadrics QsNet
II
Interconnect
Part Number: A-CPQIG-1A
May 2005
Revision/Update Information:
Version 1.1
Product Version:
HP Cluster Platform
This manual describes how to install, operate, and manage the Quadrics QsNet
II
interconnect
used in certain models of HP Cluster Platform. It also provides instructions for replacing
interconnect components and describes the available diagnostics.
Hewlett-Packard Company
Palo Alto, California