Home » HP Manuals » Servers » HP Cluster Platform Interconnects v2010 » Manual Viewer

HP Cluster Platform Interconnects v2010 HP Cluster Platform InfiniBand Interco - Page 135

x DDR IB Switch Module

Get HP Cluster Platform Interconnects v2010 PDF manuals and user guides

View all HP Cluster Platform Interconnects v2010 manuals

Add to My Manuals
Save this manual to your list of manuals

Page 135 highlights

HCA 4x DDR IB mezzanine, 105 HCA mellanox memory free PCI-Express DDR, 102 HCA mellanox memory free PCI-Express SDR, 101 HCA mellanox PCI-Express SDR, 100 HCA Mellanox PCI-X (SDR), 99 HCA port GUID, 109 HCA Topspin/Mellanox PCI-Express, 98 HCA Topspin/Mellanox PCI-X, 97 head shell, 85 hearing protection, 16 high availability, 29, 39 horizontal fan, 51 host channel adapter, 67, 93 (see also HCA) Host channel adapters, 19 (see also HCA) hot components, 16 hot swap, 76 hot-swap LED, 72 hot-swap, 50 HP documentation contact, 13 feedback, 13 URL, 13 HyperTerminal, 91 I IEC power inlet ISR 9024, 33 important information, 14 in-band management, 29 InfiniBand 4x DDR IB Switch Module, 69 4X port switch chip, 29 cabling, 85 communication speeds, 19 components, 20 connecting cables, 86 defined, 19 fabric, 19 features, 19 network features, 19 protocol, 20 InfiniBand logical link, 33 InfiniBand physical link, 33 InfiniBand port mezzanines, 30 InfiniBand port status LED ISR 9024, 31 InfiniBand® Trade Association, 19 InfiniScale III switch chip, 29, 40 installation safety considerations, 15 Installing ISR 9096, 49 ISR 9288, 49 installing cabling clamps, 63 HCA, 94 inter-integrated circuits, 30 (see also I²C) interconnect cabling, 85 interconnect administration, 26 (see also management) internally managed, 29, 39 internet protocol router, 24, 26, 77 (see also IPR) IPR ISR 9288, 77 LEDs, 78 IPR blade, 24, 26 ISR 9024 12x ports, 30 administrative cabling connections, 90 cabling procedure, 86 chassis, 29, 40 cooling, 33 DB-9 adapter, 30, 31 EEPROM, 30 EIA-232 serial port, 30 Ethernet port, optional, 33 Ethernet RJ-45 port, 31 externally managed, 29, 30 front panel, 30, 40 front panel, externally managed, 32 front panel, internally managed, 30 identifying, 22 IEC power inlet, 33 InfiniBand link LEDs, 33 InfiniBand port mezzanines, 30 InfiniBand port status LED, 31 Installing and maintaining, 29, 39 installing PSU, 36 internally managed, 29 management card, 31 master mode, 30 (see also internally managed) mini-USB, 30 operating, 36, 47 overview, 29 PCI mezzanine card, 30 PSU, 30 rack kit, 34 rack mounting, 34, 44 rear panel, 32, 42 reset button, 30, 31, 33 resetting, 31, 41 serial console interface, 30, 40 slave mode, 30 (see also externally managed) specifications, 111 status LEDs, 31, 32 troubleshooting, 36, 47 unpacking, 33 ISR 9024 management, 26 (see also management) 135

Section	Page
InfiniBand Interconnect Installation and User's Guide	1
Table of Contents	3
About This Manual	11
Organization	11
Audience	12
Related Documentation	12
Voltaire Documentation	12
HP Cluster Platform Documentation	12
Cluster Component Documentation	13
HP Encourages Your Comments	13
Text Annotations	13
Conventions	14
Safety Considerations	15
Information on Ergonomics	16
Recycling	17
1 InfiniBand Technology Overview	19
1.1 Identifying InfiniBand Fabric Components	20
1.2 ISR 9xxxx-series Interconnects	21
1.3 Identifying the ISR 9024 Interconnect (SDR)	22
1.4 Identifying the ISR 9024 S/D Interconnect (RoHS Compliant)	22
1.5 Identifying the ISR 9096 Interconnect	23
1.6 Identifying the ISR 9288 Interconnect	24
1.7 Understanding the Interconnect Management Capabilities	26
2 Installing and Maintaining the ISR 9024 Interconnect	29
2.1 Overview of the ISR 9024 Interconnect	29
2.1.1 Internally Managed ISR 9024 Front Panel	30
2.1.2 ISR 9024 Management Card	31
2.1.3 Externally Managed ISR 9024 Front Panel	32
2.1.4 ISR 9024 Rear Panel	32
2.1.5 Cooling and Airflow	33
2.2 Unpacking the ISR 9024 Interconnect	33
2.3 Mounting the ISR 9024 in the Rack	34
2.4 Installing the ISR 9024 Power Supply Unit	36
2.5 Operating the ISR 9024 Interconnect	36
2.6 Troubleshooting the ISR 9024	36
3 Installing and Maintaining the ISR 9024 S/D Interconnect (RoHS Compliant)	39
3.1 Overview of the ISR 9024 S/D Interconnect	39
3.1.1 Internally Managed ISR 9024 S/D–M Front Panel	40
3.1.1.1 ISR 9024S/D-M Management Card	41
3.1.2 Externally Managed ISR 9024 S/D Front Panel	41
3.1.3 ISR 9024 S/D Rear Panel	42
3.2 Unpacking the ISR 9024 S/D Interconnect	43
3.3 Mounting the ISR 9024 S/D in the Rack	44
3.4 The ISR 9024 S/D Fan Unit	45
3.4.1 Replacing the Fan Unit	46
3.5 Installing the ISR 9024 S/D Power Supply Units	46
3.6 Operating the ISR 9024 S/D Interconnect	47
3.7 Troubleshooting the ISR 9024 S/D	47
4 Installing and Maintaining the ISR 9096 and ISR 9288 Interconnects	49
4.1 Overview of the ISR 9XXX Interconnects	49
4.1.1 ISR ISR 9XXX Interconnect Features	49
4.1.2 ISR ISR 9XXX Interconnect Architecture	50
4.1.3 ISR ISR 9XXX Building Blocks	50
4.1.3.1 Field Replaceable Units	51
4.1.4 ISR ISR 9XXX System Firmware	51
4.1.5 ISR ISR 9XXX Control Panels and Reset Buttons	52
4.2 Unpacking the Chassis	52
4.2.1 Unpacking the ISR 9288	53
4.2.2 Unpacking the ISR 9096	55
4.2.3 Verifying the Shipment	56
4.3 Repacking the Chassis	58
4.4 Mounting the ISR 9XXX in the Rack	58
4.4.1 Resources Required to Perform an Installation	59
4.4.2 Mounting The Chassis in the Rack	59
4.4.3 Installing the Cabling Clamps	63
4.5 Standard Configurations	63
4.6 Post-installation Tasks	67
5 HP 4x DDR IB Switch Module for c-Class BladeSystems Overview	69
5.1 Installing the 4x DDR IB Switch Module	69
5.1.1 Subnet Manager	70
6 Installing Field-Replaceable Units in the ISR 9XXX Chassis	71
6.1 Guidelines for Inserting and Extracting Boards	71
6.2 Installing or Replacing the sFB-12 Fabric Board	72
6.3 Installing or Replacing sLB-24 Line Boards	73
6.4 Installing or Replacing the sMB (Management) Board	75
6.5 Installing and Replacing Power Supply Units	76
6.6 Populating the Router Blade Drawer (sRBD)	77
6.6.1 Overview of the IPR (TCP/IP) Blade	77
6.6.2 Overview of the FCR (Fibre Channel Router) Blade	78
6.6.3 Installing a Router Blade	79
6.7 Installing or Replacing the Router Blade Drawer (sRBD)	80
6.8 Installing and Replacing the sCTRL Board	81
6.9 Installing the sFU-4 Fan Module	82
6.10 Installing the sFU-8 Fan Module	83
7 Cabling the Interconnect	85
7.1 InfiniBand Cabling	85
7.1.1 General InfiniBand Cabling Guidelines	85
7.1.2 Cabling Tables	86
7.1.3 Connecting InfiniBand Cables	86
7.1.4 Cable Routing Procedure for the ISR 9024 or ISR 9024 S/D	86
7.1.5 Cable Routing Procedure for the ISR 9096	87
7.1.6 Connecting the ISR 9096 Router Blades	88
7.1.6.1 Connecting to IPR GbE Ports	88
7.1.6.2 Connecting to FCR FC Ports	88
7.1.7 Cable Routing Procedure for the ISR 9288	88
7.2 Connecting the ISR 9288 Router Blades	89
7.2.1 Connecting to IPR GbE Ports	89
7.2.2 Connecting to FCR FC Ports	90
7.3 Chassis Administrative Connections	90
7.3.1 Cabling the ISR 9024 Management Connections	90
7.3.2 Cabling the ISR 9096 and the ISR 9288 Management Connections	90
7.3.3 Connecting the Management Port Serial Interface	91
8 Installing and Maintaining HCA Cards and Mezzanines	93
8.1 Overview of the Voltaire HCA 400 PCI–X, and HCA 400 PCI-Express Cards	93
8.2 Installing the HCA	94
8.3 Topspin/Mellanox PCI-X HCA	97
8.4 Topspin/Mellanox PCI-Express HCA	98
8.5 Mellanox PCI-X HCA (SDR)	99
8.5.1 Mellanox PCI-X SDR HCA LEDs	99
8.6 Mellanox PCI-Express HCA (SDR)	100
8.6.1 Mellanox PCI-Express SDR HCA LEDs	100
8.7 Mellanox Memory Free PCI-Express HCA (SDR)	101
8.7.1 LEDs	102
8.8 Mellanox Memory Free PCI-Express HCA (DDR)	102
8.8.1 LEDs	103
8.9 Mellanox PCI-Express HCA (DDR)	103
8.9.1 Mellanox PCI-Express DDR HCA LEDs	104
8.9.2 HP HPC 4x DDR IB Mezzanine HCA	105
8.9.3 Operating System and Software Requirements	105
9 Postinstallation Troubleshooting and Diagnostics	107
9.1 Postinstallation Troubleshooting	107
9.2 Startup Checks for the ISR 9XXX	107
9.3 Debugging a Fabric Failure by Using Performance Management (PM)	108
9.4 Identifying a Leaf or Spine Port Malfunction on an ISR 9XXX	108
9.5 Detecting a Failed Port	108
A ISR 9024 Interconnect Specifications	111
B ISR 9024 S/D Interconnect Specifications (RoHS Compliant)	113
C ISR 9096 Interconnect Specifications	117
D ISR 9288 Interconnect Specifications	119
E 4x DDR IB Switch Module Specifications	123
F HCA PCI Card Specifications	125
F.1 Voltaire HCA–400 PCI Card Specifications	125
F.2 Topspin/Mellanox PCI-X HCA Specifications	126
F.3 Topspin/Mellanox PCI-Express HCA Specifications	127
F.4 Mellanox PCI-X HCA Specifications	128
F.5 Mellanox PCI-Express HCA Specifications	128
F.6 Mellanox Memory Free PCI-Express HCA (SDR) Specifications	129
F.7 Mellanox Memory Free PCI-Express HCA (DDR) Specifications	129
F.8 Mellanox PCI-Express DDR HCA Specifications	130
F.9 HPC 4x DDR IB Mezzanine HCA Specifications	130

Match case Limit results 1 per page

HCA 4x DDR IB mezzanine, 105

HCA mellanox memory free PCI-Express DDR, 102

HCA mellanox memory free PCI-Express SDR, 101

HCA mellanox PCI-Express SDR, 100

HCA Mellanox PCI-X (SDR), 99

HCA port

GUID, 109

HCA Topspin/Mellanox PCI-Express, 98

HCA Topspin/Mellanox PCI-X, 97

head shell, 85

hearing protection, 16

high availability, 29, 39

horizontal fan, 51

host channel adapter, 67, 93

(

see also

HCA)

Host channel adapters, 19

(

see also

HCA)

hot components, 16

hot swap, 76

hot-swap

LED, 72

hot–swap, 50

documentation contact, 13

feedback, 13

URL, 13

HyperTerminal, 91

IEC power inlet

ISR 9024, 33

important information, 14

in-band management, 29

InfiniBand

4x DDR IB Switch Module, 69

4X port switch chip, 29

cabling, 85

communication speeds, 19

components, 20

connecting cables, 86

defined, 19

fabric, 19

features, 19

network features, 19

protocol, 20

InfiniBand logical link, 33

InfiniBand physical link, 33

InfiniBand port mezzanines, 30

InfiniBand port status LED

ISR 9024, 31

InfiniBand® Trade Association, 19

InfiniScale III switch chip, 29, 40

installation

safety considerations, 15

Installing

ISR 9096, 49

ISR 9288, 49

installing

cabling clamps, 63

HCA, 94

inter-integrated circuits, 30

(

see also

I²C)

interconnect

cabling, 85

interconnect administration, 26

(

see also

management)

internally managed, 29, 39

internet protocol router, 24, 26, 77

(

see also

IPR)

IPR

ISR 9288, 77

LEDs, 78

IPR blade, 24, 26

ISR 9024

12x ports, 30

administrative cabling connections, 90

cabling procedure, 86

chassis, 29, 40

cooling, 33

DB-9 adapter, 30, 31

EEPROM, 30

EIA-232 serial port, 30

Ethernet port, optional, 33

Ethernet RJ-45 port, 31

externally managed, 29, 30

front panel, 30, 40

front panel, externally managed, 32

front panel, internally managed, 30

identifying, 22

IEC power inlet, 33

InfiniBand link LEDs, 33

InfiniBand port mezzanines, 30

InfiniBand port status LED, 31

Installing and maintaining, 29, 39

installing PSU, 36

internally managed, 29

management card, 31

master mode, 30

(

see also

internally managed)

mini-USB, 30

operating, 36, 47

overview, 29

PCI mezzanine card, 30

PSU, 30

rack kit, 34

rack mounting, 34, 44

rear panel, 32, 42

reset button, 30, 31, 33

resetting, 31, 41

serial console interface, 30, 40

slave mode, 30

(

see also

externally managed)

specifications, 111

status LEDs, 31, 32

troubleshooting, 36, 47

unpacking, 33

ISR 9024 management, 26

(

see also

management)

135